diff --git a/doc/_images/bi_lstm1.jpg b/doc/_images/bi_lstm1.jpg deleted file mode 100644 index adec1606d64d6e35ffe7e62abfa9a09309b05c84..0000000000000000000000000000000000000000 Binary files a/doc/_images/bi_lstm1.jpg and /dev/null differ diff --git a/doc/_images/cifar.png b/doc/_images/cifar.png deleted file mode 100644 index f54a0c58837cb3385b32dc57d02cec92666ef0f1..0000000000000000000000000000000000000000 Binary files a/doc/_images/cifar.png and /dev/null differ diff --git a/doc/_images/curve.jpg b/doc/_images/curve.jpg deleted file mode 100644 index baa35ae7f0a0b6c246f3a0d331735477ab8bcd70..0000000000000000000000000000000000000000 Binary files a/doc/_images/curve.jpg and /dev/null differ diff --git a/doc/_images/encoder-decoder-attention-model1.png b/doc/_images/encoder-decoder-attention-model1.png deleted file mode 100644 index 79f911d4ba12ac0c0d1a936c9df639c302786914..0000000000000000000000000000000000000000 Binary files a/doc/_images/encoder-decoder-attention-model1.png and /dev/null differ diff --git a/doc/_images/feature.jpg b/doc/_images/feature.jpg deleted file mode 100644 index 0e3310e4ace5613917e7779d3198ccbb3cdc5ada..0000000000000000000000000000000000000000 Binary files a/doc/_images/feature.jpg and /dev/null differ diff --git a/doc/_images/graph_construction_example_all.png b/doc/_images/graph_construction_example_all.png new file mode 100644 index 0000000000000000000000000000000000000000..261611a5721f9aa97874f7e6d897fe48cf667db2 Binary files /dev/null and b/doc/_images/graph_construction_example_all.png differ diff --git a/doc/_images/graph_construction_example_forward_backward.png b/doc/_images/graph_construction_example_forward_backward.png new file mode 100644 index 0000000000000000000000000000000000000000..4c69687f4a6a181138f3df72ce5e8aa48487b5be Binary files /dev/null and b/doc/_images/graph_construction_example_forward_backward.png differ diff --git a/doc/_images/graph_construction_example_forward_only.png b/doc/_images/graph_construction_example_forward_only.png new file mode 100644 index 0000000000000000000000000000000000000000..e668c16e0cac73acb4e5dc2b1827557ae77126b4 Binary files /dev/null and b/doc/_images/graph_construction_example_forward_only.png differ diff --git a/doc/_images/image_classification.png b/doc/_images/image_classification.png deleted file mode 100644 index 14f255805081c1b4fab27eaf336fd389fa93ca19..0000000000000000000000000000000000000000 Binary files a/doc/_images/image_classification.png and /dev/null differ diff --git a/doc/_images/lenet.png b/doc/_images/lenet.png deleted file mode 100644 index 1e6f2b32bad797f3fccb929c72a121fc935b0cbb..0000000000000000000000000000000000000000 Binary files a/doc/_images/lenet.png and /dev/null differ diff --git a/doc/_images/lstm.png b/doc/_images/lstm.png deleted file mode 100644 index aaf1fc690da2ffb8418cde5ed81848ddb5263030..0000000000000000000000000000000000000000 Binary files a/doc/_images/lstm.png and /dev/null differ diff --git a/doc/_images/network_arch.png b/doc/_images/network_arch.png deleted file mode 100644 index 4ae7864212f2a0a38102ee7ff600527ea99fec82..0000000000000000000000000000000000000000 Binary files a/doc/_images/network_arch.png and /dev/null differ diff --git a/doc/_images/paddleci.png b/doc/_images/paddleci.png new file mode 100644 index 0000000000000000000000000000000000000000..16087ce059aa3c07ce8c927d983eb86351915825 Binary files /dev/null and b/doc/_images/paddleci.png differ diff --git a/doc/_images/parameters.png b/doc/_images/parameters.png deleted file mode 100644 index 2ec67480951e21f0400bce1c34b3108dcd65c18c..0000000000000000000000000000000000000000 Binary files a/doc/_images/parameters.png and /dev/null differ diff --git a/doc/_images/plot.png b/doc/_images/plot.png deleted file mode 100644 index a31f99791c670e18bb8c62b7604ec8cb0284ffb4..0000000000000000000000000000000000000000 Binary files a/doc/_images/plot.png and /dev/null differ diff --git a/doc/_images/pprof_1.png b/doc/_images/pprof_1.png new file mode 100644 index 0000000000000000000000000000000000000000..8e9edbf377672d0ef40f2fc7bd39e746923550cb Binary files /dev/null and b/doc/_images/pprof_1.png differ diff --git a/doc/_images/pprof_2.png b/doc/_images/pprof_2.png new file mode 100644 index 0000000000000000000000000000000000000000..172ba20399ba974d27f4c072425277b69b02520b Binary files /dev/null and b/doc/_images/pprof_2.png differ diff --git a/doc/_images/rec_regression_network.png b/doc/_images/rec_regression_network.png deleted file mode 100644 index 7d2b54d4fcf560cd5b667628f0012c3822efd9b2..0000000000000000000000000000000000000000 Binary files a/doc/_images/rec_regression_network.png and /dev/null differ diff --git a/doc/_images/stacked_lstm.jpg b/doc/_images/stacked_lstm.jpg deleted file mode 100644 index 4239055050966e0095e188a8c81d860711bce29d..0000000000000000000000000000000000000000 Binary files a/doc/_images/stacked_lstm.jpg and /dev/null differ diff --git a/doc/_sources/about/index_en.rst.txt b/doc/_sources/about/index_en.rst.txt deleted file mode 100644 index 065c430cdea802ed3c9f487cd00255b85a5598a5..0000000000000000000000000000000000000000 --- a/doc/_sources/about/index_en.rst.txt +++ /dev/null @@ -1,14 +0,0 @@ -ABOUT -======= - -PaddlPaddle is an easy-to-use, efficient, flexible and scalable deep learning platform, -which is originally developed by Baidu scientists and engineers for the purpose of applying deep learning to many products at Baidu. - -PaddlePaddle is now open source but far from complete, which is intended to be built upon, improved, scaled, and extended. -We hope to build an active open source community both by providing feedback and by actively contributing to the source code. - - -Credits --------- - -We owe many thanks to `all contributors and developers `_ of PaddlePaddle! diff --git a/doc/_sources/api/index_en.rst.txt b/doc/_sources/api/index_en.rst.txt index 25c1dd00b9cbb3ab647e04cdc2b4c27c552a2332..e6f632e1a5b9c4b50b7c6aa96a120030bd6ce338 100644 --- a/doc/_sources/api/index_en.rst.txt +++ b/doc/_sources/api/index_en.rst.txt @@ -7,3 +7,4 @@ API v2/model_configs.rst v2/data.rst v2/run_logic.rst + v2/fluid.rst diff --git a/doc/_sources/api/v1/trainer_config_helpers/activations.rst.txt b/doc/_sources/api/v1/trainer_config_helpers/activations.rst.txt deleted file mode 100644 index 269e6491e7ebe3899c3fb24fca756a393043473b..0000000000000000000000000000000000000000 --- a/doc/_sources/api/v1/trainer_config_helpers/activations.rst.txt +++ /dev/null @@ -1,108 +0,0 @@ -=========== -Activations -=========== - -BaseActivation -============== - -.. automodule:: paddle.trainer_config_helpers.activations - :members: BaseActivation - :noindex: - -AbsActivation -=============== - -.. automodule:: paddle.trainer_config_helpers.activations - :members: AbsActivation - :noindex: - -ExpActivation -=============== - -.. automodule:: paddle.trainer_config_helpers.activations - :members: ExpActivation - :noindex: - -IdentityActivation -================== - -.. automodule:: paddle.trainer_config_helpers.activations - :members: IdentityActivation - :noindex: - -LinearActivation -================== - -.. automodule:: paddle.trainer_config_helpers.activations - :members: LinearActivation - :noindex: - -LogActivation -================== - -.. automodule:: paddle.trainer_config_helpers.activations - :members: LogActivation - :noindex: - -SquareActivation -================ - -.. automodule:: paddle.trainer_config_helpers.activations - :members: SquareActivation - :noindex: - -SigmoidActivation -================= - -.. automodule:: paddle.trainer_config_helpers.activations - :members: SigmoidActivation - :noindex: - -SoftmaxActivation -================= - -.. automodule:: paddle.trainer_config_helpers.activations - :members: SoftmaxActivation - :noindex: - -SequenceSoftmaxActivation -========================= - -.. automodule:: paddle.trainer_config_helpers.activations - :members: SequenceSoftmaxActivation - :noindex: - -ReluActivation -============== - -.. automodule:: paddle.trainer_config_helpers.activations - :members: ReluActivation - :noindex: - -BReluActivation -=============== - -.. automodule:: paddle.trainer_config_helpers.activations - :members: BReluActivation - :noindex: - -SoftReluActivation -================== - -.. automodule:: paddle.trainer_config_helpers.activations - :members: SoftReluActivation - :noindex: - -TanhActivation -============== - -.. automodule:: paddle.trainer_config_helpers.activations - :members: TanhActivation - :noindex: - -STanhActivation -=============== - -.. automodule:: paddle.trainer_config_helpers.activations - :members: STanhActivation - :noindex: diff --git a/doc/_sources/api/v1/trainer_config_helpers/attrs.rst.txt b/doc/_sources/api/v1/trainer_config_helpers/attrs.rst.txt deleted file mode 100644 index ac63127bf7d9db6351365ab7b58f43db12347a8e..0000000000000000000000000000000000000000 --- a/doc/_sources/api/v1/trainer_config_helpers/attrs.rst.txt +++ /dev/null @@ -1,5 +0,0 @@ -Parameter Attributes -======================= - -.. automodule:: paddle.trainer_config_helpers.attrs - :members: diff --git a/doc/_sources/api/v1/trainer_config_helpers/data_sources.rst.txt b/doc/_sources/api/v1/trainer_config_helpers/data_sources.rst.txt deleted file mode 100644 index b9dd4dda01ae59d1260356aff50ddf298d02c87f..0000000000000000000000000000000000000000 --- a/doc/_sources/api/v1/trainer_config_helpers/data_sources.rst.txt +++ /dev/null @@ -1,7 +0,0 @@ -.. _api_trainer_config_helpers_data_sources: - -DataSources -=========== - -.. automodule:: paddle.trainer_config_helpers.data_sources - :members: diff --git a/doc/_sources/api/v1/trainer_config_helpers/evaluators.rst.txt b/doc/_sources/api/v1/trainer_config_helpers/evaluators.rst.txt deleted file mode 100644 index 11dc735164284d6ed1d661fab1e7690d263b3a7c..0000000000000000000000000000000000000000 --- a/doc/_sources/api/v1/trainer_config_helpers/evaluators.rst.txt +++ /dev/null @@ -1,108 +0,0 @@ -.. _api_trainer_config_helpers_evaluators: - -========== -Evaluators -========== - -Base -==== -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: evaluator_base - :noindex: - -Classification -============== - -classification_error_evaluator ------------------------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: classification_error_evaluator - :noindex: - -auc_evaluator -------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: auc_evaluator - :noindex: - -ctc_error_evaluator -------------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: ctc_error_evaluator - :noindex: - -chunk_evaluator ---------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: chunk_evaluator - :noindex: - -precision_recall_evaluator --------------------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: precision_recall_evaluator - :noindex: - -Rank -==== - -pnpair_evaluator ----------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: pnpair_evaluator - :noindex: - -Utils -===== - -sum_evaluator -------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: sum_evaluator - :noindex: - -column_sum_evaluator --------------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: column_sum_evaluator - :noindex: - -Print -===== - -classification_error_printer_evaluator --------------------------------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: classification_error_printer_evaluator - :noindex: - -gradient_printer_evaluator --------------------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: gradient_printer_evaluator - :noindex: - -maxid_printer_evaluator ------------------------ -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: maxid_printer_evaluator - :noindex: - -maxframe_printer_evaluator ---------------------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: maxframe_printer_evaluator - :noindex: - -seqtext_printer_evaluator -------------------------- -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: seqtext_printer_evaluator - :noindex: - -value_printer_evaluator ------------------------ -.. automodule:: paddle.trainer_config_helpers.evaluators - :members: value_printer_evaluator - :noindex: - diff --git a/doc/_sources/api/v1/trainer_config_helpers/layers.rst.txt b/doc/_sources/api/v1/trainer_config_helpers/layers.rst.txt deleted file mode 100644 index 24389c2d8574dfda4bec9298776aa6b1aee51535..0000000000000000000000000000000000000000 --- a/doc/_sources/api/v1/trainer_config_helpers/layers.rst.txt +++ /dev/null @@ -1,508 +0,0 @@ -.. _api_trainer_config_helpers_layers: - -====== -Layers -====== - -Base -====== - -LayerType ---------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: LayerType - :noindex: - -LayerOutput ------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: LayerOutput - :noindex: - -Data layer -=========== - -.. _api_trainer_config_helpers_layers_data_layer: - -data_layer ----------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: data_layer - :noindex: - -Fully Connected Layers -====================== - -.. _api_trainer_config_helpers_layers_fc_layer: - -fc_layer --------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: fc_layer - :noindex: - -selective_fc_layer ------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: selective_fc_layer - :noindex: - -Conv Layers -=========== - -conv_operator -------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: conv_operator - :noindex: - -conv_projection ---------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: conv_projection - :noindex: - -conv_shift_layer ------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: conv_shift_layer - :noindex: - -img_conv_layer --------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: img_conv_layer - :noindex: - -.. _api_trainer_config_helpers_layers_context_projection: - -context_projection ------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: context_projection - :noindex: - -Image Pooling Layer -=================== - -img_pool_layer --------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: img_pool_layer - :noindex: - -spp_layer --------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: spp_layer - :noindex: - -maxout_layer ------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: maxout_layer - :noindex: - -Norm Layer -========== - -img_cmrnorm_layer ------------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: img_cmrnorm_layer - :noindex: - -batch_norm_layer ---------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: batch_norm_layer - :noindex: - -sum_to_one_norm_layer ---------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: sum_to_one_norm_layer - :noindex: - -Recurrent Layers -================ - -recurrent_layer ------------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: recurrent_layer - :noindex: - -lstmemory ---------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: lstmemory - :noindex: - -grumemory ---------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: grumemory - :noindex: - -Recurrent Layer Group -===================== - -memory ------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: memory - :noindex: - -recurrent_group ---------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: recurrent_group - :noindex: - -lstm_step_layer ---------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: lstm_step_layer - :noindex: - -gru_step_layer ---------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: gru_step_layer - :noindex: - -beam_search ------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: beam_search - :noindex: - -get_output_layer ------------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: get_output_layer - :noindex: - -Mixed Layer -=========== - -.. _api_trainer_config_helpers_layers_mixed_layer: - -mixed_layer ------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: mixed_layer - :noindex: - -.. _api_trainer_config_helpers_layers_embedding_layer: - -embedding_layer ---------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: embedding_layer - :noindex: - -scaling_projection ------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: scaling_projection - :noindex: - -dotmul_projection ------------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: dotmul_projection - :noindex: - -dotmul_operator ---------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: dotmul_operator - :noindex: - -full_matrix_projection ----------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: full_matrix_projection - :noindex: - -identity_projection -------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: identity_projection - :noindex: - - -table_projection ----------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: table_projection - :noindex: - -trans_full_matrix_projection ----------------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: trans_full_matrix_projection - :noindex: - -Aggregate Layers -================ - -.. _api_trainer_config_helpers_layers_pooling_layer: - -pooling_layer -------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: pooling_layer - :noindex: - -.. _api_trainer_config_helpers_layers_last_seq: - -last_seq --------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: last_seq - :noindex: - -.. _api_trainer_config_helpers_layers_first_seq: - -first_seq ---------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: first_seq - :noindex: - -concat_layer ------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: concat_layer - :noindex: - -seq_concat_layer ----------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: seq_concat_layer - :noindex: - -Reshaping Layers -================ - -block_expand_layer ------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: block_expand_layer - :noindex: - -.. _api_trainer_config_helpers_layers_expand_layer: - -expand_layer ------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: expand_layer - :noindex: - -repeat_layer ------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: repeat_layer - :noindex: - -rotate_layer ------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: rotate_layer - :noindex: - -seq_reshape_layer ------------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: seq_reshape_layer - :noindex: - -Math Layers -=========== - -addto_layer ------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: addto_layer - :noindex: - -linear_comb_layer ------------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: linear_comb_layer - :noindex: - -interpolation_layer -------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: interpolation_layer - :noindex: - -bilinear_interp_layer ----------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: bilinear_interp_layer - :noindex: - -power_layer ------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: power_layer - :noindex: - -scaling_layer -------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: scaling_layer - :noindex: - -slope_intercept_layer ----------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: slope_intercept_layer - :noindex: - -tensor_layer ------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: tensor_layer - :noindex: - -.. _api_trainer_config_helpers_layers_cos_sim: - -cos_sim -------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: cos_sim - :noindex: - -trans_layer ------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: trans_layer - :noindex: - -Sampling Layers -=============== - -maxid_layer ------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: maxid_layer - :noindex: - -sampling_id_layer ------------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: sampling_id_layer - :noindex: - -Slicing and Joining Layers -========================== - -pad_layer ------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: pad_layer - :noindex: - -.. _api_trainer_config_helpers_layers_cost_layers: - -Cost Layers -=========== - -cross_entropy -------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: cross_entropy - :noindex: - -cross_entropy_with_selfnorm ---------------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: cross_entropy_with_selfnorm - :noindex: - -multi_binary_label_cross_entropy --------------------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: multi_binary_label_cross_entropy - :noindex: - -mse_cost ---------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: mse_cost - :noindex: - -huber_cost ----------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: huber_cost - :noindex: - -lambda_cost ------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: lambda_cost - :noindex: - -rank_cost ---------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: rank_cost - :noindex: - -sum_cost ---------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: sum_cost - :noindex: - -crf_layer ------------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: crf_layer - :noindex: - -crf_decoding_layer -------------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: crf_decoding_layer - :noindex: - -ctc_layer ------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: ctc_layer - :noindex: - -warp_ctc_layer --------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: warp_ctc_layer - :noindex: - -nce_layer ------------ -.. automodule:: paddle.trainer_config_helpers.layers - :members: nce_layer - :noindex: - -hsigmoid ---------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: hsigmoid - :noindex: - -Check Layer -============ - -eos_layer ------------- -.. automodule:: paddle.trainer_config_helpers.layers - :members: eos_layer - :noindex: diff --git a/doc/_sources/api/v1/trainer_config_helpers/networks.rst.txt b/doc/_sources/api/v1/trainer_config_helpers/networks.rst.txt deleted file mode 100644 index edb53acbf0c31532aa34bda044066fed72eaa426..0000000000000000000000000000000000000000 --- a/doc/_sources/api/v1/trainer_config_helpers/networks.rst.txt +++ /dev/null @@ -1,123 +0,0 @@ -======== -Networks -======== - -The networks module contains pieces of neural network that combine multiple layers. - -NLP -=== - -sequence_conv_pool ------------------- -.. automodule:: paddle.trainer_config_helpers.networks - :members: sequence_conv_pool - :noindex: - -.. _api_trainer_config_helpers_network_text_conv_pool: - -text_conv_pool --------------- -.. automodule:: paddle.trainer_config_helpers.networks - :members: text_conv_pool - :noindex: - -Images -====== - -img_conv_bn_pool ----------------- -.. automodule:: paddle.trainer_config_helpers.networks - :members: img_conv_bn_pool - :noindex: - -img_conv_group --------------- -.. automodule:: paddle.trainer_config_helpers.networks - :members: img_conv_group - :noindex: - -.. _api_trainer_config_helpers_network_simple_img_conv_pool: - -simple_img_conv_pool --------------------- -.. automodule:: paddle.trainer_config_helpers.networks - :members: simple_img_conv_pool - :noindex: - -vgg_16_network ---------------- -.. automodule:: paddle.trainer_config_helpers.networks - :members: vgg_16_network - :noindex: - -Recurrent -========= - -LSTM ----- - -lstmemory_unit -`````````````` -.. automodule:: paddle.trainer_config_helpers.networks - :members: lstmemory_unit - :noindex: - -lstmemory_group -``````````````` -.. automodule:: paddle.trainer_config_helpers.networks - :members: lstmemory_group - :noindex: - -simple_lstm -``````````` -.. automodule:: paddle.trainer_config_helpers.networks - :members: simple_lstm - :noindex: - -bidirectional_lstm -`````````````````` -.. automodule:: paddle.trainer_config_helpers.networks - :members: bidirectional_lstm - :noindex: - -GRU ---- - -gru_unit -```````` -.. automodule:: paddle.trainer_config_helpers.networks - :members: gru_unit - :noindex: - -gru_group -````````` -.. automodule:: paddle.trainer_config_helpers.networks - :members: gru_group - :noindex: - -simple_gru -`````````` -.. automodule:: paddle.trainer_config_helpers.networks - :members: simple_gru - :noindex: - -simple_attention ----------------- -.. automodule:: paddle.trainer_config_helpers.networks - :members: simple_attention - :noindex: - -Miscs -===== - -dropout_layer --------------- -.. automodule:: paddle.trainer_config_helpers.networks - :members: dropout_layer - :noindex: - -outputs -------- -.. automodule:: paddle.trainer_config_helpers.networks - :members: outputs - :noindex: diff --git a/doc/_sources/api/v1/trainer_config_helpers/optimizers.rst.txt b/doc/_sources/api/v1/trainer_config_helpers/optimizers.rst.txt deleted file mode 100644 index d2f4958c92b8e3b7426945f1af07112ab4071136..0000000000000000000000000000000000000000 --- a/doc/_sources/api/v1/trainer_config_helpers/optimizers.rst.txt +++ /dev/null @@ -1,61 +0,0 @@ -.. _api_trainer_config_helpers_optimizers: - -========== -Optimizers -========== - -BaseSGDOptimizer -================ -.. automodule:: paddle.trainer_config_helpers.optimizers - :members: BaseSGDOptimizer - :noindex: - -MomentumOptimizer -================= -.. automodule:: paddle.trainer_config_helpers.optimizers - :members: MomentumOptimizer - :noindex: - -AdamOptimizer -============= -.. automodule:: paddle.trainer_config_helpers.optimizers - :members: AdamOptimizer - :noindex: - -AdamaxOptimizer -================ -.. automodule:: paddle.trainer_config_helpers.optimizers - :members: AdamaxOptimizer - :noindex: - -AdaGradOptimizer -================ -.. automodule:: paddle.trainer_config_helpers.optimizers - :members: AdaGradOptimizer - :noindex: - -DecayedAdaGradOptimizer -======================= -.. automodule:: paddle.trainer_config_helpers.optimizers - :members: DecayedAdaGradOptimizer - :noindex: - -AdaDeltaOptimizer -================= -.. automodule:: paddle.trainer_config_helpers.optimizers - :members: AdaDeltaOptimizer - :noindex: - -RMSPropOptimizer -================ -.. automodule:: paddle.trainer_config_helpers.optimizers - :members: RMSPropOptimizer - :noindex: - -.. _api_trainer_config_helpers_optimizers_settings: - -settings -======== -.. automodule:: paddle.trainer_config_helpers.optimizers - :members: settings - :noindex: diff --git a/doc/_sources/api/v1/trainer_config_helpers/poolings.rst.txt b/doc/_sources/api/v1/trainer_config_helpers/poolings.rst.txt deleted file mode 100644 index 66566809d26f59263597b5286c5b27e0bbc9415a..0000000000000000000000000000000000000000 --- a/doc/_sources/api/v1/trainer_config_helpers/poolings.rst.txt +++ /dev/null @@ -1,33 +0,0 @@ -======== -Poolings -======== - -BasePoolingType -=============== -.. automodule:: paddle.trainer_config_helpers.poolings - :members: BasePoolingType - :noindex: - -AvgPooling -========== -.. automodule:: paddle.trainer_config_helpers.poolings - :members: AvgPooling - :noindex: - -MaxPooling -========== -.. automodule:: paddle.trainer_config_helpers.poolings - :members: MaxPooling - :noindex: - -SumPooling -========== -.. automodule:: paddle.trainer_config_helpers.poolings - :members: SumPooling - :noindex: - -SquareRootNPooling -================== -.. automodule:: paddle.trainer_config_helpers.poolings - :members: SquareRootNPooling - :noindex: diff --git a/doc/_sources/api/v2/config/activation.rst.txt b/doc/_sources/api/v2/config/activation.rst.txt index eca3ce03bcdc599edca802d8dfca48d4f28275a2..5317e66b64bbd85c61f19700a9d2c1d239dee573 100644 --- a/doc/_sources/api/v2/config/activation.rst.txt +++ b/doc/_sources/api/v2/config/activation.rst.txt @@ -99,3 +99,10 @@ STanh .. automodule:: paddle.v2.activation :members: STanh :noindex: + +SoftSign +======== + +.. automodule:: paddle.v2.activation + :members: SoftSign + :noindex: diff --git a/doc/_sources/api/v2/config/evaluators.rst.txt b/doc/_sources/api/v2/config/evaluators.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..9ac972fb193a2fb525edc507f7ba1303d2c8eabe --- /dev/null +++ b/doc/_sources/api/v2/config/evaluators.rst.txt @@ -0,0 +1,110 @@ +.. _api_v2: + +========== +Evaluators +========== + +Classification +============== + +classification_error +-------------------- +.. automodule:: paddle.v2.evaluator + :members: classification_error + :noindex: + +auc +--- +.. automodule:: paddle.v2.evaluator + :members: auc + :noindex: + +ctc_error +--------- +.. automodule:: paddle.v2.evaluator + :members: ctc_error + :noindex: + +chunk +----- +.. automodule:: paddle.v2.evaluator + :members: chunk + :noindex: + +precision_recall +---------------- +.. automodule:: paddle.v2.evaluator + :members: precision_recall + :noindex: + +Rank +==== + +pnpair +------ +.. automodule:: paddle.v2.evaluator + :members: pnpair + :noindex: + +Utils +===== + +sum +--- +.. automodule:: paddle.v2.evaluator + :members: sum + :noindex: + +column_sum +---------- +.. automodule:: paddle.v2.evaluator + :members: column_sum + :noindex: + +Print +===== + +classification_error_printer +---------------------------- +.. automodule:: paddle.v2.evaluator + :members: classification_error_printer + :noindex: + +gradient_printer +---------------- +.. automodule:: paddle.v2.evaluator + :members: gradient_printer + :noindex: + +maxid_printer +------------- +.. automodule:: paddle.v2.evaluator + :members: maxid_printer + :noindex: + +maxframe_printer +---------------- +.. automodule:: paddle.v2.evaluator + :members: maxframe_printer + :noindex: + +seqtext_printer +--------------- +.. automodule:: paddle.v2.evaluator + :members: seqtext_printer + :noindex: + +value_printer +------------- +.. automodule:: paddle.v2.evaluator + :members: value_printer + :noindex: + +Detection +===== + +detection_map +------------- +.. automodule:: paddle.v2.evaluator + :members: detection_map + :noindex: diff --git a/doc/_sources/api/v2/config/layer.rst.txt b/doc/_sources/api/v2/config/layer.rst.txt index 2a02baf17ba0d1119a8d222024616ef8ae33f8d5..c3f9c18d0663a7a24880b441981875c1e4f015aa 100644 --- a/doc/_sources/api/v2/config/layer.rst.txt +++ b/doc/_sources/api/v2/config/layer.rst.txt @@ -54,18 +54,23 @@ img_conv .. _api_v2.layer_context_projection: -context_projection +context_projection ------------------ .. autoclass:: paddle.v2.layer.context_projection :noindex: +row_conv +-------- +.. autoclass:: paddle.v2.layer.row_conv + :noindex: + Image Pooling Layer =================== img_pool -------- .. autoclass:: paddle.v2.layer.img_pool - :noindex: + :noindex: spp --- @@ -77,6 +82,11 @@ maxout .. autoclass:: paddle.v2.layer.maxout :noindex: +roi_pool +-------- +.. autoclass:: paddle.v2.layer.roi_pool + :noindex: + Norm Layer ========== @@ -94,12 +104,17 @@ sum_to_one_norm --------------- .. autoclass:: paddle.v2.layer.sum_to_one_norm :noindex: - + cross_channel_norm ------------------ .. autoclass:: paddle.v2.layer.cross_channel_norm :noindex: - + +row_l2_norm +----------- +.. autoclass:: paddle.v2.layer.row_l2_norm + :noindex: + Recurrent Layers ================ @@ -130,7 +145,7 @@ recurrent_group --------------- .. autoclass:: paddle.v2.layer.recurrent_group :noindex: - + lstm_step --------- .. autoclass:: paddle.v2.layer.lstm_step @@ -145,12 +160,12 @@ beam_search ------------ .. autoclass:: paddle.v2.layer.beam_search :noindex: - + get_output ---------- .. autoclass:: paddle.v2.layer.get_output :noindex: - + Mixed Layer =========== @@ -193,6 +208,10 @@ identity_projection .. autoclass:: paddle.v2.layer.identity_projection :noindex: +slice_projection +------------------- +.. autoclass:: paddle.v2.layer.slice_projection + :noindex: table_projection ---------------- @@ -203,10 +222,15 @@ trans_full_matrix_projection ---------------------------- .. autoclass:: paddle.v2.layer.trans_full_matrix_projection :noindex: - + Aggregate Layers ================ +AggregateLevel +-------------- +.. autoclass:: paddle.v2.layer.AggregateLevel + :noindex: + .. _api_v2.layer_pooling: pooling @@ -238,6 +262,21 @@ seq_concat .. autoclass:: paddle.v2.layer.seq_concat :noindex: +seq_slice +--------- +.. autoclass:: paddle.v2.layer.seq_slice + :noindex: + +kmax_sequence_score +------------------- +.. autoclass:: paddle.v2.layer.kmax_sequence_score + :noindex: + +sub_nested_seq +-------------- +.. autoclass:: paddle.v2.layer.sub_nested_seq + :noindex: + Reshaping Layers ================ @@ -248,6 +287,11 @@ block_expand .. _api_v2.layer_expand: +ExpandLevel +----------- +.. autoclass:: paddle.v2.layer.ExpandLevel + :noindex: + expand ------ .. autoclass:: paddle.v2.layer.expand @@ -291,6 +335,16 @@ bilinear_interp .. autoclass:: paddle.v2.layer.bilinear_interp :noindex: +dot_prod +--------- +.. autoclass:: paddle.v2.layer.dot_prod + :noindex: + +out_prod +-------- +.. autoclass:: paddle.v2.layer.out_prod + :noindex: + power ----- .. autoclass:: paddle.v2.layer.power @@ -301,6 +355,16 @@ scaling .. autoclass:: paddle.v2.layer.scaling :noindex: +clip +---- +.. autoclass:: paddle.v2.layer.clip + :noindex: + +resize +------ +.. autoclass:: paddle.v2.layer.resize + :noindex: + slope_intercept --------------- .. autoclass:: paddle.v2.layer.slope_intercept @@ -318,11 +382,21 @@ cos_sim .. autoclass:: paddle.v2.layer.cos_sim :noindex: +l2_distance +----------- +.. autoclass:: paddle.v2.layer.l2_distance + :noindex: + trans ----- .. autoclass:: paddle.v2.layer.trans :noindex: +scale_shift +----------- +.. autoclass:: paddle.v2.layer.scale_shift + :noindex: + Sampling Layers =============== @@ -336,6 +410,19 @@ sampling_id .. autoclass:: paddle.v2.layer.sampling_id :noindex: +multiplex +--------- +.. autoclass:: paddle.v2.layer.multiplex + :noindex: + +Factorization Machine Layer +============================ + +factorization_machine +--------------------- +.. autoclass:: paddle.v2.layer.factorization_machine + :noindex: + Slicing and Joining Layers ========================== @@ -364,9 +451,14 @@ multi_binary_label_cross_entropy_cost .. autoclass:: paddle.v2.layer.multi_binary_label_cross_entropy_cost :noindex: -huber_cost ----------- -.. autoclass:: paddle.v2.layer.huber_cost +huber_regression_cost +------------------------- +.. autoclass:: paddle.v2.layer.huber_regression_cost + :noindex: + +huber_classification_cost +------------------------- +.. autoclass:: paddle.v2.layer.huber_classification_cost :noindex: lambda_cost @@ -374,9 +466,9 @@ lambda_cost .. autoclass:: paddle.v2.layer.lambda_cost :noindex: -mse_cost +square_error_cost -------- -.. autoclass:: paddle.v2.layer.mse_cost +.. autoclass:: paddle.v2.layer.square_error_cost :noindex: rank_cost @@ -419,10 +511,49 @@ hsigmoid .. autoclass:: paddle.v2.layer.hsigmoid :noindex: -Check Layer +smooth_l1_cost +-------------- +.. autoclass:: paddle.v2.layer.smooth_l1_cost + :noindex: + +multibox_loss +-------------- +.. autoclass:: paddle.v2.layer.multibox_loss + :noindex: + +Check Layer ============ eos --- .. autoclass:: paddle.v2.layer.eos :noindex: + +Miscs +===== + +dropout +-------------- +.. autoclass:: paddle.v2.layer.dropout + :noindex: + +Activation with learnable parameter +=================================== + +prelu +-------- +.. autoclass:: paddle.v2.layer.prelu + :noindex: + +gated_unit +----------- +.. autoclass:: paddle.v2.layer.gated_unit + :noindex: + +Detection output Layer +====================== + +detection_output +---------------- +.. autoclass:: paddle.v2.layer.detection_output + :noindex: diff --git a/doc/_sources/api/v2/config/networks.rst.txt b/doc/_sources/api/v2/config/networks.rst.txt index 6f209bc95bec7279051118bb857a96515e0371a9..048379cf01f4aec5e73e2fe3ddfa728f3c17a5d1 100644 --- a/doc/_sources/api/v2/config/networks.rst.txt +++ b/doc/_sources/api/v2/config/networks.rst.txt @@ -44,6 +44,12 @@ simple_img_conv_pool :members: simple_img_conv_pool :noindex: +small_vgg +--------- +.. automodule:: paddle.v2.networks + :members: small_vgg + :noindex: + vgg_16_network --------------- .. automodule:: paddle.v2.networks @@ -101,17 +107,26 @@ simple_gru :members: simple_gru :noindex: +simple_gru2 +``````````` +.. automodule:: paddle.v2.networks + :members: simple_gru2 + :noindex: + +bidirectional_gru +`````````````````` +.. automodule:: paddle.v2.networks + :members: bidirectional_gru + :noindex: + simple_attention ---------------- .. automodule:: paddle.v2.networks :members: simple_attention :noindex: -Miscs -===== - -dropout_layer --------------- +dot_product_attention +--------------------- .. automodule:: paddle.v2.networks - :members: dropout_layer + :members: dot_product_attention :noindex: diff --git a/doc/_sources/api/v2/data.rst.txt b/doc/_sources/api/v2/data.rst.txt index fef87c4fbdb452771ecdb361c6eeae5b32bcee14..b56c7332cc284649c7e04328e51a7faa78593a39 100644 --- a/doc/_sources/api/v2/data.rst.txt +++ b/doc/_sources/api/v2/data.rst.txt @@ -2,112 +2,9 @@ Data Reader Interface and DataSets ================================== +.. toctree:: + :maxdepth: 1 -DataTypes -========= - -.. automodule:: paddle.v2.data_type - :members: - :noindex: - -DataFeeder -========== - -.. automodule:: paddle.v2.data_feeder - :members: - :noindex: - -Reader -====== - -.. automodule:: paddle.v2.reader - :members: - :noindex: - -.. automodule:: paddle.v2.reader.creator - :members: - :noindex: - -minibatch -========= - -.. automodule:: paddle.v2.minibatch - :members: - :noindex: - -Dataset -======= - -.. automodule:: paddle.v2.dataset - :members: - :noindex: - -mnist -+++++ - -.. automodule:: paddle.v2.dataset.mnist - :members: - :noindex: - -cifar -+++++ - -.. automodule:: paddle.v2.dataset.cifar - :members: - :noindex: - -conll05 -+++++++ - -.. automodule:: paddle.v2.dataset.conll05 - :members: get_dict,get_embedding,test - :noindex: - -imdb -++++ - -.. automodule:: paddle.v2.dataset.imdb - :members: - :noindex: - -imikolov -++++++++ - -.. automodule:: paddle.v2.dataset.imikolov - :members: - :noindex: - -movielens -+++++++++ - -.. automodule:: paddle.v2.dataset.movielens - :members: - :noindex: - -.. autoclass:: paddle.v2.dataset.movielens.MovieInfo - :noindex: - -.. autoclass:: paddle.v2.dataset.movielens.UserInfo - :noindex: - -sentiment -+++++++++ - -.. automodule:: paddle.v2.dataset.sentiment - :members: - :noindex: - -uci_housing -+++++++++++ - -.. automodule:: paddle.v2.dataset.uci_housing - :members: - :noindex: - -wmt14 -+++++ - -.. automodule:: paddle.v2.dataset.wmt14 - :members: - :noindex: - + data/data_reader.rst + data/image.rst + data/dataset.rst diff --git a/doc/_sources/api/v2/data/data_reader.rst.txt b/doc/_sources/api/v2/data/data_reader.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..2ccfec9c284877a7576e9751526b169a4ac78d8e --- /dev/null +++ b/doc/_sources/api/v2/data/data_reader.rst.txt @@ -0,0 +1,36 @@ +===================== +Data Reader Interface +===================== + + +DataTypes +========= + +.. automodule:: paddle.v2.data_type + :members: + :noindex: + +DataFeeder +========== + +.. automodule:: paddle.v2.data_feeder + :members: + :noindex: + +Reader +====== + +.. automodule:: paddle.v2.reader + :members: + :noindex: + +.. automodule:: paddle.v2.reader.creator + :members: + :noindex: + +minibatch +========= + +.. automodule:: paddle.v2.minibatch + :members: + :noindex: diff --git a/doc/_sources/api/v2/data/dataset.rst.txt b/doc/_sources/api/v2/data/dataset.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..6a8ecc5bb1d855e0ded3719943ab3adb810de365 --- /dev/null +++ b/doc/_sources/api/v2/data/dataset.rst.txt @@ -0,0 +1,75 @@ +Dataset +======= + +.. automodule:: paddle.v2.dataset + :members: + :noindex: + +mnist ++++++ + +.. automodule:: paddle.v2.dataset.mnist + :members: + :noindex: + +cifar ++++++ + +.. automodule:: paddle.v2.dataset.cifar + :members: + :noindex: + +conll05 ++++++++ + +.. automodule:: paddle.v2.dataset.conll05 + :members: get_dict,get_embedding,test + :noindex: + +imdb +++++ + +.. automodule:: paddle.v2.dataset.imdb + :members: + :noindex: + +imikolov +++++++++ + +.. automodule:: paddle.v2.dataset.imikolov + :members: + :noindex: + +movielens ++++++++++ + +.. automodule:: paddle.v2.dataset.movielens + :members: + :noindex: + +.. autoclass:: paddle.v2.dataset.movielens.MovieInfo + :noindex: + +.. autoclass:: paddle.v2.dataset.movielens.UserInfo + :noindex: + +sentiment ++++++++++ + +.. automodule:: paddle.v2.dataset.sentiment + :members: + :noindex: + +uci_housing ++++++++++++ + +.. automodule:: paddle.v2.dataset.uci_housing + :members: + :noindex: + +wmt14 ++++++ + +.. automodule:: paddle.v2.dataset.wmt14 + :members: + :noindex: diff --git a/doc/_sources/api/v2/data/image.rst.txt b/doc/_sources/api/v2/data/image.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..97651ffa6be56cf3ecaca2caca38a353fa5c1f49 --- /dev/null +++ b/doc/_sources/api/v2/data/image.rst.txt @@ -0,0 +1,5 @@ +Image Interface +=============== + +.. automodule:: paddle.v2.image + :members: diff --git a/doc/_sources/api/v2/fluid.rst.txt b/doc/_sources/api/v2/fluid.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..43fc19dc492bbc119f2356034b81c65e443db2fa --- /dev/null +++ b/doc/_sources/api/v2/fluid.rst.txt @@ -0,0 +1,18 @@ +====================== +Fluid +====================== + +.. toctree:: + :maxdepth: 1 + + fluid/layers.rst + fluid/data_feeder.rst + fluid/executor.rst + fluid/initializer.rst + fluid/evaluator.rst + fluid/nets.rst + fluid/optimizer.rst + fluid/param_attr.rst + fluid/profiler.rst + fluid/regularizer.rst + diff --git a/doc/_sources/api/v2/fluid/data_feeder.rst.txt b/doc/_sources/api/v2/fluid/data_feeder.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..0fa78f7dfb04c13be7eb83b7fd35cb03f2f4a7fa --- /dev/null +++ b/doc/_sources/api/v2/fluid/data_feeder.rst.txt @@ -0,0 +1,9 @@ +=========== +DataFeeder +=========== + +DataFeeder +----------- +.. automodule:: paddle.v2.fluid.data_feeder + :members: DataFeeder + :noindex: diff --git a/doc/_sources/api/v2/fluid/evaluator.rst.txt b/doc/_sources/api/v2/fluid/evaluator.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..a23f3301d0331e0ea3733f06444515eb4680cd31 --- /dev/null +++ b/doc/_sources/api/v2/fluid/evaluator.rst.txt @@ -0,0 +1,9 @@ +=========== +Evaluator +=========== + +Evaluator +----------- +.. automodule:: paddle.v2.fluid.evaluator + :members: Evaluator + :noindex: diff --git a/doc/_sources/api/v2/fluid/executor.rst.txt b/doc/_sources/api/v2/fluid/executor.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..3a283538c120cfa1ef646c390bb71c6251c23675 --- /dev/null +++ b/doc/_sources/api/v2/fluid/executor.rst.txt @@ -0,0 +1,9 @@ +=========== +Executor +=========== + +Executor +----------- +.. automodule:: paddle.v2.fluid.executor + :members: Executor + :noindex: diff --git a/doc/_sources/api/v2/fluid/initializer.rst.txt b/doc/_sources/api/v2/fluid/initializer.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..8f587837e9873370722062404f511654a9460587 --- /dev/null +++ b/doc/_sources/api/v2/fluid/initializer.rst.txt @@ -0,0 +1,50 @@ +=========== +Initializer +=========== + + + +Initializer +----------- +.. automodule:: paddle.v2.fluid.initializer + :members: Initializer + :noindex: + + + +ConstantInitializer +------------------- +.. automodule:: paddle.v2.fluid.initializer + :members: ConstantInitializer + :noindex: + + + +UniformInitializer +------------------ +.. automodule:: paddle.v2.fluid.initializer + :members: UniformInitializer + :noindex: + + + +NormalInitializer +----------------- +.. automodule:: paddle.v2.fluid.initializer + :members: NormalInitializer + :noindex: + + +XavierInitializer +----------------- +.. automodule:: paddle.v2.fluid.initializer + :members: XavierInitializer + :noindex: + + +MSRAInitializer +--------------- +.. automodule:: paddle.v2.fluid.initializer + :members: MSRAInitializer + :noindex: + diff --git a/doc/_sources/api/v2/fluid/layers.rst.txt b/doc/_sources/api/v2/fluid/layers.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..89e5fec13bf9062dc7a7187b1334c8f5486a980b --- /dev/null +++ b/doc/_sources/api/v2/fluid/layers.rst.txt @@ -0,0 +1,302 @@ +========== +Layers +========== + + +fc +--- +.. autofunction:: paddle.v2.fluid.layers.fc + :noindex: + +embedding +--------- +.. autofunction:: paddle.v2.fluid.layers.embedding + :noindex: + +dynamic_lstm +------------ +.. autofunction:: paddle.v2.fluid.layers.dynamic_lstm + :noindex: + +data +--------- +.. autofunction:: paddle.v2.fluid.layers.data + :noindex: + +mean +--------- +.. autofunction:: paddle.v2.fluid.layers.mean + :noindex: + +mul +--------- +.. autofunction:: paddle.v2.fluid.layers.mul + :noindex: + +elementwise_add +--------------- +.. autofunction:: paddle.v2.fluid.layers.elementwise_add + :noindex: + +elementwise_div +--------------- +.. autofunction:: paddle.v2.fluid.layers.elementwise_div + :noindex: + + +dropout +--------- +.. autofunction:: paddle.v2.fluid.layers.dropout + :noindex: + + +reshape +--------- +.. autofunction:: paddle.v2.fluid.layers.reshape + :noindex: + + +sigmoid +--------- +.. autofunction:: paddle.v2.fluid.layers.sigmoid + :noindex: + + +scale +--------- +.. autofunction:: paddle.v2.fluid.layers.scale + :noindex: + + +reshape +--------- +.. autofunction:: paddle.v2.fluid.layers.reshape + :noindex: + + +transpose +--------- +.. autofunction:: paddle.v2.fluid.layers.transpose + :noindex: + + +sigmoid_cross_entropy_with_logits +--------- +.. autofunction:: paddle.v2.fluid.layers.esigmoid_cross_entropy_with_logits + :noindex: + + +cast +--------- +.. autofunction:: paddle.v2.fluid.layers.cast + :noindex: + + +concat +--------- +.. autofunction:: paddle.v2.fluid.layers.concat + :noindex: + + +sums +--------- +.. autofunction:: paddle.v2.fluid.layers.sums + :noindex: + + +linear_chain_crf +--------- +.. autofunction:: paddle.v2.fluid.layers.linear_chain_crf + :noindex: + + +assign +--------- +.. autofunction:: paddle.v2.fluid.layers.embedding + :noindex: + + +split_lod_tensor +--------- +.. autofunction:: paddle.v2.fluid.layers.split_lod_tensor + :noindex: + + +merge_lod_tensor +--------- +.. autofunction:: paddle.v2.fluid.layers.merge_lod_tensor + :noindex: + +cos_sim +--------- +.. autofunction:: paddle.v2.fluid.layers.cos_sim + :noindex: + + +cross_entropy +--------- +.. autofunction:: paddle.v2.fluid.layers.cross_entropy + :noindex: + + + +square_error_cost +--------- +.. autofunction:: paddle.v2.fluid.layers.square_error_cost + :noindex: + + +accuracy +--------- +.. autofunction:: paddle.v2.fluid.layers.accuracy + :noindex: + + +sequence_conv +--------- +.. autofunction:: paddle.v2.fluid.layers.sequence_conv + :noindex: + + +conv2d +--------- +.. autofunction:: paddle.v2.fluid.layers.conv2d + :noindex: + + +sequence_pool +--------- +.. autofunction:: paddle.v2.fluid.layers.sequence_pool + :noindex: + + +pool2d +--------- +.. autofunction:: paddle.v2.fluid.layers.pool2d + :noindex: + + +batch_norm +--------- +.. autofunction:: paddle.v2.fluid.layers.batch_norm + :noindex: + + +beam_search_decode +--------- +.. autofunction:: paddle.v2.fluid.layers.beam_search_decode + :noindex: + + +lstm +--------- +.. autofunction:: paddle.v2.fluid.layers.lstm + :noindex: + + +lod_rank_table +--------- +.. autofunction:: paddle.v2.fluid.layers.lod_rank_table + :noindex: + + +max_sequence_len +--------- +.. autofunction:: paddle.v2.fluid.layers.max_sequence_len + :noindex: + + +topk +--------- +.. autofunction:: paddle.v2.fluid.layers.topk + :noindex: + + +lod_tensor_to_array +--------- +.. autofunction:: paddle.v2.fluid.layers.lod_tensor_to_array + :noindex: + + + +array_to_lod_tensor +--------- +.. autofunction:: paddle.v2.fluid.layers.array_to_lod_tensor + :noindex: + + + + +fill_constant +--------- +.. autofunction:: paddle.v2.fluid.layers.fill_constant + :noindex: + + + +fill_constant_batch_size_like +--------- +.. autofunction:: paddle.v2.fluid.layers.fill_constant_batch_size_like + :noindex: + + +ones +--------- +.. autofunction:: paddle.v2.fluid.layers.ones + :noindex: + + +zeros +--------- +.. autofunction:: paddle.v2.fluid.layers.zeros + :noindex: + + +increment +--------- +.. autofunction:: paddle.v2.fluid.layers.increment + :noindex: + + +array_write +--------- +.. autofunction:: paddle.v2.fluid.layers.array_write + :noindex: + + + +create_array +--------- +.. autofunction:: paddle.v2.fluid.layers.create_array + :noindex: + + +less_than +--------- +.. autofunction:: paddle.v2.fluid.layers.less_than + :noindex: + + +array_read +--------- +.. autofunction:: paddle.v2.fluid.layers.array_read + :noindex: + + +shrink_memory +--------- +.. autofunction:: paddle.v2.fluid.layers.shrink_memory + :noindex: + + +array_length +--------- +.. autofunction:: paddle.v2.fluid.layers.array_length + :noindex: + + +conv2d_transpose +--------- +.. autofunction:: paddle.v2.fluid.layers.conv2d_transpose + :noindex: + diff --git a/doc/_sources/api/v2/fluid/nets.rst.txt b/doc/_sources/api/v2/fluid/nets.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..2c3d075422de29c96e25458e831133a30270dd39 --- /dev/null +++ b/doc/_sources/api/v2/fluid/nets.rst.txt @@ -0,0 +1,22 @@ +=========== +Nets +=========== + +simple_img_conv_pool +----------- +.. autofunction:: paddle.v2.fluid.nets.simple_img_conv_pool + :noindex: + + +img_conv_group +----------- +.. autofunction:: paddle.v2.fluid.nets.img_conv_group + :noindex: + + +sequence_conv_pool +----------- +.. autofunction:: paddle.v2.fluid.nets.sequence_conv_pool + :noindex: + + diff --git a/doc/_sources/api/v2/fluid/optimizer.rst.txt b/doc/_sources/api/v2/fluid/optimizer.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..233762fcdfb39e592740adef6721a556fae3feef --- /dev/null +++ b/doc/_sources/api/v2/fluid/optimizer.rst.txt @@ -0,0 +1,54 @@ +=========== +Optimizer +=========== + +Optimizer +----------- +.. automodule:: paddle.v2.fluid.optimizer + :members: Optimizer + :noindex: + + +SGDOptimizer +----------- +.. automodule:: paddle.v2.fluid.optimizer + :members: SGDOptimizer + :noindex: + + + +MomentumOptimizer +----------- +.. automodule:: paddle.v2.fluid.optimizer + :members: MomentumOptimizer + :noindex: + + + +AdagradOptimizer +----------- +.. automodule:: paddle.v2.fluid.optimizer + :members: AdagradOptimizer + :noindex: + + +AdamOptimizer +----------- +.. automodule:: paddle.v2.fluid.optimizer + :members: AdamOptimizer + :noindex: + + +AdamaxOptimizer +----------- +.. automodule:: paddle.v2.fluid.optimizer + :members: AdamaxOptimizer + :noindex: + + +DecayedAdagradOptimizer +----------- +.. automodule:: paddle.v2.fluid.optimizer + :members: DecayedAdagradOptimizer + :noindex: + diff --git a/doc/_sources/api/v2/fluid/param_attr.rst.txt b/doc/_sources/api/v2/fluid/param_attr.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..ca0c8af9e8c4f2271de7a131ad0d27c0e8635f50 --- /dev/null +++ b/doc/_sources/api/v2/fluid/param_attr.rst.txt @@ -0,0 +1,11 @@ +=========== +ParamAttr +=========== + + + +ParamAttr +----------- +.. automodule:: paddle.v2.fluid.param_attr + :members: ParamAttr + :noindex: diff --git a/doc/_sources/api/v2/fluid/profiler.rst.txt b/doc/_sources/api/v2/fluid/profiler.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..7d4042d1f41c12c4a551ba6576559d612116872a --- /dev/null +++ b/doc/_sources/api/v2/fluid/profiler.rst.txt @@ -0,0 +1,10 @@ +=========== +Profiler +=========== + + + +Profiler +----------- +.. autofunction:: paddle.v2.fluid.profiler.cuda_profiler + :noindex: diff --git a/doc/_sources/api/v2/fluid/regularizer.rst.txt b/doc/_sources/api/v2/fluid/regularizer.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..3af2b07d2ae55d99df705fbf1ad2402eee05c435 --- /dev/null +++ b/doc/_sources/api/v2/fluid/regularizer.rst.txt @@ -0,0 +1,25 @@ +=========== +Regularizer +=========== + +WeightDecayRegularizer +----------- +.. automodule:: paddle.v2.fluid.regularizer + :members: WeightDecayRegularizer + :noindex: + + +L2DecayRegularizer +----------- +.. automodule:: paddle.v2.fluid.regularizer + :members: L2DecayRegularizer + :noindex: + + + +L1DecayRegularizer +----------- +.. automodule:: paddle.v2.fluid.regularizer + :members: L1DecayRegularizer + + diff --git a/doc/_sources/api/v2/model_configs.rst.txt b/doc/_sources/api/v2/model_configs.rst.txt index a5fae7e29e56d9e236d489353a5c8967d1954641..992b559cbd87244612521d4c96f84f997d6c4196 100644 --- a/doc/_sources/api/v2/model_configs.rst.txt +++ b/doc/_sources/api/v2/model_configs.rst.txt @@ -6,6 +6,7 @@ Model Configuration config/activation.rst config/layer.rst + config/evaluators.rst config/optimizer.rst config/pooling.rst config/networks.rst diff --git a/doc/_sources/design/api.md.txt b/doc/_sources/design/api.md.txt index 8185d2af0ea264a2e7b4e28b9ed05279e4a22014..e6a4638d9100d9b07c3ee6b92b530a17eae1c162 100644 --- a/doc/_sources/design/api.md.txt +++ b/doc/_sources/design/api.md.txt @@ -3,7 +3,7 @@ ## Ingredients As our design principle is starting from the essence: how could we -allow users to express and solve their problems at neural networks. +allow users to express and solve their problems as neural networks. Some essential concepts that our API have to provide include: 1. A *topology* is an expression of *layers*. @@ -233,7 +233,7 @@ paddle.dist_train(model, num_parameter_servers=15) ``` -The pseudo code if `paddle.dist_train` is as follows: +The pseudo code of `paddle.dist_train` is as follows: ```python def dist_train(topology, parameters, trainer, reader, ...): diff --git a/doc/_sources/design/auto_gradient_check.md.txt b/doc/_sources/design/auto_gradient_check.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..f9991541bc51c6e13ffce4e9cec60f73dc800121 --- /dev/null +++ b/doc/_sources/design/auto_gradient_check.md.txt @@ -0,0 +1,146 @@ +## Auto Gradient Checker Design + +## Backgraound： +- Generally, it is easy to check whether the forward computation of an Operator is correct or not. However, backpropagation is a notoriously difficult algorithm to debug and get right: + 1. you should get the right backpropagation formula according to the forward computation. + 2. you should implement it right in CPP. + 3. it's difficult to prepare test data. + +- Auto gradient checking gets a numerical gradient by forward Operator and use it as a reference of the backward Operator's result. It has several advantages: + 1. numerical gradient checker only need forward operator. + 2. user only need to prepare the input data for forward Operator. + +## Mathematical Theory +The following two document from Stanford has a detailed explanation of how to get numerical gradient and why it's useful. + +- [Gradient checking and advanced optimization(en)](http://deeplearning.stanford.edu/wiki/index.php/Gradient_checking_and_advanced_optimization) +- [Gradient checking and advanced optimization(cn)](http://ufldl.stanford.edu/wiki/index.php/%E6%A2%AF%E5%BA%A6%E6%A3%80%E9%AA%8C%E4%B8%8E%E9%AB%98%E7%BA%A7%E4%BC%98%E5%8C%96) + + +## Numeric Gradient Implementation +### Python Interface +```python +def get_numerical_gradient(op, + input_values, + output_name, + input_to_check, + delta=0.005, + local_scope=None): + """ + Get Numeric Gradient for an operator's input. + + :param op: C++ operator instance, could be an network + :param input_values: The input variables. Should be an dictionary, whose key is + variable name, and value is numpy array. + :param output_name: The final output variable name. + :param input_to_check: The input variable with respect to which to compute the gradient. + :param delta: The perturbation value for numeric gradient method. The + smaller delta is, the more accurate result will get. But if that delta is + too small, it will suffer from numerical stability problem. + :param local_scope: The local scope used for get_numeric_gradient. + :return: The gradient array in numpy format. + """ +``` + +### Explaination: + +- Why need `output_name` + - An Operator may have multiple Output, one can get independent gradient from each Output. So caller should specify the name of the output variable. + +- Why need `input_to_check` + - One operator may have multiple inputs. Gradient Op can calculate the gradient of these inputs at the same time. But Numeric Gradient needs to calculate them one by one. So `get_numeric_gradient` is designed to calculate the gradient for one input. If you need to compute multiple inputs, you can call `get_numeric_gradient` multiple times. + + +### Core Algorithm Implementation + + +```python + # we only compute gradient of one element a time. + # we use a for loop to compute the gradient of each element. + for i in xrange(tensor_size): + # get one input element by its index i. + origin = tensor_to_check.get_float_element(i) + + # add delta to it, run op and then get the new value of the result tensor. + x_pos = origin + delta + tensor_to_check.set_float_element(i, x_pos) + y_pos = get_output() + + # plus delta to this element, run op and get the new value of the result tensor. + x_neg = origin - delta + tensor_to_check.set_float_element(i, x_neg) + y_neg = get_output() + + # restore old value + tensor_to_check.set_float_element(i, origin) + + # compute the gradient of this element and store it into a numpy array. + gradient_flat[i] = (y_pos - y_neg) / delta / 2 + + # reshape the gradient result to the shape of the source tensor. + return gradient_flat.reshape(tensor_to_check.get_dims()) +``` + +## Auto Graident Checker Framework + +Each Operator Kernel has three kinds of Gradient: + +1. Numerical gradient +2. CPU kernel gradient +3. GPU kernel gradient (if supported) + +The numerical gradient only relies on forward Operator. So we use the numerical gradient as the reference value. And the gradient checking is performed in the following three steps: + +1. calculate the numerical gradient +2. calculate CPU kernel gradient with the backward Operator and compare it with the numerical gradient +3. calculate GPU kernel gradient with the backward Operator and compare it with the numeric gradient (if supported) + +#### Python Interface + +```python + def check_grad(self, + forward_op, + input_vars, + inputs_to_check, + output_name, + no_grad_set=None, + only_cpu=False, + max_relative_error=0.005): + """ + :param forward_op: used to create backward_op + :param input_vars: numpy value of input variable. The following + computation will use these variables. + :param inputs_to_check: the input variable with respect to which to compute the gradient. + :param output_name: The final output variable name. + :param max_relative_error: The relative tolerance parameter. + :param no_grad_set: used when create backward ops + :param only_cpu: only compute and check gradient on cpu kernel. + :return: + """ +``` + +### How to check if two numpy array is close enough? +if `abs_numerical_grad` is nearly zero, then use abs error for numerical_grad + +```python +numerical_grad = ... +operator_grad = numpy.array(scope.find_var(grad_var_name(name)).get_tensor()) + +abs_numerical_grad = numpy.abs(numerical_grad) +# if abs_numerical_grad is nearly zero, then use abs error for numeric_grad, not relative +# error. +abs_numerical_grad[abs_numerical_grad < 1e-3] = 1 + +diff_mat = numpy.abs(abs_numerical_grad - operator_grad) / abs_numerical_grad +max_diff = numpy.max(diff_mat) +``` + + +#### Notes： +The Input data for auto gradient checker should be reasonable to avoid numerical stability problem. + + +#### Refs: + +- [Gradient checking and advanced optimization(en)](http://deeplearning.stanford.edu/wiki/index.php/Gradient_checking_and_advanced_optimization) +- [Gradient checking and advanced optimization(cn)](http://ufldl.stanford.edu/wiki/index.php/%E6%A2%AF%E5%BA%A6%E6%A3%80%E9%AA%8C%E4%B8%8E%E9%AB%98%E7%BA%A7%E4%BC%98%E5%8C%96) diff --git a/doc/_sources/design/block.md.txt b/doc/_sources/design/block.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..4066122c0e8dfa33776796c3d205ba5aec9e0f52 --- /dev/null +++ b/doc/_sources/design/block.md.txt @@ -0,0 +1,336 @@ +# Design Doc: Block and Scope + +## The Representation of Computation + +Both deep learning systems and programming languages help users describe computation procedures. These systems use various representations of computation: + +- Caffe, Torch, and Paddle: sequences of layers. +- TensorFlow, Caffe2, Mxnet: graph of operators. +- PaddlePaddle: nested blocks, like C++ and Java programs. + +## Block in Programming Languages and Deep Learning + +In programming languages, a block is a pair of curly braces that includes local variables definitions and a sequence of instructions or operators. + +Blocks work with control flow structures like `if`, `else`, and `for`, which have equivalents in deep learning: + +| programming languages | PaddlePaddle | +|-----------------------|-----------------------| +| for, while loop | RNN, WhileOp | +| if, if-else, switch | IfElseOp, SwitchOp | +| sequential execution | a sequence of layers | + +A key difference is that a C++ program describes a one pass computation, whereas a deep learning program describes both the forward and backward passes. + +## Stack Frames and the Scope Hierarchy + +The existence of the backward pass makes the execution of a block of PaddlePaddle different from traditional programs: + +| programming languages | PaddlePaddle | +|-----------------------|---------------------------------| +| stack | scope hierarchy | +| stack frame | scope | +| push at entering block| push at entering block | +| pop at leaving block | destroy when minibatch completes| + +1. In traditional programs: + + - When the execution enters the left curly brace of a block, the runtime pushes a frame into the stack, where it realizes local variables. + - After the execution leaves the right curly brace, the runtime pops the frame. + - The maximum number of frames in the stack is the maximum depth of nested blocks. + +1. In PaddlePaddle + + - When the execution enters a block, PaddlePaddle adds a new scope, where it realizes variables. + - PaddlePaddle doesn't pop a scope after the execution of the block because variables therein are used by the backward pass. So it has a stack forest known as a *scope hierarchy*. + - The height of the highest tree is the maximum depth of nested blocks. + - After the processing of a minibatch, PaddlePaddle destroys the scope hierarchy. + +## Use Blocks in C++ and PaddlePaddle Programs + +Let us consolidate the discussion by presenting some examples. + +### Blocks with `if-else` and `IfElseOp` + +The following C++ programs shows how blocks are used with the `if-else` structure: + +```c++ +namespace pd = paddle; + +int x = 10; +int y = 1; +int z = 10; +bool cond = false; +int o1, o2; +if (cond) { + int z = x + y; + o1 = z; + o2 = pd::layer::softmax(z); +} else { + int d = pd::layer::fc(z); + o1 = d; + o2 = d+1; +} + +``` + +An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator](./if_else_op.md) is as follows: + +```python +import paddle as pd + +x = minibatch([10, 20, 30]) # shape=[None, 1] +y = var(1) # shape=[1], value=1 +z = minibatch([10, 20, 30]) # shape=[None, 1] +cond = larger_than(x, 15) # [false, true, true] + +ie = pd.ifelse() +with ie.true_block(): + d = pd.layer.add_scalar(x, y) + ie.output(d, pd.layer.softmax(d)) +with ie.false_block(): + d = pd.layer.fc(z) + ie.output(d, d+1) +o1, o2 = ie(cond) +``` + +In both examples, the left branch computes `x+y` and `softmax(x+y)`, the right branch computes `fc(x)` and `x+1` . + +The difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances. + + +### Blocks with `for` and `RNNOp` + +The following RNN model in PaddlePaddle from the [RNN design doc](./rnn.md) : + +```python +x = sequence([10, 20, 30]) # shape=[None, 1] +m = var(0) # shape=[1] +W = var(0.314, param=true) # shape=[1] +U = var(0.375, param=true) # shape=[1] + +rnn = pd.rnn() +with rnn.step(): + h = rnn.memory(init = m) + h_prev = rnn.previous_memory(h) + a = layer.fc(W, x) + b = layer.fc(U, h_prev) + s = pd.add(a, b) + act = pd.sigmoid(s) + rnn.update_memory(h, act) + rnn.output(a, b) +o1, o2 = rnn() +``` +has its equivalent C++ program as follows + +```c++ +int* x = {10, 20, 30}; +int* m = {0}; +int* W = {0.314}; +int* U = {0.375}; + +int mem[sizeof(x) / sizeof(x[0]) + 1]; +int o1[sizeof(x) / sizeof(x[0]) + 1]; +int o2[sizeof(x) / sizeof(x[0]) + 1]; +for (int i = 1; i <= sizeof(x)/sizeof(x[0]); ++i) { + int x = x[i-1]; + if (i == 1) mem[0] = m; + int a = W * x; + int b = Y * mem[i-1]; + int s = fc_out + hidden_out; + int act = sigmoid(sum); + mem[i] = act; + o1[i] = act; + o2[i] = hidden_out; +} +``` + +## Compilation and Execution + +Like TensorFlow, a PaddlePaddle program is written in Python. The first part describes a neural network as a protobuf message, and the rest executes the message for training or inference. + +The generation of this protobuf message is similar to how a compiler generates a binary executable file. The execution of the message is similar to how the OS executes the binary file. + +## The "Binary Executable File Format" + +The definition of the protobuf message is as follows: + +```protobuf +message BlockDesc { + repeated VarDesc vars = 1; + repeated OpDesc ops = 2; +} +``` + +The step net in above RNN example would look like + +``` +BlockDesc { + vars = { + VarDesc {...} // x + VarDesc {...} // h + VarDesc {...} // fc_out + VarDesc {...} // hidden_out + VarDesc {...} // sum + VarDesc {...} // act + } + ops = { + OpDesc {...} // matmul + OpDesc {...} // add_two + OpDesc {...} // sigmoid + } +}; +``` + +Also, the RNN operator in above example is serialized into a protobuf message of type `OpDesc` and would look like: + +``` +OpDesc { + inputs = {0} // the index of x in vars of BlockDesc above + outputs = {5, 3} // indices of act and hidden_out in vars of BlockDesc above + attrs { + "states" : {1} // the index of h + "step_net" : + } +}; +``` + +This `OpDesc` value is in the `ops` field of the `BlockDesc` value representing the global block. + + +## The Compilation of Blocks + +During the generation of the Protobuf message, the Block should store VarDesc (the Protobuf message which describes Variable) and OpDesc (the Protobuf message which describes Operator). + +VarDesc in a block should have its name scope to avoid local variables affect parent block's name scope. +Child block's name scopes should inherit the parent's so that OpDesc in child block can reference a VarDesc that stored in parent block. For example: + +```python +a = pd.Variable(shape=[20, 20]) +b = pd.fc(a, params=["fc.w", "fc.b"]) + +rnn = pd.create_rnn() +with rnn.stepnet(): + x = a.as_step_input() + # reuse fc's parameter + fc_without_b = pd.get_variable("fc.w") + rnn.output(fc_without_b) + +out = rnn() +``` +The method `pd.get_variable` can help retrieve a Variable by the name. The Variable may be stored in a parent block, but might be retrieved in a child block, so block should have a variable scope that supports inheritance. + +In compiler design, the symbol table is a data structure created and maintained by compilers to store information about the occurrence of various entities such as variable names, function names, classes, etc. + +To store the definition of variables and operators, we define a C++ class `SymbolTable`, like the one used in compilers. + +`SymbolTable` can do the following: + +- store the definitions (some names and attributes) of variables and operators, +- verify if a variable was declared, +- make it possible to implement type checking (offer Protobuf message pointers to `InferShape` handlers). + + +```c++ +// Information in SymbolTable is enough to trace the dependency graph. So maybe +// the Eval() interface takes a SymbolTable is enough. +class SymbolTable { + public: + SymbolTable(SymbolTable* parent) : parent_(parent) {} + + OpDesc* NewOp(const string& name=""); + + // TODO determine whether name is generated by python or C++. + // Currently assume that a unique name will be generated by C++ if the + // argument name is left default. + VarDesc* Var(const string& name=""); + + // find a VarDesc by name, if recursive is true, find parent's SymbolTable + // recursively. + // this interface is introduced to support InferShape, find protobuf messages + // of variables and operators, pass pointers into InferShape. + // + // NOTE maybe some C++ classes such as VarDescBuilder and OpDescBuilder should + // be proposed and embedded into pybind to enable python operation on C++ pointers. + VarDesc* FindVar(const string& name, bool recursive=true); + + OpDesc* FindOp(const string& name); + + BlockDesc Compile() const; + + private: + SymbolTable* parent_; + + map ops_; + map vars_; +}; +``` + +After all the description of variables and operators is added into SymbolTable, +the block has enough information to run. + +The `Block` class takes a `BlockDesc` as input, and provides `Run` and `InferShape` functions. + + +```c++ +namespace { + +class Block : OperatorBase { +public: + Block(const BlockDesc& desc) desc_(desc) {} + + void InferShape(const framework::Scope& scope) const override { + if (!symbols_ready_) { + CreateVariables(scope); + CreateOperators(); + } + // should run InferShape first. + for (auto& op : runtime_table_.ops()) { + op->InferShape(scope); + } + } + + void Run(const framework::Scope& scope, + const platform::DeviceContext& dev_ctx) const override { + PADDLE_ENFORCE(symbols_ready_, "operators and variables should be created first."); + for (auto& op : runtime_table_.ops()) { + op->Run(scope, dev_ctx); + } + } + + void CreateVariables(const framework::Scope& scope); + void CreateOperators(); + + // some other necessary interfaces of NetOp are listed below + // ... + +private: + BlockDesc desc_; + bool symbols_ready_{false}; +}; +``` + +## The Execution of Blocks + +Block inherits from OperatorBase, which has a Run method. +Block's Run method will run its operators sequentially. + +There is another important interface called `Eval`, which takes some arguments called targets and generates a minimal graph which treats targets as the end points and creates a new Block. After `Run`, `Eval` will get the latest value and return the targets. + +The definition of Eval is as follows: + +```c++ +// clean a block description by targets using the corresponding dependency graph. +// return a new BlockDesc with minimal number of operators. +// NOTE: The return type is not a Block but the block's description so that this can be distributed +// to a cluster. +BlockDesc Prune(const BlockDesc& desc, vector targets); + +void Block::Eval(const vector& targets, + const framework::Scope& scope, + const platform::DeviceContext& dev_ctx) { + BlockDesc min_desc = Prune(desc_, targets); + Block min_block(min_desc); + min_block.Run(scope, dev_ctx); +} +``` diff --git a/doc/_sources/design/build_system/README.md.txt b/doc/_sources/design/build_system/README.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..bf0e4dddc1b640ecbce489f65820aaf8a4b3b1e7 --- /dev/null +++ b/doc/_sources/design/build_system/README.md.txt @@ -0,0 +1,152 @@ +A few months ago when we were trying to replace CMake with Bazel, @emailweixu suggested that we rewrite those handy Bazel functions using CMake. Now it seems that it's the right time to get this done, as we are facing problems from the porting of Majel and the development of new the parameter server using Go and C++. + +Here are some initial thoughts. Your comments are welcome! + +### Required CMake Function + +I think we need only the following few CMake functions to make a project description mean and clean: + +| C++ | CUDA C++ | Go | +|---|---|---| +| cc_library | nv_library | go_library | +| cc_binary | nv_binary | go_binary | +| cc_test | nv_test | go_test | + +- The `_library` functions generate .a files from source code. +- The `_binary` functions generate executable binary files. +- The `_test` functions generate executable unit test files. They work like `_binary` but links `-lgtest` and `-lgtest_main`. + +The difference between `nv_` functions and `cc_` functions is that the former use `nvcc` instead of the system-default C++ compiler. + +Both `nv_` and `cc_` functions enables C++11 (-std=c++11). + +Also, + +- to describe external dependencies, we need `external_library`. +- to build shared libraries, we need `shared_library`. + +### An Example Project + +Suppose that we have aforementioned functions defined in our `/cmake` directory. The following example `CMakeLists.txt` describes a project including the following source files: + +- tensor.h +- tensor.cc +- tensor_test.cc +- ops.h +- ops.cu +- ops_test.cu +- api.go +- api_test.go + +Suppose that ops.cu depends on CUDNN. + +```cmake +# cc_binary parses tensor.cc and figures out that target also depend +# on tensor.h. +cc_binary(tensor + SRCS + tensor.cc) + +# The dependency to target tensor implies that if any of +# tensor{.h,.cc,_test.cc} is changed, tensor_test need to be re-built. +cc_test(tensor_test + SRCS + tensor_test.cc + DEPS + tensor) + +# I don't have a clear idea what parameters external_library need to +# have. @gangliao as a CMake expert would have better ideas. +external_library(cudnn + ....) + +# Suppose that ops.cu depends on external target CUDNN. Also, ops.cu +# include global functions that take Tensor as their parameters, so +# ops depend on tensor. This implies that if any of tensor.{h.cc}, +# ops.{h,cu} is changed, ops need to be re-built. +nv_library(ops + SRCS + ops.cu + DEPS + tensor + cudnn) # cudnn is defined later. + +nv_test(ops_test + SRCS + ops_test.cu + DEPS + ops) + +# Because api.go defines a GO wrapper to ops and tensor, it depends on +# both. This implies that if any of tensor.{h,cc}, ops.{h,cu}, or +# api.go is changed, api need to be re-built. +go_library(api + SRCS + api.go + DEPS + tensor # Because ops depend on tensor, this line is optional. + ops) + +go_test(api_test + SRCS + api_test.go + DEPS + api) + + +# This builds libapi.so. shared_library might use CMake target +# api_shared so to distinguish it from above target api. +shared_library(api + DEPS + api) + +``` + +### Implementation + +As above example CMakeLists.txt executes, each function invocation adds "nodes" to a dependency graph. It also use this graph to generate CMake commands including `add_executable`, `add_dependencies`, `target_link_libraries`, and `add_test`. + +### Using Package Manager For Go + +Building Go binaries and libraries need to satisfy their dependencies, generally +we can do `go get ./...` to download and compile all external dependencies. The +problems are: + +1. `go get` will always get the latest code from the default branch of the + remote repo, so changes of dependents might break the build. This is very + different with what we already have in `cmake/external` which download a + specific version or commit id of the dependency. +1. Some locations can not access external dependencies through the internet, as mentioned + in https://github.com/PaddlePaddle/Paddle/issues/2605. Using package management + tools can package the dependencies as a "vendor" package, which can be mirrored + at many cloud file hosting, so users what to compile paddle by themselves can + download this "vendor" package from a mirror site. + +#### Choose A Suitable Tool + +As mentioned by @wangkuiyi, [Here](https://github.com/golang/go/wiki/PackageManagementTools) +list dozens of Go package managers. We choose the tool using following principles: + +- Most "active" projects with more stars, more pull requests or commits +- Widely used project + +After comparing all these projects, we shall choose between the most popular +tools: Godep and Glide. + +Here's a brief comparison between Godep and Glide +: https://github.com/Masterminds/glide/wiki/Go-Package-Manager-Comparison. There are +also many complaints about using `Godep`. There's also a new "official" pakcage +management tool has been started at: https://github.com/golang/dep to resolve +such problems, but it's currently at Alpha stage. So the best choice now is +glide obviously. + +#### Manage Go Packages + +- Dependencies: `go/glide.yaml` will store the dependencies and their versions which + is directly imported by paddle. `go/glide.lock` will store all dependencies recursively + with their commit id. Builds will "lock" to these packages if we don't `glide up` + them +- Vendor package: `go/vendor` directory will generated when running `cmake` command. `cmake` + will download the code corresponding to `go/glide.lock`. If we put a vendor folder + under `go/`, cmake will just check the commit id to the packages under the folder, + if commit id matches, there will be no download at all. diff --git a/doc_cn/_sources/design/dist/README.md.txt b/doc/_sources/design/cluster_train/README.md.txt similarity index 62% rename from doc_cn/_sources/design/dist/README.md.txt rename to doc/_sources/design/cluster_train/README.md.txt index 1788208bcabca30f66cb1c80e80f6b824c0d9579..177a5f5d54bd924fab34795219ce1f7b270c8e25 100644 --- a/doc_cn/_sources/design/dist/README.md.txt +++ b/doc/_sources/design/cluster_train/README.md.txt @@ -15,33 +15,37 @@ This poses technical challenges to PaddlePaddle: A training job will be created once user asks Paddle cloud to train a model. The training job is made up of different processes that collaboratively consume data and produce a trained model. There are three kinds of processes: -1. the *master process*, which dispatches tasks to +1. the *master server process*, which dispatches tasks to 1. one or more *trainer processes*, which run distributed training and synchronize gradients/models via -1. one or more *parameter server processes*, where each holds a shard of the global model. +1. one or more *parameter server processes*, where each holds a shard of the global model, and receive the uploaded gradients from every *trainer process*, so they can run the optimize functions to update their parameters. Their relation is illustrated in the following graph: -### Master Process +By coordinating these processes, PaddlePaddle supports use both Synchronize Stochastic Gradient Descent (sync SGD) and Asynchronous Stochastic Gradient Descent (async SGD) to train user-defined neural network topologies. -The master process will: +When training with sync SGD, parameter servers wait for all trainers to finish gradients update and then send the updated parameters to trainers, training can not proceed until the trainer received the updated parameters. This creates a synchronization point between trainers. When training with async SGD, each trainer upload gradient and download new parameters individually, without the synchronization with other trainers. Using asyc SGD will be faster in terms of time per pass, but have more noise in gradient since trainers are likely to have a stale model. + +### Master Server Process + +The master server process will: - Partition a dataset into [tasks](#task) and dispatch tasks to trainers. - Keep track of training progress on the dataset with [task queue](#task-queue). A training job will iterate on the dataset for a full pass until it goes into next pass. -#### Task +#### Task A task is a data shard to be trained. The total number of tasks will be much bigger than the total number of trainers. The number of data instances inside a task will be much bigger than the mini-batch size. #### Task Queue -The master process has three task queues to track training progress. As illustrated in the graph below, Job A and Job B both have one master process. Each master process has three task queues. +The master server has three task queues to track training progress. As illustrated in the graph below, Job A and Job B both have one master server. Each master server process has three task queues. -- The todo queue holds tasks to be dispatched. When a job starts, the master process fills in the todo queue with all tasks. +- The todo queue holds tasks to be dispatched. When a job starts, the master server fills in the todo queue with all tasks. - The pending queue holds tasks that are currently training by trainers. - the done queue holds tasks that are already trained. @@ -50,17 +54,18 @@ The life cycle of a single task is illustrated below: 1. When a new pass of training starts, all tasks will be placed in the todo queue. -1. The master process will dispatch few tasks to each trainer at a time, puts them in the pending queue and waits for completion. -1. The trainer will work on its tasks and tell the master process once a task is completed. The master process will dispatch a new task to that trainer. -1. If a task timeout. the master process will move it back to the todo queue. The timeout count will increase by one. If the timeout count is above a threshold, the task is likely to cause a trainer to crash, so it will be discarded. -1. The master process will move completed task to the done queue. When the todo queue is empty, the master process will start a new pass by moving all tasks in the done queue to todo queue and reset the timeout counter of all tasks to zero. +1. Upon trainer requests for new task, the master server will dispatch a task from todo queue to it, put the task in the pending queue and wait for completion. +1. The trainer will work on its task and tell the master server once the task is completed and ask for new task. The master server will dispatch a new task to that trainer. +1. If a task fails for any reason in trainer, or takes longer than a specific period of time, the master server will move the task back to the todo queue. The timeout count for that task will increase by one. If the timeout count is above a threshold, the task is likely to cause a trainer to crash, then it will be discarded. +1. The master server will move completed task to the done queue. When the todo queue is empty, the master server will start a new pass by moving all tasks in the done queue to todo queue and reset the timeout counter of all tasks to zero. ### Trainer Process The trainer process will: -- Receive tasks from the master. -- Work on the tasks: calculate and upload gradient to parameter servers, and update local model by downloading new parameters from parameter servers. +- Request tasks from the master. +- Work on the tasks +- Upload gradient to parameter servers, and update local model by downloading new parameters from parameter servers. ### Parameter Server Process @@ -78,7 +83,7 @@ The communication pattern between the trainers and the parameter servers depends - Synchronous Stochastic Gradient Descent (sync-SGD) Parameter server will wait for all trainer finish n-th mini-batch calculation and send their gradients before broadcasting new parameters to every trainer. Every trainer will wait for the new parameters before starting n+1-th mini-batch. - + - Asynchronous Stochastic Gradient Descent (async-SGD) There will no synchronization between different trainers, and parameter server updates its parameter as soon as it receives new gradient: @@ -89,7 +94,7 @@ The communication pattern between the trainers and the parameter servers depends ## Fault Tolerant -The training job will pause if the master processes is dead, or any of the parameter server process is dead. They will be started by [Kubernetes](https://kubernetes.io/) and recover in few minutes. Please refer to [fault recovery](#fault-recovery). +The training job will pause if the master server processes is dead, or any of the parameter server process is dead. They will be started by [Kubernetes](https://kubernetes.io/) and recover in few minutes. Please refer to [fault recovery](#fault-recovery). The training job will continue to make progress if there is at least one training process running. The strategy depends on the type of optimization algorithm: @@ -109,28 +114,26 @@ Now we will introduce how each process recovers from a failure, the graph below -### Master Process +### Master Server Process When the master is started by the Kubernetes, it executes the following steps at startup: 1. Grabs a unique *master* lock in etcd, which prevents concurrent master instantiations. 1. Recovers the task queues from etcd if they already exist, otherwise, the master will create them. -1. Watches the trainer prefix keys `/trainer/` on etcd to find the live trainers. -1. Starts dispatching the tasks to the trainers, and updates task queue using an etcd transaction to ensure lock is held during the update. +1. Write its ip address to */master/addr* so that trainers can discover it. +1. Listens to trainers' request of task, dispatch one upon request, and updates task queue using an etcd transaction to ensure lock is held during the update. -The master process will kill itself if its etcd lease expires. - -When the master process is dead for any reason, Kubernetes will restart it. It will be online again with all states recovered from etcd in few minutes. +When the master server process is dead for any reason, Kubernetes will restart it. It will be online again with all states recovered from etcd in few minutes. ### Trainer Process When the trainer is started by the Kubernetes, it executes the following steps at startup: -1. Watches the available parameter server prefix keys `/ps/` on etcd and waits until the count of parameter servers reaches the desired count. -1. Generates a unique ID, and sets key `/trainer/` with its contact address as value. The key will be deleted when the lease expires, so the master will be aware of the trainer being online and offline. -1. Waits for tasks from the master to start training. +1. Watches the available parameter server prefix keys `/ps/` on etcd and waits until the count of parameter servers reaches the desired count */ps_desired*. +1. Finds and watches */master/addr* to get master's address. +1. Requests for tasks from the master to start training. -If trainer's etcd lease expires, it will try set key `/trainer/` again so that the master process can discover the trainer again. +When a trainer fails, Kuberentes would try to restart it. The recovered trainer would fetch tasks from master and go on training. ### Parameter Server Process @@ -140,11 +143,11 @@ When the parameter server is started by Kubernetes, it executes the following st 1. Search through etcd keys `/ps/` (`/ps/0`, `/ps/1`, ...) to find the first non-existant key whose index is smaller than the total number of parameter servers. Set the key using a transaction to avoid concurrent writes. The parameter server's index is inferred from the key name. The desired number of parameter servers is 3: - + - + The third parameter server joined: - + 1. The parameter server can load parameters if there are already saved parameters in the save path (inferred from its index). @@ -153,6 +156,13 @@ When the parameter server is started by Kubernetes, it executes the following st If the parameter server's etcd lease expires, the parameter server will kill itself. +## Parameter Server Checkpointing +See [here](./checkpointing.md) + +## Store and dispatching trainning data +See [here](./data_dispatch.md) + + ## Dynamic Scaling ### Trainer Scaling diff --git a/doc/_sources/design/cluster_train/checkpointing.md.txt b/doc/_sources/design/cluster_train/checkpointing.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..c87ef2c7d2636208866d05456d5d44316d0bb200 --- /dev/null +++ b/doc/_sources/design/cluster_train/checkpointing.md.txt @@ -0,0 +1,44 @@ +## 模型参数检查点（Checkpointing） +模型数据检查点的实现，可以有效的避免parameter server的单点或多点同时故障。模型参数检查点通过定期向磁盘上保存一份存储在parameter server内存中的模型数据的完整镜像，来保证训练过程可以从中间状态重新启动。在一个不可中断并缺少备份的训练任务中，可以通过阶段性的保存每个parameter server的数据快照（snapshot）到 ***分布式存储服务*** 达到容灾的目的，比如每隔10分钟最新的快照，并删除更早的快照。在出现单点故障时，只需要恢复这台节点，或者将这台节点迁移到另一个节点并启动即可恢复训练任务。 + + + +### 快照保存的设计如下： + +说明： + +* parameter server在集群中启动后，自动挂载分布式存储目录，并把快照保存到这个目录下。 +* ***注：每个parameter server的检查点各自独立保存，暂时不考虑多个parameter server同步的保存一个特定时间点的全局检查点，因为这样做也没法保证消除随机性。*** + +检查点保存程序流程： + +1. 如果满足条件"每隔10分钟"时，parameter server会获取parameters内存的`read_lock`，启动一个新的线程开始保存检查点。如果已经正在执行保存检查点的线程，则忽略。由于对parameters的更新需要获取parameters内存的`write_lock`，所以在写入快照的过程中，parameter server会暂停参数更新并等待。 +2. parameter server生成一个UUID，向指定的目录中一个新的文件（文件名为此UUID）写入快照数据。在快照写入完成后，计算这个文件的MD5 sum。然后在etcd的`/checkpoints/[pserver_id]`中写入json内容：`{"uuid": [UUID], "md5", "MD5 sum", "timestamp": xxxx}`。 +3. 删除磁盘目录中不是当前uuid的快照文件。 +4. 释放对paramters内存的锁定，停止保存检查点的线程。 + +这里需要用户额外注意，在您的实际环境中，训练任务的运行可能会占满trainer和parameter server之间的网络带宽，如果parameter server此时还需要通过网络访问分布式存储以保存快照，可能会造成网络拥塞，而出现阶段性的运行停滞。 + +### 从快照恢复 + +在parameter server第一次启动或任意时间parameter server故障后被Kubernetes重新启动，则需要回滚到上一个检查点： + + 1. 从etcd中读取节点：`/checkpoints/[pserver_id]`获取最新的检查点的文件uuid + 1. 从磁盘文件中加载uuid文件名的检查点快照文件，并加载其中的参数 + 1. 如果上面两步出现错误，则使用启动参数定义的初始化方法初始化参数 + 1. 开始提供服务 + +## TODO List +### 推测执行/加速执行（TODO） +在异构集群中，如果存在某些trainer执行速度过慢会影响整体集群的速度（如图中Trainer 1），此时master将负责启动一个新的Trainer（Accelerate Trainer 2），使用同样的训练数据block。哪个trainer先完成block的训练，则把另一个慢速的kill掉。 + +### 动态扩容/缩容 +目前只考虑动态扩容trainer数量，可以减小系统复杂性。 + +## 术语 +* model: 指深度学习训练之后得到的所有参数，使用这个神经网络可以完成对新数据的预测 +* parameters: 神经网络中的参数，包括权重w和偏置b。一个神经网络的模型由大量的参数组成 +* shard: 分片，通常指将一个整体拆分成多份的其中的一份。 +* model shard: 将一个神经网络参数拆分成多份，每个shard分别存储在其中一台parameter server之上 +* parameter block: 多个parameter block构成一个model shard +* 单点故障: 任意时刻只可能同时有一台服务器故障。由于集群中同时存在两台机器故障的概率极低（（平均故障率*平均故障修复时间）^2）只对特殊在线系统考虑两台以上同时故障的容灾。 diff --git a/doc/_sources/design/cluster_train/data_dispatch.md.txt b/doc/_sources/design/cluster_train/data_dispatch.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..1f5d22ff5e6abcb576d16cbe7391da1967a1ab8e --- /dev/null +++ b/doc/_sources/design/cluster_train/data_dispatch.md.txt @@ -0,0 +1,160 @@ +## 训练数据的存储和分发 + +### 概念解释 + +### 流程介绍 +生产环境中的训练数据集通常体积很大，并被存储在诸如Hadoop HDFS，Ceph，AWS S3之类的分布式存储之上。这些分布式存储服务通常会把数据切割成多个分片分布式的存储在多个节点之上。这样就可以在云端执行多种数据类计算任务，包括： + +* 数据预处理任务 +* Paddle训练任务 +* 在线模型预测服务 +
+ +
+ +在上图中显示了在一个实际生产环境中的应用（人脸识别）的数据流图。生产环境的日志数据会通过实时流的方式（Kafka）和离线数据的方式（HDFS）存储，并在集群中运行多个分布式数据处理任务，比如流式数据处理（online data process），离线批处理（offline data process）完成数据的预处理，提供给paddle作为训练数据。用户也可以上传labeled data到分布式存储补充训练数据。在paddle之上运行的深度学习训练输出的模型会提供给在线人脸识别的应用使用。 + +### 训练数据存储 +我们选择[CephFS](http://docs.ceph.com/docs/master/cephfs/)作为存储系统。 + +- 无论是从[PFSClient](../file_manager/README.md)的角度，还是从[Pod](https://kubernetes.io/docs/concepts/workloads/pods/pod/)中运行任务的角度，统一用`/pfs/$DATACENTER/home/$USER`来访问用户自己的数据。 +- `/pfs/$DATACENTER/common`下存放公共数据集合 + - 做只读挂载 + +
+ +
+ +### 文件预处理 + + +在开始训练之前, 数据集需要预先被转换成PaddlePaddle分布式训练使用的存储格[RecordIO](https://github.com/PaddlePaddle/Paddle/issues/1947)。我们提供两个转换方式： + +1. 用户在本地转换好再上传 +1. 用户上传数据后，在机群上运行转换程序 + +转换生成的文件名会是以下格式： + +```text +name_prefix-aaaaa-of-bbbbb +``` + +"aaaaa"和"bbbbb"都是五位的数字，每一个文件是数据集的一个shard，"aaaaa"代表shard的index，"bbbbb"代表这个shard的最大index。 + +比如ImageNet这个数据集可能被分成1000个shard，它们的文件名是： +```text +imagenet-00000-of-00999 +imagenet-00001-of-00999 +... +imagenet-00999-of-00999 +``` + +#### 转换库 + +无论是在本地或是云端转换，我们都提供Python的转换库，接口是： +```python +def convert(output_path, reader, num_shards, name_prefix) +``` + +- `output_path`: directory in which output files will be saved. +- `reader`: a [data reader](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/reader/README.md#data-reader-interface), from which the convert program will read data instances. +- `num_shards`: the number of shards that the dataset will be partitioned into. +- `name_prefix`: the name prefix of generated files. + +`reader`每次输出一个data instance，这个instance可以是单个值，或者用tuple表示的多个值： + +```python +yield 1 # 单个值 +yield numpy.random.uniform(-1, 1, size=28*28) # 单个值 +yield numpy.random.uniform(-1, 1, size=28*28), 0 # 多个值 +``` + +每个值的类型可以是整形、浮点型数据、字符串，或者由它们组成的list，以及numpy.ndarray。如果是其它类型，会被Pickle序列化成字符串。 + +### 示例程序 + +#### 使用转换库 + +以下`reader_creator`生成的`reader`每次输出一个data instance，每个data instance包涵两个值：numpy.ndarray类型的值和整型的值： +```python +def reader_creator(): + def reader(): + for i in range(1000): + yield numpy.random.uniform(-1, 1, size=28*28), 0 # 多个值 + return reader +``` + +把`reader_creator`生成的`reader`传入`convert`函数即可完成转换： +```python +convert("./", reader_creator(), 100, random_images) +``` + +以上命令会在当前目录下生成100个文件： +```text +random_images-00000-of-00099 +random_images-00001-of-00099 +... +random_images-00099-of-00099 +``` + +#### 进行训练 + + +PaddlePaddle提供专用的[data reader creator](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/reader/README.md#python-data-reader-design-doc)，生成给定`RecordIO`文件对应的data reader。**无论在本地还是在云端，reader的使用方式都是一致的**： + +```python +# ... +reader = paddle.reader.creator.RecordIO("/pfs/datacenter_name/home/user_name/random_images-*-of-*") +batch_reader = paddle.batch(paddle.dataset.mnist.train(), 128) +trainer.train(batch_reader, ...) +``` + +以上代码的reader输出的data instance与生成数据集时，reader输出的data instance是一模一样的。 + +### 上传训练文件 + +使用下面命令，可以把本地的数据上传到存储集群中。 + +```bash +paddle pfs cp filename /pfs/$DATACENTER/home/$USER/folder/ +``` + +比如，把之前示例中转换完毕的random_images数据集上传到云端的`/home/`可以用以下指令： + +```bash +paddle pfs cp random_images-*-of-* /pfs/$DATACENTER/home/$USER/folder/ +``` + +需要`$DATACENTER`的配置写到配置文件中，例如 + +``` +# config file +[datacenter_1] +username=user +usercert=user.pem +userkey=user-key.pem +endpoint=datacenter1.paddlepaddle.org + +[datacenter_2] +username=user +usercert=user.pem +userkey=user-key.pem +endpoint=datacenter2.paddlepaddle.org +``` +## TODO +### 文件访问的权限 +控制用户权限 + +- 用户可以把自己的数据分享给别人 + +### 文件访问方式 +不用mount的方式来访问数据，而是直接用API的接口远程访问 + +例如： + +``` +f = open('/pfs/datacenter_name/home/user_name/test1.dat') +``` + + +### 支持用户自定义的数据预处理job diff --git a/doc/_sources/design/cluster_train/large_model_dist_train.md.txt b/doc/_sources/design/cluster_train/large_model_dist_train.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..0c4b5bc24c854b7062d509249bea9c50d42bd5f1 --- /dev/null +++ b/doc/_sources/design/cluster_train/large_model_dist_train.md.txt @@ -0,0 +1,101 @@ +# Alalysis of large model distributed training in Paddle + +***NOTE: This is only some note for how we implemeted this scheme in V1, not a new design.*** + +## What is it + +We often encounter cases that the embedding layer parameters(sparse) are so large that we can not store it in the trainer's memory when training. So we need to put them to several servers, and fetch them row by row instead of fetch all of the parameters. + +## How to use + +Specify command-line argument like `--loadsave_parameters_in_pserver=true --ports_num_for_sparse=1 --use_old_updater=1` when starting the paddle trainer. And also add something like `--ports_num_for_sparse=1 --pserver_num_threads=5` when starting pserver processes. + +Accrodingly, configure your embedding layers like: + +```python +SPARSE_REMOTE=True + +w1 = data_layer(name="w1", size=dict_size) +emb1 = embedding_layer(input=w1, size=32, param_attr=ParameterAttribute(sparse_update=SPARSE_REMOTE)) +w2 = data_layer(name="w2", size=dict_size) +emb2 = embedding_layer(input=w2, size=32, param_attr=ParameterAttribute(sparse_update=SPARSE_REMOTE)) +... +``` + +## Implementation details + +```c++ +enum MatType { + MAT_NORMAL, + MAT_NORMAL_SHARED, + MAT_VALUE_SHARED, + MAT_SPARSE_ROW_IDS, + MAT_SPARSE_ROW_AUTO_GROW, + MAT_CACHE_ROW, + MAT_SPARSE_ROW, + MAT_SPARSE_ROW_PREFETCH, + MAT_SPARSE_ROW_PREFETCH_FULL_SIZE, +}; +``` + +`MAT_SPARSE_ROW_PREFETCH` is what we use when configured to fetch only row of matrix when training. + +In `trainer_internal.cpp:L93 trainOneBatch`: + +```c++ + if (config_->getOptConfig().use_sparse_remote_updater()) { + REGISTER_TIMER("prefetch"); + gradientMachine_->prefetch(inArgs); + parameterUpdater_->getParametersRemote(); + } +``` + +When doing actual network forward and backward, at the beginning of each batch, the trainer will try to download one row of data from pserver. + +In `trainer/RemoteParameterUpdater.cpp`: `parameterUpdater_->getParametersRemote();`: + +```c++ +if (fullSize) { + ... +} else { +getParams = [&] { + parameterClient_->getParameterSparse( + /* recvParameterType= */ PARAMETER_VALUE, sendBackParameterType); +}; +applyL1 = [](Parameter& para, real decayRate) { + para.getMat(PARAMETER_VALUE)->applyL1(/*lr=*/1.0f, decayRate); +}; +} +``` + +Calling `parameterClient_->getParameterSparse` will do remote call to pserver's `getParameterSparse`: + +```c++ +void ParameterServer2::getParameterSparse(const SendParameterRequest& request, + std::vector& inputBuffers, + SendParameterResponse* response, + std::vector* outputBuffers) { + (void)inputBuffers; + auto& buffer = *readWriteBuffer_; + size_t numReals = 0; + for (const auto& block : request.blocks()) { + numReals += getParameterConfig(block).dims(1); + } + buffer.resize(numReals); + + VLOG(3) << "pserver: getParameterSparse, numReals=" << numReals; + + ReadLockGuard guard(parameterMutex_); + size_t offset = 0; + for (const auto& block : request.blocks()) { + size_t width = getParameterConfig(block).dims(1); + Buffer buf = {buffer.data() + offset, width}; + int type = request.send_back_parameter_type(); + sendBackParameterSparse(block, type, response, &buf, width, outputBuffers); + offset += width; + } +} +``` + +`getParameterConfig(block).dims(1)` returns the width of the current "parameter block"(a shard of parameter object), +then `getParameterSparse` remote call returns only one row of data to the client. diff --git a/doc/_sources/design/cluster_train/master_server.md.txt b/doc/_sources/design/cluster_train/master_server.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..4bf3c506f101361875043f8bfd97972b8c981a22 --- /dev/null +++ b/doc/_sources/design/cluster_train/master_server.md.txt @@ -0,0 +1,91 @@ +# Design Doc: Master Server + +For an overview of master server's role, please refer to [distributed training design doc](./README.md). In this design doc we will discuss the master server in more details. The master will be implemented in [Go](https://golang.org/). + +## Dataset + + + +A dataset is a list of files in *RecordIO* format. A RecordIO file consists of chunks, whereas each chunk consists some records. + +## Task Queue + +As mentioned in [distributed training design doc](./README.md), a *task* is a data shard that the master server assigns to the trainer process to train on. A task consists of one or multiple *chunks* from one or multiple files. The master server maintains *task queues* to track the training progress. + +### Task Queue Creation + +1. Each trainer will make an RPC call (using Go's [rpc](https://golang.org/pkg/net/rpc/) package) to the master server, telling it the RecordIO files representing the dataset specified by the user. Since every trainer will tell the master server the same dataset, only the first RPC call will be honored. + + The RPC interface is: + ```go + func (m *RPCServer) ReportDataset(Paths []string, dummy *int) error { + } + ``` +1. The master server will scan through each RecordIO file to generate the *chunk index* and know how many chunks does each file have. A chunk can be referenced by the file path and the index of the chunk within the file. The chunk index is in memory data structure that enables fast access to each chunk, and the index of the chunk with the file is an integer start from 0, representing the n-th chunk within the file. + + The definition of the chunk is: + ```go + type Chunk struct { + Idx int // index of the chunk within the file + Path string + Index recordio.Index // chunk index + } + ``` +1. Chunks are grouped into tasks, and tasks are filled into the todo queue. The pending queue and the done queue are initialized with no element. + + The definition of the task is: + ```go + type Task struct { + Index int + Chunks []Chunk + } + ``` + + The elements in the tasks queues is of type `TaskEntry`, containing a timeout counter (described in [task retry logic](#task-retry-logic)), and a task: + ```go + type TaskEntry struct { + NumTimeout int + Task Task + } + ``` + + The definition of task queues is: + ```go + type TaskQueues struct { + Todo []TaskEntry + Pending map[int]TaskEntry // map from task index to task entry + Done []TaskEntry + } + ``` + +### Task Queue Persistence + +The task queues need to be persisted on [etcd](https://github.com/coreos/etcd) for fault recovery. Since the task queues only change once a task is completed or timed out, which is not very frequent, we can afford to synchronize with etcd every time the task queues change. + +We will serialize the task queues data structure with [gob encoding](https://golang.org/pkg/encoding/gob/), compress with gzip, and save into etcd synchronously under key `/task_queues`. + +### Task Dispatch + +The trainer will make an RPC call to master to get a new task when: + +- the trainer first started, or +- the trainer finishes a task. + +The RPC interface is: +```go +func (m *RPCServer) GetTask(finished *Task, result *Task) error { +} +``` +Argument `finished` will be `nil` when the trainer is just started. + +During the RPC call the master will do the following: + +- Make a copy of the task queues, and update the copy reflecting the finished tasks and the new pending tasks. +- Synchronize the copy of task queues with etcd using a transaction conditioned on holding the master lock. +- Replace the task queues with the copy and report to the trainer with the new tasks if succeeded, or discard the copy and report the error to the trainer if failed. + +### Task Retry Logic + +When a task is dispatched to the trainer, the master will schedule a function for execution after the timeout duration (based on the moving average of task completion time). If the task entry in still in the pending queue, its timeout counter will increase by one, and the task will be moved to todo queue. If the timeout counter is above the threshold, the master will log the error and discard the task. + +Please note that since a timed out task could be completed after it has been dispatched for retry, so it is possible for a task to be processed multiple times. We do not try to prevent it from happening since it's fine to train on the same task multiple times due to the stochastic nature of the stochastic gradient decent algorithm. diff --git a/doc/_sources/design/cluster_train/pserver_client.md.txt b/doc/_sources/design/cluster_train/pserver_client.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..474b8c572cd92fc87e9f7f3f2b19d12cccd158de --- /dev/null +++ b/doc/_sources/design/cluster_train/pserver_client.md.txt @@ -0,0 +1,171 @@ +# Design Doc: The Client Library of Parameter Server + +For an overview of trainer's role, please refer to [distributed training design doc](README.md). In this design doc, we will discuss the parameter server's client library, which will manage communication with parameter servers. The library will be implemented in [Go](https://golang.org/) and made available as a static or dynamic library with a C header file. + +## Parameter Partition + +Each parameter will be partitioned into parameter blocks to make the parameters evenly distributed on parameter servers. The partition is done automatically by the client library. The *sparse parameter* require a little different treatment: + +### Sparse Parameter + +The sparse parameter is a parameter that is updated sparsely. The name is somewhat misleading, it does not have a sparse representation, it has the same representation as a dense vector. + +Because a sparse parameter is updated sparsely, the trainer will have to partition the sparse parameter. Because the parameter server will merge all sparse parameter shard into the same file when saving the parameter. It needs special naming convention: + +If a sparse parameter is partitioned into n shards, they should be named as: + +```text +name:sparse-0 +name:sparse-1 +... +name:sparse-n-1 +``` + +The library is unaware of the partition, and treat each parameter independently. Only when saving parameters, the parameter servers will merge the sparse parameters according to the naming convention. + +## Model Optimization Using Gradients + +There are two ways to perform model optimization using gradients: + +- On Client + + The client does multiple steps of forward and backward update. In each step, the gradients are calculated and a new model is generated. After some steps, the client will calculate the difference between the newest model and the old model at step 0. The difference will be updated to parameter servers. Parameter servers will just update parameters using the difference without any optimization using gradients (such as Adam and L1 regularization). + +- On Parameter Server + + The client will send accumulated gradients to parameter servers, the parameter server will do the optimization using gradients. + +## L1 and L2 Regularization + +PaddlePaddle allows L1 or L2 regularizations to be specified per parameter, so when the trainer initializes the parameter it needs include a parameter configuration when L1 or L2 regularization is necessary. + +## Parameter Initialization + +The parameters on parameter servers need to be initialized. To provide maximum flexibility, the trainer will initialize the parameters. Only one trainer will do the initialization, the other trainers will wait for the completion of initialization and get the parameters from the parameter servers. + +### Trainer Selection + +To select the trainer for initialization, every trainer will try to get a distributed lock, whoever owns the lock will do the initialization. As illustrated below: + + + +### Trainer Selection Process + +The trainer select process is encapsulated in the C API function: +```c +int paddle_begin_init_params(paddle_pserver_client* client, const char* config_proto); +``` +The selected trainer's call to `paddle_begin_init_params` will return with 1, and the other trainers' call to `paddle_begin_init_params` will return 0. `paddle_get_params` will be blocked until initialization is completed. As illustrated below: + + + +## C Interface + +```c +typedef enum { + PADDLE_ELEMENT_TYPE_INT32 = 0, + PADDLE_ELEMENT_TYPE_UINT32 = 1, + PADDLE_ELEMENT_TYPE_INT64 = 2, + PADDLE_ELEMENT_TYPE_UINT64 = 3, + PADDLE_ELEMENT_TYPE_FLOAT32 = 4, + PADDLE_ELEMENT_TYPE_FLOAT64 = 5, +} paddle_element_type; + +typedef struct { + char* name; + paddle_element_type element_type; + unsigned char* content; + int content_len; +} paddle_parameter, paddle_gradient; + +typedef int paddle_pserver_client; + +/** + * @brief creates a pserver client that talks to etcd for coordination. + */ +paddle_pserver_client paddle_new_etcd_pserver_client(char* etcd_addr); + +/** + * @brief creates a pserver client given pserver addresses. + * + * @param pserver_addrs comma-separated pserver addresses. + * @param selected if current pserver client is selected to initialize all parameter servers. + */ +paddle_pserver_client paddle_new_pserver_client(char* pserver_addrs, int selected); +void paddle_pserver_client_release(paddle_pserver_client c); + +/** + * @brief paddle_begin_init_params begins to initialize parameters on + * parameter servers. + * + * paddle_begin_init_params will be called from multiple trainers, + * only one trainer will be selected to initialize the parameters on + * parameter servers. Other trainers need to get the initialized + * parameters from parameter servers using @paddle_get_params. + * + * @return 1 if the trainer is selected to initialize parameter + * servers, otherwise 0. + */ +int paddle_begin_init_params(paddle_pserver_client client); + +/** + * @brief paddle_init_param initializes the parameter on parameter + * servers. + * + * @param param the parameter to initialize. + * @param param_config_proto the configuration for the parameter. + * @param config_len the length of param_config_proto + * @return 0 if successful, otherwise -1. On failure, the trainer + * needs to restart the entire initialization process (starting from + * @paddle_begin_init_param). Or simply exit the program and wait for + * the cluster management system to restart the trainer. + */ +int paddle_init_param(paddle_pserver_client client, paddle_parameter param, const unsigned char* param_config_proto, int config_len); + +/** + * @brief paddle_finish_init_params tells parameter servers client has + * sent all parameters to parameter servers as initialization. + * + * @return 0 if successful, otherwise -1. On failure, the trainer + * needs to restart the entire initialization process (starting from + * @paddle_begin_init_param). Or simply exit the program and wait for + * the cluster management system to restart the trainer. + */ +int paddle_finish_init_params(paddle_pserver_client client); + +/** + * @brief paddle_send_grads sends gradients to parameter servers for + * updating parameters. + * + * @param grads the array of gradients to send. + * @param len the length of the gradient array. + * @param learning_rate the learning rate for the gradients. + * @return 0 if successful, otherwise -1. + */ +int paddle_send_grads(paddle_pserver_client client, const paddle_gradient* grads, int len); + +/** + * @brief paddle_get_params gets parameters from parameter servers. + * + * paddle_get_params will block until parameters are initialized on + * the parameter servers. + * + * @param dst the destination array of parameter pointers to save to. + * The parameter pointer must be pre-popullated with required parameter name, + * and the content of parameter must be pre-allocated of the size of required + * parameter on pserver. + * @param len the length of the names array and the paddle_parameter + * array. + * @return 0 if successful, otherwise -1. + */ +int paddle_get_params(paddle_pserver_client client, paddle_parameter** dst, int len); + +/** + * @brief paddle_save_model indicates parameters to save the parameter + * to the given path + * + * @param path the path to save parameters. + * @return 0 if successful, otherwise -1. + */ +int paddle_save_model(paddle_pserver_client client, const char* path); +``` diff --git a/doc/_sources/design/cluster_train/remote_parameter_updater.md.txt b/doc/_sources/design/cluster_train/remote_parameter_updater.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..6e8e5938455b869e0f3367794c41250340b37f77 --- /dev/null +++ b/doc/_sources/design/cluster_train/remote_parameter_updater.md.txt @@ -0,0 +1,21 @@ +# Design Doc: Remote Parameter Updater for Cluster Train + +For an overview of distribute training, please refer to [distributed training design doc](README.md). In this design doc, we will discuss the parameter updater that will use parameter server cclient [The Client Library of Parameter Server Design Doc](pserver_client.md) to manage and update parameters. + +## Parameter Updater + +Parameter Updater is used by trainer to manage and update parameter, there are mainly two kind of parameter updater: local and remote, since this design is for cluster train, we will only discuss remote parameter updater here. + +### Remote Parameter Updater + +Remote Parameter Updater manage parameters through remote parameter server with the client that communicate with pserver([The Client Library of Parameter Server Design Doc](pserver_client.md)) + +In PaddlePaddle Python V2 API, trainer is implemented in python, and the trainer will hold a instance of parameter updater and call it's functions directly. In this design, we will also expose the api of RemoteParameterUpdater to python with swig. + +#### Sparse Remote Parameter Updater + +Since we will only implement dense parameter management new, the mechanism for sparse parameter will be discussed in next stage. + +### Interface Design + +TBD diff --git a/doc/_sources/design/cluster_train/save_model.md.txt b/doc/_sources/design/cluster_train/save_model.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..b755185c81ad617b9c85c47de0f5f65d2201c658 --- /dev/null +++ b/doc/_sources/design/cluster_train/save_model.md.txt @@ -0,0 +1,111 @@ +# Design Doc: Save Model + +## Overview + +The model is the output of the training process. There are two +ways from which user can obtain a model: + +- Save model triggered by user code: user code asks PaddlePaddle to + save a model. +- Convert model from the checkpoint: model being converted from + pservers' periodic checkpoint. In this way, the user can cancel a + job at any time, and still have a relatively fresh model (we + checkpoint around every 5 minutes). + +### Trainer Saving Model vs. Pservers Saving Model + +Both trainers and pservers have access to the model. So the model can +be saved from a trainer or pservers. We need to decide where the model +is saved from. + +#### Dense Update vs. Sparse Update + +There are two types of model update methods: dense update and sparse +update (when the model parameter is configured to be sparse). + +- Dense update + + Every trainer has it's own full copy of the model. Every model + update will update the entire model. + +- Sparse update + + The training input is sparse, and the trainer does not have the + entire model. It will only download the sub-model necessary related + to the input. When updating the model, only the sub-model related to + the training input is updated. + + +#### Pservers Saving Model + +The benefit of letting pservers save model is they have the entire +model all the time. However, since pservers are on different nodes, it +requires a merging process to merge model shards into the same +model. Thus requires the pservers to write models to a distributed +filesystem, making the checkpoint shards visible to the merge program. + +#### Trainer Saving Model + +The benefit of letting one trainer to save the model is it does not +require a distributed filesystem. And it's reusing the same save model +logic when training locally - except when doing sparse update, the +trainer needs to download the entire model during the saving process. + +#### Conclusion + +Given trainer saving model does not require a distributed filesystem, +and is an intuitive extension to trainer saving model when training +locally, we decide to let the trainer save the model when doing +distributed training. + + +### Convert Model from Checkpoint + +TODO + + +## Timeline + +We first implement trainer save the model. Converting the latest +snapshot to a model will be a TODO for future. + + +## Trainer Save Model + +### Trainer Election + +One trainer will be elected as the one to save the model. When using +etcd, trainer ID is a randomly generated UUID, the trainer will +contact the master server requesting to save the model, and find out +if itself is elected. When the master server is not used, unique +trainer IDs will be given by the administrator, the trainer whose ID +is "0" is elected to save the model. + +### Model Save Path + +Each trainer will be given the directory to save the model. The +elected trainer will save the model to +`given-directory/trainerID`. Since the trainer ID is unique, this +would prevent concurrent save to the same file when multiple trainers +are elected to save the model when split-brain problem happens. + +### What Happens When Model Is Saving + +It takes some time to save model, we need to define what will happen +when save model is taking place. + +When doing dense update, the trainer uses the local model. Pservers +does not need to pause model update. + +When doing sparse update. The trainer needs to download the entire +model while saving. To get the most accurate model, the model update +needs to be paused before the download starts and resumed after the +download finishes. Otherwise, the trainer gets a model that is +"polluted": some part of the model is old, some part of the model is +new. + +It's unclear that the "polluted" model will be inferior due to the +stochastic nature of deep learning, and pausing the model update will +add more complexity to the system. Since supporting sparse update is a +TODO item. We defer the evaluation of pause the model update or not +during saving model to the future. diff --git a/doc/_sources/design/cluster_train/submit-job.md.txt b/doc/_sources/design/cluster_train/submit-job.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..8377d5489dc64bd2fdc5bb4f7bc737e7b489000d --- /dev/null +++ b/doc/_sources/design/cluster_train/submit-job.md.txt @@ -0,0 +1,127 @@ +# Submit a Distributed Training Job + +The user can submit a distributed training job with Python code, rather than with a command-line interface. + +## Runtime Environment On Kubernetes + +For a distributed training job, there is two Docker image called *runtime Docker image* and *base Docker image*. The runtime Docker image is the Docker image that gets scheduled by Kubernetes to run during training. The base Docker image is for building the runtime Docker image. + +### Base Docker Image + +Usually, the base Docker image is PaddlePaddle product Docker image including paddle binary files and python package. And of course, users can specify any image name hosted on any docker registry which users have the access right. + +### Runtime Docker Image + +The trainer package which user upload and some Python dependencies are packaged into a runtime Docker image based on base Docker image. + +- Handle Python Dependencies + + You need to provide requirements.txt file in your `trainer-package` folder. Example: + + ```txt + pillow + protobuf==3.1.0 + ``` + More [details](https://pip.readthedocs.io/en/1.1/requirements.html) about requirements, an example project looks like: + ```bash + paddle_example + |-quick_start + |-trainer.py + |-dataset.py + |-requirements.txt + ``` + +## Submit Distributed Training Job With Python Code + + +- `paddle.job.dist_train()` will call the Job Server API `/v1/packages` to upload the trainer package and save them on CephFS, and then call `/v1/trainer/job` to submit the PaddlePaddle distributed job. +- `/v1/trainer/job` will start a building job for preparing the runtime Docker image. When the building job is finished, Job Server will submit the PaddlePaddle distributed job to Kubernetes. +- *NOTE*: For the first version, we will not prepare the runtime Docker image, instead, the package is uploaded to Paddle Cloud, and Paddle Cloud will mount the package in a temporary folder into the base Docker image. We will not support custom Python dependencies in the first version as well. + +You can call `paddle.job.dist_train` and provide distributed training configuration as the parameters: +```python +paddle.job.dist_train( + trainer=dist_trainer(), + paddle_job=PaddleJob( + job_name = "paddle-cloud", + entry_point = "python %s"%__file__, + trainer_package = "/example/word2vec", + image = "yancey1989/paddle-job", + trainers = 10, + pservers = 3, + trainer_cpu = 1, + trainer_gpu = 1, + trainer_mem = "10G", + pserver_cpu = 1, + pserver_mem = "2G" + )) +``` + +The parameter `trainer` of `paddle.job.dist_train` is a function and you can implement it as follows: +```python +def dist_trainer(): + def trainer_creator(): + trainer = paddle.v2.trainer.SGD(...) + trainer.train(...) + return trainer_creator +``` + +The pseudo code of `paddle.job.dist_train` is as follows: +```python +def dist_train(trainer, paddle_job): + # if the code is running on cloud, set PADDLE_ON_CLOUD=YES + if os.getenv("RUNNING_ON_CLOUD", "NO") == "NO": + #submit the paddle job + paddle_job.submit() + else: + #start the training + trainer() +``` +### PaddleJob Parameters +parameter | type | explanation + --- | --- | --- +job_name | str | the unique name for the training job +entry_point | str | entry point for startup trainer process +trainer_package | str | trainer package file path which user have the access right +image|str|the [base image](#base-docker-image) for building the [runtime image](#runtime-docker-image) +pservers|int| Parameter Server process count +trainers|int| Trainer process count +pserver_cpu|int| CPU count for each Parameter Server process +pserver_mem|str| memory allocated for each Parameter Server process, a plain integer using one of these suffixes: E, P, T, G, M, K +trainer_cpu|int| CPU count for each Trainer process +trainer_mem|str| memory allocated for each Trainer process, a plain integer using one of these suffixes: E, P, T, G, M, K +trainer_gpu|int| GPU count for each Trainer process, if you only want CPU, do not set this parameter + +### Deploy Parameter Server, Trainer and Master Process + - Deploy PaddlePaddle Parameter Server processes, it's a Kubernetes ReplicaSet. + - Deploy PaddlePaddle Trainer processes, it's a Kubernetes Job. + - Deploy PaddlePaddle Master processes, it's a Kubernetes ReplicaSet. + +## Job Server + +- RESTful API + + Job server provides RESTful HTTP API for receiving the trainer package and displaying + PaddlePaddle job related informations. + - `POST /v1/package` receive the trainer package and save them on CephFS + - `POST /v1/trainer/job` submit a trainer job + - `GET /v1/jobs/` list all jobs + - `GET /v1/jobs/` the status of a job + - `DELETE /v1/jobs/` delete a job + - `GET /v1/version` job server version + +- Build Runtime Docker Image on Kubernetes + + `paddle.job.dist_train` will upload the trainer package to Job Server, save them on the distributed filesystem, and then start up a job for building the runtime Docker image that gets scheduled by Kubernetes to run during training. + + There are some benefits for building runtime Docker image on JobServer: + - On Paddle Cloud, users will run the trainer code in a Jupyter Notebook which is a Kubernetes Pod, if we want to execute `docker build` in the Pod, we should mount the host's `docker.sock` to the Pod, user's code will connect the host's Docker Engine directly, it's not safe. + - Users only need to upload the training package files, does not need to install docker engine, docker registry as dependencies. + - If we want to change another image type, such as RKT, users do not need to care about it. + +- Deploy Parameter Server, Trainer and Master Processes + + `POST /v1/trainer/job` receives the distributed training parameters, and deploy the job as follows: + - Deploy PaddlePaddle Parameter Server processes, it's a Kubernetes ReplicaSet. + - Deploy PaddlePaddle Trainer processes, it's a Kubernetes Job. + - Deploy PaddlePaddle Master processes, it's a Kubernetes ReplicaSet. diff --git a/doc/_sources/design/evaluator.md.txt b/doc/_sources/design/evaluator.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..11cc129d56905a9ee666da92fbe6f8559c6d325a --- /dev/null +++ b/doc/_sources/design/evaluator.md.txt @@ -0,0 +1,58 @@ +## Evaluator Design + +### Problem Statement + +During training or inference, we provide an evaluation function to measure the model performance, for example, accuracy, precision, etc. In the operator based framework design, the data passes through the network pipeline batch by batch. As a result, inside the operator, we only calculate the metrics for one minibatch. Thus, we need to provide a mechanism to calculate the metrics for each N pass/batch the user wants. + +### Evaluator Design +Currently, every operation is expressed in the graph. We divide the evaluator process into three steps. + +1. Initialize the metric state and add it into the block. + +2. Calculate the concerned metrics for every mini-batch. The single evaluator operator is only responsible for calculating the necessary statistics for one mini-batch. For example, the accuracy operator only calculates the accuracy for a minibatch data if run once. + + +3. Merge the mini-batch statistics to form the evaluation result for multiple mini-batches. When it comes to distributed training/Multi-GPU training, aggregate the value from different devices. + +### Implementation +This design is shown in the Python API. +Each metric operator needs to caculate the metric statistic and return the batch-aware states. Python side is responsible for accumulating the states for each pass. + + +```python +class Evaluator(object): + """ + Evaluator Base class. + """ + def __init__(self, name, **kwargs): + """ + Different evaluator may has different metric states. E.g, Accuracy need two variables, total and right sample counts. + Auc need four variables, `true_positives`, + `true_negatives`, `false_positives` and `false_negatives`. So every evaluator should create its needed variables and append to main_program + + The initialization of Evaluator should be responsible for: + create metric states and append to the main_program + """ + pass + + def _update_ops(self, input, label, **kwargs) + """ + Add mini-batch evaluator caculate operators to the main_program. + Add increment operator to accumulate the metric states. + """ + + + def reset(self, executor, reset_program=None): + """ + Reset metric states at the begin of each pass/user specified batch number. + Execute the reset_program to reset the states. + """ + + + def eval(self, executor, eval_program=None): + """ + Merge the mini-batch statistics to form the evaluation result for multiple mini-batches. + Execute the eval_program and return the result. + """ + return eval_result +``` diff --git a/doc/_sources/design/executor.md.txt b/doc/_sources/design/executor.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..b5fb6c5c3c1da3c112ce63878322083dd5c42b70 --- /dev/null +++ b/doc/_sources/design/executor.md.txt @@ -0,0 +1,23 @@ +# Executor Design Doc + +## Motivation + +We use executor to do the runtime evaluation of a `ProgramDesc`. + +## Overview + +An executor takes a `ProgramDesc`, a `block_id` and a `Scope`. The `ProgramDesc` is a list of blocks and each block contains the protobuf definition of all the parameters and operators. The `block_id` specifies the entrance block. And the `Scope` is the container of all the variable instance, which is persistent throughout different runs. + +### What does executor do? + +It evaluates all the operators in the `block_id`th block of a `ProgramDesc`. + +### What does executor NOT do? + +It does not do runtime optimization, meaning intelligently parse the dependency of each op a choose which one to be run and in which order they should be run. + +It does not do graph partitioning, meaning dividing the `ProgramDesc` into several small pieces and executing them on different devices. + +## Implementation + +`Executor` evaluates a `ProgramDesc`. Essentially, it instantiates Variables and Operators, then run all the operators in sequence. [[code]](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/executor.cc) diff --git a/doc/_sources/design/file_manager/README.md.txt b/doc/_sources/design/file_manager/README.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..3df10d801e568834729f902aace483d033340e2d --- /dev/null +++ b/doc/_sources/design/file_manager/README.md.txt @@ -0,0 +1,87 @@ +# FileManager设计文档 +## 目标 +在本文档中，我们设计说明了名为FileManager系统，方便用户上传自己的训练数据以进行分布式训练 + +主要功能包括： + +- 提供常用的命令行管理命令管理文件和目录 +- 支持大文件的断点上传、下载 + +## 名词解释 +- PFS：是`Paddlepaddle cloud File System`的缩写，是对用户文件存储空间的抽象，与之相对的是local filesystem。目前我们用CephFS来搭建。 +- [CephFS](http://docs.ceph.com/docs/master/cephfs/)：一个POSIX兼容的文件系统。 +- Chunk：逻辑划上文件分块的单位。 + +## 模块 +### 架构图 + + +### PFSClient +- 功能： 详细设计[link](./pfs/pfsclient.md) + - 提供用户管理文件的命令 + - 需要可以跨平台执行 + +- 双向验证 + PFSClient需要和Ingress之间做双向验证[tls](#tls)，所以用户需要首先在`cloud.paddlepaddle.org`上注册一下，申请用户空间，并且把系统生成的CA(certificate authority)、Key、CRT(CA signed certificate)下载到本地，然后才能使用PFSClient。 + +### [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) +- 功能： + 提供七层协议的反向代理、基于粘性会话的负载均衡功能。 + +- 透传用户身份的办法 + Ingress需要把PFSClient的身份信息传给PFSServer，配置的方法参考[link](http://www.integralist.co.uk/posts/clientcertauth.html#3) + +### PFSServer +PFSServer提供RESTful API接口，接收处理PFSClient端的文件管理请求，并且把结果返回PFSClient端。 + +RESTful API + +- /api/v1/files + - `GET /api/v1/files`: Get metadata of files or directories. + - `POST /api/v1/files`: Create files or directories. + - `PATCH /api/v1/files`: Update files or directories. + - `DELETE /api/v1/files`: Delete files or directories. + +- /api/v1/file/chunks + - `GET /api/v1/storage/file/chunks`: Get chunks's metadata of a file. + +- /api/v1/storage/files + - `GET /api/v1/storage/files`: Download files or directories. + - `POST /api/v1/storage/files`: Upload files or directories. + +- /api/v1/storage/file/chunks + - `GET /api/v1/storage/file/chunks`: Download chunks's data. + - `POST /api/v1/storage/file/chunks`: Upload chunks's data. + +## 文件传输优化 + +### 分块文件传输 +用户文件可能是比较大的，上传到Cloud或者下载到本地的时间可能比较长，而且在传输的过程中也可能出现网络不稳定的情况。为了应对以上的问题，我们提出了Chunk的概念，一个Chunk由所在的文件偏移、数据、数据长度及校验值组成。文件的上传和下载都是通过对Chunk的操作来实现的。由于Chunk比较小（默认256K），完成一个传输动作完成的时间也比较短，不容易出错。PFSClient需要在传输完毕最后一个Chunk的时候检查destination文件的MD5值是否和source文件一致。 + +一个典型的Chunk如下所示： + +``` +type Chunk struct { + fileOffset int64 + checksum uint32 + len uint32 + data []byte +} +``` + +### 生成sparse文件 +当destination文件不存在或者大小和source文件不一致时，可以用[Fallocate](https://Go.org/pkg/syscall/#Fallocate)生成sparse文件，然后就可以并发写入多个Chunk。 + +### 覆盖不一致的部分 +文件传输的的关键在于需要PFSClient端对比source和destination的文件Chunks的checksum是否保持一致，不一致的由PFSClient下载或者传输Chunk完成。这样已经传输成功的部分就不用重新传输了。 + +## 用户使用流程 +参考[link](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/cluster_train/data_dispatch.md) + +## 框架生成 +用[swagger](https://github.com/swagger-api/swagger-codegen)生成PFSClient和PFSServer的框架部分，以便我们可以把更多的精力放到逻辑本身上。 + +## 参考文档 +- [TLS complete guide](https://github.com/k8sp/tls/blob/master/tls.md) +- [aws.s3](http://docs.aws.amazon.com/cli/latest/reference/s3/) +- [linux man document](https://linux.die.net/man/) diff --git a/doc/_sources/design/file_manager/pfs/pfsclient.md.txt b/doc/_sources/design/file_manager/pfs/pfsclient.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..56bc70c54bbc92b78d66e04fb495b1300cf8ebe0 --- /dev/null +++ b/doc/_sources/design/file_manager/pfs/pfsclient.md.txt @@ -0,0 +1,129 @@ +# PFSClient + +## Description +The `pfs` command is a Command Line Interface to manage your files on PaddlePaddle Cloud + +## Synopsis +``` +paddle [options] pfs [parameters] +``` + +## Options +``` +--profile (string) + Use a specific profile from your credential file. + +--help (string) + Display more information about command + +--version + Output version information and exit + +--debug + Show detailed debugging log + +--only-show-errors (boolean) + Only errors and warnings are displayed. All other output is suppressed. +``` + +## Path Arguments +When using a command, we need to specify path arguments. There are two path argument type: `localpath` and `pfspath`. + +A `pfspath` begin with `/pfs`, eg: `/pfs/$DATACENTER/home/$USER/folder`. + +[Here](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/cluster_train/data_dispatch.md#上传训练文件) is how to config datacenters. + +## order of Path Arguments +Commonly, if there are two path arguments, the first is the source, and the second is the destination. + +## Subcommonds +- rm - remove files or directories + +``` +Synopsis: + rm [-r] [-v] ... + +Options: + -r + Remove directories and their contents recursively + -v + Cause rm to be verbose, showing files after they are removed. + +Examples: + paddle pfs rm /pfs/$DATACENTER/home/$USER/file + paddle pfs rm -r /pfs/$DATACENTER/home/$USER/folder +``` +- mv - move (rename) files + +``` +Synopsis: + mv [-f | -n] [-v] + mv [-f | -n] [-v] ... + mv [-f | -n] [-v] + mv [-f | -n] [-v] ... + mv [-f | -n] [-v] + mv [-f | -n] [-v] ... + +Options: + -f + Do not prompt for confirmation before overwriting the destination path. (The -f option overrides previous -n options.) + -n + Do not overwrite an existing file. (The -n option overrides previous -f options.) + -v + Cause mv to be verbose, showing files after they are moved. + +Examples: + paddle pfs mv ./text1.txt /pfs/$DATACENTER/home/$USER/text1.txt +``` +- cp - copy files or directories + +``` +Synopsis: + cp [-r] [-f | -n] [-v] [--preserve--links] + cp [-r] [-f | -n] [-v] [--preserve--links] ... + cp [-r] [-f | -n] [-v] [--preserve--links] + cp [-r] [-f | -n] [-v] [--preserve--links] ... + cp [-r] [-f | -n] [-v] [--preserve--links] + cp [-r] [-f | -n] [-v] [--preserve--links] ... + +Options: + -r + Copy directories recursively + -f + Do not prompt for confirmation before overwriting the destination path. (The -f option overrides previous -n options.) + -n + Do not overwrite an existing file. (The -n option overrides previous -f options.) + -v + Cause cp to be verbose, showing files after they are copied. + --preserve--links + Reserve links when copy links + +Examples: + paddle pfs cp ./file /pfs/$DATACENTER/home/$USER/file + paddle pfs cp /pfs/$DATACENTER/home/$USER/file ./file +``` +- ls- list files + +``` +Synopsis: + ls [-r] ... + +Options: + -R + List directory(ies) recursively + +Examples: + paddle pfs ls /pfs/$DATACENTER/home/$USER/file + paddle pfs ls /pfs/$DATACENTER/home/$USER/folder +``` + +- mkdir - mkdir directory(ies) +Create intermediate directory(ies) as required. + +``` +Synopsis: + mkdir ... + +Examples: + paddle pfs mkdir /pfs/$DATACENTER/home/$USER/folder +``` diff --git a/doc/_sources/design/float16.md.txt b/doc/_sources/design/float16.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..1ea95ed6b5d6792171569b6ff76d09be92fcb13e --- /dev/null +++ b/doc/_sources/design/float16.md.txt @@ -0,0 +1,105 @@ +# Design Doc: float16 + +## Why float16 +Half precision (float16) is a binary floating-point format that occupies 16 bits in memory. float16 is half the size of traditional 32-bit single precision format (float) and has lower precision and smaller range. + +When high precision computation is not required, using float16 data type could potentially + +- reduce storage space, memory bandwidth, and power usages; +- increase the chance of data fitting into a smaller cache of lower latency; +- provide arithmetic speed up if supported by hardware. + +## Survey of current float16 support +A brief survey of float16 support on different compilers, hardwares, and libraries can be found below. Interested readers can refer to [link1](https://github.com/PaddlePaddle/Paddle/issues/4853) and [link2](https://github.com/Xreki/Xreki.github.io/blob/master/multi_data_types_in_dl_framework/ppt/float16_and_quantized_type.md) for more info. + +The goal of float16 is to serve as a key for the executor to find and run the correct version of compute method specialized for float16 in operator kernel. It should be compatible with various natively supported float16 implementations including `__half` for cuda, `float16_t` for ARM, and `Eigen::half` for Eigen to make writing customized float16 kernels easier. + +### Compiler +- nvcc supports `__half` data type after CUDA 7.5. +- `__fp16` or `float16_t` is supported as storage type for gcc >= 6.1 and clang >= 3.4. +- `__fp16` or `float16_t` is supported as arithmetic type for gcc >= 7.1 and clang >= 3.9. + +### Hardware +- `__half` is supported on GPU with compute capability >= 5.3. +- `__fp16` is supported as storage type for ARMv7-A, ARMv8-A, and above. +- `__fp16` is supported as arithmetic type after ARMv8.2-A (currently, the only microarchitecture implementing ARMv8.2-A is ARM Cortex-A75, which is announced in May 2017. There seems to be no application processors currently available on market that adopts this architecture. It is reported that Qualcomm Snapdragon 845 uses Cortex-A75 design and will be available in mobile devices in early 2018). + +### Libraries +- [Eigen](https://github.com/RLovelett/eigen) >= 3.3 supports float16 calculation on both GPU and CPU using the `Eigen::half` class. It is mostly useful for Nvidia GPUs because of the overloaded arithmetic operators using cuda intrinsics. It falls back to using software emulation on CPU for calculation and there is no special treatment to ARM processors. +- [ARM compute library](https://github.com/ARM-software/ComputeLibrary) >= 17.02.01 supports NEON FP16 kernels (requires ARMv8.2-A CPU). + +### CUDA version issue +There are currently three versions of CUDA that supports `__half` data type, namely, CUDA 7.5, 8.0, and 9.0. +CUDA 7.5 and 8.0 define `__half` as a simple struct that has a `uint16_t` data (see [`cuda_fp16.h`](https://github.com/ptillet/isaac/blob/9212ab5a3ddbe48f30ef373f9c1fb546804c7a8c/include/isaac/external/CUDA/cuda_fp16.h)) as follows: +``` +typedef struct __align__(2) { + unsigned short x; +} __half; + +typedef __half half; +``` +This struct does not define any overloaded arithmetic operators. So you have to directly use `__hadd` instead of `+` to correctly add two half types: +``` +__global__ void Add() { + half a, b, c; + c = __hadd(a, b); // correct + c = a + b; // compiler error: no operator "+" matches these operands +} +``` +CUDA 9.0 provides a major update to the half data type. The related code can be found in the updated [`cuda_fp16.h`](https://github.com/ptillet/isaac/blob/master/include/isaac/external/CUDA/cuda_fp16.h) and the newly added [`cuda_fp16.hpp`](https://github.com/ptillet/isaac/blob/master/include/isaac/external/CUDA/cuda_fp16.hpp). + +Essentially, CUDA 9.0 renames the original `__half` type in 7.5 and 8.0 as `__half_raw`, and defines a new `__half` class type that has constructors, conversion operators, and also provides overloaded arithmetic operators such as follows: +``` +typedef struct __CUDA_ALIGN__(2) { + unsigned short x; +} __half_raw; + + +struct __CUDA_ALIGN__(2) __half { +protected: + unsigned short __x; +public: + // constructors and conversion operators from/to + // __half_raw and other built-in data types +} + +typedef __half half; + +__device__ __forceinline__ +__half operator+(const __half &lh, const __half &rh) { + return __hadd(lh, rh); +} + +// Other overloaded operators +``` +This new design makes `c = a + b` work correctly for CUDA half data type. + +## Implementation +The float16 class holds a 16-bit `uint16_t` data internally. +``` +struct float16 { + uint16_t x; +}; +``` + +float16 supports the following features: + - constructors / assignment operators that take input from primitive data types including bool, integers of various length, float, and double. + - constructors / assignment operators that take input from `__half` on cuda, `float16_t` on ARM, and `Eigen::half` on Eigen. + - conversion operators to primitive data types and half precision data types on cuda, ARM and Eigen. + - overloaded arithmetic operators for cuda, arm, and non-arm cpu, respectively. These operators will take advantage of the cuda and ARM intrinsics on the corresponding hardware. + +To support the above features, two fundamental conversion functions are provided: +``` +float16 float_to_half_rn(float f); // convert to half precision in round-to-nearest-even mode +float half_to_float(float16 h); +``` +which provides one-to-one conversion between float32 and float16. These twos functions will do different conversion routines based on the current hardware. CUDA/ARM instrinsics will be used when the corresonding hardware is available. If the hardware or compiler level does not support float32 to float16 conversion, software emulation will be performed to do the conversion. + +## To do +After float16 class is available, some of the future items are below: + +- Update pybind/tensor_py.h to bind c++ float16 with numpy float16. + +- Modify `GetKernelType()` method in `framework/operator.h` to make it compatible with float16. + +- Create a type-casting operator that can convert the data type in tensor between float16 and other types. diff --git a/doc/_sources/design/functions_operators_layers.md.txt b/doc/_sources/design/functions_operators_layers.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..984b59f4c6971dfb6f46dfe342f2751f392c0e88 --- /dev/null +++ b/doc/_sources/design/functions_operators_layers.md.txt @@ -0,0 +1,100 @@ +# Design Doc: Functions, Operators, and Layers + +In a DL system, we can compose one or more fine grained operators into a coarse grained one. For example, the FC layer can be composed of a multiplication operator and an add operator. + +Historically, some fine grained operations are known as operators, and some coarse level ones are known as layers. But we need a well-defined separation. + +In general, operators are those very fine grained operations, e.g., mul and add. In the implementation, we can write them as C++ functions: + +```c++ +template T add(T x, T y) { return x + y; } +template T mul(T x, T y) { return x * y; } +``` + +Then we can wrap them into operators which are C++ classes and can be created from Python bindings by name. A C macro can do this. For example, the following macro invocation + +```c++ +#define MAKE_FUNCTION_OPERATOR(mul); +``` + +generates + +```c++ +template class mulOp : public OperatorBase {...}; +REGISTER_OP(mulOp, "mul"); +``` + +so that in Python we can create operator mul by: + +```python +X1 = Var() +X2 = Var() +Y = Var() +paddle.cpp.create_operator("mul", input=[X1, X2], output=Y) +``` + +Also, at the same time, we can compose a coarse level C++ operator class by composing functions `mul` and `add`: + +```c++ +template +class FCOp : public OperatorBase { + public: + void Run(...) { + add(mul(Input("X"), Input("W")), Input("b"); + } +}; +REGISTER_OP(FCOp, "fc"); +``` + +We need to support such composition in Python as well. To do so, we need a higher level Python wrapping of operator creation than `paddle.cpp.create_operator`. This higher level operator API should be compatible with the layer API. + +Let's explain using an example. Suppose that we are going to compose the FC using mul and add in Python, we'd like to have Python functions `mul` and `add` defined in module `operator`: + +```python +def operator.mul(X1, X2): + O = Var() + paddle.cpp.create_operator("mul", input={X1, Y1}, output=O) + return O + +def operator.add(X1, X2): + O = Var() + paddle.cpp.create_operator("add", input={X1, X2}, output=O) + return O +``` + +Above code snippets are automatically generated. Given them, users can define + +```python +def layer.fc(X): + W = Var() + b = Var() + return operator.add(operator.mul(X, W), b) +``` + +If we don't have `operator.mul` and `operator.add`, the definiton of `layer.fc` would be complicated: + +```python +def layer.fc(X): + W = Var() + b = Var() + O1 = Var() + paddle.cpp.create_operator("mul", input=[X, W], output=O1) + O2 = Var() + paddle.cpp.create_operator("add", input=[O1, b], output=O2) + return O2 +``` + +We'd like to have Python bindings to operators in package `paddle.operator`, and Python compositions of operators in package `paddle.layer`. So we have the following concepts in above illustrative example: + + +| C++ functions/functors | mul | add | | | +|------------------------|--------------|--------------|-------------|----------| +| C++ operator class | mulOp | addOp | FCOp | | +| Python binding | operator.mul | operator.add | operator.fc | | +| Python function | | | | layer.fc | + + +This is how we differentiate layer and operators in PaddlePaddle: + +- those defined in C++ and have a lightweighted Python wrapper in module `operators` are operators; whereas +- those who don't have C++ implementations but a Python implementation that compose C++ operators are known as layers. diff --git a/doc/_sources/design/gan_api.md.txt b/doc/_sources/design/gan_api.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..fb41df8615f73d9fd4c32995eab265833eac1a55 --- /dev/null +++ b/doc/_sources/design/gan_api.md.txt @@ -0,0 +1,253 @@ +# Design for GAN + +GAN (General Adversarial Net [https://arxiv.org/abs/1406.2661]) is an important model for unsupervised learning and widely used in many areas. + +It applies several important concepts in machine learning system design, including building and running subgraphs, dependency tracing, different optimizers in one executor and so forth. + +In our GAN design, we wrap it as a user-friendly easily customized python API to design different models. We take the conditional DC-GAN (Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks [https://arxiv.org/abs/1511.06434]) as an example due to its good performance on image generation. + +

+
+Figure 1. The overall running logic of GAN. The black solid arrows indicate the forward pass; the green dashed arrows indicate the backward pass of generator training; the red dashed arrows indicate the backward pass of the discriminator training. The BP pass of the green (red) arrow should only update the parameters in the green (red) boxes. The diamonds indicate the data providers. d\_loss and g\_loss marked in red and green are the two targets we would like to run. +

+ +The operators, layers and functions required/optional to build a GAN demo is summarized in https://github.com/PaddlePaddle/Paddle/issues/4563. + +

+
+Figure 2. Photo borrowed from the original DC-GAN paper. +

+ +## The Conditional-GAN might be a class. +This design we adopt the popular open source design in https://github.com/carpedm20/DCGAN-tensorflow and https://github.com/rajathkmp/DCGAN. It contains following data structure: + +- DCGAN(object): which contains everything required to build a GAN model. It provides following member functions methods as API: + +- __init__(...): Initialize hyper-parameters (like conv dimension and so forth), and declare model parameters of discriminator and generator as well. + +- generator(z, y=None): Generate a fake image from input noise z. If the label y is provided, the conditional GAN model will be chosen. +Returns a generated image. + +- discriminator(image): +Given an image, decide if it is from a real source or a fake one. +Returns a 0/1 binary label. + +- build_model(self): +build the whole GAN model, define training loss for both generator and discrimator. + +## Discussion on Engine Functions required to build GAN +- Trace the tensor and variable dependency in the engine executor. (Very critical, otherwise GAN can'be be trained correctly) +- Different optimizers responsible for optimizing different loss. + +To be more detailed, we introduce our design of DCGAN as following: + +### Class member Function: Initializer +- Set up hyper-parameters, including condtional dimension, noise dimension, batch size and so forth. +- Declare and define all the model variables. All the discriminator parameters are included in the list self.theta_D and all the generator parameters are included in the list self.theta_G. +```python +class DCGAN(object): + def __init__(self, y_dim=None): + + # hyper parameters + self.y_dim = y_dim # conditional gan or not + self.batch_size = 100 + self.z_dim = z_dim # input noise dimension + + # define parameters of discriminators + self.D_W0 = pd.Variable(shape=[3,3, 1, 128], data=pd.gaussian_normal_randomizer()) + self.D_b0 = pd.Variable(np.zeros(128)) # variable also support initialization using a numpy data + self.D_W1 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer()) + self.D_b1 = pd.Variable(np.zeros(128)) # variable also support initialization using a numpy data + self.D_W2 = pd.Varialble(np.random.rand(128, 1)) + self.D_b2 = pd.Variable(np.zeros(128)) + self.theta_D = [self.D_W0, self.D_b0, self.D_W1, self.D_b1, self.D_W2, self.D_b2] + + # define parameters of generators + self.G_W0 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer()) + self.G_b0 = pd.Variable(np.zeros(128)) # variable also support initialization using a numpy data + self.G_W1 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer()) + self.G_b1 = pd.Variable(np.zeros(128)) # variable also support initialization using a numpy data + self.G_W2 = pd.Varialble(np.random.rand(128, 1)) + self.G_b2 = pd.Variable(np.zeros(128)) + self.theta_G = [self.G_W0, self.G_b0, self.G_W1, self.G_b1, self.G_W2, self.G_b2] +``` + +### Class member Function: Generator +- Given a noisy input z, returns a fake image. +- Concatenation, batch-norm, FC operations required; +- Deconv layer required, which is missing now... +```python +class DCGAN(object): + def generator(self, z, y = None): + # input z: the random noise + # input y: input data label (optional) + # output G_im: generated fake images + + if not self.y_dim: + z = pd.layer.concat(1, [z, y]) + + G_h0 = pd.layer.fc(z, self.G_w0, self.G_b0) + G_h0_bn = pd.layer.batch_norm(G_h0) + G_h0_relu = pd.layer.relu(G_h0_bn) + + G_h1 = pd.layer.deconv(G_h0_relu, self.G_w1, self.G_b1) + G_h1_bn = pd.layer.batch_norm(G_h1) + G_h1_relu = pd.layer.relu(G_h1_bn) + + G_h2 = pd.layer.deconv(G_h1_relu, self.G_W2, self.G_b2)) + G_im = pd.layer.tanh(G_im) + return G_im +``` + +### Class member function: Discriminator +- Given a noisy input z, returns a fake image. +- Concatenation, Convolution, batch-norm, FC, Leaky-ReLU operations required; +```python +class DCGAN(object): + def discriminator(self, image): + # input image: either generated images or real ones + # output D_h2: binary logit of the label + + D_h0 = pd.layer.conv2d(image, w=self.D_w0, b=self.D_b0) + D_h0_bn = pd.layer.batchnorm(h0) + D_h0_relu = pd.layer.lrelu(h0_bn) + + D_h1 = pd.layer.conv2d(D_h0_relu, w=self.D_w1, b=self.D_b1) + D_h1_bn = pd.layer.batchnorm(D_h1) + D_h1_relu = pd.layer.lrelu(D_h1_bn) + + D_h2 = pd.layer.fc(D_h1_relu, w=self.D_w2, b=self.D_b2) + return D_h2 +``` + +### Class member function: Build the model +- Define data readers as placeholders to hold the data; +- Build generator and discriminators; +- Define two training losses for discriminator and generator, respectively. +If we have execution dependency engine to back-trace all tensors, the module building our GAN model will be like this: +```python +class DCGAN(object): + def build_model(self): + if self.y_dim: + self.y = pd.data(pd.float32, [self.batch_size, self.y_dim]) + self.images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size]) + self.faked_images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size]) + self.z = pd.data(tf.float32, [None, self.z_size]) + + # step 1: generate images by generator, classify real/fake images with discriminator + if self.y_dim: # if conditional GAN, includes label + self.G = self.generator(self.z, self.y) + self.D_t = self.discriminator(self.images) + # generated fake images + self.sampled = self.sampler(self.z, self.y) + self.D_f = self.discriminator(self.G) + else: # original version of GAN + self.G = self.generator(self.z) + self.D_t = self.discriminator(self.images) + # generate fake images + self.sampled = self.sampler(self.z) + self.D_f = self.discriminator(self.images) + + # step 2: define the two losses + self.d_loss_real = pd.reduce_mean(pd.cross_entropy(self.D_t, np.ones(self.batch_size)) + self.d_loss_fake = pd.reduce_mean(pd.cross_entropy(self.D_f, np.zeros(self.batch_size)) + self.d_loss = self.d_loss_real + self.d_loss_fake + + self.g_loss = pd.reduce_mean(pd.cross_entropy(self.D_f, np.ones(self.batch_szie)) +``` + +If we do not have dependency engine but blocks, the module building our GAN model will be like this: +```python +class DCGAN(object): + def build_model(self, default_block): + # input data in the default block + if self.y_dim: + self.y = pd.data(pd.float32, [self.batch_size, self.y_dim]) + self.images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size]) + # self.faked_images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size]) + self.z = pd.data(tf.float32, [None, self.z_size]) + + # step 1: generate images by generator, classify real/fake images with discriminator + with pd.default_block().g_block(): + if self.y_dim: # if conditional GAN, includes label + self.G = self.generator(self.z, self.y) + self.D_g = self.discriminator(self.G, self.y) + else: # original version of GAN + self.G = self.generator(self.z) + self.D_g = self.discriminator(self.G, self.y) + self.g_loss = pd.reduce_mean(pd.cross_entropy(self.D_g, np.ones(self.batch_szie)) + + with pd.default_block().d_block(): + if self.y_dim: # if conditional GAN, includes label + self.D_t = self.discriminator(self.images, self.y) + self.D_f = self.discriminator(self.G, self.y) + else: # original version of GAN + self.D_t = self.discriminator(self.images) + self.D_f = self.discriminator(self.G) + + # step 2: define the two losses + self.d_loss_real = pd.reduce_mean(pd.cross_entropy(self.D_t, np.ones(self.batch_size)) + self.d_loss_fake = pd.reduce_mean(pd.cross_entropy(self.D_f, np.zeros(self.batch_size)) + self.d_loss = self.d_loss_real + self.d_loss_fake +``` +Some small confusion and problems with this design: +- D\_g and D\_f are actually the same thing, but has to be written twice; i.e., if we want to run two sub-graphs conceptually, the same codes have to be written twice if they are shared by the graph. +- Requires ability to create a block anytime, rather than in if-else or rnn only; + +## Main function for the demo: +Generally, the user of GAN just need to the following things: +- Define an object as DCGAN class; +- Build the DCGAN model; +- Specify two optimizers for two different losses with respect to different parameters. +```python +# pd for short, should be more concise. +from paddle.v2 as pd +import numpy as np +import logging + +if __name__ == "__main__": + # dcgan class in the default graph/block + # if we use dependency engine as tensorflow + # the codes, will be slightly different like: + # dcgan = DCGAN() + # dcgan.build_model() + with pd.block() as def_block: + dcgan = DCGAN() + dcgan.build_model(def_block) + + # load mnist data + data_X, data_y = self.load_mnist() + + # Two subgraphs required!!! + with pd.block().d_block(): + d_optim = pd.train.Adam(lr = .001, beta= .1) + d_step = d_optim.minimize(dcgan.d_loss, dcgan.theta_D) + with pd.block.g_block(): + g_optim = pd.train.Adam(lr = .001, beta= .1) + g_step = pd.minimize(dcgan.g_loss, dcgan.theta_G) + + # executor + sess = pd.executor() + + # training + for epoch in xrange(10000): + for batch_id in range(N / batch_size): + idx = ... + # sample a batch + batch_im, batch_label = data_X[idx:idx+batch_size], data_y[idx:idx+batch_size] + # sample z + batch_z = np.random.uniform(-1., 1., [batch_size, z_dim]) + + if batch_id % 2 == 0: + sess.run(d_step, + feed_dict = {dcgan.images: batch_im, + dcgan.y: batch_label, + dcgan.z: batch_z}) + else: + sess.run(g_step, + feed_dict = {dcgan.z: batch_z}) +``` + +# More thinking about dependency engine v.s. block design: +- What if we just want to run an intermediate result? Do we need to run the whole block/graph? +- Should we call eval() to get the fake images in the first stage? And then train the discriminator in the second stage? diff --git a/doc/_sources/design/graph.md.txt b/doc/_sources/design/graph.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..7519a65df835a39fe14f6ef45530afff170191ff --- /dev/null +++ b/doc/_sources/design/graph.md.txt @@ -0,0 +1,70 @@ +# Design Doc: Computations as a Graph + +A primary goal of the refactorization of PaddlePaddle is a more flexible representation of deep learning computation, in particular, a graph of operators and variables, instead of sequences of layers as before. + +This document explains that the construction of a graph as three steps: + +- construct the forward part +- construct the backward part +- construct the optimization part + +## The Construction of a Graph + +Let us take the problem of image classification as a simple example. The application program that trains the model looks like: + +```python +x = layer.data("images") +l = layer.data("label") +y = layer.fc(x) +cost = layer.mse(y, l) +optimize(cost) +train(cost, reader=mnist.train()) +``` + +### Forward Part + +The first four lines of above program build the forward part of the graph. + +![](images/graph_construction_example_forward_only.png) + +In particular, the first line `x = layer.data("images")` creates variable x and a Feed operator that copies a column from the minibatch to x. `y = layer.fc(x)` creates not only the FC operator and output variable y, but also two parameters, W and b, and the initialization operators. + +Initialization operators are kind of "run-once" operators -- the `Run` method increments a class data member counter so to run at most once. By doing so, a parameter wouldn't be initialized repeatedly, say, in every minibatch. + +In this example, all operators are created as `OpDesc` protobuf messages, and all variables are `VarDesc`. These protobuf messages are saved in a `BlockDesc` protobuf message. + +### Backward Part + +The fifth line `optimize(cost)` calls two functions, `ConstructBackwardGraph` and `ConstructOptimizationGraph`. + +`ConstructBackwardGraph` traverses the forward graph in the `BlockDesc` protobuf message and builds the backward part. + +![](images/graph_construction_example_forward_backward.png) + +According to the chain rule of gradient computation, `ConstructBackwardGraph` would + +1. create a gradient operator G for each operator F, +1. make all inputs, outputs, and outputs' gradient of F as inputs of G, +1. create gradients for all inputs of F, except for those who don't have gradients, like x and l, and +1. make all these gradients as outputs of G. + +### Optimization Part + +For each parameter, like W and b created by `layer.fc`, marked as double circles in above graphs, `ConstructOptimizationGraph` creates an optimization operator to apply its gradient. Here results in the complete graph: + +![](images/graph_construction_example_all.png) + +## Block and Graph + +The word block and graph are interchangable in the desgin of PaddlePaddle. A [Block](https://github.com/PaddlePaddle/Paddle/pull/3708) is a metaphore of the code and local variables in a pair of curly braces in programming languages, where operators are like statements or instructions. A graph of operators and variables is a representation of the block. + +A Block keeps operators in an array `BlockDesc::ops` + +```protobuf +message BlockDesc { + repeated OpDesc ops = 1; + repeated VarDesc vars = 2; +} +``` + +in the order that they appear in user programs, like the Python program at the beginning of this article. We can imagine that in `ops`, we have some forward operators, followed by some gradient operators, and then some optimization operators. diff --git a/doc/_sources/design/graph_survey.md.txt b/doc/_sources/design/graph_survey.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..6c6db08f463ae0a2b94fc4546f123a1d7c151870 --- /dev/null +++ b/doc/_sources/design/graph_survey.md.txt @@ -0,0 +1,232 @@ +## Survey on Graph + +Neural network framework often provides symbolic API for users to write network topology conveniently. This doc manily focus on symbolic API in most popular neural network frameworks, and try to find out how to parse symbolic configuration to a portable file, such as protobuf or json. + +### Mxnet + +The core concept of symbolic API is `Symbol`. Mxnet implements `Symbol` class in C++, and export to Python using C-API. Please refer to the comments in Mxnet: + + +`Symbol` is help class used to represent the operator node in Graph. +`Symbol` acts as an interface for building graphs from different components like Variable, Functor and Group. `Symbol` is also exported to python front-end (while Graph is not) to enable quick test and deployment. Conceptually, symbol is the final operation of a graph and thus including all the information required (the graph) to evaluate its output value. + + +A simple network topology wrote by Symbol is as follows: + +```python +def get_symbol(num_classes=10, **kwargs): + data = mx.symbol.Variable('data') + data = mx.symbol.Flatten(data=data) + fc1 = mx.symbol.FullyConnected(data = data, name='fc1', num_hidden=128) + act1 = mx.symbol.Activation(data = fc1, name='relu1', act_type="relu") + fc2 = mx.symbol.FullyConnected(data = act1, name = 'fc2', num_hidden = 64) + act2 = mx.symbol.Activation(data = fc2, name='relu2', act_type="relu") + fc3 = mx.symbol.FullyConnected(data = act2, name='fc3', num_hidden=num_classes) + mlp = mx.symbol.SoftmaxOutput(data = fc3, name = 'softmax') + return mlp +``` + + + +Varible here is actually a Symbol. Every basic Symbol will correspond to one Node, and every Node has its own NodeAttr. There is a op field in NodeAttr class, when a Symbol represents Variable(often input data), the op field is null. + +Symbol contains a data member, std::vector outputs, and NodeEntry cantains a poniter to Node. We can follow the Node pointer to get all the Graph. + +And Symbol can be saved to a Json file. + +Here is a detailed example: + +``` +>>> import mxnet as mx +>>> data = mx.symbol.Variable('data') +>>> print data.debug_str() +Variable:data + +>>> data = mx.symbol.Flatten(data=data) +>>> print data.debug_str() +Symbol Outputs: + output[0]=flatten0(0) +Variable:data +-------------------- +Op:Flatten, Name=flatten0 +Inputs: + arg[0]=data(0) version=0 + +>>> fc1 = mx.symbol.FullyConnected(data = data, name='fc1', num_hidden=128) +>>> print fc1.debug_str() +Symbol Outputs: + output[0]=fc1(0) +Variable:data +-------------------- +Op:Flatten, Name=flatten0 +Inputs: + arg[0]=data(0) version=0 +Variable:fc1_weight +Variable:fc1_bias +-------------------- +Op:FullyConnected, Name=fc1 +Inputs: + arg[0]=flatten0(0) + arg[1]=fc1_weight(0) version=0 + arg[2]=fc1_bias(0) version=0 +Attrs: + num_hidden=128 + +``` + + +### TensorFlow + + +The core concept of symbolic API is `Tensor`. Tensorflow defines `Tensor` in Python. Please refer to the comments in TensorFlow: + +A `Tensor` is a symbolic handle to one of the outputs of an `Operation`. It does not hold the values of that operation's output, but instead provides a means of computing those values in a TensorFlow [Session](https://www.tensorflow.org/api_docs/python/tf/Session). + +A simple example is as follows: + +```python + # Build a dataflow graph. + c = tf.constant([[1.0, 2.0], [3.0, 4.0]]) + d = tf.constant([[1.0, 1.0], [0.0, 1.0]]) + e = tf.matmul(c, d) + + # Construct a `Session` to execute the graph. + sess = tf.Session() + + # Execute the graph and store the value that `e` represents in `result`. + result = sess.run(e) +``` + + +The main method of `Tensor` is as follows: + + +```python +@property +def op(self): + """The `Operation` that produces this tensor as an output.""" + return self._op + +@property +def dtype(self): + """The `DType` of elements in this tensor.""" + return self._dtype + +@property +def graph(self): + """The `Graph` that contains this tensor.""" + return self._op.graph + +@property +def name(self): + """The string name of this tensor.""" + if not self._op.name: + raise ValueError("Operation was not named: %s" % self._op) + return "%s:%d" % (self._op.name, self._value_index) + +@property +def device(self): + """The name of the device on which this tensor will be produced, or None.""" + return self._op.device +``` + + +Tensor can be taken as target to run by session. Tensor contains all the information of Graph, and tracks data dependency. + + +Here is a detailed example: + + +``` +>>> import tensorflow as tf +>>> c = tf.constant([[1.0, 2.0], [3.0, 4.0]]) +>>> print c.graph + +>>> d = tf.constant([[1.0, 1.0], [0.0, 1.0]]) +>>> print d.graph + +>>> e = tf.matmul(c, d) +>>> print e.graph + +``` + +### Dynet + + +The core concept of symbolic API is `Expression`, and Dynet defines `Expression` class in C++. + + +A simple example is as follows: + +```cpp +ComputationGraph cg; +Expression W = parameter(cg, pW); + +Expression in = input(cg, xs[i]); +Expression label = input(cg, ys[i]); +Expression pred = W * in; +Expression loss = square(pred - label); +``` + +The input data and parameter are also represented by Expression. Every basci Expression corresponds to a Node. And input data is also a Node. + +Expression has a data member ComputationGraph, and ComputationGraph will be modified in users' configuring process. Expression can be a running target, beacuse Expression contains all dependency. + + +Here is a detailed example: + +write topology in C++ + +``` +ComputationGraph cg; +Expression W = parameter(cg, pW); +cg.print_graphviz(); + +Expression pred = W * xs[i]; +cg.print_graphviz(); + +Expression loss = square(pred - ys[i]); +cg.print_graphviz(); +``` + +compile and print + +``` +# first print +digraph G { + rankdir=LR; + nodesep=.05; + N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"]; +} +# second print +digraph G { + rankdir=LR; + nodesep=.05; + N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"]; + N1 [label="v1 = v0 * -0.98"]; + N0 -> N1; +} +# third print +digraph G { + rankdir=LR; + nodesep=.05; + N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"]; + N1 [label="v1 = v0 * -0.98"]; + N0 -> N1; + N2 [label="v2 = -1.88387 - v1"]; + N1 -> N2; + N3 [label="v3 = -v2"]; + N2 -> N3; + N4 [label="v4 = square(v3)"]; + N3 -> N4; +} +``` + +### Conclusion + + +Actually, Symbol/Tensor/Expression in Mxnet/TensorFlow/Dynet are the same level concepts. We use a unified name Expression here, this level concept has following features: + +- Users wirte topoloy with symbolic API, and all return value is Expression, including input data and parameter. +- Expression corresponds with a global Graph, and Expression can also be composed. +- Expression tracks all dependency and can be taken as a run target diff --git a/doc/_sources/design/if_else_op.md.txt b/doc/_sources/design/if_else_op.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..26d140f06db4ecefa86be015eaa731ffddc6910c --- /dev/null +++ b/doc/_sources/design/if_else_op.md.txt @@ -0,0 +1,51 @@ +# The `IfElse` Operator + +PaddlePaddle's `IfElse` operator differs from TensorFlow's: + +- the TensorFlow version takes a scalar boolean value as the condition so that the whole mini-batch goes to either the true or the false branch, whereas +- the PaddlePaddle version takes a vector of boolean value as the condition, and instances corresponding to true values go to the true branch, those corresponding to false values go to the false branch. + +## Example + +The following PaddlePaddle program shows the usage of the IfElse operator: + +```python +import paddle as pd + +x = minibatch([10, 20, 30]) # shape=[None, 1] +y = var(1) # shape=[1], value=1 +z = minibatch([10, 20, 30]) # shape=[None, 1] +cond = larger_than(x, 15) # [false, true, true] + +ie = pd.ifelse() +with ie.true_block(): + d = pd.layer.add(x, y) + ie.output(d, pd.layer.softmax(d)) +with ie.false_block(): + d = pd.layer.fc(z) + ie.output(d, d+1) +o1, o2 = ie(cond) +``` + +A challenge to implement the `IfElse` operator is to infer those variables to be split, or, say, to identify the variable of the mini-batch or those derived from the mini-batch. + +An equivalent C++ program is as follows: + +```c++ +namespace pd = paddle; + +int x = 10; +int y = 1; +int z = 10; +bool cond = false; +int o1, o2; +if (cond) { + int d = x + y; + o1 = z; + o2 = pd::layer::softmax(z); +} else { + int d = pd::layer::fc(z); + o1 = d; + o2 = d+1; +} +``` diff --git a/doc/_sources/design/infer_var_type.md.txt b/doc/_sources/design/infer_var_type.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..d9d5397becba2ef1806d9341cd49cd9aabbf4a6a --- /dev/null +++ b/doc/_sources/design/infer_var_type.md.txt @@ -0,0 +1,78 @@ +# Design Doc: InferVarType + +## The Problem Posed + +The variable in our design can hold variant types. Such as `LoDTensor` and `SelectedRows`. An operator should be able to inference the variable types of its output. + +For example, a `lookup table` operator takes two `LoDTensor`; one is a float tensor as the embedding table, the other is an int tensor as word ID. The gradient operator of `lookup table` will generate a `SelectedRows` as its output. A `sum` operator can take both `LoDTensor` and `SelectedRows` as its inputs and will generate a `LoDTensor` if any of its inputs is `LoDTensor`, otherwise, the `sum` operator will generate `SelectedRows` as its output. + +The variable type will be constant at runtime. Every variable's type can either be set by the user (input data and parameter) or be inferred by the operator in compile time. + +## Proposed Solution + +The `InferVarType` is a compile-time function which is registered to each operator. The inferface of that function is: + + +```c++ +using InferVarTypeFN = std::function< + void (const OpDescBind& /*op_desc*/, BlockDescBind* /*block*/)>; +``` + +It takes an operator description as its input and will write the output variable type and store them in block description. + +The `InferVarTypeFN` will be registered in `OpInfo`, to replace `infer_var_type_` field. The `OpInfo` should be + +```cpp +struct OpInfo { + InferVarTypeFN infer_var_type_; + ... +}; +``` + +The default `InferVarType` will set output type as `LoDTensor`. It can be done by `GetInferVarType()`. + +```cpp +void DefaultInferVarType(const OpDescBind& op_desc, BlockDescBind* block) { + // set the output type of variable as `LoDTensor`. + // ... +} + +struct OpInfo { + InferVarTypeFN infer_var_type_; + InferVarTypeFN GetInferVarType() const { + if (infer_var_type_) { + return infer_var_type_; + } else { + return DefaultInferVarType; + } + } +}; +``` + +## Register InferVarType + +We provide a thin base class for registering an `InferVarTypeFN`. To use a base class will ease the implementation of registry since we can detect the registry entry is an `InferVarTypeFN` or not. + +```cpp +class VarTypeInferer { +public: + virtual void operator()(const OpDescBind& op_desc, BlockDescBind* block) const = 0; +} +``` + +Operator developers can write the specialize `VarTypeInferer` as follow. + +```cpp +class SpecialVarTypeInferer : public VarTypeInferer { +public: + virtual void operator()(const OpDescBind& op_desc, BlockDescBind* block) const { + // .. own logic + } +} +``` + +Then user can register the `InferVarType` just like `GradOpDescMaker` and `OpInfoMaker`. + +``` +REGISTER_OPERATOR(some_op, OpType, SpecialVarTypeInferer, ...); +``` diff --git a/doc/_sources/design/model_format.md.txt b/doc/_sources/design/model_format.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..e29129fddf775939c9f7a8b49d850d523e6e5a45 --- /dev/null +++ b/doc/_sources/design/model_format.md.txt @@ -0,0 +1,36 @@ +# Design Doc: Model Format + +## Motivation + +A model is an output of the training process. One complete model consists of two parts, the **topology** and the **parameters**. In order to support industrial deployment, the model format must be self-complete and must not expose any training source code. + +As a result, In PaddlePaddle, the **topology** is represented as a [ProgramDesc](https://github.com/PaddlePaddle/Paddle/blob/1c0a4c901c9fc881d120249c703b15d1c50dae7d/doc/design/program.md), which describes the model structure. The **parameters** contain all the trainable weights in the model. We must support large size parameters and efficient serialization/deserialization of parameters. + +## Implementation + +The topology is saved as a plain text in a detailed self-contain protobuf file. + +The parameters are saved as a binary file. As we all know, the protobuf message has a limit of [64M size](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.io.coded_stream#CodedInputStream.SetTotalBytesLimit.details). We have done a [benchmark experiment](https://github.com/PaddlePaddle/Paddle/pull/4610), which shows that protobuf is not fit for the task. + +As a result, we design a particular format for tensor serialization. By default, an arbitrary tensor in Paddle is a [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md), and has a description information proto of [LoDTensorDesc](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L99). We save the DescProto as the byte string header. It contains all the necessary information, such as the `dims`, and the `LoD` information in [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/1c0a4c901c9fc881d120249c703b15d1c50dae7d/paddle/framework/lod_tensor.md). A tensor stores values in a continuous memory buffer. For speed we dump the raw memory to disk and save it as the byte string content. So, the binary format of one tensor is, + +The table below shows a tensor's byte view in detail. Note that all the signed values are written in the little-endian format. + +|field name | type | description | +| --- | --- | --- | +| version | uint32_t | Version of saved file. Always 0 now. | +| tensor desc length | uint32_t | TensorDesc(Protobuf message) length in bytes. | +| tensor desc | void* | TensorDesc protobuf binary message | +| tensor data | void* | Tensor's data in binary format. The length of `tensor_data` is decided by `TensorDesc.dims()` and `TensorDesc.data_type()` | +| lod_level | uint64_t | Level of LoD | +| length of lod[0] | uint64_t | [Optional] length of lod[0] in bytes. | +| data of lod[0] | uint64_t* | [Optional] lod[0].data() | +| ... | ... | ... | + + + +## Summary + +- We introduce a model format. +- The model represented by its forward-pass computation procedure is saved in a **ProgramDesc** protobuf message. +- A bunch of specified format binary tensors describe the **parameters**. diff --git a/doc_cn/_sources/design/multi_language_interface/why_plain_c.md.txt b/doc/_sources/design/multi_language_interface/00.why_plain_c.md.txt similarity index 94% rename from doc_cn/_sources/design/multi_language_interface/why_plain_c.md.txt rename to doc/_sources/design/multi_language_interface/00.why_plain_c.md.txt index a3f41ca7b93de8a55d927c88812802ef12246182..a1443093342c5a3ed698fb6b52a751dfc7cb5319 100644 --- a/doc_cn/_sources/design/multi_language_interface/why_plain_c.md.txt +++ b/doc/_sources/design/multi_language_interface/00.why_plain_c.md.txt @@ -58,32 +58,32 @@ typedef void* paddle_matrix; typedef int paddle_error; extern "C" -paddle_error paddle_matrix_shape(paddle_matrix matrix, - uint64_t* width, - uint64_t* height); +paddle_error paddle_matrix_get_shape(paddle_matrix matrix, + uint64_t* width, + uint64_t* height); ``` 而在CPP里面实现这个C的接口，文件 `paddle_matrix.cpp` ```cpp -#include "paddle/math/matrix.hpp" +#include "paddle/math/matrix.h" extern "C" paddle_error paddle_matrix_shape(paddle_matrix matrix, uint64_t *width, uint64_t *height) { - auto m = (paddle::math::matrix*)(matrix); + auto m = (paddle::capi::CMatrix*)(matrix); *width = m->width(); *height = m->height(); } ``` -其中`paddle/math/matrix.hpp`文件内容为: +其中`paddle/capi/CMatrix.hpp`文件内容为: ```cpp namespace paddle { namespace math { -class Matrix { - //... +class CMatrix { + std::shared_ptr mat; }; } // namespace math @@ -113,6 +113,6 @@ class Matrix { | 手写多语言绑定 | 不使用SWIG | 使用SWIG需要多语言绑定的开发人员熟练掌握SWIG配置，社区参与困难。SWIG生成的代码不能保证多语言代码风格的一致性 | -## 简单实现 +## 实现 -TBD +参考[Inference implementation](01.inference_implementation.md) diff --git a/doc/_sources/design/multi_language_interface/01.inference_implementation.md.txt b/doc/_sources/design/multi_language_interface/01.inference_implementation.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..9820284523246a062581f322616d196f575c9d29 --- /dev/null +++ b/doc/_sources/design/multi_language_interface/01.inference_implementation.md.txt @@ -0,0 +1,131 @@ +# C-API 模型推断实现文档 + +本文档描述Paddle C-API的实现细节。Paddle C-API是多语言API的基础部分。Paddle需要暴露的API很多。先实现模型推断的API，通过模型推断API的实现作为一个样例，来进行讨论。至于为什么需要C-API，请参考[Why Plain C](./00.why_plain_c.md)。 + +## Table of Contents + * [C-API 模型推断实现文档](#c-api-模型推断实现文档) + * [暴露接口原则](#暴露接口原则) + * [目录结构](#目录结构) + * [实现方式](#实现方式) + * [capi.h](#capih) + * [具体某种类型的头文件](#具体某种类型的头文件) + * [capi_private.h](#capi_privateh) + * [具体某种类型的实现文件](#具体某种类型的实现文件) + * [libpaddle_capi_shared.{so, dylib}](#libpaddle_capi_sharedso-dylib) + * [libpaddle_capi_whole.a](#libpaddle_capi_wholea) + * [examples](#examples) + * [编译选项](#编译选项) + + +## 暴露接口原则 + +1. 所有的接口均为C接口。即使用`extern "C"` +2. 除构造某种类型的函数(`paddle_matrix_create`等)，其他函数均返回`paddle_error`。且调用时不能抛出异常或出现运行时错误。 +3. 所有类型名为`paddle_类型名`，所有与类型相关的函数，函数名为`paddle_类型名_函数名` +4. 如果某一个Paddle Core概念(GradientMachine/Matrix)需要被暴露到其他语言，那么 + * 为了暴露的接口尽量简单。只暴露概念的接口，而不暴露概念的实现。即暴露`GradientMachine`或者`Matrix`但不暴露`RecurrentGradientMachine`和`CpuSparseMatrix`。 + * 暴露这个概念必要函数。`必要`是指，即完成某一个任务的最少函数。 +5. 不在`capi`接口层做过多封装。 + * 如果某一个Paddle概念必须要暴露，但是又过于琐碎。不在`capi`这一层进行封装，而是直接修改Paddle Core。让Paddle核心中，这一概念不再琐碎。 + + +## 目录结构 + +```text +Paddle + `-- paddle + `-- capi + `-- examples # The example project for C-API. + `-- tests # unittests for C-API + `-- capi.h # C-API header file. + `-- capi_private.h # The shared header file between implementation sources. + `-- matrix.{h, cpp} + `-- gradient_machine.{h, cpp} + `-- ... +``` + + +Paddle的C-API目录结构如上图表所示。这个目录中除了`capi_private.h`之外的所有头文件，均会被安装到include/paddle路径下。C-API生成的二进制文件会被安装到`lib`目录下。即，安装后的目录结构为 + +```text +`-- include + `-- paddle + `-- capi.h + `-- matrix.h + `-- gradient_machine.h + `-- ... +`-- lib + `-- libpaddle_capi_shared.{so, dylib} # In mac, dynamic libary's file name extention is `dylib` + `-- libpaddle_capi_whole.a # static library for all symbols of Paddle. +``` + +## 实现方式 + +下面分别介绍某一类文件的实现方式。 + +### capi.h + +`capi.h`是用户使用C-API时所唯一需要引入的头文件。在`capi.h`中，引入了类型的头文件，`matrix.h`, `gradient_machine.h`。在引入其他类型的头文件时，使用相对路径的引用方式。即`#include "matrix.h"` + +### 具体某种类型的头文件 + +具体某种类型的头文件，即例如`matrix.h`，`gradient_machine.h`等。在这些头文件中，包含了某种类型的类型定义和暴露的全部函数。 + +这个头文件不假设其他文件的引用顺序，即使用户直接引用某种类型的头文件，也不应该报错(虽然不鼓励这样)。如果某一个类型需要引用另一个类型，例如`gradient_machine`需要引用`matrix`，则直接引入另一种类型的头文件，即`#include "matrix.h"`。 + +### capi_private.h + +`capi_prviate.h`是各个实现中共享的头文件，他主要包含了实际暴露的类型结构。在用户使用C-API时，Paddle的类型全部退化成`void *`，即`typedef paddle_matrix void*`。但，对于每种C-API暴露的类型，均是在`capi_private.h`中实现的结构体。 + +```cpp +struct CMatrix { + int type = MatrixType; + std::shared_ptr mat; +}; +``` + +通常，这个结构体包含两个项目。 + +* `type`是一个类型的标志。对于每种类型，type字段均不尽相同。这样，即使C-API接受的类型全是`void *`，我们也可以确定每一个参数的类型。 + + ```cpp + void some_c_api_function(void* some_instance) { + int* type = (int *) some_instance; + switch (*type) { + case MatrixType: + CMatrix* mat = (CMatrix *) some_instance; + ... + ... + } + } + ``` +* 这个结构体中的另一个项目是，Paddle Core中这一类型接口的智能指针(shared_ptr)。 + * 使用智能指针的原因是: 用户可以安全的释放某个C-API的实例，而不必在意Paddle Core是否还在使用这个实例。 + * 例如，用户通过C-API获得了神经网络的参数实例。当用户使用完这个参数后，直接删除这个参数即可。即便Paddle Core中的模型还在使用这个参数，这个参数也不会一并删除。 + +### 具体某种类型的实现文件 + +具体某种类型的实现文件，即`matrix.cpp`, `gradient_machine.cpp`等文件。在这些文件中，使用C++ 11实现了C-API的接口，并且使用`extern "C"`导出这些接口。在实现过程中，对输入参数的安全性进行了必要的判断，并将C-API接口的参数转发给`Paddle Core`。 + +### libpaddle\_capi_shared.{so, dylib} + +`libpaddle_capi_shared`是C-API导出的动态库。这个动态库的连接参数与Paddle的其他二进制(例如`paddle_trainer`)类似。用户可以直接使用这个动态库来引入Paddle C-API。具体使用方法为`-lpaddle_capi_shared`。 + +### libpaddle\_capi_whole.a + +`libpaddle_capi_whole`是C-API导出的静态库。这个静态库包含了Paddle的全部符号。他是将`libpaddle_gserver.a`, `libpaddle_math.a`, `libpaddle_capi.a`等全部静态库中的目标文件全部打包后产生的文件。具体使用方法为`--whole-archive -lpaddle_capi_whole --no-whole-archive`。 + + +### examples + +在样例中，使用`C99`开发了模型预测的样例代码。具体请参考[example/README.md](../../../paddle/capi/examples/README.md)。 + +## 编译选项 + +C-API的编译选项默认关闭，打开这个编译选项，需要在cmake的时候，设置 + +```bash +cmake ${YOUR_SOURCE_ROOT} -DWITH_C_API=ON -DWITH_PYTHON=OFF -DWITH_SWIG_PY=OFF +``` + +编译C-API的时候推荐Paddle不嵌入Python解释器，也不生成`SWIG`接口，具体原因参考[Why Plain C](./00.why_plain_c.md)。 diff --git a/doc/_sources/design/ops/rnn.md.txt b/doc/_sources/design/ops/rnn.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..2f4854793fa1f0b02e4dc17b51a48a972be61c06 --- /dev/null +++ b/doc/_sources/design/ops/rnn.md.txt @@ -0,0 +1,153 @@ +# RNNOp design + +This document describes the RNN (Recurrent Neural Network) operator and how it is implemented in PaddlePaddle. The RNN op requires that all instances in a mini-batch have the same length. We will have a more flexible dynamic RNN operator in the future. + +## RNN Algorithm Implementation + +

+ +

+ +The above diagram shows an RNN unrolled into a full network. + +There are several important concepts here: + +- *step-net*: the sub-graph that runs at each step. +- *memory*, $h_t$, the state of the current step. +- *ex-memory*, $h_{t-1}$, the state of the previous step. +- *initial memory value*, the memory of the first (initial) step. + +### Step-scope + +There could be local variables defined in each step-net. PaddlePaddle runtime realizes these variables in *step-scopes* which are created for each step. + +

+
+Figure 2 illustrates the RNN's data flow +

+ +Please be aware that every step runs the same step-net. Each step does the following: + +1. Creates the step-scope. +2. Initializes the local variables including step-outputs, in the step-scope. +3. Runs the step-net, which uses the above mentioned variables. + +The RNN operator will compose its output from step outputs in each of the step scopes. + +### Memory and Ex-memory + +Let's give more details about memory and ex-memory using a simple example: + +$$ +h_t = U h_{t-1} + W x_t +$$, + +where $h_t$ and $h_{t-1}$ are the memory and ex-memory (previous memory) of step $t$ respectively. + +In the implementation, we can make an ex-memory variable either "refer to" the memory variable of the previous step, +or copy the memory value of the previous step to the current ex-memory variable. + +### Usage in Python + +For more information on Block, please refer to the [design doc](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/block.md). + +We can define an RNN's step-net using a Block: + +```python +import paddle as pd + +X = some_op() # x is some operator's output and is a LoDTensor +a = some_op() + +# declare parameters +W = pd.Variable(shape=[20, 30]) +U = pd.Variable(shape=[20, 30]) + +rnn = pd.create_rnn_op(output_num=1) +with rnn.stepnet(): + x = rnn.add_input(X) + # declare a memory (rnn's step) + h = rnn.add_memory(init=a) + # h.pre_state(), the previous memory of rnn + new_state = pd.add_two( pd.matmul(W, x) + pd.matmul(U, h.pre_state())) + # update current memory + h.update(new_state) + # indicate that h variables in all step scopes should be merged + rnn.add_outputs(h) + +out = rnn() +``` + +Python API functions in above example: + +- `rnn.add_input`: indicates that the parameter is a variable that will be segmented into step-inputs. +- `rnn.add_memory`: creates a variable used as the memory. +- `rnn.add_outputs`: marks the variables that will be concatenated across steps into the RNN output. + +### Nested RNN and LoDTensor + +An RNN whose step-net includes other RNN operators is known as an *nested RNN*. + +For example, we could have a 2-level RNN, where the top level corresponds to paragraphs, and the lower level corresponds to sentences. Each step of the higher level RNN also receives an input from the corresponding step of the lower level, and additionally the output from the previous time step at the same level. + +The following figure illustrates feeding in text into the lower level, one sentence at a step, and the feeding in step outputs to the top level. The final top level output is about the whole text. + +

+ +

+ +```python +import paddle as pd + +W = pd.Variable(shape=[20, 30]) +U = pd.Variable(shape=[20, 30]) + +W0 = pd.Variable(shape=[20, 30]) +U0 = pd.Variable(shape=[20, 30]) + +# a is output of some op +a = some_op() + +# chapter_data is a set of 128-dim word vectors +# the first level of LoD is sentence +# the second level of LoD is a chapter +chapter_data = pd.Variable(shape=[None, 128], type=pd.lod_tensor, level=2) + +def lower_level_rnn(paragraph): + ''' + x: the input + ''' + rnn = pd.create_rnn_op(output_num=1) + with rnn.stepnet(): + sentence = rnn.add_input(paragraph, level=0) + h = rnn.add_memory(shape=[20, 30]) + h.update( + pd.matmul(W, sentence) + pd.matmul(U, h.pre_state())) + # get the last state as sentence's info + rnn.add_outputs(h) + return rnn + +top_level_rnn = pd.create_rnn_op(output_num=1) +with top_level_rnn.stepnet(): + paragraph_data = rnn.add_input(chapter_data, level=1) + low_rnn = lower_level_rnn(paragraph_data) + paragraph_out = low_rnn() + + h = rnn.add_memory(init=a) + h.update( + pd.matmul(W0, paragraph_data) + pd.matmul(U0, h.pre_state())) + top_level_rnn.add_outputs(h) + +# output the last step +chapter_out = top_level_rnn(output_all_steps=False) +``` + +In the above example, the construction of the `top_level_rnn` calls `lower_level_rnn`. The input is an LoD Tensor. The top level RNN segments input text data into paragraphs, and the lower level RNN segments each paragraph into sentences. + +By default, the `RNNOp` will concatenate the outputs from all the time steps. +If the `output_all_steps` is set to False, it will only output the final time step. + + +

+ +

diff --git a/doc/_sources/design/ops/sequence_decoder.md.txt b/doc/_sources/design/ops/sequence_decoder.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..9db5fb8e9a9f89b004bf71ddc064cd976c0d0bee --- /dev/null +++ b/doc/_sources/design/ops/sequence_decoder.md.txt @@ -0,0 +1,229 @@ +# Design: Sequence Decoder Generating LoDTensors +In tasks such as machine translation and visual captioning, +a [sequence decoder](https://github.com/PaddlePaddle/book/blob/develop/08.machine_translation/README.md) is necessary to generate sequences, one word at a time. + +This documentation describes how to implement the sequence decoder as an operator. + +## Beam Search based Decoder +The [beam search algorithm](https://en.wikipedia.org/wiki/Beam_search) is necessary when generating sequences. It is a heuristic search algorithm that explores the paths by expanding the most promising node in a limited set. + +In the old version of PaddlePaddle, the C++ class `RecurrentGradientMachine` implements the general sequence decoder based on beam search, due to the complexity involved, the implementation relies on a lot of special data structures that are quite trivial and hard to be customized by users. + +There are a lot of heuristic tricks in the sequence generation tasks, so the flexibility of sequence decoder is very important to users. + +During the refactoring of PaddlePaddle, some new concepts are proposed such as: [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md) and [TensorArray](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/tensor_array.md) that can better support the sequence usage, and they can also help make the implementation of beam search based sequence decoder **more transparent and modular** . + +For example, the RNN states, candidates IDs and probabilities of beam search can be represented all as `LoDTensors`; +the selected candidate's IDs in each time step can be stored in a `TensorArray`, and `Packed` to the sentences translated. + +## Changing LoD's absolute offset to relative offsets +The current `LoDTensor` is designed to store levels of variable-length sequences. It stores several arrays of integers where each represents a level. + +The integers in each level represent the begin and end (not inclusive) offset of a sequence **in the underlying tensor**, +let's call this format the **absolute-offset LoD** for clarity. + +The relative-offset LoD can retrieve any sequence very quickly but fails to represent empty sequences, for example, a two-level LoD is as follows +```python +[[0, 3, 9] + [0, 2, 3, 3, 3, 9]] +``` +The first level tells that there are two sequences: +- the first's offset is `[0, 3)` +- the second's offset is `[3, 9)` + +while on the second level, there are several empty sequences that both begin and end at `3`. +It is impossible to tell how many empty second-level sequences exist in the first-level sequences. + +There are many scenarios that rely on empty sequence representation, for example in machine translation or visual captioning, one instance has no translation or the empty candidate set for a prefix. + +So let's introduce another format of LoD, +it stores **the offsets of the lower level sequences** and is called **relative-offset** LoD. + +For example, to represent the same sequences of the above data + +```python +[[0, 3, 6] + [0, 2, 3, 3, 3, 9]] +``` + +the first level represents that there are two sequences, +their offsets in the second-level LoD is `[0, 3)` and `[3, 5)`. + +The second level is the same with the relative offset example because the lower level is a tensor. +It is easy to find out the second sequence in the first-level LoD has two empty sequences. + +The following examples are based on relative-offset LoD. + +## Usage in a simple machine translation model +Let's start from a simple machine translation model that is simplified from the [machine translation chapter](https://github.com/PaddlePaddle/book/tree/develop/08.machine_translation) to draw a blueprint of what a sequence decoder can do and how to use it. + +The model has an encoder that learns the semantic vector from a sequence, and a decoder which uses the sequence encoder to generate new sentences. + +**Encoder** +```python +import paddle as pd + +dict_size = 8000 +source_dict_size = dict_size +target_dict_size = dict_size +word_vector_dim = 128 +encoder_dim = 128 +decoder_dim = 128 +beam_size = 5 +max_length = 120 + +# encoder +src_word_id = pd.data( + name='source_language_word', + type=pd.data.integer_value_sequence(source_dict_dim)) +src_embedding = pd.embedding(size=source_dict_size, size=word_vector_dim) + +src_word_vec = pd.lookup(src_embedding, src_word_id) + +encoder_out_seq = pd.gru(input=src_word_vec, size=encoder_dim) + +encoder_ctx = pd.last_seq(encoder_out_seq) +# encoder_ctx_proj is the learned semantic vector +encoder_ctx_proj = pd.fc( + encoder_ctx, size=decoder_dim, act=pd.activation.Tanh(), bias=None) +``` + +**Decoder** + +```python +def generate(): + decoder = pd.while_loop() + with decoder.step(): + decoder_mem = decoder.memory(init=encoder_ctx) # mark the memory + generated_ids = decoder.memory() # TODO init to batch_size s + generated_scores = decoder.memory() # TODO init to batch_size 1s or 0s + + target_word = pd.lookup(trg_embedding, gendrated_ids) + # expand encoder_ctx's batch to fit target_word's lod + # for example + # decoder_mem.lod is + # [[0 1 3], + # [0 1 3 6]] + # its tensor content is [a1 a2 a3 a4 a5] + # which means there are 2 sentences to translate + # - the first sentence has 1 translation prefixes, the offsets are [0, 1) + # - the second sentence has 2 translation prefixes, the offsets are [1, 3) and [3, 6) + # the target_word.lod is + # [[0, 1, 6] + # [0, 2, 4, 7, 9 12]] + # which means 2 sentences to translate, each has 1 and 5 prefixes + # the first prefix has 2 candidates + # the following has 2, 3, 2, 3 candidates + # the encoder_ctx_expanded's content will be + # [a1 a1 a2 a2 a3 a3 a3 a4 a4 a5 a5 a5] + encoder_ctx_expanded = pd.lod_expand(encoder_ctx, target_word) + decoder_input = pd.fc( + act=pd.activation.Linear(), + input=[target_word, encoder_ctx], + size=3 * decoder_dim) + gru_out, cur_mem = pd.gru_step( + decoder_input, mem=decoder_mem, size=decoder_dim) + scores = pd.fc( + gru_out, + size=trg_dic_size, + bias=None, + act=pd.activation.Softmax()) + # K is an config + topk_scores, topk_ids = pd.top_k(scores, K) + topk_generated_scores = pd.add_scalar(topk_scores, generated_scores) + + selected_ids, selected_generation_scores = decoder.beam_search( + topk_ids, topk_generated_scores) + + # update the states + decoder_mem.update(cur_mem) # tells how to update state + generated_ids.update(selected_ids) + generated_scores.update(selected_generation_scores) + + decoder.output(selected_ids) + decoder.output(selected_generation_scores) + +translation_ids, translation_scores = decoder() +``` +The `decoder.beam_search` is an operator that, given the candidates and the scores of translations including the candidates, +returns the result of the beam search algorithm. + +In this way, users can customize anything on the input or output of beam search, for example: + +1. Make the corresponding elements in `topk_generated_scores` zero or some small values, beam_search will discard this candidate. +2. Remove some specific candidate in `selected_ids`. +3. Get the final `translation_ids`, remove the translation sequence in it. + +The implementation of sequence decoder can reuse the C++ class: [RNNAlgorithm](https://github.com/Superjom/Paddle/blob/68cac3c0f8451fe62a4cdf156747d6dc0ee000b3/paddle/operators/dynamic_recurrent_op.h#L30), +so the python syntax is quite similar to that of an [RNN](https://github.com/Superjom/Paddle/blob/68cac3c0f8451fe62a4cdf156747d6dc0ee000b3/doc/design/block.md#blocks-with-for-and-rnnop). + +Both of them are two-level `LoDTensors`: + +- The first level represents `batch_size` of (source) sentences. +- The second level represents the candidate ID sets for translation prefix. + +For example, 3 source sentences to translate, and has 2, 3, 1 candidates. + +Unlike an RNN, in sequence decoder, the previous state and the current state have different LoD and shape, and an `lod_expand` operator is used to expand the LoD of the previous state to fit the current state. + +For example, the previous state: + +* LoD is `[0, 1, 3][0, 2, 5, 6]` +* content of tensor is `a1 a2 b1 b2 b3 c1` + +the current state is stored in `encoder_ctx_expanded`: + +* LoD is `[0, 2, 7][0 3 5 8 9 11 11]` +* the content is + - a1 a1 a1 (a1 has 3 candidates, so the state should be copied 3 times for each candidates) + - a2 a2 + - b1 b1 b1 + - b2 + - b3 b3 + - None (c1 has 0 candidates, so c1 is dropped) + +The benefit from the relative offset LoD is that the empty candidate set can be represented naturally. + +The status in each time step can be stored in `TensorArray`, and `Pack`ed to a final LoDTensor. The corresponding syntax is: + +```python +decoder.output(selected_ids) +decoder.output(selected_generation_scores) +``` + +The `selected_ids` are the candidate ids for the prefixes, and will be `Packed` by `TensorArray` to a two-level `LoDTensor`, where the first level represents the source sequences and the second level represents generated sequences. + +Packing the `selected_scores` will get a `LoDTensor` that stores scores of each translation candidate. + +Packing the `selected_generation_scores` will get a `LoDTensor`, and each tail is the probability of the translation. + +## LoD and shape changes during decoding +

+ +

+ +According to the image above, the only phase that changes the LoD is beam search. + +## Beam search design +The beam search algorithm will be implemented as one method of the sequence decoder and has 3 inputs: + +1. `topk_ids`, the top K candidate ids for each prefix. +2. `topk_scores`, the corresponding scores for `topk_ids` +3. `generated_scores`, the score of the prefixes. + +All of these are LoDTensors, so that the sequence affiliation is clear. Beam search will keep a beam for each prefix and select a smaller candidate set for each prefix. + +It will return three variables: + +1. `selected_ids`, the final candidate beam search function selected for the next step. +2. `selected_scores`, the scores for the candidates. +3. `generated_scores`, the updated scores for each prefix (with the new candidates appended). + +## Introducing the LoD-based `Pack` and `Unpack` methods in `TensorArray` +The `selected_ids`, `selected_scores` and `generated_scores` are LoDTensors that exist at each time step, +so it is natural to store them in arrays. + +Currently, PaddlePaddle has a module called `TensorArray` which can store an array of tensors. It is better to store the results of beam search in a `TensorArray`. + +The `Pack` and `UnPack` in `TensorArray` are used to pack tensors in the array to an `LoDTensor` or split the `LoDTensor` to an array of tensors. +It needs some extensions to support the packing or unpacking an array of `LoDTensors`. diff --git a/doc/_sources/design/optimizer.md.txt b/doc/_sources/design/optimizer.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..202b4b65103c0b7c536a9cb466c4120ce134d8c3 --- /dev/null +++ b/doc/_sources/design/optimizer.md.txt @@ -0,0 +1,91 @@ +## Optimizer Design + +### The Problem + +A PaddlePaddle program, or a block, is a sequence of operators operating variables. A training program needs to do three kinds of works: + +1. the forward pass, which computes intermediate results and the cost(s), +1. the backward pass, which derives gradients from intermediate results and costs, and +1. the optimization pass, which update model parameters to optimize the cost(s). + +These works rely on three kinds of operators: + +1. forward operators, +1. gradient operators, and +1. optimization operators. + +It's true that users should be able to create all these operators manually by calling some low-level API, but it would be much more convenient if they could only describe the forward pass and let PaddlePaddle create the backward and optimization operators automatically. + +In this design, we propose a high-level API that automatically derives the optimisation pass and operators from the forward pass. + + +### High-level Python API to describe the training process + +1. User write code to describe the network: + + ```python + images = layer.data("images") + labels = layer.data("labels") + w1 = pd.var("w1") + b1 = pd.var("b1") + hidden = layer.fc(images, w=w1, b=b1) + cost = layer.mse(hidden, labels) + ``` + + The above code snippet will create forward operators in [Block](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/block.md). + + +2. Users create a certain kind of Optimizer with some argument. + + ```python + optimizer = AdagradOptimizer(learing_rate=0.001) + ``` + +3. Users use the optimizer to `minimize` a certain `cost` through updating parameters in parameter_list. + + ```python + opt_op_list = optimizer.minimize(cost, parameter_list=[w1, b1]) + ``` + The above code snippet will create gradient and optimization operators in Block. The return value of `minimize()` is list of optimization operators that will be run by session. + +4. Users use Session/Executor to run this opt_op_list as target to do training. + + ```python + sess.run(target= opt_op_list, ...) + ``` + +#### Optimizer Python interface: + +```python +class Optimizer(object): + """Optimizer Base class. + + """ + + def __init__(self): + pass + + def create_optimization_pass(self, parameters_and_grads): + """Add optimization operators to update gradients to variables. + + Args: + parameters_and_grads: a list of (variable, gradient) pair to update. + + Returns: + optmization_op_list: a list of optimization operator that will update parameter using gradient. + """ + return None + + def minimize(self, loss, parameter_list): + """Add operations to minimize `loss` by updating `parameter_list`. + + This method combines interface `append_backward_ops()` and + `create_optimization_pass()` into one. + """ + params_grads = self.create_backward_pass(loss, parameter_list) + update_ops = self.create_optimization_pass(params_grads) + return update_ops + +``` + +Users can inherit the Optimizer above to create their own Optimizer with some special logic, such as AdagradOptimizer. diff --git a/doc/_sources/design/parameter_average.md.txt b/doc/_sources/design/parameter_average.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..2c4edee9fe31d502ea62b9fe5c8757c0a4c5e79f --- /dev/null +++ b/doc/_sources/design/parameter_average.md.txt @@ -0,0 +1,72 @@ +# Averaging Parameter in PaddlePaddle + +## Why Averaging +In a large scale machine learning setup where the size of the training data is huge, it could take us a large number of iterations over the training data before we can achieve the optimal values of parameters of our model. Looking at the problem setup, it is desirable if we can obtain the optimal values of parameters by going through the data in as few passes as we can. + +Polyak and Juditsky (1992) showed that the test performance of simple average of parameters obtained by Stochastic Gradient Descent (SGD) is as good as that of parameter values that are obtained by training the model over and over again, over the training dataset. + +Hence, to accelerate the speed of Stochastic Gradient Descent, Averaged Stochastic Gradient Descent (ASGD) was proposed in Polyak and Juditsky (1992). For ASGD, the running average of parameters obtained by SGD, is used as the estimator for
. The averaging is done as follows: + +
+ +We propose averaging for any optimizer similar to how ASGD performs it, as mentioned above. + +### How to perform Parameter Averaging in PaddlePaddle + +Parameter Averaging in PaddlePaddle works in the following way during training : +1. It will take in an instance of a normal optimizer as an input, e.g. RMSPropOptimizer +2. The optimizer itself is responsible for updating the parameters. +3. The ParameterAverageOptimizer maintains a separate copy of the parameters for itself: + 1. In concept, the values of this copy are the average of the values of the parameters in the most recent N batches. + 2. However, saving all the N instances of the parameters in memory is not feasible. + 3. Therefore, an approximation algorithm is used. + +Hence, overall we have have two copies of the parameters: one for the optimizer itself, and one for the ParameterAverageOptimizer. The former should be used in back propagation, while the latter should be used during testing and should be saved. + +During the testing/ saving the model phase, we perform the following steps: +1. Perform the delayed operations. +2. Save current values of the parameters to a temporary variable. +3. Replace the values of the parameters with the averaged values. +4. Perform testing and/or save the parameters. +5. Restore the values of the parameters once done. + +### How to implement Averaging of Parameter in PaddlePaddle + +We can add the ParameterAverageOptimizer op to the graph through Python API. Using this approach, we manually add this op to the graph and direct the output of the optimizer op to this op during training. + + **Advantages**: + - Allows for greater flexibility to the users of PaddlePaddle. Using this approach, the users can plug different optimizers into ParameterAverageOptimizer by passing in the optimizer to the op. + - Makes it easy for the users to customize and extend the framework. + + **Disadvantages**: + - Implementation requires re-writing the averaging methodology in Python. + +### Low-Level implementation + +In the new design, we propose to create a new operation for averaging parameter updates (ParameterAverageOptimizer). For now, we can add an op that takes in the following as input: +- the optimizer +- the window_size to keep the updates + +The ParameterAverageOptimizer op can be like any other operator with its own CPU/GPU implementation either using Eigen or separate CPU and GPU kernels. As the initial implementation, we can implement the kernel using Eigen following the abstraction pattern implemented for [Operators](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/rmsprop_op.h). We also want to support the case when the Trainer/Optimizer runs on the GPU while ParameterAverageOptimizer runs on a CPU. + +The idea of building an op for averaging is in sync with the refactored PaddlePaddle philosophy of using operators to represent any computation unit. The way the op will be added to the computation graph will be decided by the [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md#layer-function) in Python API. + +### Python API implementation for ParameterAverageOptimizer + +Based on Polyak and Juditsky (1992), we can generalize the averaging of updates to any optimizer. The input to the op would be the following: +- Any optimizer (RMSProp , AdaGrad etc.) +- A window size. The op keeps accumulating updated parameter values over a window of N batches and takes an average. Move the averaged value to a buffer when window is full to avoid loss of precision. + +Using the ParameterAverageOptimizer op, any user can add the operation to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support averaging. As per the PaddlePaddle [Python API design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md), the layer functions are responsible for creating operators, operator parameters and variables. Since ParameterAverageOptimizer will be an operator, it makes sense to create it in the layer functions. +We will have a wrapper written in Python that will support the functionality and implement the actual core computation in C++ core as we have done for other [Optimizers](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/rmsprop_op.cc) + +#### Creation of the ParameterAverageOptimizer operator +There are two ways for creating the ParameterAverageOptimizer op: +1. We create the op immediately while building the computation graph. +2. We add the op in a lazy manner, just before the backward pass, similar to the way the optimization ops are added. + +The proposal is to add the op immediately while building the computation graph. + +#### High-level API + +In PaddlePaddle Python API, users will primarily rely on [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md#layer-function) to create neural network layers. Hence, we also need to provide parameter average functionality in layer functions. diff --git a/doc/_sources/design/parameters_in_cpp.md.txt b/doc/_sources/design/parameters_in_cpp.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..a7ac3f17c44ca94a669a8f1e283b291bceb42317 --- /dev/null +++ b/doc/_sources/design/parameters_in_cpp.md.txt @@ -0,0 +1,41 @@ +# Design Doc: The C++ Class `Parameters` + +`Parameters` is a concept we designed in PaddlePaddle V2 API. `Parameters` is a container of parameters, which makes PaddlePaddle capable of sharing parameter between topologies. We described usages of `Parameter` in [api.md](./api.md). + +We used Python to implement Parameters when designing V2 API before. There are several defects for the current implementation: +* We just use `memcpy` to share Parameters between topologies, but this is very inefficient. +* We did not support sharing Parameters while training. We just trigger `memcpy` when start training. + +It is necessary that we implement Parameters in CPP side. However, it could result a code refactoring for PaddlePaddle, because PaddlePaddle was designed for training only one topology before, i.e., each GradientMachine contains its Parameter as a data member. In current PaddlePaddle implementation, there are three concepts associated with `Parameters`: + +1. `paddle::Parameter`. A `Parameters` is a container for `paddle::Parameter`. +It is evident that we should use `paddle::Parameter` when developing `Parameters`. +However, the `Parameter` class contains many functions and does not have a clear interface. +It contains `create/store Parameter`, `serialize/deserialize`, `optimize(i.e SGD)`, `randomize/zero`. +When we developing `Parameters`, we only use `create/store Parameter` functionality. +We should extract functionalities of Parameter into many classes to clean PaddlePaddle CPP implementation. + +2. `paddle::GradientMachine` and its sub-classes, e.g., `paddle::MultiGradientMachine`, `paddle::NeuralNetwork`. +We should pass `Parameters` to `paddle::GradientMachine` when `forward/backward` to avoid `memcpy` between topologies. +Also, we should handle multi-GPU/CPU training, because `forward` and `backward` would perform on multi-GPUs and multi-CPUs. +`Parameters` should dispatch the parameter value to each device, and gather the parameter gradient from each device. + +3. `paddle::ParameterUpdater`. The ParameterUpdater is used to update parameters in Paddle. +So `Parameters` should be used by `paddle::ParameterUpdater`, and `paddle::ParameterUpdater` should optimize `Parameters` (by SGD). + + +The step by step approach for implementation Parameters in PaddlePaddle C++ core is listed below. Each step should be a PR and could be merged into PaddlePaddle one by one. + +1. Clean `paddle::Parameter` interface. Extract the functionalities of `paddle::Parameter` to prepare for the implementation of Parameters. + +2. Implementation a `Parameters` class. It just stores the `paddle::Parameter` inside. Make `GradientMachine` uses `Parameters` as a class member. + +3. Make `Parameters` support Multi-CPU and Multi-GPU training to prepare for sharing `Parameter` between topologies. +Because we need share `Parameters` between topologies, it is `Parameters`'s response to exchange Parameters between GPUs. +`GradientMachine` should not handle how to exchange Parameters because `GradientMachine` only used to train one topology and we need to support train many topologies in Paddle, i.e., there could be many GradientMachines use one `Parameters`. + * We should use a global function to exchange Parameters between GPUs, not a member function in `Parameters`. The `MultiGradientMachine` invoke this function, which uses `Parameters` as this function inputs. + * The MultiGradientMachine contains many functionalities. Extracting the Parameters exchanging logic could make MultiGradientMachine clearer and simpler. + +4. Make `Parameters` as an argument for `forward/backward` function, not a data member for `GradientMachine`. For example, `forward` could be `forward(const Parameters& params, ...)` and `backward` could be `backward(Parameters* params, ...)`. After this step, Paddle could share `Parameters` between topologies. + +5. `ParameterUpdater` is invoked by `GradientMachine` and `Trainer`, but it updates `Parameters`. In the end of this code refactoring, we could change `ParameterUpdater` directly uses `Parameters` to make `ParameterUpdater`'s implementation clear. diff --git a/doc/_sources/design/program.md.txt b/doc/_sources/design/program.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..bd2456787c4e336d357a65255a8274a7c9e465cc --- /dev/null +++ b/doc/_sources/design/program.md.txt @@ -0,0 +1,139 @@ +# Design Doc: PaddlePaddle Programs + +## Compile and Execution + +A PaddlePaddle program consists of two parts -- the first generates a `ProgramDesc` protobuf message that describes the program, and the second runs this message using a C++ class `Executor`. + +A simple example PaddlePaddle program can be found in [graph.md](./graph.md): + +```python +x = layer.data("images") +l = layer.data("label") +y = layer.fc(x) +cost = layer.mse(y, l) +optimize(cost) +train(cost, reader=mnist.train()) +``` + +The first five lines of the following PaddlePaddle program generates, or, compiles, the `ProgramDesc` message. The last line runs it. + +## Programs and Blocks + +The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program. + +- program: some nested blocks +- [block](./block.md): + - some local variable definitions, and + - a sequence of operators + +The concept of block comes from usual programs. For example, the following C++ program has three blocks: + +```c++ +int main() { // block 0 + int i = 0; + if (i < 10) { // block 1 + for (int j = 0; j < 10; j++) { // block 2 + } + } + return 0; +} +``` + +The following PaddlePaddle program has three blocks: + +```python +import paddle as pd // block 0 + +x = minibatch([10, 20, 30]) # shape=[None, 1] +y = var(1) # shape=[1], value=1 +z = minibatch([10, 20, 30]) # shape=[None, 1] +cond = larger_than(x, 15) # [false, true, true] + +ie = pd.ifelse() +with ie.true_block(): // block 1 + d = pd.layer.add_scalar(x, y) + ie.output(d, pd.layer.softmax(d)) +with ie.false_block(): // block 2 + d = pd.layer.fc(z) + ie.output(d, d+1) +o1, o2 = ie(cond) +``` + +## `BlockDesc` and `ProgramDesc` + +All protobuf messages are defined in `framework.proto`. + +`BlockDesc` is straight-forward -- it includes local variable definitions, `vars`, and a sequence of operators, `ops`. + +```protobuf +message BlockDesc { + required int32 parent = 1; + repeated VarDesc vars = 2; + repeated OpDesc ops = 3; +} +``` + +The parent ID indicates the parent block so that operators in a block can refer to variables defined locally and also those defined in their ancestor blocks. + +All hierarchical blocks in a program are flattened and stored in an array. The block ID is the index of the block in this array. + +```protobuf +message ProgramDesc { + repeated BlockDesc blocks = 1; +} +``` + + +### Global Block + +The global block is the first one in the above array. + +## Operators that Use Blocks + +In the above example, the operator `IfElseOp` has two blocks -- the true branch and the false branch. + +The definition of `OpDesc` shows that an operator could have some attributes: + +```protobuf +message OpDesc { + AttrDesc attrs = 1; + ... +} +``` + +and an attribute could be of type block, which is, in fact, a block ID as described above: + +``` +message AttrDesc { + required string name = 1; + + enum AttrType { + INT = 1, + STRING = 2, + ... + BLOCK = ... + } + required AttrType type = 2; + + optional int32 block = 10; // when type == BLOCK + ... +} +``` + +## InferShape + +With this design, the InferShape function should take the following parameters: + +```c++ +void InferShape(int current_block, + int current_operator, + ProgramDesc* program // might change VarDesc values. + ) { + ... +} +``` + +where + +- `current_block` indices into `ProgramDesc::blocks`, +- `current_operator` indices into `BlockDesc::ops`. diff --git a/doc/_sources/design/prune.md.txt b/doc/_sources/design/prune.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..4a5cf10c79a554779137f0cce5494fdd96ef6b7a --- /dev/null +++ b/doc/_sources/design/prune.md.txt @@ -0,0 +1,63 @@ +# Prune + +## Motivation + +We want to support running inference, training and checkpointing in one `ProgramDesc`. We implement +`void Prune(const ProgramDesc* input, ProgramDesc* output)` function, which takes a `ProgramDesc` +and generate a pruned `ProgramDesc`. + +## Challenge + +Pruning need to support both variables and operators being evaluation targets. Consider the following +different situations. + +```python +# Case 1: run foward pass. +cost_np = session.run(target=cost) +# Case 2: run backward passing. +opts_np, _ = session.run(target=[cost, opt]) +# Case 3: run checkpointing +_ = session.run(target=checkpoint) +``` + +## Solution + +To support evaluation of operators, we add `is_target` field in the `OpDesc`. + +```c++ +message OpDesc { + required string type = 3; + repeated Var inputs = 1; + repeated Var outputs = 2; + repeated Attr attrs = 4; + optional bool is_target = 5 [ default = false ]; +}; +``` + +To support evaluation of variables, we add [fetch_op](https://github.com/PaddlePaddle/Paddle/pull/4599). +For each variable in the `target`, we insert a `fetch_op` into the `ProgramDesc` with `variable` being +`fetch_op`'s input. Then we also set `fetch_op` is a target. + +### Algorithm + +If an operator needs to be run, it must fall into one of the following cases: + +1. It is the target. +2. It is depended by some other ops, meaning its output is some other op's input. + +The first case can be checked by `op_desc.is_traget()` . The second case can be implement as + +```c++ +bool HasDependentVar(const OpDesc& op_desc, const std::set& dependent_vars) { + for (auto& var : op_desc.outputs()) { + for (auto& argu : var.arguments()) { + if (dependent_vars.count(argu) != 0) { + return true; + } + } + } + return false; +} +``` + +Then the whole algorithm can be implemented as the following [code](https://github.com/tonyyang-svail/Paddle/blob/prune_impl/paddle/framework/prune.cc). diff --git a/doc/_sources/design/python_api.md.txt b/doc/_sources/design/python_api.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..cb5fdc765b7126fc66a1c8978d4b96c0dc5a9f2c --- /dev/null +++ b/doc/_sources/design/python_api.md.txt @@ -0,0 +1,284 @@ +# Design Doc: Python API + +Due to the refactorization of the PaddlePaddle core, we need Python classes to construct corresponding protobuf messages that describe a DL program. + +| Python classes | Protobuf messages | +| --- | --- | +| Program | ProgramDesc | +| Block | BlockDesc | +| Operator | OpDesc | +| Variable | VarDesc | + +Please be aware that these Python classes need to maintain some construction-time information, which are not part of the protobuf messages. + +## Core Concepts + +### Program + +A `ProgramDesc` describes a [DL program](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/program.md), which is composed of an array of `BlockDesc`s. The `BlockDesc`s in a `ProgramDesc` can have a tree-like hierarchical structure. However, the `ProgramDesc` onlys stores a flattened array of `BlockDesc`s. A `BlockDesc` refers to its parent block by its index in the array. For example, operators in the step block of an RNN operator need to be able to access variables in its ancestor blocks. + +Whenever we create a block, we need to set its parent block to the current block, hence the Python class `Program` needs to maintain a data member `current_block`. + +```python +class Program(objects): + def __init__(self): + self.desc = core.NewProgram() # a C++ ProgramDesc pointer. + self.blocks = vector() + self.blocks.append(Block(self, -1)) # the global block + self.current_block = 0 # initialized to the global block + + def global_block(): + return self.blocks[0] + + def current_block(): + return self.get_block(self.current_block) + + def rollback(): + self.current_block = self.current_block().parent_idx + + def create_block(): + new_block_idx = len(self.block) + self.blocks.append(Block(self, self.current_block)) + self.current_block = new_block_idx + return current_block() +``` + +`Program` is an accessor to the protobuf message `ProgramDesc`, which is created in C++ space, because the InferShape function is in C++, which manipulates `VarDesc` messages, which are in turn members of `BlockDesc`, which is a member of `ProgramDesc`. + +`Program` creates the first block as the global block in its constructor. All parameters and their initializer operators are in the global block. + +### Block + +A [Block](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/block.md) includes + +1. a map from variable names to an instance of the Python `Variable` class, and +1. a list of `Operator` instances. + +```python +class Block(objects): + def __init__(self, program, parent_idx): + self.desc = core.NewBlock(program.desc) + self.program = program + self.vars = map() + self.ops = vector() + self.parent_idx = parent_idx + + def create_var(self, ...): + return Variable(self, ...) + + def _create_global_var(self, ...): + program.global_block().create_var(...) + + def create_parameter(self, name, ...): + # Parameter is a subclass of variable. See Parameter section for details. + self.vars[name] = Parameter(self._create_global_var(...), ...) + return self.vars[name] + + def append_operator(self, ...): + self.ops.append(Operator(self, ...)) + + def prepend_operator(self, ...): # Parameter's ctor prepands initialize operators. + self.ops.prepend(Operator(self, ...)) +``` + +`create_parameter` is necessary because parameters are global variables, defined in the global block, but can be created in some sub-blocks. For example, an FC layer in the step block of an RNN operator. + +`prepend_operator` is necessary because the constructor of `Parameter` needs to create the initialize (or load) operator of the parameter, and would like to put it in the *preamble* of the global block. + +### Operator + +The `Operator` class fills in the `OpDesc` message and calls the C++ function `InferShape` to infer the output shapes from the input shapes. + +```python +class Operator(object): + def __init__(self, + block, # Block + type, # string + inputs, # dict + outputs,# dict + attrs # dict + ): + self.desc = core.NewOpDesc(block.desc, type, inputs, outputs, attrs) + core.infer_shape(self.desc, inputs, outputs) + + def type(self): + return self.desc.type() +``` + +`Operator` creates the `OpDesc` message in C++ space, so that it can call the `InferShape` function, which is in C++. + +### Variable + +Operators take Variables as its inputs and outputs. + +```python +class Variable(object): + def __init__(self, + block=None, # Block + name=None, # string + shape, # tuple + dtype="float32", # string + lod_level=None # int + ): + if name is None: + name = unique_name_generator() + self.name = name + self.block = block + self.desc = core.NewVarDesc(block.desc, name, shape, lod_level) + self.writer = None +``` + +Please be aware of `self.writer`, that tracks operator who creates the variable. It possible that there are more than one operators who write a variable, but in Python space, each write to a variable is represented by a Variable class. This is guaranteed by the fact that **`core.NewVarDesc` must NOT create a new `VarDesc` message if its name already exists in the specified block**. + +### Parameter + +A parameter is a global variable with an initializer (or load) operator. + +```python +class Parameter(Variable): + def __init__(self, + block=None, # Block + name=None, # string + shape, # tuple + dtype="float32", # string + lod_level=None # int + trainable, # bool + initialize_op_attrs, + optimize_op_attrs): + super(Parameter, self).__init__(block, name, shape, dtype, lod_level) + self.trainable = trainable + self.optimize_op_attrs = optimize_op_attrs + block.prepend(Operator(block, # Block + initialize_op_attrs['type'], # string + None, # no inputs + self, # output is the parameter + initialize_op_attrs) +``` + +When users create a parameter, they can call + +```python +program.create_parameter( + ..., + init_attr={ + type: "uniform_random", + min: -1.0, + max: 1.0, + }) +) +``` + +In above example, `init_attr.type` names an initialize operator. It can also name the load operator + +```python +init_attr={ + type: "load", + filename: "something.numpy", +} +``` + +`optimize_op_attrs` is not in the `VarDesc` message, but kept in the Python instance, as it will be used in the Python space when creating the optimize operator's `OpDesc`, and will be in the `OpDesc` message. + +## Layer Function + +A layer is a Python function that creates some operators and variables. Layers simplify the work of application programmers. + +Layer functions take `Variable` and configuration parameters as its input and return the output variable(s). + +For example, `FullyConnected` take one or more variable as its input. The input could be input data or another layer's output. There are many configuration options for a `FullyConnected` layer, such as layer size, activation, parameter names, initialization strategies of parameters, and so on. The `FullyConnected` layer will return an output variable. + + +### Necessity for reusing code between layer functions + +There are a lot of code that can be reused. Such as + +* Give the default value of configuration. e.g., default initialize strategy for parameters is uniform random with `min = -1.0`, `max = 1.0`. and default initialize strategy for bias is to fill zero. +* Append the activation operator. +* Create a temporary variable. +* Create parameter. +* Generate a unique name. +* Add a bias. +* ... + +A mechanism to reuse code between layer functions is necessary. It will be around [150 lines of code](https://github.com/PaddlePaddle/Paddle/pull/4724/files#diff-823b27e07e93914ada859232ae23f846R12) if we write a `FullyConnected` layer without any helper functions. + + + +### Comparision between global functions and helper class + +The `FullyConnected` layer will be as follow when we provide global functions: + +```python +def fc_layer(input, size, param_attr=None, bias_attr=None, act=None, name=None): + if name is None: + name = unique_name("fc") + input = multiple_input(input) + param_attr = default_param_attr(param_attr) + param_attr = multiple_param_attr(param_attr, len(input)) + + # mul + mul_results = [] + for ipt, attr in zip(input, param_attr): + shape = ipt.shape[1:] + [size] + w = g_program.global_block().create_parameter(shape, ipt.dtype, name, attr) + tmp = create_tmp_var(name) + g_program.current_block().append_op("mul", {ipt, w}, {tmp}) + mul_results.append(tmp) + + # add sum + ... + # add bias + ... + # add activation + ... + return out +``` + +We can provide many helpers functions for layer developers. However, there are several disadvantages for global helper functions: + +1. We need a namespace for these methods, then layer developers can quickly figure out what method they can use. +2. Global functions will force layer developers to pass its parameter time by time. + +So we provide a helper class, `LayerHelper`, to share code between layer functions. The `FullyConnected` Layer will be as follow. + +```python +def fc_layer(input, size, param_attr=None, bias_attr=None, act=None, name=None): + helper = LayerHelper(locals()) # pass all parameter to LayerHelper + + mul_results = [] + for ipt, param in helper.iter_multiple_input_and_param(): + w = helper.create_parameter(shape=ipt.shape[1:] + [size], dtype = ipt.dtype) + tmp = helper.create_tmp_variable() + helper.append_op('mul', {ipt, w}, {tmp}) + mul_results.append(tmp) + + pre_bias = helper.add_sum(mul_results) + pre_activation = helper.add_bias(pre_bias) + return helper.add_activation(pre_activation) +``` + +We not only use the fewer lines of code to write `fc_layer` but also make the code clearer to understand. At the same time, layer developers can figure out what function they can invoke by typing `helper.` in a python editor. + + +### Implementation of layer helper + +We just keep all parameters of a layer function as a dictionary in layer helper as a private data member. Every method of layer helper will look up the dictionary after it is invoked. In that way, we can implement a layer helper for all layer functions even some layer does not contain some operator. For example, The `activation` is used by the FullyConnected layer or convolution layers, but a cross-entropy layer does not use it. The example code of `add_activation` are: + +```python +class LayerHelper(object): + def __init__(self, **kwargs): # kwargs is short for `keyword arguments` + self.kwargs = kwargs + + def add_activation(self, input_var): + act = self.kwargs.get("act", None) # default value is None + if act is None: # do nothing if no act + return input_var + + tmp = self.create_tmp_var(self) + self.append_op(type=act, input=input_var, output=tmp) + return tmp +``` + +## Optimizer + +[Optimizer Design Doc](./optimizer.md) diff --git a/doc/_sources/design/reader/README.md.txt b/doc/_sources/design/reader/README.md.txt index f21f7af520df5171798326818ecb97c3bcd14a12..2cd4b6225b61cf374458e40afabad7745f61ba71 100644 --- a/doc/_sources/design/reader/README.md.txt +++ b/doc/_sources/design/reader/README.md.txt @@ -1,25 +1,25 @@ # Python Data Reader Design Doc -At training and testing time, PaddlePaddle programs need to read data. To ease the users' work to write data reading code, we define that +During the training and testing phases, PaddlePaddle programs need to read data. To help the users write code that performs reading input data, we define the following: -- A *reader* is a function that reads data (from file, network, random number generator, etc) and yields data items. -- A *reader creator* is a function that returns a reader function. -- A *reader decorator* is a function, which accepts one or more readers, and returns a reader. -- A *batch reader* is a function that reads data (from *reader*, file, network, random number generator, etc) and yields a batch of data items. +- A *reader*: A function that reads data (from file, network, random number generator, etc) and yields the data items. +- A *reader creator*: A function that returns a reader function. +- A *reader decorator*: A function, which takes in one or more readers, and returns a reader. +- A *batch reader*: A function that reads data (from *reader*, file, network, random number generator, etc) and yields a batch of data items. -and provide function which converts reader to batch reader, frequently used reader creators and reader decorators. +and also provide a function which can convert a reader to a batch reader, frequently used reader creators and reader decorators. ## Data Reader Interface -Indeed, *data reader* doesn't have to be a function that reads and yields data items. It can be any function with no parameter that creates a iterable (anything can be used in `for x in iterable`): +*Data reader* doesn't have to be a function that reads and yields data items. It can just be any function without any parameters that creates an iterable (anything can be used in `for x in iterable`) as follows: ``` iterable = data_reader() ``` -Element produced from the iterable should be a **single** entry of data, **not** a mini batch. That entry of data could be a single item, or a tuple of items. Item should be of [supported type](http://www.paddlepaddle.org/doc/ui/data_provider/pydataprovider2.html?highlight=dense_vector#input-types) (e.g., numpy 1d array of float32, int, list of int) +The item produced from the iterable should be a **single** entry of data and **not** a mini batch. The entry of data could be a single item or a tuple of items. Item should be of one of the [supported types](http://www.paddlepaddle.org/doc/ui/data_provider/pydataprovider2.html?highlight=dense_vector#input-types) (e.g., numpy 1d array of float32, int, list of int etc.) -An example implementation for single item data reader creator: +An example implementation for single item data reader creator is as follows: ```python def reader_creator_random_image(width, height): @@ -29,7 +29,7 @@ def reader_creator_random_image(width, height): return reader ``` -An example implementation for multiple item data reader creator: +An example implementation for multiple item data reader creator is as follows: ```python def reader_creator_random_image_and_label(width, height, label): def reader(): @@ -40,9 +40,10 @@ def reader_creator_random_image_and_label(width, height, label): ## Batch Reader Interface -*batch reader* can be any function with no parameter that creates a iterable (anything can be used in `for x in iterable`). The output of the iterable should be a batch (list) of data items. Each item inside the list must be a tuple. +*Batch reader* can be any function without any parameters that creates an iterable (anything can be used in `for x in iterable`). The output of the iterable should be a batch (list) of data items. Each item inside the list should be a tuple. + +Here are some valid outputs: -Here are valid outputs: ```python # a mini batch of three data items. Each data item consist three columns of data, each of which is 1. [(1, 1, 1), @@ -52,26 +53,28 @@ Here are valid outputs: # a mini batch of three data items, each data item is a list (single column). [([1,1,1],), ([2,2,2],), -([3,3,3],), +([3,3,3],)] ``` Please note that each item inside the list must be a tuple, below is an invalid output: ```python # wrong, [1,1,1] needs to be inside a tuple: ([1,1,1],). - # Otherwise it's ambiguous whether [1,1,1] means a single column of data [1, 1, 1], - # or three column of datas, each of which is 1. + # Otherwise it is ambiguous whether [1,1,1] means a single column of data [1, 1, 1], + # or three columns of data, each of which is 1. [[1,1,1], [2,2,2], [3,3,3]] ``` -It's easy to convert from reader to batch reader: +It is easy to convert from a reader to a batch reader: + ```python mnist_train = paddle.dataset.mnist.train() mnist_train_batch_reader = paddle.batch(mnist_train, 128) ``` -Also easy to create custom batch reader: +It is also straight forward to create a custom batch reader: + ```python def custom_batch_reader(): while True: @@ -85,7 +88,8 @@ mnist_random_image_batch_reader = custom_batch_reader ## Usage -batch reader, mapping from item(s) read to data layer, batch size and number of total pass will be passed into `paddle.train`: +Following is how we can use the reader with PaddlePaddle: +The batch reader, a mapping from item(s) to data layer, the batch size and the number of total passes will be passed into `paddle.train` as follows: ```python # two data layer is created: @@ -99,13 +103,13 @@ paddle.train(batch_reader, {"image":0, "label":1}, 128, 10, ...) ## Data Reader Decorator -*Data reader decorator* takes a single or multiple data reader, returns a new data reader. It is similar to a [python decorator](https://wiki.python.org/moin/PythonDecorators), but it does not use `@` syntax. +The *Data reader decorator* takes in a single reader or multiple data readers and returns a new data reader. It is similar to a [python decorator](https://wiki.python.org/moin/PythonDecorators), but it does not use `@` in the syntax. -Since we have a strict interface for data readers (no parameter, return a single data item). Data reader can be used flexiable via data reader decorators. Following are a few examples: +Since we have a strict interface for data readers (no parameters and return a single data item), a data reader can be used in a flexible way using data reader decorators. Following are a few examples: ### Prefetch Data -Since reading data may take time and training can not proceed without data. It is generally a good idea to prefetch data. +Since reading data may take some time and training can not proceed without data, it is generally a good idea to prefetch the data. Use `paddle.reader.buffered` to prefetch data: @@ -117,9 +121,9 @@ buffered_reader = paddle.reader.buffered(paddle.dataset.mnist.train(), 100) ### Compose Multiple Data Readers -For example, we want to use a source of real images (reusing mnist dataset), and a source of random images as input for [Generative Adversarial Networks](https://arxiv.org/abs/1406.2661). +For example, if we want to use a source of real images (say reusing mnist dataset), and a source of random images as input for [Generative Adversarial Networks](https://arxiv.org/abs/1406.2661). -We can do: +We can do the following : ```python def reader_creator_random_image(width, height): @@ -139,13 +143,13 @@ false_reader = reader_creator_bool(False) reader = paddle.reader.compose(paddle.dataset.mnist.train(), data_reader_creator_random_image(20, 20), true_reader, false_reader) # Skipped 1 because paddle.dataset.mnist.train() produces two items per data entry. -# And we don't care second item at this time. +# And we don't care about the second item at this time. paddle.train(paddle.batch(reader, 128), {"true_image":0, "fake_image": 2, "true_label": 3, "false_label": 4}, ...) ``` ### Shuffle -Given shuffle buffer size `n`, `paddle.reader.shuffle` will return a data reader that buffers `n` data entries and shuffle them before a data entry is read. +Given the shuffle buffer size `n`, `paddle.reader.shuffle` returns a data reader that buffers `n` data entries and shuffles them before a data entry is read. Example: ```python @@ -154,21 +158,21 @@ reader = paddle.reader.shuffle(paddle.dataset.mnist.train(), 512) ## Q & A -### Why reader return only a single entry, but not a mini batch? +### Why does a reader return only a single entry, and not a mini batch? -Always returning a single entry make reusing existing data readers much easier (e.g., if existing reader return not a single entry but 3 entries, training code will be more complex because it need to handle cases like batch size 2). +Returning a single entry makes reusing existing data readers much easier (for example, if an existing reader returns 3 entries instead if a single entry, the training code will be more complicated because it need to handle cases like a batch size 2). -We provide function `paddle.batch` to turn (single entry) reader into batch reader. +We provide a function: `paddle.batch` to turn (a single entry) reader into a batch reader. -### Why do we need batch reader, isn't train take reader and batch_size as arguments sufficient? +### Why do we need a batch reader, isn't is sufficient to give the reader and batch_size as arguments during training ? -In most of the case, train taking reader and batch_size as arguments would be sufficent. However sometimes user want to customize order of data entries inside a mini batch. Or even change batch size dynamically. +In most of the cases, it would be sufficient to give the reader and batch_size as arguments to the train method. However sometimes the user wants to customize the order of data entries inside a mini batch, or even change the batch size dynamically. For these cases using a batch reader is very efficient and helpful. -### Why use a dictionary but not a list to provide mapping? +### Why use a dictionary instead of a list to provide mapping? -We decided to use dictionary (`{"image":0, "label":1}`) instead of list (`["image", "label"]`) is because that user can easily resue item (e.g., using `{"image_a":0, "image_b":0, "label":1}`) or skip item (e.g., using `{"image_a":0, "label":2}`). +Using a dictionary (`{"image":0, "label":1}`) instead of a list (`["image", "label"]`) gives the advantage that the user can easily reuse the items (e.g., using `{"image_a":0, "image_b":0, "label":1}`) or even skip an item (e.g., using `{"image_a":0, "label":2}`). -### How to create custom data reader creator +### How to create a custom data reader creator ? ```python def image_reader_creator(image_path, label_path, n): @@ -192,7 +196,7 @@ paddle.train(paddle.batch(reader, 128), {"image":0, "label":1}, ...) ### How is `paddle.train` implemented -An example implementation of paddle.train could be: +An example implementation of paddle.train is: ```python def train(batch_reader, mapping, batch_size, total_pass): diff --git a/doc/_sources/design/refactor/distributed_architecture.md.txt b/doc/_sources/design/refactor/distributed_architecture.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..d9fe7d6bbb0eeb73fcdca3ee749a4f10bcdda682 --- /dev/null +++ b/doc/_sources/design/refactor/distributed_architecture.md.txt @@ -0,0 +1,144 @@ +# Design Doc: Distributed Training Architecture + +## Abstract + +PaddlePaddle version 0.10.0 uses the "trainer-parameter server" architecture. We run multiple instances of trainers (where each trainer runs the same model) and parameter servers for distributed training. This architecture serves well, but has few limitations: + +1. There is a need to write special code that handles tasks which should only be run on a single trainer. E.g., initializing the model, saving the model etc. + +2. Model parallelism is hard: It would need all the if-else branches conditioned on the trainer ID to partition the model onto the trainers, and eventually manually writing out the inter-model-shard communication code to communicate between different trainers. + +3. The user can not directly specify the parameter update rule: This would need to modify the parameter server code and compile a new binary. This makes things more complicated for researchers: A lot of extra effort is required to make this work. Besides, the training job submission program may not allow running arbitrary binaries. + +This design doc discusses PaddlePaddle's new distributed training architecture that addresses the above mentioned limitations. + +## Analysis + +The assumption is that the user writes the trainer program in either Python or C++. + +### Limitation 1 + +There are two basic functionalities in the trainer program: + +1. The training logic such as loading / saving the model and printing out the logs. +2. The neural network definition such as the definition of the data layer, the fully connected layer, the cost function and the + optimizer. + +When we train using PaddlePaddle v0.10.0 in a distributed fashion, multiple instances of the same Python code are run on different nodes, hence both: the +training logic as well as the neural network computation logic, is replicated. + +The tasks that only need to be run once belong to the training logic. Hence if we only replicate the neural network computation part, and do **not** +replicate the training logic, the limitation mentioned above can be avoided. + +### Limitation 2 + +Model parallelism means that a single model is partitioned into different components and each node runs one of the component separately. This comes at the extra cost of managing the +inter-model-shard communication between nodes. + +PaddlePaddle should ideally be able to modify the neural network computation and figure out the support for model parallelism automatically. However, the +computation is only specified in Python code which sits outside of PaddlePaddle, hence PaddlePaddle can not support the feature in this setup. + +Similar to how a compiler uses an intermediate representation (IR) so that the programmer does not need to manually optimize their code for most of the cases, we can have an intermediate representation in PaddlePaddle as well. The compiler optimizes the IR as follows: + + + +PaddlePaddle can support model parallelism by converting the IR so that the user no longer needs to manually perform the computation and operations in the Python component: + + + +The IR for PaddlePaddle after refactoring is called a `Block`, it specifies the computation dependency graph and the variables used in the computation. + +### Limitation 3 + +The user can not directly specify the parameter update rule for the parameter server in the Python module, since the parameter server does not use the same computation definition as the trainer. Instead, the update rule is baked inside the parameter server. The user can not specify the update rule explicitly. + +This could be fixed by making the parameter server run the same computation definition as the trainer (the user's Python module). For a detailed explanation, refer to this document - +[Design Doc: Operation Graph Based Parameter Server](./parameter_server.md) + +## Distributed Training Architecture + +The revamped distributed training architecture can address the above discussed limitations. Below is the illustration of how it does so: + + + +The major components in the architecture are: *PaddlePaddle Python*, *PaddlePaddle converter* and *PaddlePaddle runtime*. + +### PaddlePaddle Python + +PaddlePaddle Python is the Python library that user's Python code invokes, to read the data. build the neural network topology, start training, etc. + +```Python +paddle.init() +input = paddle.op.recordIO("/home/data/mnist.recordio") # file stored on the cluster +img, label = input[0], input[1] +hidden = paddle.layer.fc(input=img, size=200, act=paddle.activation.Tanh()) +prediction = paddle.layer.fc(input=img, size=10, act=paddle.activation.Softmax()) +cost = paddle.layer.classification_cost(input=prediction, label=label) +optimizer = paddle.optimizer.SGD(cost, learning_rate=0.01) +session = paddle.session.NewRemote(num_trainer=3, num_ps=2, GPU_per_trainer=1) +for i in range(1000): + _, cost_val = session.eval(targets=[cost, optimizer]) + print cost_val +``` + +The above code is what a typical Python trainer code is, the neural network topology is built using the helper functions such as `paddle.layer.fc`. Training is done by calling `session.eval` iteratively. + +#### session.eval + +As shown in the graph, `session.eval` sends the IR and the evaluation inputs or targets to the PaddlePaddle cluster for evaluation. +The targets can be any variable in the computation graph. When the target is say, the `optimizer` variable, the neural network will be optimized once. When the target is the `cost` variable, `session.eval` returns the cost value. Based on what the target is, an appropriate action is taken. + +The Python `session` is a wrapper of the C++ `Session` class. For more information about `Session`, refer to this document - [Design Doc: Session](./session.md). + +### PaddlePaddle Converter + +The PaddlePaddle converter automatically converts the IR in the request (IR and evaluation inputs/targets) from PaddlePaddle Python to partitioned IRs and dispatches the new IRs and evaluation inputs/targets to different PaddlePaddle runtimes. Below are the steps that are followed : + +1. Add a `feed` OP that feeds the eval inputs, and a `fetch` OP that fetches the eval targets to the IR. + +2. Extract a new computation (sub)graph with the `feed` and `fetch` OPs as the boundary. The runtime does not need to run the OP that is not dependent on the `fetch` OP. + +3. Optimize the computation graph. + +4. Place the OPs in the graph onto different devices on different PaddlePaddle runtime according to a placement algorithm and the device constraints specified by the user. + +5. Partition the graph according to runtime boundaries and add `send` / `recv` OP pair on the runtime boundaries. + +6. Dispatch the partitioned graph to different PaddlePaddle runtimes. + +7. PaddlePaddle runtimes with the `fetch` OP reports evaluation results back to the converter, the converter reports the evaluation results back to the PaddlePaddle Python. + +The output IRs will be cached to optimize the conversion latency. + + +#### Placement Algorithm + +Our first implementation will only support "trainer-parameter server" placement: the parameters, initializers, and optimizers are all placed on the PaddlePaddle runtimes with the parameter server role. Everything else will be placed on the PaddlePaddle runtimes with the trainer role. This has the same functionality as the "trainer-parameter server" architecture of PaddlePaddle v0.10.0, but is more generic and flexible. + +In the future, a more general placement algorithm should be implemented, which makes placements according to the input IR, and a model of device computation time and device communication time. Model parallelism requires the generic placement algorithm. + + +### PaddlePaddle Runtime + +The PaddlePaddle runtime owns multiple devices (e.g., CPUs, GPUs) and runs the IR. The runtime does not need to do OP placement since it is already done by the converter. + + +### Local Training Architecture + +The local training architecture will be the same as the distributed training architecture, the difference is that everything runs locally, and there is just one PaddlePaddle runtime: + + + + +### Training Data + +In PaddlePaddle v0.10.0, training data is typically read with a [data reader](../reader/README.md) from Python. This approach is no longer efficient when training in a distributed fashion since the Python process no longer runs on the same node with the trainer processes. The Python reader will need to read from the distributed filesystem (assuming it has the required access) and send to the trainers, doubling the network traffic. + +When doing distributed training, the user can still use Python data reader: the training data are sent with `session.eval`. However this should be used for debugging purpose only. The users are encouraged to use the read data OPs. + + +## References: + +[1] [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45166.pdf) + +[2] [TensorFlow: A System for Large-Scale Machine Learning](https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf) diff --git a/doc/_sources/design/refactor/parameter_server.md.txt b/doc/_sources/design/refactor/parameter_server.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..fa3c5d7990213cf2b0d236e66e592dd2699da876 --- /dev/null +++ b/doc/_sources/design/refactor/parameter_server.md.txt @@ -0,0 +1,106 @@ +# Design Doc: Operation Graph Based Parameter Server + +## Abstract + +We propose an approach to implement the parameter server. In this +approach, there is no fundamental difference between the trainer and +the parameter server: they both run subgraphs, but subgraphs of +different purposes. + +## Background + +The previous implementations of the parameter server does not run a +subgraph. parameter initialization, optimizer computation, network +communication and checkpointing are implemented twice on both the +trainer and the parameter server. + +It would be great if we can write code once and use them on both the +trainer and the parameter server: reduces code duplication and +improves extensibility. Given that after the current refactor, we are +representing everything as a computing graph on the +trainer. Representing everything as a computing graph on the parameter +server becomes a natural extension. + +## Design + +### Graph Converter + +The *graph converter* converts the user-defined operation (OP) graph +into subgraphs to be scheduled on different nodes with the following +steps: + +1. OP placement: the OPs will be placed on different nodes according + to heuristic that minimizes estimated total computation + time. Currently we will use a simple heuristic that puts parameter + varable on parameter server workers and everything else on trainer + workers. + +1. Add communication OPs to enable the communication between nodes. + +We will need these OPs: *Send*, *Recv*, *Enqueue*, *Dequeue*. + +Below is an example of converting the user defined graph to the +subgraphs for the trainer and the parameter server: + + + +After converting: + + + +1. The parameter variable W and it's optimizer subgraph are placed on the parameter server. +1. Operators are added to the subgraphs. + - *Send* sends data to the connected *Recv* operator. The + scheduler on the receive node will only schedule *Recv* operator + to run when the *Send* operator has ran (the *Send* OP will mark + the *Recv* OP runnable automatically). + - *Enueue* enqueues the input variable, it can block until space + become available in the queue. + - *Dequeue* outputs configurable numbers of tensors from the + queue. It will block until the queue have the required number of + tensors. + + +### Benefits + +- Model parallelism become easier to implement: it's an extension to + the trainer - parameter server approach. we already have the + communication OPs, but need to extend the graph converter's + placement functionality. + +- User-defined optimizer is easier to add - user can now express it as + a subgraph. + +- No more duplication logic inside the trainer and the parameter + server mentioned in the background section. + +### Challenges + +- It might be hard for the graph converter to cut a general graph + (without any hint for which subgraph is the optimizer). We may need + to label which subgraph inside the OP graph is the optimizer. + +- It's important to balance the parameter shards of on multiple + parameter server. If a single parameter is very big (some + word-embedding, fully connected, softmax layer), we need to + automatically partition the single parameter onto different + parameter servers when possible (only element-wise optimizer depends + on the parameter variable). + +### Discussion + +- In the "Aync SGD" figure, the "W" variable on the parameter server + could be read and wrote concurrently, what is our locking strategy? + E.g., each variable have a lock cpp method to be invoked by every + OP, or, have a lock OP. + +- Can the Enqueue OP be implemented under our current tensor design + (puts the input tensor into the queue tensor)? + +- *Dequeue* OP will have variable numbers of output (depends on the + `min_count` attribute), does our current design support it? (similar + question for the *Add* OP) + + +### References: +[1] [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45166.pdf) diff --git a/doc/_sources/design/refactor/session.md.txt b/doc/_sources/design/refactor/session.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..1d9a26683c14f54e3b5fe41675cd03b5620646b8 --- /dev/null +++ b/doc/_sources/design/refactor/session.md.txt @@ -0,0 +1,180 @@ +# Design Doc: Session + +## Abstract + +The *session* object encapsulates the environment in which the +computation graph is executed. + +We will have the *local* session and *remote* session, they offer the +same [interface](#interface). The local session encapsulates the local +runtime environment and the remote session encapsulates the cluster +runtime environment. + +The local runtime environment contains: + +1. computation devices (i.e., CPU, GPU) handles, and +1. the [scope](../scope.md) which holds all variables. + +The remote runtime environment contains: + +1. computation devices (i.e., CPU and GPU on node 0, 1) in a cluster, + and +1. the distributed [scope](../scope.md) in a cluster which holds all + variables. + +The user can create a remote session on Paddle Cloud and evaluate the +computation graph with it. In this way, the user can control the +remote computation resource in a cluster from his local computer. + + +## Background + +The current design has an implicit global session in which +`paddle.eval()` is executed. The pain point is: + +Since the user is not able to explicitly switch between runtime +environments, the user cannot run a topology in two independent +environments. + +For example, in reinforcement learning, the user may want to have a +stale model for inference and a fresh model for training, and only +replace the stale model with the fresh model periodically. + +Furthermore, we have no concept that encapsulates a remote environment +that executes a computation graph. + +We need the session object to address above issues. + + +## Session + +A session is an object that owns the runtime environment. All +computations are executed through `session.eval()`. + + +### Interface + +```python +eval( + targets, + feed_dict=None, +) +``` + +Evaluates the target Operations or Variables in `targets`. + +- *targets*: the evaluation targets. Can be a single Operation or + Variable, or a list with the Operations or Variables as + elements. The value returned by `eval()` has the same shape as the + `target` argument. + + The PaddlePaddle program is represented by + the [ProgramDesc](../design/program.md), `eval()` will infer the + ProgramDesc from the given targets and run the PaddlePaddle + program. Please + see + [this graph](./distributed_architecture.md#local-training-architecture) for + the detailed illustration for the local session + and + [this graph](./distributed_architecture.md#distributed-training-architecture) for + the detailed illustration for the remote session. + +- *feed_dict*: a dictionary that contains the tensors which override + the edges of the computation graph. + + feed_dict not only can provide the input data, it can override any + OP's input as well: + + ```python + a = pd.constant(2.0, name="a") + b = pd.variable(name="b") + c = pd.mul(a,b) + sess.eval(targets=c, feed_dict={"b":3.0}) # returns 6.0 + ``` + +```python +close() +``` + +Closes the session and releases the scope that the session owns. + + +### Create a Local Session + +```python +session( + devices=None +) +``` + +Creates a new session. One session owns one global scope, so creating +multiple sessions will create different scopes. + +- *devices*: a single `string` or a list of `string` of device names, + the corresponding devices will be the computation devices for + `eval()`. If not specified, all available devices (e.g., all GPUs) + will be used. The user doesn't need to specify the CPU device since + it will be always used. Multiple sessions can use the same device. + + +#### Example + +```Python +a = paddle.constant(1.0) +b = paddle.constant(2.0) +c = a + b +sess = paddle.session(devices=["gpu:0", "gpu:1", "fpga:0"]) +sess.eval(c) +sess.close() +``` + +### Create a Remote Session + +```python +create_cloud_job( + name, + num_trainer, + mem_per_trainer, + gpu_per_trainer, + cpu_per_trainer, + num_ps, + mem_per_ps, + cpu_per_ps, +) +``` + +Creates a Paddle Cloud job. Fails if the job name exists. + +```python +get_cloud_job( + name +) +``` + +Gets a Paddle Cloud job. + +```python +remote_session( + job +) +``` + +- *job*: the Paddle Cloud job. + +#### Example + +```Python +reader = paddle.reader.recordio("/pfs/home/peter/mnist-train-*") # data stored on Paddle Cloud +image = reader.column(0) +label = reader.column(1) +fc1 = paddle.op.fc(image, size=256, act="sigmoid") +fc2 = paddle.op.fc(fc1, size=10, act="softmax") +cost = paddle.op.cross_entropy(fc2, label) +opt = paddle.optimizer.sgd(cost) + +job = paddle.create_cloud_job("test", 3, "1G", 1, 1, 2, "1G", 1) +sess = paddle.remote_ession(job) +for i in range(1000): + sess.eval(opt) +sess.close() +``` diff --git a/doc/_sources/design/refactorization.md.txt b/doc/_sources/design/refactorization.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..f93d6155e1764386b01d2f0df3f141ab75cd55d4 --- /dev/null +++ b/doc/_sources/design/refactorization.md.txt @@ -0,0 +1,249 @@ +# Design Doc: Refactorization Overview + +The goals of refactoring include: + +1. Making it easy for external contributors to write new elementary computation operations. +1. Making the codebase clean and readable. +1. Designing a new computation representation -- a computation graph of operators and variables. +1. Implementing auto-scalability and auto fault recoverable distributed computing with the help of computation graphs. + +## Computation Graphs + +1. PaddlePaddle represents the computation, training and inference of Deep Learning models, by computation graphs. + + 1. Please refer to [computation graphs](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/graph.md) for a concrete example. + +1. Users write Python programs to describe the graphs and run them (locally or remotely). + +1. A graph is composed of *variables* and *operators*. + +1. The description of graphs must be serializable/deserializable, so that: + + 1. It can be sent to the cloud for distributed execution, and + 1. It can be sent to clients for mobile or enterprise deployment. + +1. The Python program does two things + + 1. *Compilation* runs a Python program to generate a protobuf message representation of the graph and send it to + 1. the C++ library `libpaddle.so` for local execution, + 1. the master process of a distributed training job for training, or + 1. the server process of a Kubernetes serving job for distributed serving. + 1. *Execution* executes the graph by constructing instances of class [`Variable`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24) and [`OperatorBase`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L70), according to the protobuf message. + +## Description and Realization of Computation Graph + +At compile time, the Python program generates a protobuf message representation of the graph, or a description of the graph. + +At runtime, the C++ program realizes the graph and runs it. + +| | Representation (protobuf messages) | Realization (C++ class objects) | +|---|---|---| +|Data|[VarDesc](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L107)|[Variable](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24)| +|Operation|[OpDesc](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L35)|[Operator](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L64)| +|Block|BlockDesc|Block| + +The word *graph* is interchangeable with *block* in this document. A graph consists of computation steps and local variables similar to a C++/Java program block, or a pair of parentheses(`{` and `}`). + +## Compilation and Execution + +1. Run a Python program to describe the graph. In particular, the Python application program does the following: + + 1. Create `VarDesc` to represent local/intermediate variables, + 1. Create operators and set attributes, + 1. Validate attribute values, + 1. Infer the type and the shape of variables, + 1. Plan memory-reuse for variables, + 1. Generate the backward graph + 1. Add optimization operators to the computation graph. + 1. Optionally, split the graph for distributed training. + +1. The invocation of `train` or [`infer`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/inference.py#L108) methods in the Python program does the following: + + 1. Create a new Scope instance in the [scope hierarchy](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md) for each run of a block, + 1. realize local variables defined in the BlockDesc message in the new scope, + 1. a scope is similar to the stack frame in programming languages, + + 1. Create an instance of class `Block`, in which, + 1. realize operators in the BlockDesc message, + + 1. Run the Block by calling + 1. `Block::Eval(vector* targets)` for forward and backward computations, or + 1. `Block::Eval(vector* targets)` for optimization. + + +## Intermediate Representation (IR) + +```text +Compile Time -> IR -> Runtime +``` + +### Benefits of IR + +- Optimization + ```text + Compile Time -> IR -> Optimized IR -> Runtime + ``` +- Automatically send partitioned IR to different nodes. + - Automatic Data Parallelism + ```text + Compile Time + |-> Single GPU IR + |-> [trainer-IR-0, trainer-IR-1, pserver-IR] + |-> Node-0 (runs trainer-IR-0) + |-> Node-1 (runs trainer-IR-1) + |-> Node-2 (runs pserver-IR) + ``` + - Automatic Model Parallelism (planned for future) + +--- + +# Operator/OpWithKernel/OpKernel + +![class_diagram](http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/49caf1fb70820fb4a6c217634317c9306f361f36/op_op_with_kern_class_diagram.dot) + +--- + +# Operator +![class_diagram](http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/dd598e8f1976f5759f58af5e5ef94738a6b2e661/op.dot) + +* `Operator` is the fundamental building block of the user interface. + * Operator stores input/output variable names and attributes. + * The `InferShape` interface is used to infer the shape of the output variables based on the shapes of the input variables. + * Use `Run` to compute the `output` variables from the `input` variables. + +--- + +# OpWithKernel/Kernel + +![class_diagram](http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/9d7f4eba185cf41c8e2fbfb40ae21890dbddcd39/op_with_kernel.dot) + +* `OpWithKernel` inherits `Operator`. +* `OpWithKernel` contains a Kernel map. + * `OpWithKernel::Run` get device's kernel, and invoke `OpKernel::Compute`. + * `OpKernelKey` is the map key. Only device place now, but may be data type later. + +--- + +# Why separate Kernel and Operator + +* Separate GPU and CPU code. + * Make Paddle capable of running without GPU. +* Make one operator (which is a user interface) and create many implementations. + * For example, same multiplication op can have different implementations kernels such as FP16 kernel, FP32 kernel, MKL, eigen kernel. +--- + +# Libraries for Kernel development + +* `Eigen::Tensor` contains basic math and element-wise functions. + * Note that `Eigen::Tensor` has broadcast implementation. + * Limit the number of `tensor.device(dev) = ` in your code. +* `thrust::transform` and `std::transform`. + * `thrust` has the same API as C++ standard library. Using `transform`, one can quickly implement customized element-wise kernels. + * `thrust`, in addition, supports more complex APIs, like `scan`, `reduce`, `reduce_by_key`. +* Hand-writing `GPUKernel` and `CPU` code + * Do not write in header (`.h`) files. CPU Kernel should be in cpp source (`.cc`) and GPU kernels should be in cuda (`.cu`) files. (GCC cannot compile GPU code.) +--- +# Operator Registration + +## Why is registration necessary? +We need a method to build mappings between Op type names and Op classes. + +## How is registration implemented? +Maintaining a map, whose key is the type name and the value is the corresponding Op constructor. + +--- +# The Registry Map + +### `OpInfoMap` + +`op_type(string)` -> `OpInfo` + +`OpInfo`: + +- **`creator`**: The Op constructor. +- **`grad_op_type`**: The type of the gradient Op. +- **`proto`**: The Op's Protobuf, including inputs, outputs and required attributes. +- **`checker`**: Used to check attributes. + +--- +# Related Concepts + +### Op_Maker +It's constructor takes `proto` and `checker`. They are completed during Op_Maker's construction. ([ScaleOpMaker](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/scale_op.cc#L37)) + +### Register Macros +```cpp +REGISTER_OP(op_type, op_class, op_maker_class, grad_op_type, grad_op_class) +REGISTER_OP_WITHOUT_GRADIENT(op_type, op_class, op_maker_class) +``` + +--- +# Registration Process +1. Write an Op class and its gradient Op class, if required. +2. Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator. +3. Invoke the macro `REGISTER_OP`. This macro will + 1. Call maker class to complete `proto` and `checker` + 2. Using the completed `proto` and `checker`, it will add a new key-value pair to the `OpInfoMap` + +--- +# Backward Module (1/2) +### Create Backward Operator +- Mapping from forward Op to backward Op +![backward](https://gist.githubusercontent.com/dzhwinter/a6fbd4623ee76c459f7f94591fd1abf0/raw/61026ab6e518e66bde66a889bc42557a1fccff33/backward.png) + +--- +# Backward Module (2/2) +### Build Backward Network +- **Input**: a graph of forward operators +- **Output**: a graph of backward operators +- **Corner cases in construction** + - Shared Variables => insert an `Add` operator to combine gradients + - No Gradient => insert a `fill_zero_grad` operator + - Recursive NetOp => call `Backward` recursively + - RNN Op => recursively call `Backward` on stepnet + - RNN Op => recursively call `Backward` on stepnet + + +--- +# Scope, Variable, Tensor + +* `Tensor` is an n-dimension array with type. + * Only dims and data pointers are stored in `Tensor`. + * All operations on `Tensor` are written in `Operator` or global functions. + * Variable length Tensor design [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md) +* `Variable` instances are the inputs and the outputs of an operator, not just `Tensor`. + * `step_scopes` in RNN is a variable and not a tensor. +* `Scope` is where variables are stored. + * map + * `Scope` has a hierarchical structure. The local scope can get variables from its parent scope. + +--- +# Block (in design) +## the difference between original RNNOp and Block +- As an operator is more intuitive than `RNNOp`, +- Offers a new interface `Eval(targets)` to deduce the minimal block to `Run`, +- Fits the compile-time/ runtime separation design paradigm. + - During the compilation, `SymbolTable` stores `VarDesc`s and `OpDesc`s and serialize to a `BlockDesc` + - When graph executes, a Block with `BlockDesc` is passed. It then creates `Op` and `Var` instances and then invokes `Run`. + +--- +# Milestone +- Take Paddle/books as the main line, the requirement of the models motivates framework refactoring, +- Model migration + - Framework development gives **priority support** to model migration, for example, + - the MNIST demo needs a Python interface, + - the RNN models require the framework to support `LoDTensor`. + - Determine some timelines, + - Frequently used Ops need to be migrated first, + - Different models can be migrated in parallel. +- Improve the framework at the same time +- Accept imperfection, concentrate on solving the specific problem at the right price. + +--- +# Control the migration quality +- Compare the performance of migrated models with old ones. +- Follow the google C++ style guide. +- Build the automatic workflow of generating Python/C++ documentations. + - The documentation of layers and ops should be written inside the code. + - Take the documentation quality into account when submitting pull requests. + - Preview the documentations, read and improve them from a user's perspective. diff --git a/doc/_sources/design/register_grad_op.md.txt b/doc/_sources/design/register_grad_op.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..8d973eb53178c3e889c845144553a453e11f067c --- /dev/null +++ b/doc/_sources/design/register_grad_op.md.txt @@ -0,0 +1,92 @@ +# Design Doc: Gradient Operators Registration + + +## The Problem Posed + +Currently, for each C++ operator class definition, a *gradient operator creator* function is registered, which takes as input a C++ operator instance and returns the corresponding gradient operator instance. + +However, we noticed two problems with the current design: + +1. As we decided to separate the *compilation* and the *execution* phases, we need to change the creator to take an `OpDesc` protobuf message in a `ProgramDesc` and inserts corresponding `OpDesc` messages into the `ProgramDesc` message. + +1. For some operators, the gradient computation can be written in terms of existing operators. For example, the gradient of *minus* operator consists of two operators -- an *identity* operator followed by a *scale* operator. Hence the registration mechanism needs to support mapping from an operator to a set of operators for the gradient computation. + +## The Current Implementation + +Instances of the C++ class `OpInfo` are stored an associative map whose key is the operator type. The `grad_op_type` indicates the associated gradient operator type. An operator can create the gradient operator by invoking `OpInfo::creator_` of the gradient operator. The pseudo code is as follows + +```cpp +struct OpInfo { + std::function creator_; + std::string grad_op_type_; + ... +}; + +map OpInfoMap; + +OperatorBase* CreateGradientOperator(const OperatorBase& op) { + return OpInfoMap.at(op.Type()).creator_(...); +} +``` + +## Proposed Solution + +The mapping relationship between an operator and its gradient operators is a function. The interface of this function is: + +```cpp +// (OpDesc) --> vector +std::function(const OpDescBind&)>; +``` + +The function takes an `OpDescBind` of the forward operator and returns one or many gradient operator descriptions. `OpDescBind` is a C++ wrapper for the protobuf message `OpDesc` for rapid manipulation of `OpDesc`. + +The `GradOpDescMaker` will be registered in `OpInfo` and will replace the `grad_op_type_` field. The `OpInfo` should look like + +```cpp +struct OpInfo { + std::function>(const OpDescBind&)> grad_op_maker_; + ... +}; +``` + +The `grad_op_maker_ ` is a `nullptr` if the operator does not have any associated gradient operators. + +We propose a base class called `GradOpDescMakerBase` to let operator developers generate `Gradient Operators` easily. The public interface of that class is + +```cpp +class GradOpDescMakerBase { +public: + GradOpDescMakerBase(const OpDescBind& ); + virtual std::vector> operator()()const = 0; +}; +``` + +We can convert `GradOpDescMakerBase` to `std::function>(const OpDescBind&)>` by + +```cpp +using GradOpMaker = ...; +std::function(const OpDescBind&)> func; +func = [] (const OpDescBind& fwd_op) { + GradOpMaker maker(fwd_op); + return maker(); +}; +``` + +We can write many helper functions since the `GradOpDescMakerBase` is a class now. The basic helper functions get the variables of `Input`, `Output`, `InputGradient` and `OutputGradient` in the forwarding operator. + +We should change register macros at the same time. In the current solution, there is no difference between forwarding operators and backward operators. So `REGISTER_OP` just register one operator. If the `REGISTER_OPERATOR ` contains `OpProtoAndCheckerMaker` and `GradOpDescMaker`, we just list them in the same macro. It can be done by a macro contains `__VA_ARGS__`. + +The user interface should be + +```cpp +vector MinusOpGradMaker(OpDesc) {...} +REGISTER_OPERATOR(minus, MinusOp, MinusOpProtoAndCheckerMaker, SumOpGradMaker); +// Developers can still manually implement gradient operator. +REGISTER_OPERATOR(minus_grad, MinusGradOp); +``` + +The interface of current `REGISTER_OP` macro could not be changed. In `REGISTER_OP`, it will invoke `REGISTER_OPERATOR` two times and generate GradOpDescMaker inside. + +```cpp +REGISTER_OP(minus, MinusOp, MinusOpProtoAndCheckerMaker, minus_grad, MinusGradOp); +``` diff --git a/doc/_sources/design/regularization.md.txt b/doc/_sources/design/regularization.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..21280ac898feb4dd5e5a5d9e88d121e856850f0b --- /dev/null +++ b/doc/_sources/design/regularization.md.txt @@ -0,0 +1,72 @@ +# Regularization in PaddlePaddle + +## Introduction to Regularization +A central problem in machine learning is how to design an algorithm that will perform well not just on the training data, but also on new data. A frequently faced problem is the problem of **overfitting**, where the model does not make reliable predictions on new unseen data. **Regularization** is the process of introducing additional information in order to prevent overfitting. This is usually done by adding extra penalties to the loss function that restricts the parameter spaces that an optimization algorithm can explore. + +### Parameter Norm Penalties +Most common regularization approaches in deep learning are based on limiting the capacity of the models by adding a parameter norm penalty to the objective function `J`. This is given as follows: + +
+ +The parameter `alpha` is a hyperparameter that weights the relative contribution of the norm penalty term, `omega`, relative to the standard objective function `J`. + +The most commonly used norm penalties are the L2 norm penalty and the L1 norm penalty. These are given as follows: + +##### L2 Regularization: +
+ +##### L1 Regularization +
+ +A much more detailed mathematical background of regularization can be found [here](http://www.deeplearningbook.org/contents/regularization.html). + +## Regularization Survey + +A detailed survey of regularization in various deep learning frameworks can be found [here](https://github.com/PaddlePaddle/Paddle/wiki/Regularization-Survey). + +## Proposal for Regularization in PaddlePaddle + +### Low-Level implementation + +In the new design, we propose to create new operations for regularization. For now, we can add 2 ops that correspond to the most frequently used regularizations: +- L2_regularization_op +- L1_regularization_op + +These ops can be like any other ops with their own CPU/GPU implementations either using Eigen or separate CPU and GPU kernels. As the initial implementation, we can implement their kernels using Eigen following the abstraction pattern implemented for [Activation Ops](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/accuracy_op.h). This abstraction pattern can make it very easy to implement new regularization schemes other than L1 and L2 norm penalties. + +The idea of building ops for regularization is in sync with the refactored Paddle philosophy of using operators to represent any computation unit. The way these ops will be added to the computation graph, will be decided by the [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md#layer-function) in Python API. + +### Computation Graph + +Below is an example of a really simple feed forward neural network. + +
+ +The Python API will modify this computation graph to add regularization operators. The modified computation graph will look as follows: + +
+    +### Python API implementation for Regularization + +Using the low level ops, `L2_regularization_op` and `L1_regularization_op`, any user can add regularization to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support regularization. An example of such an API can be seen in [Keras](https://keras.io/regularizers/). As per the PaddlePaddle [Python API design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md), the layer functions are responsible for creating operators, operator parameters and variables. Since regularization is a property of parameters, it makes sense to create these in the layer functions. + +#### Creation of Regularization ops +There are two possibilities for creating the regularization ops: +1. We create these ops immediately while building the computation graph. +2. We add these ops in a lazy manner, just before the backward, similar to the way the optimization ops are added. + +The proposal is to add these ops in a lazy manner just before the backward pass. + +#### Storage of Regularization attributes + +Since we want to create the regularization ops in a lazy manner, the regularization attributes (type of regularization and weight of regularization penalty) can be stored as attributes of the [`Parameter`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/framework.py#L421) class. This is because regularization is a property of the parameters and storing regularization properties with Parameters also allows for shared parameters. + +#### High-level API + +In PaddlePaddle Python API, users will primarily rely on [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md#layer-function) to create neural network layers. Hence, we also need to provide regularization functionality in layer functions. The design of these APIs can be postponed for later right now. A good reference for these APIs can be found in [Keras](https://keras.io/regularizers/) and also by looking at Tensorflow in [`tf.contrib.layers`](https://www.tensorflow.org/api_guides/python/contrib.layers). + + + + + + diff --git a/doc/_sources/design/releasing_process.md.txt b/doc/_sources/design/releasing_process.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..14c081ea84282e52a2e36475c3c0ea755122d154 --- /dev/null +++ b/doc/_sources/design/releasing_process.md.txt @@ -0,0 +1,68 @@ +# PaddlePaddle发行规范 + +PaddlePaddle使用git-flow branching model做分支管理，使用[Semantic Versioning](http://semver.org/)标准表示PaddlePaddle版本号。 + +PaddlePaddle每次发新的版本，遵循以下流程: + +1. 从`develop`分支派生出新的分支，分支名为`release/版本号`。例如，`release/0.10.0` +1. 将新分支的版本打上tag，tag为`版本号rc.Patch号`。第一个tag为`0.10.0rc1`，第二个为`0.10.0rc2`，依次类推。 +1. 对这个版本的提交，做如下几个操作: + * 修改`python/setup.py.in`中的版本信息,并将`istaged`字段设为`True`。 + * 编译这个版本的Docker发行镜像，发布到dockerhub。如果失败，修复Docker编译镜像问题，Patch号加一，返回第二步 + * 编译这个版本的Ubuntu Deb包。如果失败，修复Ubuntu Deb包编译问题，Patch号加一，返回第二步。 + * 使用Regression Test List作为检查列表，测试Docker镜像/ubuntu安装包的功能正确性 + * 如果失败，记录下所有失败的例子，在这个`release/版本号`分支中，修复所有bug后，Patch号加一，返回第二步 + * 编译这个版本的python wheel包，并发布到pypi。 + * 由于pypi.python.org目前遵循[严格的命名规范PEP 513](https://www.python.org/dev/peps/pep-0513)，在使用twine上传之前，需要重命名wheel包中platform相关的后缀，比如将`linux_x86_64`修改成`manylinux1_x86_64`。 + * pypi上的package名称为paddlepaddle和paddlepaddle_gpu，如果要上传GPU版本的包，需要修改build/python/setup.py中，name: "paddlepaddle_gpu"并重新打包wheel包：`python setup.py bdist_wheel`。 + * 上传方法： + ``` + cd build/python + pip install twine + twine upload dist/[package to upload] + ``` +1. 第三步完成后，将`release/版本号`分支合入master分支，并删除`release/版本号`分支。将master分支的合入commit打上tag，tag为`版本号`。同时再将`master`分支合入`develop`分支。最后删除`release/版本号`分支。 +1. 编译master分支的Docker发行镜像，发布到dockerhub。编译ubuntu的deb包，发布到github release页面 +1. 协同完成Release Note的书写 + + +需要注意的是: + +* `release/版本号`分支一旦建立，一般不允许再从`develop`分支合入`release/版本号`。这样保证`release/版本号`分支功能的封闭，方便测试人员测试PaddlePaddle的行为。 +* 在`release/版本号`分支存在的时候，如果有bugfix的行为，需要将bugfix的分支同时merge到`master`, `develop`和`release/版本号`这三个分支。 + +## PaddlePaddle 分支规范 + +PaddlePaddle开发过程使用[git-flow](http://nvie.com/posts/a-successful-git-branching-model/)分支规范，并适应github的特性做了一些区别。 + +* PaddlePaddle的主版本库遵循[git-flow](http://nvie.com/posts/a-successful-git-branching-model/)分支规范。其中: + * `master`分支为稳定(stable branch)版本分支。每一个`master`分支的版本都是经过单元测试和回归测试的版本。 + * `develop`分支为开发(develop branch)版本分支。每一个`develop`分支的版本都经过单元测试，但并没有经过回归测试。 + * `release/版本号`分支为每一次Release时建立的临时分支。在这个阶段的代码正在经历回归测试。 + +* 其他用户的fork版本库并不需要严格遵守[git-flow](http://nvie.com/posts/a-successful-git-branching-model/)分支规范，但所有fork的版本库的所有分支都相当于特性分支。 + * 建议，开发者fork的版本库使用`develop`分支同步主版本库的`develop`分支 + * 建议，开发者fork的版本库中，再基于`develop`版本fork出自己的功能分支。 + * 当功能分支开发完毕后，向PaddlePaddle的主版本库提交`Pull Reuqest`，进而进行代码评审。 + * 在评审过程中，开发者修改自己的代码，可以继续在自己的功能分支提交代码。 + +* BugFix分支也是在开发者自己的fork版本库维护，与功能分支不同的是，BugFix分支需要分别给主版本库的`master`、`develop`与可能有的`release/版本号`分支，同时提起`Pull Request`。 + +## PaddlePaddle回归测试列表 + +本列表说明PaddlePaddle发版之前需要测试的功能点。 + +### PaddlePaddle Book中所有章节 + +PaddlePaddle每次发版本首先要保证PaddlePaddle Book中所有章节功能的正确性。功能的正确性包括验证PaddlePaddle目前的`paddle_trainer`训练和纯使用`Python`训练模型正确性。 + +| | 新手入门章节 | 识别数字 | 图像分类 | 词向量 | 情感分析 | 语意角色标注 | 机器翻译 | 个性化推荐 | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | +| API.V2 + Docker + GPU | | | | | | | | | +| API.V2 + Docker + CPU | | | | | | | | | +| `paddle_trainer` + Docker + GPU | | | | | | | | | +| `paddle_trainer` + Docker + CPU | | | | | | | | | +| API.V2 + Ubuntu + GPU | | | | | | | | | +| API.V2 + Ubuntu + CPU | | | | | | | | | +| `paddle_trainer` + Ubuntu + GPU | | | | | | | | | +| `paddle_trainer` + Ubuntu + CPU | | | | | | | | | diff --git a/doc/_sources/design/scope.md.txt b/doc/_sources/design/scope.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..4da76eebb74abcd26ec2b8671399e6bc4fb58574 --- /dev/null +++ b/doc/_sources/design/scope.md.txt @@ -0,0 +1,124 @@ +# Design of Scope in Paddle + +## Overview + +Scope is an important concept in programming languages, which defines a program region that a set of bindings between names and entities applies. In a specific scope, a valid name is uniquely associated with an entity, such as a variable. And in another scope, this name may refer to other entity or nothing at all. It clearly restricts the visibility and validity of names in a program. Hence **Scope** is introduced to PaddlePaddle to manage variables in context. But different from the original abstract concept, Scope now becomes an object with two important attributes: + +- Scope is an association of a name to variable. +- Variables in a parent scope can be retrieved from local scope. + +A detailed explanation of these two attributes goes as following. + + +## Scope is an association of a name to variable. + +Scope is an association of a name to variable. All variables belong to `Scope`. You need to specify a scope to run a Net, i.e., `net.Run(&scope)`. One net can run in different scopes and update different variable in the scope. + + +1. Scope only contains a map of a name to variable. + + All parameters, data, states in a Net should be variables and stored inside a scope. Each op should get inputs and outputs to do computation from a scope, such as data buffer, state (momentum) etc. + +1. Variable can only be created by Scope and a variable can only be got from Scope. User cannot create or get a variable outside a scope. This is a constraints of our framework, and will keep our framework simple and clear. + +1. Scope only contains methods that are used to Create and Get Variables. Scope do not contain Operators and have no information to run them. + `Net` is designed to drive the computation and Scope only contains a map of variables. There is no computation logic inside a `Scope`. Scope just handles the lifetime management of variables. + - `Create` is used to create a Variable by its name and add the mapping relation. + - `Get` is used to find a Variable by name. + +1. Every variable only belongs to one certain Scope. + + Variable can not belong to many scopes. If you want to use variables from parent scope, you can use `parent scope`. + +1. Scope should destruct all Variables inside it when itself is destructed. User can never store `Variable` pointer somewhere else. + + Because Variable can only be got from Scope. When destroying Scope, we also need to destroy all the Variables in it. If user store `Variable` pointer to private data member or some global variable, the pointer will be an invalid pointer when associated `Scope` is destroyed. + +```cpp +class Scope { + public: + Variable* Var(const std::string& name); + const Variable* FindVar(const std::string& name) const; + + private: + std::unordered_map> vars_; +}; +``` + + +## Parent scope and local scope + +Just like [scope](https://en.wikipedia.org/wiki/Scope_(computer_science)) in programming languages, `Scope` in the neural network can also be a local scope. There are two attributes about local scope. + +1. We can create local variables in a local scope. When that local scope is destroyed, all local variables should also be destroyed. +2. Variables in a parent scope can be retrieved from local scopes of that parent scope, i.e., when user get a variable from a scope, it will try to search this variable in current scope. If there is no such variable in the local scope, `scope` will keep searching from its parent, until the variable is found or there is no parent. + +```cpp +class Scope { + public: + Scope(const std::shared_ptr& scope): parent_(scope) {} + + Variable* FindVar(const std::string& name) const { + auto it = vars_.find(name); + if (it != vars_.end()) { + return it->second.get(); + } else if (parent_ != nullptr) { + return parent_->FindVar(name); + } else { + return nullptr; + } + } + + private: + std::shared_ptr parent_ {nullptr}; +}; +``` + +In `Scope` class, there is a private data member called `parent_`. `parent_` is a smart pointer to its parent scope. When user `Get` a variable by its `name`, the `name` will be searched inside the current scope. If the variable cannot be found locally and parent scope is not a `nullptr`, the variable will be searched inside that parent scope. `parent_` pointer's default value is `nullptr`. It means that the scope is a global scope when `parent_` is nullptr. + +A local scope is very useful when we implement Recurrent Neural Network. Each timestep of an RNN should be a `Net`. Each `Net` of timestep (`StepNet` for short) should use an independent local scope. Just like variables in a while loop is inside a local scope in programming languages. By using a single `StepNet` and changing local scope, we can implement an RNN easily. + +# Interface Design + +```cpp +class Variable { + private: + Variable() = default; + friend class Scope; +}; + +class Scope { + private: + Scope(const std::shared_ptr& parent = nullptr); + + public: + static std::shared_ptr Create(const std::shared_ptr& parent = nullptr); + + // return nullptr if not found. + Variable* FindVar(const std::string& name) const; + + // return if already contains same name variable. + Variable* Var(const std::string& name); + + private: + std::shared_ptr parent_; + std::unordered_map> vars_; +}; +``` +## Only scope can create a variable + +To ensure `only scope can create a variable`, we should mark `Variable`'s constructor as a private member function, and Scope is a friend class of Variable. And then only `Var` can construct `Variable`. + +## When scope destroyed, all variables inside this scope should be destroyed together + +The scope hold unique pointers for all variables. User can `FindVar` from scope, but he should not hold this pointer as a member variable. Because when scope is destroyed, all variables inside this scope will be destroyed together. + +## Sharing a parent scope + +Local scope contains a `parent_` pointer. It is a linked-list for scopes. Using a `shared_ptr` because when a local scope is using, its parents cannot be destroyed. + +Also, as the parent scope is a `shared_ptr`, we can only `Create()` a scope shared pointer. We cannot construct a scope variable, because it cannot be passed to other scope as `parent` pointer. + +## Orthogonal interface + +`FindVar` will return `nullptr` when `name` is not found. It can be used as `Contains` method. `Var` will return an `Error` when there is a name conflict locally. Combine `FindVar` and `Var`, we can implement `Var` easily. diff --git a/doc/_sources/design/selected_rows.md.txt b/doc/_sources/design/selected_rows.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..1a98839a957612b91b2276b58818623ecc62d1d5 --- /dev/null +++ b/doc/_sources/design/selected_rows.md.txt @@ -0,0 +1,74 @@ +# Design Doc: Selected Rows + +`SelectedRows` is a type of sparse tensor data type, which is designed to support `embedding` operators. The gradient of embedding table is a sparse tensor. Only a few rows are non-zero values in this tensor. It is straight-forward to represent a sparse tensor by the following sparse tensor data structure: + +```cpp +class SelectedRows { + private: + vector rows_; + Tensor value_; + int height_; +}; +``` + +The field `height_` is the first dimension of `SelectedRows`. The `rows` are the indices of the non-zero rows of `SelectedRows`. The `value_` field is an N-dim tensor of shape `[rows.size() /* NUM_ROWS */, ...]`, which supplies values for each row. The dimension of `SelectedRows` satisfies `[height_] + value_.shape[1:]`. + +Suppose that a SelectedRows-typed variable `x` has many rows, but only two of them have values -- row 73 is `[1, 2]` and row 84 is `[3, 4]`, the `SelectedRows` representation would be: + +``` +x = SelectedRow { + rows = [73, 84], + value = [[1, 2], [3,4]] +} +``` + + +## SelectedRows in Protobuf + +`SelectedRows` is a type of `Variable`. `VarDesc` in protobuf should describe the `SelectedRows` information. Only the tensor dimension of a `SelectedRows` will be described in compile-time because the `rows_` and `value_` are dependent on the training data. +So we use `TensorDesc` to unify `data_type` and `dims`. A LodTensorDesc contains a `TensorDesc` and `lod_level`. The description of `SelectedRows` is a Tensor description. + +```proto +message TensorDesc { + required DataType data_type = 1; + repeated int64 dims = 2; // [UNK, 640, 480] is saved as [-1, 640, 480] +} + +message LodTensorDesc { + required TensorDesc tensor = 1; + optional int lod_level = 2; +} + +message VarDesc { + required string name = 1; + enum VarType { + LOD_TENSOR = 0; + SELECTED_ROWS = 1; + } + required VarType type = 2; + optional LodTensorDesc lod_desc = 3; + optional TensorDesc selected_rows_desc = 4; + optional bool persistable = 5 [ default = false ]; +} +``` + +## InferShape for Selected Rows + +Just like `LoD` information, `InferShape` method will infer the output tensor type as well. The operator should decide whether its output is a `SelectedRows` or `Dense` tensor. + +For example, the gradient operator of `TableLookup` will always generate `SelectedRows`. Its `InferShape` method should be like following + +```cpp +void TableLookupGrad::InferShape(context) { + ... + context.SetDataType("Embedding.Grad", kSelectedRows); +} +``` + + +## Sparse Operators + +There are several operators that need to be written to support `SelectedRows`. These are: + +1. Operators which generate `SelectedRows` gradient. e.g. Gradient of `TableLookupOp`. +2. Optimize operators which support `SelectedRows` gradient. e.g. `SGD` or `AdaGrad` for `SelectedRows`. However, there should be only one `SGD` operator. `OpWithKernel::Run` should select a suitable kernel for both `dense` tensor or `SelectedRows`. diff --git a/doc/_sources/design/simple_op_design.md.txt b/doc/_sources/design/simple_op_design.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..c7aeed7f9b4637e1c29d530f37b42d12500af82f --- /dev/null +++ b/doc/_sources/design/simple_op_design.md.txt @@ -0,0 +1,202 @@ +## Interaction between C++ and Python + +Users employ API in Python to describe their own network, however, the network construction actually happens in C++. so Protobuf is introduced to send the message between Python and C++. + +The Interaction between Python and C++ can be simplified as two steps: + +1. C++ tells Python how many Ops there are, and what parameter do users need to offer to initialize a new Op. Python then builds API for each Op at compile time. + +2. Users invoke APIs built by Python and provide necessary parameters. These parameters will be sent to C++ for finishing the Op construction task. + +### Message from C++ to Python + +We define a Protobuf message class `OpProto` to hold message needed in the first step. What should an `OpProto` contain? This question is equivalent to “What message do we need to offer, to build a Python API which is legal and user oriented and can use to describe a whole Op.” + +Following message are necessary: + +1. Op's name, and its simple comment. +2. Input and output variable number; each variable's name, type, and comment. +3. Op's attributes; each attribute includes name, type, comment, **default value** and **value range**. + +So `OpProto` can be defined as follows: + +```proto +enum AttrType { + INT = 1; + FLOAT = 2; + STRING = 3; + INTS = 4; + FLOATS = 5; + STRINGS = 6; +}; + +message AttrValue { + AttrType type = 1; + optional int iv = 2; + optional float fv = 3; + optional string sv = 4; + repeated int ivs = 5; + repeated float fvs = 6; + repeated string svs = 7; +}; + +message AttrProto { + required string name = 1; + required string comment = 2; + required AttrType type = 3; +}; + +message VarProto { + required string name = 1; + required string comment = 2; + required bool is_tensor = 3; +}; + +message OpProto { + repeated VarProto inputs = 1; + repeated VarProto outputs = 2; + repeated AttrProto attrs = 3; + required string type = 4; + required string comment = 5; +}; +``` + +To generate Python code automatically: + +```python +def create_python_ops_creatation_functions(): + op_protos = paddle.framework.OpRegistry.get_all_op_proto() + for type_name in op_protos: + op_proto = op_protos[type_name] + def __impl__(**kwargs): # User must use key word args in Paddle API + inputs = [kwargs.get(ipt.name, "") for ipt in op_proto.inputs] + outputs = [kwargs.get(opt.name, "") for opt in op_proto.outputs] + attrs = [cast_to_op_attr(attr, kwargs.get(attr.name, None)) for attr in op_proto.attrs] + opdesc = （input, outputs, type_name, attrs） + return paddle.framework.OpRegistry.CreateOp(opdesc) + __impl__.__doc__ = create_doc_string(op_proto) + globals()[type_name] = __impl__ + +create_python_ops_creatation_functions() +``` + +### Message from Python to C++ + +To hold message needed in the above second step, we define Protobuf message class `OpDesc`. It is used to hold user-specified parameters in Op describing. + +```proto +message OpDesc { + required string type = 1; + repeated string inputs = 2; + repeated string outputs = 3; + map attrs = 4; +}; +``` + +## OpProto Register + +Every Op has its own `OpProto`. For using convenience, we need to register them and record all their messages. For each `Op` class, we define a corresponding `OpMaker` class, in whose constructor we implement the `OpProto`'s building process. `OpMaker`'s constructor will be invoked by another function `OpRegistry::RegisterOp()`. + +```cpp +class OpProtoMaker { +public: + OpProtoMaker(OpProto* proto): proto_(proto) {} +protected: + OpProto* proto_; + void AddInput(const std::string& name, const std::string& desc) {...} + void AddAttr(const std::string& name, const std::string& desc, TypeId type) {...} + void AddComment(const std::string& comment) { ... } +}; + +class OpRegistry { +public: + using OpCreator = std::function; + + template + static void RegisterOp(const std::string& name) { + gCreators_[name] = [](const OpDesc& desc) { + return new OpType(desc); + }; + OpProto& opProto = gProtos_[name]; + OpMaker()(&opProto); + } + + static map gCreators_; + static map gProtos_; +}; + +template +class OpRegister { + public: + OpRegister(std::string type) { + OpRegistry::RegisterOp(type); + } +}; + +#define REGISTER_OP(op_class, op_maker_class, type_name) \ + class op_class##Register { \ + private: \ + const static OpRegister<#op_class, #op_maker_class> reg; \ + }; \ + const Register op_class##Register::reg(#type_name); + +class CosineOp { +// ... +} + +struct CosineOpProtoMaker : public OpProtoMaker { + CosineOpProtoMaker(OpProto* proto) : OpProtoMaker(proto) { + AddInput("input", "input of cosine op"); + AddAttr("scale", "scale of cosine op", float).Default(1.0).GreaterThan(0.0); + AddType("cos"); + AddComment("This is cos op"); + } +} + +REGISTER_OP(CosineOp, CosineOpProtoMaker, cos); +``` + +In `REGISTER_OP(CosineOp, CosineOpProtoMaker, cos)`, we register not only `CosineOp` but also `CosineOpProto`. As fields of `CosineOpProto`, the default value and value range of `scale` are also registered here. + +## Python API + +Python APIs are divided into two types, high-level API and low-level API. + +### High-Level API + +High-level API is called by users directly, so it should keep its style consistent with existing V2 APIs. + +Here is a sample about how a define a fc layer: + +```python +hd = fc_layer(input=data, size=56, with_bias=True, activation="sigmoid"); +``` + +`hd` is the output of `fc_layer` and it's a `variable`. It can be further sent into other layers as input. + +The definition of `fc_layer()`: + +```python +def fc_layer(input, size, with_bias, activation): + attr_map = {"size":size} + check_attrs(attr_map) + w = make_variable('w') + if with_bias: + b = make_variable('b') + else: + b = None + fc_output = make_variable('fc_output'); + fc_op(input, w, b, fc_output, attr_map) + act_output = make_variable('sigmod_output'); + if activation == "sigmod": + sigmod_op(fc_output, act_output); + elif: + # ... + return act_output; +``` + +### Low Leval API + +In above sample, `fc_op` and `sigmod_op` are low-level API. They build `OpDesc` and invoke corresponding C++ code. + +*TODO* diff --git a/doc/_sources/design/tensor_array.md.txt b/doc/_sources/design/tensor_array.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..37e4f7b90f94fa3eb015e733999cd84c96b2239c --- /dev/null +++ b/doc/_sources/design/tensor_array.md.txt @@ -0,0 +1,271 @@ +# Design for TensorArray +This design doc presents the necessity of a new C++ class `TensorArray`. +In addition to the very simple C++ implementation + +```c++ +class TensorArray { + public: + explicit TensorArray(const LoDTensor&); + explicit TensorArray(size_t size); + + private: + vector values_; +}; +``` + +We also need to expose it to PaddlePaddle's Python API, +because users would want to use it with our very flexible operators `WhileLoop`. +An example for a RNN based on dynamic operators is + +```python +input = pd.data(...) +num_steps = Var(12) + +TensorArray states(size=num_steps) +TensorArray step_inputs(unstack_from=input) +TensorArray step_outputs(size=num_steps) + +W = Tensor(...) +U = Tensor(...) +default_state = some_op() + +step = Var(1) + +wloop = paddle.create_whileloop(loop_vars=[step]) +with wloop.frame(): + wloop.break_if(pd.equal(step, num_steps) + pre_state = states.read(step-1, default_state) + step_input = step_inputs.read(step) + state = pd.sigmoid(pd.matmul(U, pre_state) + pd.matmul(W, step_input)) + states.write(step, state) + step_outputs.write(step, state) # output state + step.update(state+1) + +output = step_outputs.stack() +``` + +## Background +Steps are one of the core concepts of RNN. In each time step of RNN, there should be several input segments, states, and output segments; all these components act like arrays, for example, call `states[step_id]` will get the state in `step_id`th time step. + +An RNN can be implemented with the following pseudocode + +```c++ +Array states; +Array input_segments; +Array output_segments; +Parameter W, U; + +step = 1 +seq_len = 12 +while_loop { + if (step == seq_len) break; + states[step] = sigmoid(W * states[step-1] + U * input_segments[step]); + output_segments[step] = states[step] // take state as output + step++; +} +``` +According to the [RNN roadmap](https://github.com/PaddlePaddle/Paddle/issues/4561), there are several different RNNs that PaddlePaddle will eventually support. + +Currently, the basic RNN implementation supported by PaddlePaddle is the `recurrent_op` which takes tensors as input and splits them into `input_segments`. + + +Since a tensor cannot store variable-length sequences directly, PaddlePaddle implements the tensor with level of details (`LoDTensor` for short). +Segmenting the `LoDTensor` is much more complicated than splitting a tensor, that makes it necessary to refactor the `recurrent_op` with `LoDTensor` segmenting support. + +As the next step in RNN support, `dynamic_recurrent_op` should be introduced to handle inputs with variable-length sequences. + +The implementation is similar to `recurrent_op`. +The key difference is the way **the original input `LoDTensors` and outupts are split to get the `input_segments` and the `output_segments`.** + + +Though it can't be built over `recurrent_op` or `dynamic_recurrent_op` directly, +the logic behind splitting a tensor or a LoD tensor into `input_segments` remains the same. + +## Why `TensorArray` +The logic behind splitting the inputs to segments, states and outputs is similar and can be shared in a seperate module. + +The array of `states`, `input_segments` and `output_segments` would be exposed to users when writing a dynamic RNN model similar to the above pseudo codes. + +So there should be an array-like container, which can store the segments of a tensor or LoD tensor. + +**This container can store an array of tensors and provides several methods to split a tensor or a LoD tensor** . +This is where the notion of `TensorArray` comes from. + +## Introduce TensorArray to uniform all the three RNNs +TensorArray as a new concept is borrowed from TensorFlow, +it is meant to be used with dynamic iteration primitives such as `while_loop` and `map_fn`. + +This concept can be used to support our new design of dynamic operations, and help to refactor some existing variant-sentence-related layers, +such as `recurrent_op`, `RecurrentGradientMachine`. + +In [our design for dynamic RNN](https://github.com/PaddlePaddle/Paddle/pull/4401), +`TensorArray` is used to segment inputs and store states in all time steps. +By providing some methods similar to a C++ array, +the definition of some state-based dynamic models such as RNN can be more natural and highly flexible. + +## Dynamic-operations on TensorArray + +`TensorArray` will be used directly when defining dynamic models, so some operators listed below should be implemented + +```python +# several helper operators for TensorArray +def tensor_array_stack(ta, tensor): + ''' + get a tensor array `ta`, return a packed `tensor`. + ''' + pass + +def tensor_array_unstack(tensor, ta): + ''' + get a `tensor`, unstack it and get a tensor array `ta`. + ''' + pass + +def tensor_array_write(ta, index, tensor, data_shared): + ''' + get a `tensor` and a scalar tensor `index`, write `tensor` into index-th + value of the tensor array `ta`. + `data_shared` is an attribute that specifies whether to copy or reference the tensors. + ''' + pass + +def tensor_array_read(ta, index, tensor): + ''' + get a tensor array `ta`, a scalar tensor `index`, read the index-th value of + `ta` and return as the `tensor`. + ''' + pass + +def tensor_array_size(ta, tensor): + ''' + get a tensor array `ta`, return the size of `ta` and return as the scalar `tensor`. + ''' + pass +``` + +It is trivial for users to use so many low-level operators, so some helper methods should be proposed in python wrapper to make `TensorArray` easier to use, +for example + +```python +class TensorArray: + def __init__(self, name): + self.name = name + self.desc = TensorArrayDesc() + + def stack(self, name=None): + ''' + Pack the values in a `TensorArray` into a tensor with rank one higher + than each tensor in `values`. + `stack` can be used to split tensor into time steps for RNN or whileloop. + + @name: str + the name of the variable to output. + ''' + tensor = Var(name) + tensor_array_stack(self.name, tensor) + return tensor + + def unstack(self, input): + ''' + Unpacks the given dimension of a rank-`R` tensor into rank-`(R-1)` tensors. + `unstack` can be used to concatenate all the time steps for RNN or whileloop. + + @input: str + the name of input tensor + ''' + tensor_array_unstack(tensor, self.name) + + def write(self, index, value, data_shared=True): + ''' + Write value into index of the TensorArray. + If `data_shared` is set to True, than the index-th value in TensorArray will + be shared with the tensor passed in. + + @index: str + name of a scalar tensor + @value: str + name of a tensor + @data_shared: bool + ''' + tensor_array_write(self.name, index, value, data_shared) + + def read(self, index, output): + ''' + Read the value at location `index` in the `TensorArray`. + + @index: str + name of a scalar tensor + @output: + name of a output variable + ''' + tensor_array_read(self.name, index, output) + + + def size(self, output): + ''' + Return the number of values. + + @output: str + name of a scalar tensor + ''' + tensor_array_size(self.name, output) +``` + +## LoDTensor-related Supports +The `RecurrentGradientMachine` in Paddle serves as a flexible RNN layer; it takes varience-length sequences as input, and output sequences too. + +Since each step of RNN can only take a tensor-represented batch of data as input, +some preprocess should be taken on the inputs such as sorting the sentences by their length in descending order and cut each word and pack to new batches. + +Such cut-like operations can be embedded into `TensorArray` as general methods called `unpack` and `pack`, +these two operations are similar to `stack` and `unstack` except that they operate on variable-length sequences formated as a LoD tensor rather than a tensor. + +Some definitions are like + +```python +def unpack(level): + ''' + Split LodTensor in some `level` and generate batches, if set `sort_by_length`, + will sort by length. + + Returns: + - a new `TensorArray`, whose values are LodTensors and represents batches + of data. + - an int32 Tensor, which stores the map from the new batch's indices to + original LoDTensor + ''' + pass + +def pack(level, indices_map): + ''' + Recover the original LoD-arranged LoDTensor with the values in a `TensorArray` + and `level` and `indices_map`. + ''' + pass +``` + +With these two methods, a varience-length sentence supported RNN can be implemented like + +```c++ +// input is the varient-length data +LodTensor sentence_input(xxx); +TensorArray ta; +Tensor indice_map; +Tensor boot_state = xxx; // to initialize rnn's first state +TensorArray::unpack(input, 1/*level*/, true/*sort_by_length*/, &ta, &indice_map); +TessorArray step_outputs; +TensorArray states; + +for (int step = 0; step = ta.size(); step++) { + auto state = states.read(step); + // rnnstep is a function which acts like a step of RNN + auto step_input = ta.read(step); + auto step_output = rnnstep(step_input, state); + step_outputs.write(step_output, true/*data_shared*/); +} + +// rnn_output is the final output of an rnn +LoDTensor rnn_output = ta.pack(ta, indice_map); +``` +the code above shows that by embedding the LoDTensor-related preprocess operations into `TensorArray`, +the implementation of a RNN that supports varient-length sentences is far more concise than `RecurrentGradientMachine` because the latter mixes all the codes together, hard to read and extend. diff --git a/doc/_sources/design/var_desc.md.txt b/doc/_sources/design/var_desc.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..0b2958c1b10ef6a6ce51aa75f61e15a7f2d94b3f --- /dev/null +++ b/doc/_sources/design/var_desc.md.txt @@ -0,0 +1,69 @@ +## Background +PaddlePaddle divides the description of neural network computation graph into two stages: compile time and runtime. + +PaddlePaddle use proto message to describe compile time graph because + +1. Computation graph should be able to be saved to a file. +1. In distributed training, the graph will be serialized and send to multiple workers. + +The computation graph is constructed by Data Node and Operation Node. The concept to represent them is in the table below. + +| |compile time|runtime| +|---|---|---| +|Data|VarDesc(proto)|Variable(cpp)| +|Operation|OpDesc(proto)|Operator(cpp)| + + +## Definition of VarDesc + +A VarDesc should have a name, and value. The are two kinds of variable type in compile time, they are `LoDTensor` and `SelectedRows`. + +```proto +message VarDesc { + required string name = 1; + enum VarType { + LOD_TENSOR = 0; + SELECTED_ROWS = 1; + } + required VarType type = 2; + optional LoDTensorDesc lod_desc = 3; + optional TensorDesc selected_rows_desc = 4; + optional bool persistable = 5 [ default = false ]; +} +``` + +## Definition of TensorDesc + +```proto +enum DataType { + BOOL = 0; + INT16 = 1; + INT32 = 2; + INT64 = 3; + FP16 = 4; + FP32 = 5; + FP64 = 6; +} + +message TensorDesc { + required DataType data_type = 1; + repeated int64 dims = 2; // [UNK, 640, 480] is saved as [-1, 640, 480] +} +``` + +A TensorDesc describes `SelectedRows` and `LoDTensor`. For details of `SelectedRows`, please reference [`SelectedRows`](./selected_rows.md). + +## Definition of LodTensorDesc + +```proto +message LoDTensorDesc { + required TensorDesc tensor = 1; + optional int lod_level = 2; +} +``` + +A LoDTensorDesc contains a tensor and a lod_level. + +## Definition of Variable in Python + +For Variable in Python, please reference [`Python API`](./python_api.md). diff --git a/doc/_sources/getstarted/basic_usage/index_en.rst.txt b/doc/_sources/getstarted/basic_usage/index_en.rst.txt deleted file mode 100644 index 6775da20c2f51000f305b095d40abd27b8fa6c0e..0000000000000000000000000000000000000000 --- a/doc/_sources/getstarted/basic_usage/index_en.rst.txt +++ /dev/null @@ -1,101 +0,0 @@ -Simple Linear Regression -======================== - -PaddlePaddle is a deep learning platform open-sourced by Baidu. With PaddlePaddle, you can easily train a classic neural network within a couple lines of configuration, or you can build sophisticated models that provide state-of-the-art performance on difficult learning tasks like sentiment analysis, machine translation, image caption and so on. - -Problem Background ------------------- - -Now, to give you a hint of what using PaddlePaddle looks like, let's start with a fundamental learning problem - `simple linear regression `_: you have observed a set of two-dimensional data points of ``X`` and ``Y``, where ``X`` is an explanatory variable and ``Y`` is corresponding dependent variable, and you want to recover the underlying correlation between ``X`` and ``Y``. Linear regression can be used in many practical scenarios. For example, ``X`` can be a variable about house size, and ``Y`` a variable about house price. You can build a model that captures relationship between them by observing real estate markets. - -Prepare the Data ------------------ - -Suppose the true relationship can be characterized as ``Y = 2X + 0.3``, let's see how to recover this pattern only from observed data. Here is a piece of python code that feeds synthetic data to PaddlePaddle. The code is pretty self-explanatory, the only extra thing you need to add for PaddlePaddle is a definition of input data types. - - .. code-block:: python - - # dataprovider.py - from paddle.trainer.PyDataProvider2 import * - import random - - # define data types of input: 2 real numbers - @provider(input_types=[dense_vector(1), dense_vector(1)],use_seq=False) - def process(settings, input_file): - for i in xrange(2000): - x = random.random() - yield [x], [2*x+0.3] - -Train a NeuralNetwork ----------------------- - -To recover this relationship between ``X`` and ``Y``, we use a neural network with one layer of linear activation units and a square error cost layer. Don't worry if you are not familiar with these terminologies, it's just saying that we are starting from a random line ``Y' = wX + b`` , then we gradually adapt ``w`` and ``b`` to minimize the difference between ``Y'`` and ``Y``. Here is what it looks like in PaddlePaddle: - - .. code-block:: python - - # trainer_config.py - from paddle.trainer_config_helpers import * - - # 1. read data. Suppose you saved above python code as dataprovider.py - data_file = 'empty.list' - with open(data_file, 'w') as f: f.writelines(' ') - define_py_data_sources2(train_list=data_file, test_list=None, - module='dataprovider', obj='process',args={}) - - # 2. learning algorithm - settings(batch_size=12, learning_rate=1e-3, learning_method=MomentumOptimizer()) - - # 3. Network configuration - x = data_layer(name='x', size=1) - y = data_layer(name='y', size=1) - y_predict = fc_layer(input=x, param_attr=ParamAttr(name='w'), size=1, act=LinearActivation(), bias_attr=ParamAttr(name='b')) - cost = mse_cost(input=y_predict, label=y) - outputs(cost) - -Some of the most fundamental usages of PaddlePaddle are demonstrated: - -- The first part shows how to feed data into PaddlePaddle. In general cases, PaddlePaddle reads raw data from a list of files, and then do some user-defined process to get real input. In this case, we only need to create a placeholder file since we are generating synthetic data on the fly. - -- The second part describes learning algorithm. It defines in what ways adjustments are made to model parameters. PaddlePaddle provides a rich set of optimizers, but a simple momentum based optimizer will suffice here, and it processes 12 data points each time. - -- Finally, the network configuration. It usually is as simple as "stacking" layers. Three kinds of layers are used in this configuration: - - **Data Layer**: a network always starts with one or more data layers. They provide input data to the rest of the network. In this problem, two data layers are used respectively for ``X`` and ``Y``. - - **FC Layer**: FC layer is short for Fully Connected Layer, which connects all the input units to current layer and does the actual computation specified as activation function. Computation layers like this are the fundamental building blocks of a deeper model. - - **Cost Layer**: in training phase, cost layers are usually the last layers of the network. They measure the performance of current model, and provide guidence to adjust parameters. - -Now that everything is ready, you can train the network with a simple command line call: - - .. code-block:: bash - - paddle train --config=trainer_config.py --save_dir=./output --num_passes=30 - - -This means that PaddlePaddle will train this network on the synthectic dataset for 30 passes, and save all the models under path ``./output``. You will see from the messages printed out during training phase that the model cost is decreasing as time goes by, which indicates we are getting a closer guess. - - -Evaluate the Model -------------------- - -Usually, a different dataset that left out during training phase should be used to evalute the models. However, we are lucky enough to know the real answer: ``w=2, b=0.3``, thus a better option is to check out model parameters directly. - -In PaddlePaddle, training is just to get a collection of model parameters, which are ``w`` and ``b`` in this case. Each parameter is saved in an individual file in the popular ``numpy`` array format. Here is the code that reads parameters from last pass. - - .. code-block:: python - - import numpy as np - import os - - def load(file_name): - with open(file_name, 'rb') as f: - f.read(16) # skip header for float type. - return np.fromfile(f, dtype=np.float32) - - print 'w=%.6f, b=%.6f' % (load('output/pass-00029/w'), load('output/pass-00029/b')) - # w=1.999743, b=0.300137 - - .. image:: parameters.png - :align: center - -Although starts from a random guess, you can see that value of ``w`` changes quickly towards 2 and ``b`` changes quickly towards 0.3. In the end, the predicted line is almost identical with real answer. - -There, you have recovered the underlying pattern between ``X`` and ``Y`` only from observed data. diff --git a/doc/_sources/getstarted/build_and_install/build_from_source_en.md.txt b/doc/_sources/getstarted/build_and_install/build_from_source_en.md.txt deleted file mode 100644 index 69f4501f370dcc9d603ec54a63d68568d66e832e..0000000000000000000000000000000000000000 --- a/doc/_sources/getstarted/build_and_install/build_from_source_en.md.txt +++ /dev/null @@ -1,222 +0,0 @@ -Installing from Sources -========================== - -* [1. Download and Setup](#download) -* [2. Requirements](#requirements) -* [3. Build on Ubuntu](#ubuntu) -* [4. Build on Centos](#centos) - - -## Download and Setup -You can download PaddlePaddle from the [github source](https://github.com/PaddlePaddle/Paddle). - -```bash -git clone https://github.com/PaddlePaddle/Paddle paddle -cd paddle -``` -## Requirements - -To compile the source code, your computer must be equipped with the following dependencies. - -- **Compiler**: GCC >= 4.8 or Clang >= 3.3 (AppleClang >= 5.1) and gfortran compiler -- **CMake**: CMake >= 3.0 (at least CMake 3.4 on Mac OS X) -- **BLAS**: MKL, OpenBlas or ATLAS -- **Python**: only support Python 2.7 - -**Note:** For CUDA 7.0 and CUDA 7.5, GCC 5.0 and up are not supported! -For CUDA 8.0, GCC versions later than 5.3 are not supported! - -### Options - -PaddlePaddle supports some build options. - - - - - - - - - - - - - - - - - - - - - - - - - - -
OptionalDescription
WITH_GPUCompile PaddlePaddle with NVIDIA GPU
WITH_AVXCompile PaddlePaddle with AVX intrinsics
WITH_DSOCompile PaddlePaddle with dynamic linked CUDA
WITH_TESTINGCompile PaddlePaddle with unit testing
WITH_SWIG_PYCompile PaddlePaddle with inference api
WITH_STYLE_CHECKCompile PaddlePaddle with style check
WITH_PYTHONCompile PaddlePaddle with python interpreter
WITH_DOUBLECompile PaddlePaddle with double precision
WITH_RDMACompile PaddlePaddle with RDMA support
WITH_TIMERCompile PaddlePaddle with stats timer
WITH_PROFILERCompile PaddlePaddle with GPU profiler
WITH_DOCCompile PaddlePaddle with documentation
WITH_COVERAGECompile PaddlePaddle with code coverage
COVERALLS_UPLOADPackage code coverage data to coveralls
ON_TRAVISExclude special unit test on Travis CI
- - -**Note:** - - The GPU version works best with Cuda Toolkit 8.0 and cuDNN v5. - - Other versions like Cuda Toolkit 7.0, 7.5 and cuDNN v3, v4 are also supported. - - **To utilize cuDNN v5, Cuda Toolkit 7.5 is prerequisite and vice versa.** - -As a simple example, consider the following: - -1. **BLAS Dependencies(optional)** - - CMake will search BLAS libraries from system. If not found, OpenBLAS will be downloaded, built and installed automatically. - To utilize preinstalled BLAS， you can simply specify MKL, OpenBLAS or ATLAS via `MKL_ROOT`, `OPENBLAS_ROOT` or `ATLAS_ROOT`. - - ```bash - # specify MKL - cmake .. -DMKL_ROOT= - # or specify OpenBLAS - cmake .. -DOPENBLAS_ROOT= - ``` - -2. **Doc Dependencies(optional)** - - To generate PaddlePaddle's documentation, install dependencies and set `-DWITH_DOC=ON` as follows: - - ```bash - pip install 'sphinx>=1.4.0' - pip install sphinx_rtd_theme recommonmark - - # install doxygen on Ubuntu - sudo apt-get install doxygen - # install doxygen on Mac OS X - brew install doxygen - - # active docs in cmake - cmake .. -DWITH_DOC=ON` - ``` - -## Build on Ubuntu 14.04 - -### Install Dependencies - -- **Paddle Dependencies** - - ```bash - # necessary - sudo apt-get update - sudo apt-get install -y git curl gcc g++ gfortran make build-essential automake - sudo apt-get install -y python python-pip python-numpy libpython-dev bison - sudo pip install 'protobuf==3.1.0.post1' - - # install cmake 3.4 - curl -sSL https://cmake.org/files/v3.4/cmake-3.4.1.tar.gz | tar -xz && \ - cd cmake-3.4.1 && ./bootstrap && make -j4 && sudo make install && \ - cd .. && rm -rf cmake-3.4.1 - ``` - -- **GPU Dependencies (optional)** - - To build GPU version, you will need the following installed: - - 1. a CUDA-capable GPU - 2. A supported version of Linux with a gcc compiler and toolchain - 3. NVIDIA CUDA Toolkit (available at http://developer.nvidia.com/cuda-downloads) - 4. NVIDIA cuDNN Library (availabel at https://developer.nvidia.com/cudnn) - - The CUDA development environment relies on tight integration with the host development environment, - including the host compiler and C runtime libraries, and is therefore only supported on - distribution versions that have been qualified for this CUDA Toolkit release. - - After downloading cuDNN library, issue the following commands: - - ```bash - sudo tar -xzf cudnn-7.5-linux-x64-v5.1.tgz -C /usr/local - sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn* - ``` - Then you need to set LD\_LIBRARY\_PATH, PATH environment variables in ~/.bashrc. - - ```bash - export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH - export PATH=/usr/local/cuda/bin:$PATH - ``` - -### Build and Install - -As usual, the best option is to create build folder under paddle project directory. - -```bash -mkdir build && cd build -``` - -Finally, you can build and install PaddlePaddle: - -```bash -# you can add build option here, such as: -cmake .. -DCMAKE_INSTALL_PREFIX= -# please use sudo make install, if you want to install PaddlePaddle into the system -make -j `nproc` && make install -# set PaddlePaddle installation path in ~/.bashrc -export PATH=/bin:$PATH -# install PaddlePaddle Python modules. -sudo pip install /opt/paddle/share/wheels/*.whl -``` -## Build on Centos 7 - -### Install Dependencies - -- **CPU Dependencies** - - ```bash - # necessary - sudo yum update - sudo yum install -y epel-release - sudo yum install -y make cmake3 python-devel python-pip gcc-gfortran swig git - sudo pip install wheel numpy - sudo pip install 'protobuf>=3.0.0' - ``` - -- **GPU Dependencies (optional)** - - To build GPU version, you will need the following installed: - - 1. a CUDA-capable GPU - 2. A supported version of Linux with a gcc compiler and toolchain - 3. NVIDIA CUDA Toolkit (available at http://developer.nvidia.com/cuda-downloads) - 4. NVIDIA cuDNN Library (availabel at https://developer.nvidia.com/cudnn) - - The CUDA development environment relies on tight integration with the host development environment, - including the host compiler and C runtime libraries, and is therefore only supported on - distribution versions that have been qualified for this CUDA Toolkit release. - - After downloading cuDNN library, issue the following commands: - - ```bash - sudo tar -xzf cudnn-7.5-linux-x64-v5.1.tgz -C /usr/local - sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn* - ``` - Then you need to set LD\_LIBRARY\_PATH, PATH environment variables in ~/.bashrc. - - ```bash - export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH - export PATH=/usr/local/cuda/bin:$PATH - ``` - -### Build and Install - -As usual, the best option is to create build folder under paddle project directory. - -```bash -mkdir build && cd build -``` - -Finally, you can build and install PaddlePaddle: - -```bash -# you can add build option here, such as: -cmake3 .. -DCMAKE_INSTALL_PREFIX= -# please use sudo make install, if you want to install PaddlePaddle into the system -make -j `nproc` && make install -# set PaddlePaddle installation path in ~/.bashrc -export PATH=/bin:$PATH -# install PaddlePaddle Python modules. -sudo pip install /opt/paddle/share/wheels/*.whl -``` diff --git a/doc/_sources/getstarted/build_and_install/build_from_source_en.rst.txt b/doc/_sources/getstarted/build_and_install/build_from_source_en.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..f194f84ce7c961bb8644d7c077a7c71730220ea2 --- /dev/null +++ b/doc/_sources/getstarted/build_and_install/build_from_source_en.rst.txt @@ -0,0 +1,159 @@ +Build from Sources +========================== + +.. _build_step: + +How To Build +---------------- + +PaddlePaddle mainly uses `CMake `_ and GCC, G++ as compile +tools. We recommend you to use our pre-built Docker image to run the build +to avoid installing dependencies by yourself. We have several build environment +Docker images `here `_ . + +If you choose not to use Docker image for your build, you need to install the +below `Compile Dependencies`_ before run the build. + +Then run: + +.. code-block:: bash + + git clone https://github.com/PaddlePaddle/Paddle.git + cd Paddle + # run the following command to build a CPU-Only binaries if you are using docker + docker run -it -v $PWD:/paddle -e "WITH_GPU=OFF" -e "WITH_TESTING=OFF" paddlepaddle/paddle_manylinux_devel:cuda8.0_cudnn5 bash -x /paddle/paddle/scripts/docker/build.sh + # else run these commands + mkdir build + cd build + cmake -DWITH_GPU=OFF -DWITH_TESTING=OFF .. + make + +When the compile finishes, you can get the output whl package under +build/python/dist, then you can choose to install the whl on local +machine or copy it to the target machine. + +.. code-block:: bash + + pip install build/python/dist/*.whl + + +.. _run_test: + +Run Tests +---------------- + +If you wish to run the tests, you may follow the below steps: + +When using Docker, set :code:`RUN_TEST=ON` and :code:`WITH_TESTING=ON` will run test immediately after the build. +Set :code:`WITH_GPU=ON` Can also run tests on GPU. + +.. code-block:: bash + + docker run -it -v $PWD:/paddle -e "WITH_GPU=OFF" -e "WITH_TESTING=ON" -e "RUN_TEST=ON" paddlepaddle/paddle_manylinux_devel:cuda8.0_cudnn5 bash -x paddle/paddle/scripts/docker/build.sh + +If you don't use Docker, just run ctest will start the tests: + +.. code-block:: bash + + mkdir build + cd build + cmake -DWITH_GPU=OFF -DWITH_TESTING=ON .. + make + ctest + # run a single test like test_mul_op + ctest -R test_mul_op + + +.. _compile_deps: + +Compile Dependencies +---------------- + +PaddlePaddle need the following dependencies when compiling, other dependencies +will be downloaded automatically. + +.. csv-table:: PaddlePaddle Compile Dependencies + :header: "Dependency", "Version", "Description" + :widths: 10, 15, 30 + + "CMake", ">=3.5", "" + "GCC", "4.8.2", "Recommend devtools2 for CentOS" + "Python", "2.7.x", "Need libpython2.7.so" + "pip", ">=9.0", "" + "numpy", "", "" + "SWIG", ">=2.0", "" + "Go", ">=1.8", "Optional" + + +.. _build_options: + +Build Options +---------------- + +Build options include whether build binaries for CPU or GPU, which BLAS +library to use etc. You may pass these settings when running cmake. +For detailed cmake tutorial please refer to `here `_ 。 + +.. _build_options_bool: + +Bool Type Options +---------------- + +You can add :code:`-D` argument to pass such options, like: + +.. code-block:: bash + + cmake .. -DWITH_GPU=OFF + +.. csv-table:: Bool Type Options + :header: "Option", "Description", "Default" + :widths: 1, 7, 2 + + "WITH_GPU", "Build with GPU support", "ON" + "WITH_C_API", "Build only CAPI", "OFF" + "WITH_DOUBLE", "Build with double precision", "OFF" + "WITH_DSO", "Dynamically load CUDA libraries", "ON" + "WITH_AVX", "Build with AVX support", "ON" + "WITH_PYTHON", "Build with integrated Python interpreter", "ON" + "WITH_STYLE_CHECK", "Check code style when building", "ON" + "WITH_TESTING", "Build unit tests", "ON" + "WITH_DOC", "Build documentations", "OFF" + "WITH_SWIG_PY", "Build Python SWIG interface for V2 API", "Auto" + "WITH_GOLANG", "Build fault-tolerant parameter server written in go", "ON" + "WITH_MKL", "Use MKL as BLAS library, else use OpenBLAS", "ON" + + +BLAS ++++++ + +PaddlePaddle supports `MKL `_ and +`OpenBlAS `_ as BLAS library。By default it uses MKL. +If you are using MKL and your machine supports AVX2, MKL-DNN will also be downloaded +and used, for more `details `_ . + +If you choose not to use MKL, then OpenBlAS will be used. + +CUDA/cuDNN ++++++++++++ + +PaddlePaddle will automatically find CUDA and cuDNN when compiling and running. +parameter :code:`-DCUDA_ARCH_NAME=Auto` can be used to detect SM architecture +automatically in order to speed up the build. + +PaddlePaddle can build with any version later than cuDNN v5.1, and we intend to +keep on with latest cuDNN versions. Be sure to run with the same version of cuDNN +you built. + +Pass Compile Options +++++++++++++++ + +You can pass compile options to use intended BLAS/CUDA/Cudnn libraries. +When running cmake command, it will search system paths like +:code:`/usr/lib:/usr/local/lib` and then search paths that you +passed to cmake, i.e. + +.. code-block:: bash + + cmake .. -DWITH_GPU=ON -DWITH_TESTING=OFF -DCUDNN_ROOT=/opt/cudnnv5 + +**NOTE: These options only take effect when running cmake for the first time, you need to clean the cmake cache or clean the build directory (** :code:`rm -rf` **) if you want to change it.** diff --git a/doc/_sources/getstarted/build_and_install/docker_install_en.rst.txt b/doc/_sources/getstarted/build_and_install/docker_install_en.rst.txt index 03df497506099d2fb758bd0ab437d2c082f2b537..d7acc7aeb744b19d83acb520d07c8551168dd096 100644 --- a/doc/_sources/getstarted/build_and_install/docker_install_en.rst.txt +++ b/doc/_sources/getstarted/build_and_install/docker_install_en.rst.txt @@ -1,270 +1,146 @@ -PaddlePaddle in Docker Containers +Run in Docker Containers ================================= -Docker container is currently the only officially-supported way to -running PaddlePaddle. This is reasonable as Docker now runs on all -major operating systems including Linux, Mac OS X, and Windows. -Please be aware that you will need to change `Dockers settings -`_ to make full use -of your hardware resource on Mac OS X and Windows. +Run PaddlePaddle in Docker container so that you don't need to care about +runtime dependencies, also you can run under Windows system. You can get +tutorials at `here `_ . -Working With Docker -------------------- +If you are using Windows, please refer to +`this `_ +tutorial to start running docker under windows. -Docker is simple as long as we understand a few basic concepts: +After you've read above tutorials you may proceed the following steps. -- *image*: A Docker image is a pack of software. It could contain one or more programs and all their dependencies. For example, the PaddlePaddle's Docker image includes pre-built PaddlePaddle and Python and many Python packages. We can run a Docker image directly, other than installing all these software. We can type +.. _docker_pull: - .. code-block:: bash - - docker images +Pull PaddlePaddle Docker Image +------------------------------ - to list all images in the system. We can also run +Run the following command to download the latest Docker images: .. code-block:: bash - - docker pull paddlepaddle/paddle:0.10.0rc2 - to download a Docker image, paddlepaddle/paddle in this example, - from Dockerhub.com. + docker pull paddlepaddle/paddle -- *container*: considering a Docker image a program, a container is a - "process" that runs the image. Indeed, a container is exactly an - operating system process, but with a virtualized filesystem, network - port space, and other virtualized environment. We can type +For users in China, we provide a faster mirror: .. code-block:: bash - docker run paddlepaddle/paddle:0.10.0rc2 - - to start a container to run a Docker image, paddlepaddle/paddle in this example. + docker pull docker.paddlepaddle.org/paddle -- By default docker container have an isolated file system namespace, - we can not see the files in the host file system. By using *volume*, - mounted files in host will be visible inside docker container. - Following command will mount current dirctory into /data inside - docker container, run docker container from debian image with - command :code:`ls /data`. +Download GPU version images: .. code-block:: bash - docker run --rm -v $(pwd):/data debian ls /data - -Usage of CPU-only and GPU Images ----------------------------------- - -We package PaddlePaddle's compile environment into a Docker image, -called the develop image, it contains all compiling tools that -PaddlePaddle needs. We package compiled PaddlePaddle program into a -Docker image as well, called the production image, it contains all -runtime environment that running PaddlePaddle needs. For each version -of PaddlePaddle, we release both of them. Production image includes -CPU-only version and a CUDA GPU version and their no-AVX versions. - -We put the docker images on `dockerhub.com -`_. You can find the -latest versions under "tags" tab at dockerhub.com. If you are in -China, you can use our Docker image registry mirror to speed up the -download process. To use it, please replace all paddlepaddle/paddle in -the commands to docker.paddlepaddle.org/paddle. - -1. Production images, this image might have multiple variants: - - - GPU/AVX：:code:`paddlepaddle/paddle:-gpu` - - GPU/no-AVX：:code:`paddlepaddle/paddle:-gpu-noavx` - - CPU/AVX：:code:`paddlepaddle/paddle:` - - CPU/no-AVX：:code:`paddlepaddle/paddle:-noavx` - - Please be aware that the CPU-only and the GPU images both use the - AVX instruction set, but old computers produced before 2008 do not - support AVX. The following command checks if your Linux computer - supports AVX: - - .. code-block:: bash - - if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi - - - To run the CPU-only image as an interactive container: - - .. code-block:: bash - - docker run -it --rm paddlepaddle/paddle:0.10.0rc2 /bin/bash - - Above method work with the GPU image too -- the recommended way is - using `nvidia-docker `_. - - Please install nvidia-docker first following this `tutorial - `_. - - Now you can run a GPU image: - - .. code-block:: bash - - nvidia-docker run -it --rm paddlepaddle/paddle:0.10.0rc2-gpu /bin/bash - -2. development image :code:`paddlepaddle/paddle:-dev` - - This image has packed related develop tools and runtime - environment. Users and developers can use this image instead of - their own local computer to accomplish development, build, - releasing, document writing etc. While different version of paddle - may depends on different version of libraries and tools, if you - want to setup a local environment, you must pay attention to the - versions. The development image contains: - - - gcc/clang - - nvcc - - Python - - sphinx - - woboq - - sshd - - Many developers use servers with GPUs, they can use ssh to login to - the server and run :code:`docker exec` to enter the docker - container and start their work. Also they can start a development - docker image with SSHD service, so they can login to the container - and start work. - - -Train Model Using Python API ----------------------------- - -Our official docker image provides a runtime for PaddlePaddle -programs. The typical workflow will be as follows: - -Create a directory as workspace: - -.. code-block:: bash - - mkdir ~/workspace - -Edit a PaddlePaddle python program using your favourite editor - -.. code-block:: bash - - emacs ~/workspace/example.py + docker pull paddlepaddle/paddle:latest-gpu + docker pull docker.paddlepaddle.org/paddle:latest-gpu -Run the program using docker: +Choose between different BLAS version: -.. code-block:: bash - - docker run --rm -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2 python /workspace/example.py - -Or if you are using GPU for training: - -.. code-block:: bash - - nvidia-docker run --rm -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2-gpu python /workspace/example.py - -Above commands will start a docker container by running :code:`python -/workspace/example.py`. It will stop once :code:`python -/workspace/example.py` finishes. - -Another way is to tell docker to start a :code:`/bin/bash` session and -run PaddlePaddle program interactively: - -.. code-block:: bash - - docker run -it -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2 /bin/bash - # now we are inside docker container - cd /workspace - python example.py - -Running with GPU is identical: - -.. code-block:: bash - - nvidia-docker run -it -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2-gpu /bin/bash - # now we are inside docker container - cd /workspace - python example.py - - -Develop PaddlePaddle or Train Model Using C++ API ---------------------------------------------------- - -We will be using PaddlePaddle development image since it contains all -compiling tools and dependencies. - -1. Build PaddlePaddle develop image + .. code-block:: bash - Use following command to build PaddlePaddle develop image: + # image using MKL by default + docker pull paddlepaddle/paddle + # image using OpenBLAS + docker pull paddlepaddle/paddle:latest-openblas - .. code-block:: bash - git clone https://github.com/PaddlePaddle/Paddle.git && cd Paddle - docker build -t paddle:dev . +If you want to use legacy versions, choose a tag from +`DockerHub `_ +and run: -2. Build PaddlePaddle production image + .. code-block:: bash - There are two steps for building production image, the first step is to run: + docker pull paddlepaddle/paddle:[tag] + # i.e. + docker pull docker.paddlepaddle.org/paddle:0.10.0-gpu - .. code-block:: bash +.. _docker_run: - docker run -v $(pwd):/paddle -e "WITH_GPU=OFF" -e "WITH_AVX=OFF" -e "WITH_TEST=ON" paddle:dev +Launch your training program in Docker +------------------------------ - The above command will compile PaddlePaddle and create a Dockerfile for building production image. All the generated files are in the build directory. "WITH_GPU" controls if the generated production image supports GPU. "WITH_AVX" controls if the generated production image supports AVX. "WITH_TEST" controls if the unit test will be generated. +Assume that you have already written a PaddlePaddle program +named :code:`train.py` under directory :code:`/home/work` (refer to +`PaddlePaddleBook `_ +for more samples), then run the following command: - The second step is to run: + .. code-block:: bash - .. code-block:: bash + cd /home/work + docker run -it -v $PWD:/work paddlepaddle/paddle /work/train.py - docker build -t paddle:prod -f build/Dockerfile ./build +In the above command, :code:`-it` means run the container interactively; +:code:`-v $PWD:/work` means mount the current directory ($PWD will expand +to current absolute path in Linux) under :code:`/work` in the container. +:code:`paddlepaddle/paddle` to specify image to use; finnally +:code:`/work/train.py` is the command to run inside docker. - The above command will generate the production image by copying the compiled PaddlePaddle program into the image. +Also, you can go into the container shell, run or debug your code +interactively: -3. Run unit test + .. code-block:: bash + docker run -it -v $PWD:/work paddlepaddle/paddle /bin/bash + cd /work + python train.py - Following command will run unit test: +**NOTE: We did not install vim in the default docker image to reduce the image size, you can run** :code:`apt-get install -y vim` **to install it if you need to edit python files.** - .. code-block:: bash - - docker run -it -v $(pwd):/paddle paddle:dev bash -c "cd /paddle/build && ctest" +.. _docker_run_book: PaddlePaddle Book ------------------ -The Jupyter Notebook is an open-source web application that allows -you to create and share documents that contain live code, equations, -visualizations and explanatory text in a single browser. - -PaddlePaddle Book is an interactive Jupyter Notebook for users and developers. -We already exposed port 8888 for this book. If you want to +You can create a container serving PaddlePaddle Book using Jupyter Notebook in +one minute using Docker. PaddlePaddle Book is an interactive Jupyter Notebook +for users and developers.If you want to dig deeper into deep learning, PaddlePaddle Book definitely is your best choice. We provide a packaged book image, simply issue the command: -.. code-block:: bash + .. code-block:: bash - docker run -p 8888:8888 paddlepaddle/book + docker run -p 8888:8888 paddlepaddle/book Then, you would back and paste the address into the local browser: -.. code-block:: text + .. code-block:: text - http://localhost:8888/ + http://localhost:8888/ That's all. Enjoy your journey! +.. _docker_run_gpu: -Documentation -------------- +Train with Docker with GPU +------------------------------ -Paddle Docker images include an HTML version of C++ source code -generated using `woboq code browser -`_. This makes it easy -for users to browse and understand the C++ source code. +We recommend using +`nvidia-docker `_ +to run GPU training jobs. Please ensure you have latest +GPU driver installed before move on. -As long as we give the Paddle Docker container a name, we can run an -additional Nginx Docker container to serve the volume from the Paddle -container: + .. code-block:: bash -.. code-block:: bash + nvidia-docker run -it -v $PWD:/work paddledev/paddle:latest-gpu /bin/bash - docker run -d --name paddle-cpu-doc paddle: - docker run -d --volumes-from paddle-cpu-doc -p 8088:80 nginx +**NOTE: If you don't have nvidia-docker installed, try the following method to mount CUDA libs and devices into the container.** + .. code-block:: bash + + export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')" + export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}') + docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:latest-gpu + +**About AVX:** + +AVX is a kind of CPU instruction can accelerate PaddlePaddle's calculations. +The latest PaddlePaddle Docker image turns AVX on by default, so, if your +computer doesn't support AVX, you'll probably need to +`build <./build_from_source_en.rst>`_ with :code:`WITH_AVX=OFF`. -Then we can direct our Web browser to the HTML version of source code -at http://localhost:8088/paddle/ +The following command will tell you whether your computer supports AVX. + + .. code-block:: bash + + if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi diff --git a/doc/_sources/getstarted/build_and_install/index_en.rst.txt b/doc/_sources/getstarted/build_and_install/index_en.rst.txt index 1bfd4f75c0b9b82d61d28a30f03181f7be159f24..32d66d63dd5b2a30d5de4a088dc80b680830cb84 100644 --- a/doc/_sources/getstarted/build_and_install/index_en.rst.txt +++ b/doc/_sources/getstarted/build_and_install/index_en.rst.txt @@ -1,23 +1,34 @@ Install and Build ================= -Install PaddlePaddle ----------------------- +.. _install_steps: -.. toctree:: - :maxdepth: 1 +Install Steps +++++++++ + +You can choose either pip or Docker to complete your install: + +.. toctree:: + :maxdepth: 1 + + pip_install_en.rst + docker_install_en.rst + ../../howto/dev/build_en.md - docker_install_en.rst - ubuntu_install_en.rst Build from Source ----------------- .. warning:: - Please use :code:`deb` package or :code:`docker` image to install paddle. The building guide is used for hacking or contributing PaddlePaddle source code. + We recommend to directly install via above installation steps, you'll only need to build PaddlePaddle from source when you need a modifed binary. .. toctree:: :maxdepth: 1 build_from_source_en.md + +FAQ +++++++++++ + +`FAQ `_ diff --git a/doc/_sources/getstarted/build_and_install/pip_install_en.rst.txt b/doc/_sources/getstarted/build_and_install/pip_install_en.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..70f601a11c610e0a2b5dcc8b73d2c3ea19e195e1 --- /dev/null +++ b/doc/_sources/getstarted/build_and_install/pip_install_en.rst.txt @@ -0,0 +1,104 @@ +Install Using pip +================================ + +You can use current widely used Python package management +tool `pip `_ +to install PaddlePaddle. This method can be used in +most of current Linux systems or MacOS. + +.. _pip_install: + +Install Using pip +------------------------------ + +Run the following command to install PaddlePaddle on the current +machine, it will also download requirements. + + .. code-block:: bash + + pip install paddlepaddle + + +If you wish to install GPU version, just run: + + .. code-block:: bash + + pip install paddlepaddle-gpu + +If you wish to install the latest develop branch PaddlePaddle, +you can download the latest whl package from our CI system. Access +the below links, log in as guest, then click at the "Artifact" +tab, you'll find the download link of whl packages. + +If the links below shows up the login form, just click "Log in as guest" to start the download: + +.. image:: paddleci.png + :scale: 50 % + :align: center + +.. csv-table:: whl package of each version + :header: "version", "cp27-cp27mu", "cp27-cp27m", "C-API" + :widths: 1, 3, 3, 3 + + "cpu_avx_mkl", "`paddlepaddle-0.10.0-cp27-cp27mu-linux_x86_64.whl `_", "`paddlepaddle-0.10.0-cp27-cp27m-linux_x86_64.whl `_", "`paddle.tgz `_" + "cpu_avx_openblas", "`paddlepaddle-0.10.0-cp27-cp27mu-linux_x86_64.whl `_", "`paddlepaddle-0.10.0-cp27-cp27m-linux_x86_64.whl `_", "Not Available" + "cuda7.5_cudnn5_avx_mkl", "`paddlepaddle-0.10.0-cp27-cp27mu-linux_x86_64.whl `_", "`paddlepaddle-0.10.0-cp27-cp27m-linux_x86_64.whl `_", "`paddle.tgz `_" + "cuda8.0_cudnn5_avx_mkl", "`paddlepaddle-0.10.0-cp27-cp27mu-linux_x86_64.whl `_", "`paddlepaddle-0.10.0-cp27-cp27m-linux_x86_64.whl `_", "`paddle.tgz `_" + "cuda8.0_cudnn7_avx_mkl", "`paddlepaddle-0.10.0-cp27-cp27mu-linux_x86_64.whl `_", "`paddlepaddle-0.10.0-cp27-cp27m-linux_x86_64.whl `_", "`paddle.tgz `_" + +.. _pip_dependency: + +Runtime Dependency +------------------------------ + +PaddlePaddle installation packages (whl) does not only contain .py files, +but also binaries built from C++ code. We ensure that PaddlePaddle can +run on current mainline Linux distributions, like CentOS 6, Ubuntu 14.04 +and MacOS 10.12. + +PaddlePaddle whl packages are trying to satisfy +`manylinux1 `_ +standard, which uses CentOS 5 as default build environment. But CUDA libraries +seems only run on CentOS 6 at least, also, CentOS 5 is about to end its lifetime, +so we use CentOS 6 as default build environment. + +.. csv-table:: PaddlePaddle Runtime Deps + :header: "Dependency", "version", "description" + :widths: 10, 15, 30 + + "OS", "Linux, MacOS", "CentOS 6 or later，Ubuntu 14.04 or later，MacOS 10.12 or later" + "Python", "2.7.x", "Currently Python3 is not supported" + "libc.so", "GLIBC_2.7", "glibc at least include GLIBC_2.7 symbols" + "libstdc++.so", "GLIBCXX_3.4.11, CXXABI_1.3.3", "At least include GLIBCXX_3.4.11, CXXABI_1.3.3 symbols" + "libgcc_s.so", "GCC_3.3", "At least include GCC_3.3 symbols" + +.. _pip_faq: + +FAQ +------------------------------ + +- paddlepaddle*.whl is not a supported wheel on this platform. + + The main cause of this issue is that your current platform is + not supported. Please check that you are using Python 2.7 series. + Besides, pypi only supports manylinux1 standard, you'll need to + upgrade your pip to >9.0.0. Then run the below command: + + .. code-block:: bash + + pip install --upgrade pip + + If the problem still exists, run the following command: + + .. code-block:: bash + + python -c "import pip; print(pip.pep425tags.get_supported())" + + Then you'll get supported package suffixes, then check if it matches + the file name of the whl package. You can find default whl package at + `here `_ + + If your system supports linux_x86_64 but the whl package is manylinux1_x86_64, + you'll need to update pip to the latest version; If your system supports + manylinux1_x86_64 but the whl package is linux_x86_64 you can rename the + file to manylinux1_x86_64 suffix and then install. diff --git a/doc/_sources/getstarted/build_and_install/ubuntu_install_en.rst.txt b/doc/_sources/getstarted/build_and_install/ubuntu_install_en.rst.txt deleted file mode 100644 index ea8042085bf458be96e71017d229d88ad867695b..0000000000000000000000000000000000000000 --- a/doc/_sources/getstarted/build_and_install/ubuntu_install_en.rst.txt +++ /dev/null @@ -1,25 +0,0 @@ -Debian Package installation guide -================================= - -PaddlePaddle supports :code:`deb` pacakge. The installation of this :code:`deb` package is tested in ubuntu 14.04, but it should be support other debian based linux, too. - -There are four versions of debian package, :code:`cpu`, :code:`gpu`, :code:`cpu-noavx`, :code:`gpu-noavx`. And :code:`noavx` version is used to support CPU which does not contain :code:`AVX` instructions. The download url of :code:`deb` package is \: https://github.com/baidu/Paddle/releases/ - - -After downloading PaddlePaddle deb packages, you can use :code:`gdebi` install. - -.. code-block:: bash - - gdebi paddle-*.deb - -If :code:`gdebi` is not installed, you can use :code:`sudo apt-get install gdebi` to install it. - -Or you can use following commands to install PaddlePaddle. - -.. code-block:: bash - - dpkg -i paddle-*.deb - apt-get install -f - -And if you use GPU version deb package, you need to install CUDA toolkit and cuDNN, and set related environment variables(such as LD_LIBRARY_PATH) first. It is normal when `dpkg -i` get errors. `apt-get install -f` will continue install paddle, and install dependences. - diff --git a/doc/_sources/getstarted/index_en.rst.txt b/doc/_sources/getstarted/index_en.rst.txt index 9f771e93e8b63eb98e31ec12667bd1aa007af20e..d14e3f5c0cc90792fce9cb82e65da482c44dc433 100644 --- a/doc/_sources/getstarted/index_en.rst.txt +++ b/doc/_sources/getstarted/index_en.rst.txt @@ -1,9 +1,61 @@ GET STARTED ============ +.. _quick_install: + +Quick Install +---------------------- + +You can use pip to install PaddlePaddle with a single command, supports +CentOS 6 above, Ubuntu 14.04 above or MacOS 10.12, with Python 2.7 installed. +Simply run the following command to install: + + .. code-block:: bash + + pip install paddlepaddle + +If you need to install GPU version, run: + + .. code-block:: bash + + pip install paddlepaddle-gpu + +For more details about installation and build: + .. toctree:: :maxdepth: 1 build_and_install/index_en.rst -- `Deep Learning 101 `_ + +.. _quick_start: + +Quick Start +++++++++ + +Create a new file called housing.py, and paste this Python +code: + + + .. code-block:: python + + import paddle.v2 as paddle + + # Initialize PaddlePaddle. + paddle.init(use_gpu=False, trainer_count=1) + + # Configure the neural network. + x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13)) + y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear()) + + # Infer using provided test data. + probs = paddle.infer( + output_layer=y_predict, + parameters=paddle.dataset.uci_housing.model(), + input=[item for item in paddle.dataset.uci_housing.test()()]) + + for i in xrange(len(probs)): + print 'Predicted price: ${:,.2f}'.format(probs[i][0] * 1000) + +Run :code:`python housing.py` and voila! It should print out a list of predictions +for the test housing data. diff --git a/doc/_sources/howto/deep_model/rnn/index_en.rst.txt b/doc/_sources/howto/deep_model/rnn/index_en.rst.txt index 13a153b05c578e0af82ee29db5ea27fd4b6d6f59..7adc79873d699fdfd5a85034bcef964dd1f19132 100644 --- a/doc/_sources/howto/deep_model/rnn/index_en.rst.txt +++ b/doc/_sources/howto/deep_model/rnn/index_en.rst.txt @@ -1,2 +1,7 @@ RNN Models ========== + +.. toctree:: + :maxdepth: 1 + + rnn_config_en.rst diff --git a/doc/_sources/howto/deep_model/rnn/rnn_config_en.rst.txt b/doc/_sources/howto/deep_model/rnn/rnn_config_en.rst.txt index 73f5d5371fcd3ce95253cad47b0d8e738284441c..f92edd108ff5c10a31b5f181f0f6dcb7a3f119f3 100644 --- a/doc/_sources/howto/deep_model/rnn/rnn_config_en.rst.txt +++ b/doc/_sources/howto/deep_model/rnn/rnn_config_en.rst.txt @@ -3,34 +3,11 @@ RNN Configuration This tutorial will guide you how to configure recurrent neural network in PaddlePaddle. PaddlePaddle supports highly flexible and efficient recurrent neural network configuration. In this tutorial, you will learn how to: -- prepare sequence data for learning recurrent neural networks. - configure recurrent neural network architecture. - generate sequence with learned recurrent neural network models. -We will use vanilla recurrent neural network, and sequence to sequence model to guide you through these steps. The code of sequence to sequence model can be found at :code:`demo/seqToseq`. - -===================== -Prepare Sequence Data -===================== - -PaddlePaddle does not need any preprocessing to sequence data, such as padding. The only thing that needs to be done is to set the type of the corresponding type to input. For example, the following code snippets defines three input. All of them are sequences, and the size of them are :code:`src_dict`, :code:`trg_dict`, and :code:`trg_dict`: - -.. code-block:: python - - settings.input_types = [ - integer_value_sequence(len(settings.src_dict)), - integer_value_sequence(len(settings.trg_dict)), - integer_value_sequence(len(settings.trg_dict))] - - -Then at the :code:`process` function, each :code:`yield` function will return three integer lists. Each integer list is treated as a sequence of integers: - -.. code-block:: python - - yield src_ids, trg_ids, trg_ids_next - - -For more details description of how to write a data provider, please refer to :ref:`api_pydataprovider2` . The full data provider file is located at :code:`demo/seqToseq/dataprovider.py`. +We will use vanilla recurrent neural network, and sequence to sequence model to guide you through these steps. The code of sequence to sequence model can be found at `book/08.machine_translation `_ . +And the data preparation of this model can be found at `python/paddle/v2/dataset/wmt14.py `_ =============================================== Configure Recurrent Neural Network Architecture @@ -42,7 +19,7 @@ Simple Gated Recurrent Neural Network Recurrent neural network process a sequence at each time step sequentially. An example of the architecture of LSTM is listed below. -.. image:: ../../../tutorials/sentiment_analysis/src/bi_lstm.jpg +.. image:: src/bi_lstm.jpg :align: center Generally speaking, a recurrent network perform the following operations from :math:`t=1` to :math:`t=T`, or reversely from :math:`t=T` to :math:`t=1`. @@ -75,19 +52,19 @@ Its **output function** simply takes :math:`x_t` as the output. act=None, rnn_layer_attr=None): def __rnn_step__(ipt): - out_mem = memory(name=name, size=size) - rnn_out = mixed_layer(input = [full_matrix_projection(ipt), - full_matrix_projection(out_mem)], - name = name, - bias_attr = rnn_bias_attr, - act = act, - layer_attr = rnn_layer_attr, - size = size) + out_mem = paddle.layer.memory(name=name, size=size) + rnn_out = paddle.layer.mixed(input = [paddle.layer.full_matrix_projection(input=ipt), + paddle.layer.full_matrix_projection(input=out_mem)], + name = name, + bias_attr = rnn_bias_attr, + act = act, + layer_attr = rnn_layer_attr, + size = size) return rnn_out - return recurrent_group(name='%s_recurrent_group' % name, - step=__rnn_step__, - reverse=reverse, - input=input) + return paddle.layer.recurrent_group(name='%s_recurrent_group' % name, + step=__rnn_step__, + reverse=reverse, + input=input) PaddlePaddle uses memory to construct step function. **Memory** is the most important concept when constructing recurrent neural networks in PaddlePaddle. A memory is a state that is used recurrently in step functions, such as :math:`x_{t+1} = f_x(x_t)`. One memory contains an **output** and a **input**. The output of memory at the current time step is utilized as the input of the memory at the next time step. A memory can also has a **boot layer**, whose output is utilized as the initial value of the memory. In our case, the output of the gated recurrent unit is employed as the output memory. Notice that the name of the layer :code:`rnn_out` is the same as the name of :code:`out_mem`. This means the output of the layer :code:`rnn_out` (:math:`x_{t+1}`) is utilized as the **output** of :code:`out_mem` memory. @@ -101,7 +78,7 @@ Sequence to Sequence Model with Attention ----------------------------------------- We will use the sequence to sequence model with attention as an example to demonstrate how you can configure complex recurrent neural network models. An illustration of the sequence to sequence model with attention is shown in the following figure. -.. image:: ../../../tutorials/text_generation/encoder-decoder-attention-model.png +.. image:: src/encoder-decoder-attention-model.png :align: center In this model, the source sequence :math:`S = \{s_1, \dots, s_T\}` is encoded with a bidirectional gated recurrent neural networks. The hidden states of the bidirectional gated recurrent neural network :math:`H_S = \{H_1, \dots, H_T\}` is called *encoder vector* The decoder is a gated recurrent neural network. When decoding each token :math:`y_t`, the gated recurrent neural network generates a set of weights :math:`W_S^t = \{W_1^t, \dots, W_T^t\}`, which are used to compute a weighted sum of the encoder vector. The weighted sum of the encoder vector is utilized to condition the generation of the token :math:`y_t`. @@ -113,43 +90,52 @@ We also project the encoder vector to :code:`decoder_size` dimensional space, ge .. code-block:: python # Define the data layer of the source sentence. - src_word_id = data_layer(name='source_language_word', size=source_dict_dim) + src_word_id = paddle.layer.data( + name='source_language_word', + type=paddle.data_type.integer_value_sequence(source_dict_dim)) # Calculate the word embedding of each word. - src_embedding = embedding_layer( + src_embedding = paddle.layer.embedding( input=src_word_id, size=word_vector_dim, - param_attr=ParamAttr(name='_source_language_embedding')) + param_attr=paddle.attr.ParamAttr(name='_source_language_embedding')) # Apply forward recurrent neural network. - src_forward = grumemory(input=src_embedding, size=encoder_size) + src_forward = paddle.networks.simple_gru( + input=src_embedding, size=encoder_size) # Apply backward recurrent neural network. reverse=True means backward recurrent neural network. - src_backward = grumemory(input=src_embedding, - size=encoder_size, - reverse=True) + src_backward = paddle.networks.simple_gru( + input=src_embedding, size=encoder_size, reverse=True) # Mix the forward and backward parts of the recurrent neural network together. - encoded_vector = concat_layer(input=[src_forward, src_backward]) + encoded_vector = paddle.layer.concat(input=[src_forward, src_backward]) # Project encoding vector to decoder_size. - encoder_proj = mixed_layer(input = [full_matrix_projection(encoded_vector)], - size = decoder_size) + encoded_proj = paddle.layer.mixed( + size=decoder_size, + input=paddle.layer.full_matrix_projection(encoded_vector)) # Compute the first instance of the backward RNN. - backward_first = first_seq(input=src_backward) + backward_first = paddle.layer.first_seq(input=src_backward) # Project the first instance of backward RNN to decoder size. - decoder_boot = mixed_layer(input=[full_matrix_projection(backward_first)], size=decoder_size, act=TanhActivation()) + decoder_boot = paddle.layer.mixed( + size=decoder_size, + act=paddle.activation.Tanh(), + input=paddle.layer.full_matrix_projection(backward_first)) The decoder uses :code:`recurrent_group` to define the recurrent neural network. The step and output functions are defined in :code:`gru_decoder_with_attention`: .. code-block:: python - group_inputs=[StaticInput(input=encoded_vector,is_seq=True), - StaticInput(input=encoded_proj,is_seq=True)] - trg_embedding = embedding_layer( - input=data_layer(name='target_language_word', - size=target_dict_dim), - size=word_vector_dim, - param_attr=ParamAttr(name='_target_language_embedding')) + group_input1 = paddle.layer.StaticInput(input=encoded_vector, is_seq=True) + group_input2 = paddle.layer.StaticInput(input=encoded_proj, is_seq=True) + group_inputs = [group_input1, group_input2] + trg_embedding = paddle.layer.embedding( + input=paddle.layer.data( + name='target_language_word', + type=paddle.data_type.integer_value_sequence(target_dict_dim)), + size=word_vector_dim, + param_attr=paddle.attr.ParamAttr(name='_target_language_embedding')) + group_inputs.append(trg_embedding) group_inputs.append(trg_embedding) # For decoder equipped with attention mechanism, in training, @@ -158,9 +144,10 @@ The decoder uses :code:`recurrent_group` to define the recurrent neural network. # StaticInput means the same value is utilized at different time steps. # Otherwise, it is a sequence input. Inputs at different time steps are different. # All sequence inputs should have the same length. - decoder = recurrent_group(name=decoder_group_name, - step=gru_decoder_with_attention, - input=group_inputs) + decoder = paddle.layer.recurrent_group( + name=decoder_group_name, + step=gru_decoder_with_attention, + input=group_inputs) The implementation of the step function is listed as below. First, it defines the **memory** of the decoder network. Then it defines attention, gated recurrent unit step function, and the output function: @@ -171,27 +158,32 @@ The implementation of the step function is listed as below. First, it defines th # Defines the memory of the decoder. # The output of this memory is defined in gru_step. # Notice that the name of gru_step should be the same as the name of this memory. - decoder_mem = memory(name='gru_decoder', - size=decoder_size, - boot_layer=decoder_boot) + decoder_mem = paddle.layer.memory( + name='gru_decoder', size=decoder_size, boot_layer=decoder_boot) # Compute attention weighted encoder vector. - context = simple_attention(encoded_sequence=enc_vec, - encoded_proj=enc_proj, - decoder_state=decoder_mem) + context = paddle.networks.simple_attention( + encoded_sequence=enc_vec, + encoded_proj=enc_proj, + decoder_state=decoder_mem) # Mix the current word embedding and the attention weighted encoder vector. - decoder_inputs = mixed_layer(inputs = [full_matrix_projection(context), - full_matrix_projection(current_word)], - size = decoder_size * 3) + decoder_inputs = paddle.layer.mixed( + size=decoder_size * 3, + input=[ + paddle.layer.full_matrix_projection(input=context), + paddle.layer.full_matrix_projection(input=current_word) + ]) # Define Gated recurrent unit recurrent neural network step function. - gru_step = gru_step_layer(name='gru_decoder', - input=decoder_inputs, - output_mem=decoder_mem, - size=decoder_size) + gru_step = paddle.layer.gru_step( + name='gru_decoder', + input=decoder_inputs, + output_mem=decoder_mem, + size=decoder_size) # Defines the output function. - out = mixed_layer(input=[full_matrix_projection(input=gru_step)], - size=target_dict_dim, - bias_attr=True, - act=SoftmaxActivation()) + out = paddle.layer.mixed( + size=target_dict_dim, + bias_attr=True, + act=paddle.activation.Softmax(), + input=paddle.layer.full_matrix_projection(input=gru_step)) return out @@ -207,45 +199,37 @@ After training the model, we can use it to generate sequences. A common practice - :code:`eos_id`: the end token. Every sentence ends with the end token. - :code:`beam_size`: the beam size used in beam search. - :code:`max_length`: the maximum length of the generated sentences. - -* use :code:`seqtext_printer_evaluator` to print text according to index matrix and dictionary. This function needs to set: - - - :code:`id_input`: the integer ID of the data, used to identify the corresponding output in the generated files. - - :code:`dict_file`: the dictionary file for converting word id to word. - - :code:`result_file`: the path of the generation result file. The code is listed below: .. code-block:: python - group_inputs=[StaticInput(input=encoded_vector,is_seq=True), - StaticInput(input=encoded_proj,is_seq=True)] + group_input1 = paddle.layer.StaticInput(input=encoded_vector, is_seq=True) + group_input2 = paddle.layer.StaticInput(input=encoded_proj, is_seq=True) + group_inputs = [group_input1, group_input2] # In generation, decoder predicts a next target word based on # the encoded source sequence and the last generated target word. # The encoded source sequence (encoder's output) must be specified by # StaticInput which is a read-only memory. # Here, GeneratedInputs automatically fetchs the last generated word, # which is initialized by a start mark, such as . - trg_embedding = GeneratedInput( - size=target_dict_dim, - embedding_name='_target_language_embedding', - embedding_size=word_vector_dim) + trg_embedding = paddle.layer.GeneratedInput( + size=target_dict_dim, + embedding_name='_target_language_embedding', + embedding_size=word_vector_dim) group_inputs.append(trg_embedding) - beam_gen = beam_search(name=decoder_group_name, - step=gru_decoder_with_attention, - input=group_inputs, - bos_id=0, # Beginnning token. - eos_id=1, # End of sentence token. - beam_size=beam_size, - max_length=max_length) + beam_gen = paddle.layer.beam_search( + name=decoder_group_name, + step=gru_decoder_with_attention, + input=group_inputs, + bos_id=0, # Beginnning token. + eos_id=1, # End of sentence token. + beam_size=beam_size, + max_length=max_length) - seqtext_printer_evaluator(input=beam_gen, - id_input=data_layer(name="sent_id", size=1), - dict_file=trg_dict_path, - result_file=gen_trans_file) - outputs(beam_gen) + return beam_gen -Notice that this generation technique is only useful for decoder like generation process. If you are working on sequence tagging tasks, please refer to :ref:`semantic_role_labeling` for more details. +Notice that this generation technique is only useful for decoder like generation process. If you are working on sequence tagging tasks, please refer to `book/06.understand_sentiment `_ for more details. -The full configuration file is located at :code:`demo/seqToseq/seqToseq_net.py`. +The full configuration file is located at `book/08.machine_translation/train.py `_ . diff --git a/doc/_sources/howto/dev/build_en.md.txt b/doc/_sources/howto/dev/build_en.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..91c41ef8ce3abdec5d69a9cbcebbc49b17d8f663 --- /dev/null +++ b/doc/_sources/howto/dev/build_en.md.txt @@ -0,0 +1,124 @@ +# Build using Docker + +## What Developers Need + +To contribute to PaddlePaddle, you need + +1. A computer -- Linux, BSD, Windows, MacOS, and +1. Docker. + +Nothing else. Not even Python and GCC, because you can install all build tools into a Docker image. We run all the tools by running this image. + +## General Process + +1. Retrieve source code. + + ```bash + git clone https://github.com/paddlepaddle/paddle + ``` + +2. Install build tools into a Docker image. + + ```bash + cd paddle; docker build -t paddle:dev . + ``` + + Please be aware of the `.` at the end of the command, which refers to the [`./Dockerfile` file](https://github.com/PaddlePaddle/Paddle/blob/develop/Dockerfile). `docker build` follows instructions in this file to create a Docker image named `paddle:dev`, and installs building tools into it. + +3. Build from source. + + This following command starts a Docker container that executes the Docker image `paddle:dev`, mapping the current directory to `/paddle/` in the container, and runs the default entry-point [`build.sh`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build.sh) as specified in the Dockefile. `build.sh` invokes `cmake` and `make` to build PaddlePaddle source code, which had been mapped to `/paddle`, and writes outputs to `/paddle/build`, which maps to `build` in the current source directory on the computer. + + ```bash + docker run -v $PWD:/paddle paddle:dev + ``` + + Above command builds a CUDA-enabled version. If we want to build a CPU-only version, we can type + + ```bash + docker run -e WITH_GPU=OFF -v $PWD:/paddle paddle:dev + ``` + +4. Run unit tests. + + To run all unit tests using the first GPU of a node: + + ```bash + NV_GPU=0 nvidia-docker run -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest" + ``` + + If we used `WITH_GPU=OFF` at build time, it generates only CPU-based unit tests, and we don't need nvidia-docker to run them. We can just run + + ```bash + docker run -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest" + ``` + + Sometimes we want to run a specific unit test, say `memory_test`, we can run + + ```bash + nvidia-docker run -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest -V -R memory_test" + ``` + +5. Clean Build. + + Sometimes, we might want to clean all thirt-party dependents and built binaries. To do so, just + + ```bash + rm -rf build + ``` + +## Docker, Or Not? + +- What is Docker? + + If you haven't heard of it, consider it something like Python's virtualenv. + +- Docker or virtual machine? + + Some people compare Docker with VMs, but Docker doesn't virtualize any hardware nor running a guest OS, which means there is no compromise on the performance. + +- Why Docker? + + Using a Docker image of build tools standardizes the building environment, which makes it easier for others to reproduce your problems and to help. + + Also, some build tools don't run on Windows or Mac or BSD, but Docker runs almost everywhere, so developers can use whatever computer they want. + +- Can I choose not to use Docker? + + Sure, you don't have to install build tools into a Docker image; instead, you can install them in your local computer. This document exists because Docker would make the development way easier. + +- How difficult is it to learn Docker? + + It takes you ten minutes to read [an introductory article](https://docs.docker.com/get-started) and saves you more than one hour to install all required build tools, configure them, especially when new versions of PaddlePaddle require some new tools. Not even to mention the time saved when other people trying to reproduce the issue you have. + +- Can I use my favorite IDE? + + Yes, of course. The source code resides on your local computer, and you can edit it using whatever editor you like. + + Many PaddlePaddle developers are using Emacs. They add the following few lines into their `~/.emacs` configure file: + + ```emacs + (global-set-key "\C-cc" 'compile) + (setq compile-command + "docker run --rm -it -v $(git rev-parse --show-toplevel):/paddle paddle:dev") + ``` + + so they could type `Ctrl-C` and `c` to build PaddlePaddle from source. + +- Does Docker do parallel building? + + Our building Docker image runs a [Bash script](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build.sh), which calls `make -j$(nproc)` to starts as many processes as the number of your CPU cores. + +## Some Gotchas + +- Docker requires sudo + + An owner of a computer has the administrative privilege, a.k.a., sudo, and Docker requires this privilege to work properly. If you use a shared computer for development, please ask the administrator to install and configure Docker. We will do our best to support rkt, another container technology that doesn't require sudo. + +- Docker on Windows/MacOS builds slowly + + On Windows and MacOS, Docker containers run in a Linux VM. You might want to give this VM some more memory and CPUs so to make the building efficient. Please refer to [this issue](https://github.com/PaddlePaddle/Paddle/issues/627) for details. + +- Not enough disk space + + Examples in this article uses option `--rm` with the `docker run` command. This option ensures that stopped containers do not exist on hard disks. We can use `docker ps -a` to list all containers, including stopped. Sometimes `docker build` generates some intermediate dangling images, which also take disk space. To clean them, please refer to [this article](https://zaiste.net/posts/removing_docker_containers/). diff --git a/doc/_sources/howto/dev/contribute_to_paddle_en.md.txt b/doc/_sources/howto/dev/contribute_to_paddle_en.md.txt index 9b0d3e83c0dc264650eda73e6801c60a75439b4a..a60453ff4e3bba6e6cb3b3de915dd69afd3a1ec3 100644 --- a/doc/_sources/howto/dev/contribute_to_paddle_en.md.txt +++ b/doc/_sources/howto/dev/contribute_to_paddle_en.md.txt @@ -1,146 +1,157 @@ # Contribute Code -We sincerely appreciate your contributions. You can use fork and pull request -workflow to merge your code. +We sincerely appreciate your contribution. This document explains our workflow and work style. -## Code Requirements -- Your code must be fully documented by - [doxygen](http://www.stack.nl/~dimitri/doxygen/) style. -- Make sure the compiler option WITH\_STYLE\_CHECK is on and the compiler - passes the code style check. -- All code must have unit test. -- Pass all unit tests. +## Workflow -The following tutorial guides you into submitting your contibution. +PaddlePaddle uses this [Git branching model](http://nvie.com/posts/a-successful-git-branching-model/). The following steps guide usual contributions. -## [Creating a Fork](https://help.github.com/articles/fork-a-repo/) +1. Fork -Just head over to the GitHub page and click the "Fork" button. -It's just that simple. + Our development community has been growing fastly; it doesn't make sense for everyone to write into the official repo. So, please file Pull Requests from your fork. To make a fork, just head over to the GitHub page and click the ["Fork" button](https://help.github.com/articles/fork-a-repo/). -## Clone +1. Clone -Paddle is currently using [git-flow branching model](http://nvie.com/posts/a-successful-git-branching-model/). -The **develop** is the main branch, and other user's branches are feature branches. + To make a copy of your fork to your local computers, please run -Once you've created a fork, you can use your favorite git client to clone your -repo or just head straight to the command line: + ```bash + git clone https://github.com/your-github-account/paddle + cd paddle + ``` -```shell -# Clone your fork to your local machine -git clone --branch develop https://github.com/USERNAME/Paddle.git -``` -If your repository doesn't contain **develop** branch, just create it by your own. - -```shell -git clone https://github.com/USERNAME/Paddle.git Paddle -cd Paddle -git checkout -b develop # create develop branch. -git remote add upstream https://github.com/PaddlePaddle/Paddle.git # add upstream to baidu/Paddle -git pull upstream develop # update to upstream -``` +1. Create the local feature branch -Then you can start to develop by making a local developement branch + For daily works like adding a new feature or fixing a bug, please open your feature branch before coding: -```shell -git checkout -b MY_COOL_STUFF_BRANCH -``` + ```bash + git checkout -b my-cool-stuff + ``` -## Using `pre-commit` hook +1. Commit -Paddle developers use [pre-commit](http://pre-commit.com/) tool to manage git -pre-commit hooks. It can help us format source codes (cpp, python), check some -basic thing before commit (only one EOL for each file, do not add a huge file -in git). `pre-commit` tests is a part of unit tests in Travis-CI now, every -PR doesn't fit hook can not be merged into Paddle. + Before issuing your first `git commit` command, please install [`pre-commit`](http://pre-commit.com/) by running the following commands: -To use [pre-commit](http://pre-commit.com/), you should install it by -`pip install pre-commit`, and currently, Paddle uses `clang-format` to format -c/cpp sources. Please make sure clang-format 3.8+ installed. + ```bash + pip install pre-commit + pre-commit install + ``` -Then just run `pre-commit install` in your Paddle clone directory. When you -commit your code, the pre-commit hook will check the local code if there is -anything not suitable to commit, and so on. + Our pre-commit configuration requires clang-format 3.8 for auto-formating C/C++ code and yapf for Python. -## Commit + Once installed, `pre-commit` checks the style of code and documentation in every commit. We will see something like the following when you run `git commit`: -Commit your changes by following command lines: + ``` + ➜ git commit + CRLF end-lines remover...............................(no files to check)Skipped + yapf.................................................(no files to check)Skipped + Check for added large files..............................................Passed + Check for merge conflicts................................................Passed + Check for broken symlinks................................................Passed + Detect Private Key...................................(no files to check)Skipped + Fix End of Files.....................................(no files to check)Skipped + clang-formater.......................................(no files to check)Skipped + [my-cool-stuff c703c041] add test file + 1 file changed, 0 insertions(+), 0 deletions(-) + create mode 100644 233 + ``` -```shell -# show the working tree status -git status -# add modified files -git add xx -env EDITOR=vim git commit # You can write your comments by vim/nano/emacs. -``` -The first line of commit infomation is the title. The second and later lines -are the details if any. +1. Build and test -## Keeping Fork Up to Date + Users can build PaddlePaddle natively on Linux and Mac OS X. But to unify the building environment and to make it easy for debugging, the recommended way is [using Docker](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/dev/build_en.md). -Before pull your request, you should sync your code from the latest PaddlePaddle. -To do this, you'll need to add a remote at first: +1. Keep pulling -```shell -# see the current configured remote repository -git remote -v -# add upstream repository -git remote add upstream https://github.com/PaddlePaddle/Paddle.git -# verify the new upstream -git remote -v -``` + An experienced Git user pulls from the official repo often -- daily or even hourly, so they notice conflicts with others work early, and it's easier to resolve smaller conflicts. -Update your fork with the latest upstream changes: + ```bash + git remote add upstream https://github.com/PaddlePaddle/Paddle + git pull upstream develop + ``` -```shell -git pull --rebase upstream develop -``` +1. Push and file a pull request -If there are no unique commits locally, git will simply perform a fast-forward. -However, if you have been making changes (in the vast majority of cases you -probably shouldn't be), you may have to deal with conflicts. + You can "push" your local work into your forked repo: -Now, your local master branch is up-to-date with everything modified upstream. + ```bash + git push origin my-cool-stuff + ``` -## Push to GitHub + The push allows you to create a pull request, requesting owners of this [official repo](https://github.com/PaddlePaddle/Paddle) to pull your change into the official one. -```shell -# push to your repository in Github -git push -u origin MY_COOL_STUFF_BRANCH # create remote branch MY_COOL_STUFF_BRANCH to origin. -``` + To create a pull request, please follow [these steps](https://help.github.com/articles/creating-a-pull-request/). -## Pull Request + If your change is for fixing an issue, please write ["Fixes "](https://help.github.com/articles/closing-issues-using-keywords/) in the description section of your pull request. Github would close the issue when the owners merge your pull request. -Go to the page for your fork on GitHub, select your development branch, -and click the **pull request button**. + Please remember to specify some reviewers for your pull request. If you don't know who are the right ones, please follow Github's recommendation. -## Update your pull request with the lastest version -During the code review, your pull request may become stale because new commits in -baidu/Paddle. GitHub allows autmotic update if there is no conflict. You can do this -by clicking the "Update Branch" button in your pull request page. However, in the case -of conflict, you need to do the update manually. You need to do the following on -your local repository: -```shell -git checkout MY_COOL_STUFF_BRANCH -git pull upstream develop -# You may need to resolve the conflict according to the git prompt. -# Make and test your code. -git push origin MY_COOL_STUFF_BRANCH -``` -Now your Pull Request is updated with the latest version. +1. Delete local and remote branches + + To keep your local workspace and your fork clean, you might want to remove merged branches: + + ```bash + git push origin :my-cool-stuff + git checkout develop + git pull upstream develop + git branch -d my-cool-stuff + ``` + +### Code Review + +- Please feel free to ping your reviewers by sending them the URL of your pull request via IM or email. Please do this after your pull request passes the CI. + +- Please answer reviewers' every comment. If you are to follow the comment, please write "Done"; please give a reason otherwise. + +- If you don't want your reviewers to get overwhelmed by email notifications, you might reply their comments by [in a batch](https://help.github.com/articles/reviewing-proposed-changes-in-a-pull-request/). + +- Reduce the unnecessary commits. Some developers commit often. It is recommended to append a sequence of small changes into one commit by running `git commit --amend` instead of `git commit`. + + +## Coding Standard -## Revise your pull request +### Code Style -When you revise your pull request according to reviewer's comments, please use 'git commit' instead of 'git commit --amend' to commit your changes so that the reviewers can see the difference between the new pull requrest and the old pull request. +Our C/C++ code follows the [Google style guide](http://google.github.io/styleguide/cppguide.html). -The possible commands are +Our Python code follows the [PEP8 style guide](https://www.python.org/dev/peps/pep-0008/). -```shell -git checkout MY_COOL_STUFF_BRANCH -git pull upstream develop # update local to newest code base. -# May be some conflicts will occured. -# And develop your cool stuff -env EDITOR=vim git commit # add your revise log -git push origin MY_COOL_STUFF_BRANCH +Our build process helps to check the code style. In [`build.sh`](https://github.com/PaddlePaddle/Paddle/blob/b84e8226514b8bb4405c3c28e54aa5077193d179/paddle/scripts/docker/build.sh#L42), the entry point of our [builder Docker image](https://github.com/PaddlePaddle/Paddle/blob/b84e8226514b8bb4405c3c28e54aa5077193d179/Dockerfile#L88), the CMake argument `WITH_STYLE_CHECK` is set to `ON` by default. This flag is on + +Please install pre-commit, which automatically reformat the changes to C/C++ and Python code whenever we run `git commit`. To check the whole codebase, we can run the command `pre-commit run -a`, as in the [`check_style.sh` file](https://github.com/PaddlePaddle/Paddle/blob/b84e8226514b8bb4405c3c28e54aa5077193d179/paddle/scripts/travis/check_style.sh#L30), which is invoked by [our Travis CI configuration](https://github.com/PaddlePaddle/Paddle/blob/b84e8226514b8bb4405c3c28e54aa5077193d179/.travis.yml#L43). + +### Unit Tests + +Please remember to add related unit tests. + +- For C/C++ code, please follow [`google-test` Primer](https://github.com/google/googletest/blob/master/googletest/docs/Primer.md). + +- For Python code, please use [Python's standard `unittest` package](http://pythontesting.net/framework/unittest/unittest-introduction/). + + +### Writing Logs + +We use [glog](https://github.com/google/glog) for logging in our C/C++ code. + +For general information, please use `LOG`. For debug information, please use [`VLOG`](http://htmlpreview.github.io/?https://github.com/google/glog/blob/master/doc/glog.html#verbose). The reason is at [here](https://groups.google.com/a/chromium.org/d/msg/chromium-dev/3NDNd1KzXeY/AZKMMx37fdQJ). + +`VLOG` requires a *verbose level* parameter. For example: + +```c++ +VLOG(3) << "Operator FC is taking " << num_inputs << "inputs." +``` + +When we run a PaddlePaddle application or test, we can specify a verbose threshold. For example: + +```bash +GLOG_vmodule=buddy_allocator=2 \ +GLOG_v=10 \ +python \ +../python/paddle/v2/framework/tests/test_recurrent_op.py ``` + +This will enable VLOG messages generated by `buddy_allocator.{h,cc}` and in the verbose range of 0 to 3, so you will see above example VLOG message, which is in level 3. This suggests that we output overall messages in lower verbose levels, so they display with higher probability. When coding C++, please follow the verbose level convention as follows: + +- verbose level 1: [framework](https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/framework) +- verbose level 3: [operators](https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/operators) +- verbose level 5: [memory](https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/memory), [platform](https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/platform) +- verbose level 7: [math](https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/math) diff --git a/doc/_sources/howto/dev/new_layer_en.rst.txt b/doc/_sources/howto/dev/new_layer_en.rst.txt index 46481f5ead33dc6a26507e021fd9ae0f8316e940..110a9fb38f890a766bb4480e91feb22d3b0838a5 100644 --- a/doc/_sources/howto/dev/new_layer_en.rst.txt +++ b/doc/_sources/howto/dev/new_layer_en.rst.txt @@ -29,7 +29,7 @@ Fully connected layer takes a dense input vector with dimension :math:`D_i`. It where :math:`f(.)` is an nonlinear *activation* function, such as sigmoid, tanh, and Relu. -The transformation matrix :math:`W` and bias vector :math:`b` are the *parameters* of the layer. The *parameters* of a layer are learned during training in the *backward pass*. The backward pass computes the gradients of the output function with respect to all parameters and inputs. The optimizer can use chain rule to compute the gradients of the loss function with respect to each parameter. +The transformation matrix :math:`W` and bias vector :math:`b` are the *parameters* of the layer. The *parameters* of a layer are learned during training in the *backward pass*. The backward pass computes the gradients of the output function with respect to all parameters and inputs. The optimizer can use chain rule to compute the gradients of the loss function with respect to each parameter. Suppose our loss function is :math:`c(y)`, then @@ -37,7 +37,7 @@ Suppose our loss function is :math:`c(y)`, then \frac{\partial c(y)}{\partial x} = \frac{\partial c(y)}{\partial y} \frac{\partial y}{\partial x} -Suppose :math:`z = f(W^T x + b)`, then +Suppose :math:`z = W^T x + b`, then .. math:: @@ -48,7 +48,7 @@ This derivative can be automatically computed by our base layer class. Then, for fully connected layer, we need to compute: .. math:: - + \frac{\partial z}{\partial x} = W, \frac{\partial z_j}{\partial W_{ij}} = x_i, \frac{\partial z}{\partial b} = \mathbf 1 where :math:`\mathbf 1` is an all one vector, :math:`W_{ij}` is the number at the i-th row and j-th column of the matrix :math:`W`, :math:`z_j` is the j-th component of the vector :math:`z`, and :math:`x_i` is the i-th component of the vector :math:`x`. @@ -322,7 +322,7 @@ All the gradient check unit tests are located in :code:`paddle/gserver/tests/tes /* weight */ true); } } - + If you are creating a new file for the test, such as :code:`paddle/gserver/tests/testFCGrad.cpp`, you need to add the file to :code:`paddle/gserver/tests/CMakeLists.txt`. An example is given below. All the unit tests will run when you execute the command :code:`make tests`. Notice that some layers might need high accuracy for the gradient check unit tests to work well. You need to configure :code:`WITH_DOUBLE` to `ON` when configuring cmake. .. code-block:: bash diff --git a/doc/_sources/howto/dev/new_op_en.md.txt b/doc/_sources/howto/dev/new_op_en.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..1e88e1f5b4df710f1b69f0305d8d8a2921c4249a --- /dev/null +++ b/doc/_sources/howto/dev/new_op_en.md.txt @@ -0,0 +1,342 @@ +# How to write a new operator + + - [Background](#background) + - [Implementing C++ Types](#implementing-c++-types) + - [Defining ProtoMaker](#defining-protoMaker) + - [Defining Operator](#defining-operator) + - [Registering Operator](#registering-operator) + - [Compilation](#compilation) + - [Python Binding](#python-binding) + - [Unit Tests](#unit-tests) + - [Testing Forward Operators](#testing-forward-operators) + - [Testing Backward Operators](#testing-backward-operators) + - [Compiling and Running](#compiling-and-running) + - [Remarks](#remarks) +## Background + +Here are the base types needed. For details, please refer to the design docs. + +- `framework::OperatorBase`: Operator (Op)base class. +- `framework::OpKernel`: Base class for Op computation. +- `framework::OperatorWithKernel`: Inherited from OperatorBase, describing an operator with computation. +- `class OpProtoAndCheckerMaker`: Describes an Operator's input, output, attributes and description, mainly used to interface with Python API. + +An operator can be differentiated by whether in has kernel methods. An operator with kernel inherits from `OperatorWithKernel` while the ones without inherit from `OperatorBase`. This tutorial focuses on implementing operators with kernels. In short, an operator includes the following information: + + + Information | Where is it defined +-------------- | :---------------------- +OpProtoMake definition | `.cc`files, Backward Op does not need an OpProtoMake interface. +Op definition | `.cc` files +Kernel implementation | The kernel methods shared between CPU and GPU are defined in `.h` files. CPU-specific kernels live in `.cc` files, while GPU-specific kernels are implemented in `.cu`files. +Registering the Op | Ops are registered in `.cc` files; For Kernel registration, `.cc` files contain the CPU implementation, while `.cu` files contain the GPU implementation. + + +New Operator implementations are added to the list [paddle/operators](https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/operators), with file names in the format `*_op.h` (if applicable), `*_op.cc`, `*_op.cu` (if applicable).** The system will use the naming scheme to automatically build operators and their corresponding Python extensions. ** + + +Let's take matrix multiplication operator, [MulOp](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc), as an example to introduce the writing of an Operator with Kernel. + + +## Implementing C++ Types + + +### 1. Defining Class ProtoMaker + +Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output. + +First, define `ProtoMaker` to describe the Operator's input, output, and additional comments: + +```cpp +class MulOpMaker : public framework::OpProtoAndCheckerMaker { + public: + MulOpMaker(framework::OpProto *proto, framework::OpAttrChecker *op_checker) + : OpProtoAndCheckerMaker(proto, op_checker) { + AddInput("X", "(Tensor), 2D tensor of size (M x K)"); + AddInput("Y", "(Tensor), 2D tensor of size (K x N)"); + AddOutput("Out", "(Tensor), 2D tensor of size (M x N)"); + AddComment(R"DOC( +Two Element Mul Operator. +The equation is: Out = X * Y +)DOC"); + } +}; +``` + +[`MulOpMaker`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc#L43)is inherited from`framework::OpProtoAndCheckerMaker`, consisting of 2 variables in the constructor： + + - `framework::OpProto` stores Operator input and variable attribute, used for generating Python API interfaces. + - `framework::OpAttrChecker` is used to validate variable attributes. + +The constructor utilizes `AddInput`, `AddOutput`, and `AddComment`, so that the corresponding information will be added to `OpProto`. + +The code above adds two inputs `X` and `Y` to `MulOp`, an output `Out`, and their corresponding descriptions, in accordance to Paddle's [naming convention](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/name_convention.md). + + +An additional example [`ScaleOp`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/scale_op.cc#L37) is implemented as follows: + +```cpp +template +class ScaleOpMaker : public framework::OpProtoAndCheckerMaker { + public: + ScaleOpMaker(framework::OpProto *proto, framework::OpAttrChecker *op_checker) + : OpProtoAndCheckerMaker(proto, op_checker) { + AddInput("X", "The input tensor of scale operator.").NotInGradient(); + AddOutput("Out", "The output tensor of scale operator.").NotInGradient(); + AddComment(R"DOC(Scale operator +The equation is: Out = scale*X +)DOC"); + AddAttr("scale", "scale of scale operator.").SetDefault(1.0); + } +}; +``` + +There are two changes in this example: + +- `AddInput("X","...").NotInGradient()` expresses that input `X` is not involved in `ScaleOp`'s corresponding computation. If an input to an operator is not participating in back-propagation, please explicitly set `.NotInGradient()`. + +- `AddAttr("scale", "...").SetDefault(1.0);` adds `scale`constant as an attribute, and sets the default value to 1.0. + + +### 2. Defining Operator + +The following code defines the interface for MulOp: + +```cpp +class MulOp : public framework::OperatorWithKernel { + public: + using framework::OperatorWithKernel::OperatorWithKernel; + + protected: + void InferShape(const framework::InferShapeContext &ctx) const override { + auto dim0 = ctx.Input("X")->dims(); + auto dim1 = ctx.Input("Y")->dims(); + PADDLE_ENFORCE_EQ(dim0.size(), 2, + "input X(%s) should be a tensor with 2 dims, a matrix", + ctx.op_.Input("X")); + PADDLE_ENFORCE_EQ(dim1.size(), 2, + "input Y(%s) should be a tensor with 2 dims, a matrix", + ctx.op_.Input("Y")); + PADDLE_ENFORCE_EQ( + dim0[1], dim1[0], + "First matrix's width must be equal with second matrix's height."); + ctx.Output("Out")->Resize({dim0[0], dim1[1]}); + } +}; +``` + +[`MulOp`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc#L22) is inherited from `OperatorWithKernel`. Its `public` member + +```cpp +using framework::OperatorWithKernel::OperatorWithKernel; +``` + +expresses an operator constructor using base class `OperatorWithKernel`, alternatively written as + +```cpp +MulOp(const std::string &type, const framework::VariableNameMap &inputs, + const framework::VariableNameMap &outputs, + const framework::AttributeMap &attrs) + : OperatorWithKernel(type, inputs, outputs, attrs) {} +``` + +`InferShape` interface needs to be re-written.`InferShape` is a constant method and cannot modify Op's member variables, its constant member `const framework::InferShapeContext &ctx` can be used to extract input, output, and attributes. It functions to + + - 1). validate and error out early: it checks input data dimensions and types. + - 2). configures the tensor shape in the output. + +Usually `OpProtoMaker` and `Op`'s type definitions are written in `.cc` files, which also include the registration methods introduced later. + +### 3. Defining OpKernel + +`MulKernel` inherits `framework::OpKernel`, which includes the following templates: + +- `typename Place` denotes device type. When different devices, namely the CPU and the GPU, share the same kernel, this template needs to be added. If they don't share kernels, this must not be added. An example of a non-sharing kernel is [`OnehotCrossEntropyOpKernel`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43). + +- `typename T` denotes data type, such as `float` or `double`. + +`MulKernel` types need to rewrite the interface for `Compute`. +- `Compute` takes one input variable `const framework::ExecutionContext& context`. +- Compared with `InferShapeContext`, `ExecutionContext` includes device types, and can similarly extract input, output, and attribute variables. +- `Compute` implements the computation logics of an `OpKernel`. + +`MulKernel`'s implementation of `Compute` is as follows: + + ```cpp + template + class MulKernel : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext& context) const override { + auto* X = context.Input("X"); + auto* Y = context.Input("Y"); + auto* Z = context.Output("Out"); + Z->mutable_data(context.GetPlace()); + auto* device_context = + const_cast(context.device_context_); + math::matmul(*X, false, *Y, false, 1, Z, 0, device_context); + } + }; + ``` + +Note that **different devices (CPU, GPU)share an Op definition; whether or not they share the same `OpKernel` depends on whether `Compute` calls functions that support both devices.** + +`MulOp`'s CPU and GPU share the same `Kernel`. A non-sharing `OpKernel` example can be seen in [`OnehotCrossEntropyOpKernel`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43). + +To ease the writing of `OpKernel` compute, and for reusing code cross-device, [`Eigen-unsupported Tensor`](https://bitbucket.org/eigen/eigen/src/default/unsupported/Eigen/CXX11/src/Tensor/README.md?fileviewer=file-view-default) module is used to implement `Compute` interface. To learn about how the Eigen library is used in PaddlePaddle, please see [usage document](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/dev/use_eigen_cn.md). + + +This concludes the forward implementation of an operator. Next its operation and kernel need to be registered in a `.cc` file. + +The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. **Note that a backward operator does not include a `ProtoMaker`**. + +### 4. Registering Operator + +- In `.cc` files, register forward and backward operator classes and the CPU kernel. + + ```cpp + namespace ops = paddle::operators; + REGISTER_OP(mul, ops::MulOp, ops::MulOpMaker, mul_grad, ops::MulOpGrad); + REGISTER_OP_CPU_KERNEL(mul, ops::MulKernel); + REGISTER_OP_CPU_KERNEL(mul_grad, + ops::MulGradKernel); + ``` + + In that code block, + + - `REGISTER_OP` registers the `ops::MulOp` class, type named `mul`, its type `ProtoMaker` is `ops::MulOpMaker`, registering `ops::MulOpGrad` as `mul_grad`. + - `REGISTER_OP_WITHOUT_GRADIENT` registers an operator without gradient. + - `REGISTER_OP_CPU_KERNEL` registers `ops::MulKernel` class and specialized template types `paddle::platform::CPUPlace` and `float`, which also registers `ops::MulGradKernel`. + + +- Registering GPU Kernel in `.cu` files + - Note that if GPU Kernel is implemented using the `Eigen unsupported` module, then on top of `.cu`, a macro definition `#define EIGEN_USE_GPU` is needed, such as + + ```cpp + // if use Eigen unsupported module before include head files + #define EIGEN_USE_GPU + + namespace ops = paddle::operators; + REGISTER_OP_GPU_KERNEL(mul, ops::MulKernel); + REGISTER_OP_GPU_KERNEL(mul_grad, + ops::MulGradKernel); + ``` + +### 5. Compilation + +Run the following commands to compile. + +``` +make mul_op +``` + +## Python Binding + +The system will automatically bind to Python and link it to a generated library. + +## Unit Tests + +Unit tests for an operator include + +1. comparing a forward operator's implementations on different devices, + +2. comparing a backward operator's implementation on different devices, and + +3. a scaling test for the backward operator. + +Here, we introduce the [unit tests for `MulOp`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/tests/test_mul_op.py). + +### Testing Forward Operators + +A forward operator unit test inherits `unittest.TestCase` and defines metaclass `__metaclass__ = OpTestMeta`. More concrete tests are performed in `OpTestMeta`. Testing a forward operator requires the following: + +1. Defining input, output and relevant attributes in `setUp` method. + +2. Generating random input data. + +3. Implementing the same computation logic in a Python script: + + ```python + import unittest + import numpy as np + from gradient_checker import GradientChecker, create_op + from op_test_util import OpTestMeta + + class TestMulOp(unittest.TestCase): + __metaclass__ = OpTestMeta + + def setUp(self): + self.type = "mul" + self.inputs = { + 'X': np.random.random((32, 84)).astype("float32"), + 'Y': np.random.random((84, 100)).astype("float32") + } + self.outputs = {'Out': np.dot(self.inputs['X'], self.inputs['Y'])} + ``` +Get its output, and compare it with the forward operator's own output. + +The code above first loads required packages. In addition, we have + +- `self.type = "mul" ` defines the type that is identical to what the operator's registered type. +- `self.inputs` defines input, with type `numpy.array` and initializes it. +- `self.outputs` defines output and completes the same operator computation in the Python script, and returns its result from the Python script. + +### Testing Backward Operators + +A backward operator unit test inherits `GradientChecker`, which inherits `unittest.TestCase`. As a result, **a backward operator unit test needs to be have the prefix `test_`**. + +```python +class TestMulGradOp(GradientChecker): + def setUp(self): + self.op = create_op("mul") + self.inputs = { + 'X': np.random.random((32, 84)).astype("float32"), + 'Y': np.random.random((84, 100)).astype("float32") + } + + def test_check_grad_normal(self): + # mul op will enlarge the relative error + self.check_grad(['X', 'Y'], 'Out', max_relative_error=0.5) + + def test_check_grad_ingore_x(self): + self.check_grad( + ['Y'], 'Out', max_relative_error=0.5, no_grad_set=set("X")) + + def test_check_grad_ingore_y(self): + self.check_grad( + ['X'], 'Out', max_relative_error=0.5, no_grad_set=set('Y')) +``` + +Some key points in the code above include: + +- `create_op("mul")` creates the backward operator's corresponding forward operator. +- `test_normal` calls `check_grad` to validate scaling tests' correctness and stability through numeric methods. + - The first variable `["X", "Y"]` appoints `X` and `Y` to be scale tested. + - The second variable `"Out"` points to the network's final output target `Out`. + - The third variable `max_relative_error` points to the maximum relative tolerance error during scaling tests. +- `test_check_grad_ingore_x` and `test_check_grad_ingore_y`branches test the cases where there is only one scaling input. + +### Compiling and Running + + +Any new unit testing file of the format `test_*.py` added to the director `python/paddle/v2/framework/tests` is automatically added to the project to compile. + +Note that **unlike the compile test for Ops, running unit tests requires compiling the entire project** and requires compiling with flag `WITH_TESTING` on i.e. `cmake paddle_dir -DWITH_TESTING=ON`. + +After successfully compiling the project, run the following command to run unit tests: + +```bash +make test ARGS="-R test_mul_op -V" +``` + +Or, + +```bash +ctest -R test_mul_op +``` + +## Remarks + +- Every `*_op.h` (if applicable), `*_op.cc`, and `*_op.cu` (if applicable) must be created for a unique Op. Compiling will fail if multiple operators are included per file. +- The type with which an operator is registered needs to be identical to the Op's name. Registering `REGISTER_OP(B, ...)` in `A_op.cc` will cause unit testing failures. +- If the operator does not implement a GPU kernel, please refrain from creating an empty `*_op.cu` file, or else unit tests will fail. +- If multiple operators rely on some shared methods, a file NOT named `*_op.*` can be created to store them, such as `gather.h`. diff --git a/doc/_sources/howto/dev/use_eigen_en.md.txt b/doc/_sources/howto/dev/use_eigen_en.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..e169106e12f5d62696f1f0e7163562793b32c18c --- /dev/null +++ b/doc/_sources/howto/dev/use_eigen_en.md.txt @@ -0,0 +1,146 @@ +## How to use Eigen in Paddle + +Essentially, a neural network is a compute graph. T data needed for the computation is stored in `Tensor`s and its computation procedure is described by `Operator`s. An `Operator` calls the `Compute` interface in its corresponding `OpKernel` and operates on the `Tensor`. + + +### Eigen Tensor Module + +The Eigen Tensor module supports powerful element-wise computation. In addition, a piece of code written using it can be run on both the CPU and the GPU. + +Note that Eigen Tensor is still being actively developed, so its tests are not completely covered and its documentation may be sparse. + +For details on Eigen Tensor module, please see [doc 1](https://github.com/RLovelett/eigen/blob/master/unsupported/Eigen/CXX11/src/Tensor/README.md) and [doc 2](https://bitbucket.org/eigen/eigen/src/default/unsupported/Eigen/CXX11/src/Tensor/README.md). + + +### paddle::framework::Tensor + +Paddle Tensor's is defined in the framework directory with the following interface: + +```cpp +class Tensor { + public: + /*! Return a pointer to mutable memory block. */ + template + inline T* data(); + + /** + * @brief Return a pointer to mutable memory block. + * @note If not exist, then allocation. + */ + template + inline T* mutable_data(platform::Place place); + + /** + * @brief Return a pointer to mutable memory block. + * + * @param[in] dims The dimensions of the memory block. + * @param[in] place The place of the memory block. + * + * @note If not exist, then allocation. + */ + template + inline T* mutable_data(DDim dims, platform::Place place); + + /*! Resize the dimensions of the memory block. */ + inline Tensor& Resize(const DDim& dims); + + /*! Return the dimensions of the memory block. */ + inline const DDim& dims() const; + + private: + /*! holds the memory block if allocated. */ + std::shared_ptr holder_; + + /*! points to dimensions of memory block. */ + DDim dim_; +}; +``` + +`Placeholder` is used to delay memory allocation; that is, we can first define a tensor, using `Resize` to configure its shape, and then call `mutuable_data` to allocate the actual memory. + +```cpp +paddle::framework::Tensor t; +paddle::platform::CPUPlace place; +// set size first +t.Resize({2, 3}); +// allocate memory on CPU later +t.mutable_data(place); +``` + +### paddle::framework::Tensor Usage +`AddOp` demonstrates Tensor's usage. + +- InferShape + +When computing a neural network's compute graph, first call every `Operator`'s `InferShape` method, and use `Resize` to configure the size of the output tensor. + +```cpp +void InferShape(const framework::InferShapeContext &ctx) const override { + PADDLE_ENFORCE_EQ(ctx.Input("X")->dims(), + ctx.Input("Y")->dims(), + "Two input of Add Op's dimension must be same."); + ctx.Output("Out")->Resize(ctx.Input("X")->dims()); +} +``` + + +- Run + +```cpp +void Compute(const framework::ExecutionContext& context) const override { + auto* input0 = context.Input("X"); + auto* input1 = context.Input("Y"); + auto* output = context.Output("Out"); + + output->mutable_data(context.GetPlace()); + + auto x = EigenVector::Flatten(*input0); + auto y = EigenVector::Flatten(*input1); + auto z = EigenVector::Flatten(*output); + + auto place = context.GetEigenDevice(); + + z.device(place) = x + y; +} +``` + + +### paddle::framework::Tensor到EigenTensor的转换 + +As shown above, in actual computation, we need to transform the input and output `Tensor`s into formats Eigen supports. We show some functions in [eigen.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/eigen.h) to implement the transformation from `paddle::framework::Tensor`to `EigenTensor/EigenMatrix/EigenVector/EigenScalar`. + +Using EigenTensor as an example: + +```cpp +Tensor t; +float* p = t.mutable_data(make_ddim({1, 2, 3}), platform::CPUPlace()); +for (int i = 0; i < 1 * 2 * 3; i++) { + p[i] = static_cast(i); +} + +EigenTensor::Type et = EigenTensor::From(t); +``` + +`From` is an interfacing method provided by the EigenTensor template, which implements the transformation from a `paddle::framework::Tensor` object to an EigenTensor. Since `rank` is a template parameter, it needs to be explicitly specified at the time of the transformation. + +In Eigen, tensors with different ranks are different types, with `Vector` bring a rank-1 instance. Note that `EigenVector::From` uses a transformation from an 1-dimensional Paddle tensor to a 1-dimensional Eigen tensor while `EigenVector::Flatten` reshapes a paddle tensor and flattens it into a 1-dimensional Eigen tensor. Both resulting tensors are still typed EigenVector. + +For more transformations, see the [unit tests](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/eigen_test.cc) in the `eigen_test.cc` file. + + + +### Implementing Computation + +While computing, the device interface is needed from the EigenTensors on the left hand side of the assignments. Note that the computation between EigenTensors only changes the data originally inthe Tensor and does not change all the shape information associated with the Tensor. + +```cpp +auto x = EigenVector::Flatten(*input0); +auto y = EigenVector::Flatten(*input1); +auto z = EigenVector::Flatten(*output); +auto place = context.GetEigenDevice(); +z.device(place) = x + y; +``` + +In this code segment, input0/input1/output can be Tensors of arbitrary dimension. We are calling Flatten from EigenVector, transforming a tensor of any dimension into a 1-dimensional EigenVector. After completing computation, input0/input1/output will retain the same shape information, and they can be resized using the `Resize` interface. + +Because the Eigen Tensor module is under-documented, please refer to `OpKernel`'s computation code in TensorFlow's [kernel module documentation](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/kernels). diff --git a/doc/_sources/howto/dev/write_docs_en.rst.txt b/doc/_sources/howto/dev/write_docs_en.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..b3ef07eb1d0012827df8e6a4f27c5fa643649492 --- /dev/null +++ b/doc/_sources/howto/dev/write_docs_en.rst.txt @@ -0,0 +1,80 @@ +################## +Contribute Documentation +################## + +PaddlePaddle supports English documentation ``doc`` and Chinese documentation ``doc_cn``. +Both are compiled by `cmake`_ and `sphinx`_ , the compiled documentations will be stored under ``doc`` and ``doc_cn`` directories. +When using the PaddlePaddle.org to compile documentations, the compiled documentations will be stored under a consolidated directory: .ppo_workspace/content + +How to Build Documentations +============ + +We recommend using PaddlePaddle.org tool to build documentation + + +Use PaddlePaddle.org tool +-------------- +This is the recommended method to build documentation. It can compile documentation and preview the documentation in a web browser. + +The tool uses Docker, please install it on your system. Please check Docker official website on how to install Docker. You may use the following commands to activate the tool + +.. code-block:: bash + + mkdir paddlepaddle # Create paddlepaddle working directory + cd paddlepaddle + + # Clone the content repositories. You may only clone the contents you need + git clone https://github.com/PaddlePaddle/Paddle.git + git clone https://github.com/PaddlePaddle/book.git + git clone https://github.com/PaddlePaddle/models.git + git clone https://github.com/PaddlePaddle/Mobile.git + + # Please specify the working directory through -v + docker run -it -p 8000:8000 -v `pwd`:/var/content paddlepaddle/paddlepaddle.org:latest + +Note: PaddlePaddle.org will read the content repos specified in the -v (volume) flag of the docker run command +Use a web browser and navigate to http://localhost:8000, click the buttons to compile the documentation +The compiled documentations will be stored in /.ppo_workspace/content + + +If you don't wish to use Docker, you can also activate the tool through Django. Use the following the commands to set up + +.. code-block:: bash + + mkdir paddlepaddle # Create paddlepaddle working directory + cd paddlepaddle + + # Clone the content repositories and PaddlePaddle.org + git clone https://github.com/PaddlePaddle/Paddle.git + git clone https://github.com/PaddlePaddle/book.git + git clone https://github.com/PaddlePaddle/models.git + git clone https://github.com/PaddlePaddle/Mobile.git + git clone https://github.com/PaddlePaddle/PaddlePaddle.org.git + + # Please specify the PaddlePaddle working directory. In the current setting, it should be pwd + export CONTENT_DIR= + export ENV='' + cd PaddlePaddle.org/portal/ + pip install -r requirements.txt + python manage.py runserver + +Use a web browser and navigate to http://localhost:8000, click the buttons to compile the documentation +The compiled documentations will be stored in /.ppo_workspace/content + +If you want to learn more on the PaddlePaddle.org, please `click here `_ 。 + +How to write Documentations +============ + +PaddlePaddle uses `sphinx`_ to compile documentations，Please check sphinx official website for more detail. + + +How to update www.paddlepaddle.org +============================ + +Please create PRs and submit them to github, please check `Contribute Code `_ 。 +PaddlePaddle develop branch will update the documentation once the PR is merged. User may check latest `Chinese Docs `_ and +`English Docs `_ 。 + +.. _cmake: https://cmake.org/ +.. _sphinx: http://www.sphinx-doc.org/en/1.4.8/ diff --git a/doc/_sources/howto/index_en.rst.txt b/doc/_sources/howto/index_en.rst.txt index 1fbfcd260b912078f00ed5b720ed607db725c4e2..61bf25ccd12eeedffc747fdd4ce84fa4adde07ee 100644 --- a/doc/_sources/howto/index_en.rst.txt +++ b/doc/_sources/howto/index_en.rst.txt @@ -20,6 +20,7 @@ Development dev/new_layer_en.rst dev/contribute_to_paddle_en.md + dev/write_docs_en.rst Configuration ------------- diff --git a/doc/_sources/howto/optimization/cpu_profiling.md.txt b/doc/_sources/howto/optimization/cpu_profiling.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..1775374cf6e518586c28bbd8e04946c74df7e4c5 --- /dev/null +++ b/doc/_sources/howto/optimization/cpu_profiling.md.txt @@ -0,0 +1,197 @@ +This tutorial introduces techniques we use to profile and tune the +CPU performance of PaddlePaddle. We will use Python packages +`cProfile` and `yep`, and Google's `perftools`. + +Profiling is the process that reveals performance bottlenecks, +which could be very different from what's in the developers' mind. +Performance tuning is done to fix these bottlenecks. Performance optimization +repeats the steps of profiling and tuning alternatively. + +PaddlePaddle users program AI applications by calling the Python API, which calls +into `libpaddle.so.` written in C++. In this tutorial, we focus on +the profiling and tuning of + +1. the Python code and +1. the mixture of Python and C++ code. + +## Profiling the Python Code + +### Generate the Performance Profiling File + +We can use Python standard +package, [`cProfile`](https://docs.python.org/2/library/profile.html), +to generate Python profiling file. For example: + +```bash +python -m cProfile -o profile.out main.py +``` + +where `main.py` is the program we are going to profile, `-o` specifies +the output file. Without `-o`, `cProfile` would outputs to standard +output. + +### Look into the Profiling File + +`cProfile` generates `profile.out` after `main.py` completes. We can +use [`cprofilev`](https://github.com/ymichael/cprofilev) to look into +the details: + +```bash +cprofilev -a 0.0.0.0 -p 3214 -f profile.out main.py +``` + +where `-a` specifies the HTTP IP, `-p` specifies the port, `-f` +specifies the profiling file, and `main.py` is the source file. + +Open the Web browser and points to the local IP and the specifies +port, we will see the output like the following: + +``` + ncalls tottime percall cumtime percall filename:lineno(function) + 1 0.284 0.284 29.514 29.514 main.py:1() + 4696 0.128 0.000 15.748 0.003 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/executor.py:20(run) + 4696 12.040 0.003 12.040 0.003 {built-in method run} + 1 0.144 0.144 6.534 6.534 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/__init__.py:14() +``` + +where each line corresponds to Python function, and the meaning of +each column is as follows: + +| column | meaning | +| --- | --- | +| ncalls | the number of calls into a function | +| tottime | the total execution time of the function, not including the + execution time of other functions called by the function | +| percall | tottime divided by ncalls | +| cumtime | the total execution time of the function, including the execution time of other functions being called | +| percall | cumtime divided by ncalls | +| filename:lineno(function) | where the function is defined | + +### Identify Performance Bottlenecks + +Usually, `tottime` and the related `percall` time is what we want to +focus on. We can sort above profiling file by tottime: + +```text + 4696 12.040 0.003 12.040 0.003 {built-in method run} + 300005 0.874 0.000 1.681 0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/dataset/mnist.py:38(reader) + 107991 0.676 0.000 1.519 0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:219(__init__) + 4697 0.626 0.000 2.291 0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp) + 1 0.618 0.618 0.618 0.618 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/__init__.py:1() +``` + +We can see that the most time-consuming function is the `built-in +method run`, which is a C++ function in `libpaddle.so`. We will +explain how to profile C++ code in the next section. At this +moment, let's look into the third function `sync_with_cpp`, which is a +Python function. We can click it to understand more about it: + +``` +Called By: + + Ordered by: internal time + List reduced from 4497 to 2 due to restriction <'sync_with_cpp'> + +Function was called by... + ncalls tottime cumtime +/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp) <- 4697 0.626 2.291 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp) +/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp) <- 4696 0.019 2.316 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:487(clone) + 1 0.000 0.001 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:534(append_backward) + + +Called: + + Ordered by: internal time + List reduced from 4497 to 2 due to restriction <'sync_with_cpp'> +``` + +The lists of the callers of `sync_with_cpp` might help us understand +how to improve the function definition. + +## Profiling Python and C++ Code + +### Generate the Profiling File + +To profile a mixture of Python and C++ code, we can use a Python +package, `yep`, that can work with Google's `perftools`, which is a +commonly-used profiler for C/C++ code. + +In Ubuntu systems, we can install `yep` and `perftools` by running the +following commands: + +```bash +apt update +apt install libgoogle-perftools-dev +pip install yep +``` + +Then we can run the following command + +```bash +python -m yep -v main.py +``` + +to generate the profiling file. The default filename is +`main.py.prof`. + +Please be aware of the `-v` command line option, which prints the +analysis results after generating the profiling file. By examining the + the print result, we'd know that if we stripped debug +information from `libpaddle.so` at build time. The following hints +help make sure that the analysis results are readable: + +1. Use GCC command line option `-g` when building `libpaddle.so` so to + include the debug information. The standard building system of + PaddlePaddle is CMake, so you might want to set + `CMAKE_BUILD_TYPE=RelWithDebInfo`. + +1. Use GCC command line option `-O2` or `-O3` to generate optimized + binary code. It doesn't make sense to profile `libpaddle.so` + without optimization, because it would anyway run slowly. + +1. Profiling the single-threaded binary file before the + multi-threading version, because the latter often generates tangled + profiling analysis result. You might want to set environment + variable `OMP_NUM_THREADS=1` to prevents OpenMP from automatically + starting multiple threads. + +### Examining the Profiling File + +The tool we used to examine the profiling file generated by +`perftools` is [`pprof`](https://github.com/google/pprof), which +provides a Web-based GUI like `cprofilev`. + +We can rely on the standard Go toolchain to retrieve the source code +of `pprof` and build it: + +```bash +go get github.com/google/pprof +``` + +Then we can use it to profile `main.py.prof` generated in the previous +section: + +```bash +pprof -http=0.0.0.0:3213 `which python` ./main.py.prof +``` + +Where `-http` specifies the IP and port of the HTTP service. +Directing our Web browser to the service, we would see something like +the following: + +![result](./pprof_1.png) + +### Identifying the Performance Bottlenecks + +Similar to how we work with `cprofilev`, we'd focus on `tottime` and +`cumtime`. + +![kernel_perf](./pprof_2.png) + +We can see that the execution time of multiplication and the computing +of the gradient of multiplication takes 2% to 4% of the total running +time, and `MomentumOp` takes about 17%. Obviously, we'd want to +optimize `MomentumOp`. + +`pprof` would mark performance critical parts of the program in +red. It's a good idea to follow the hints. diff --git a/doc/_sources/howto/usage/cluster/cluster_train_en.md.txt b/doc/_sources/howto/usage/cluster/cluster_train_en.md.txt index c60876721cbf5565d6e48c8061811aacada748cd..baa97c0c02ae490fff8587071bd2d4adfb5325e3 100644 --- a/doc/_sources/howto/usage/cluster/cluster_train_en.md.txt +++ b/doc/_sources/howto/usage/cluster/cluster_train_en.md.txt @@ -1,129 +1,220 @@ -# Run Distributed Training +# PaddlePaddle Distributed Training + +* [Introduction](#introduction) +* [Preparations](#preparations) +* [Command-line arguments](#command-line-arguments) + * [Starting parameter server](#starting-parameter-server) + * [Starting trainer](#starting-trainer) + * [Prepare Training Dataset](#prepare-training-dataset) + * [Prepare Training program](#prepare-training-program) +* [Use cluster platforms or cluster management tools](#use-cluster-platforms-or-cluster-management-tools) + * [Cluster Training Using Fabric](#cluster-training-using-fabric) + * [Prepare a Linux cluster](#prepare-a-linux-cluster) + * [Launching Cluster Job](#launching-cluster-job) + * [Kill Cluster Job](#kill-cluster-job) + * [Check Cluster Training Result](#check-cluster-training-result) + * [Check Model Output](#check-model-output) + * [Cluster Training Using OpenMPI](#cluster-training-using-openmpi) + * [Prepare an OpenMPI cluster](#prepare-an-openmpi-cluster) + * [Launching Cluster Job](#launching-cluster-job-1) + * [Cluster Training Using Kubernetes](#cluster-training-using-kubernetes) + +## Introduction + +In this article, we'll explain how to run distributed training jobs with PaddlePaddle on different types of clusters. The diagram below shows the main architecture of a distributed trainning job: + + + +- Data shard: training data will be split into multiple partitions, trainers use the partitions of the whole dataset to do the training job. +- Trainer: each trainer reads the data shard, and train the neural network. Then the trainer will upload calculated "gradients" to parameter servers, and wait for parameters to be optimized on the parameter server side. When that finishes, the trainer download optimized parameters and continues its training. +- Parameter server: every parameter server stores part of the whole neural network model data. They will do optimization calculations when gradients are uploaded from trainers, and then send updated parameters to trainers. + +PaddlePaddle can support both synchronize stochastic gradient descent (SGD) and asynchronous SGD. + +When training with synchronize SGD, PaddlePaddle uses an internal "synchronize barrier" which makes gradients update and parameter download in strict order. On the other hand, asynchronous SGD won't wait for all trainers to finish upload at a single step, this will increase the parallelism of distributed training: parameter servers do not depend on each other, they'll do parameter optimization concurrently. Parameter servers will not wait for trainers, so trainers will also do their work concurrently. But asynchronous SGD will introduce more randomness and noises in the gradient. + +## Preparations +1. Prepare your computer cluster. It's normally a bunch of Linux servers connected by LAN. Each server will be assigned a unique IP address. The computers in the cluster can be called "nodes". +2. Install PaddlePaddle on every node. If you are going to take advantage of GPU cards, you'll also need to install proper driver and CUDA libraries. To install PaddlePaddle please read [this build and install](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/getstarted/build_and_install) document. We strongly recommend using [Docker installation](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst). + +After installation, you can check the version by typing the below command (run a docker container if using docker: `docker run -it paddlepaddle/paddle:[tag] /bin/bash`): + +```bash +$ paddle version +PaddlePaddle 0.10.0rc, compiled with + with_avx: ON + with_gpu: OFF + with_double: OFF + with_python: ON + with_rdma: OFF + with_timer: OFF +``` -In this article, we explain how to run distributed Paddle training jobs on clusters. We will create the distributed version of the single-process training example, [recommendation](https://github.com/baidu/Paddle/tree/develop/demo/recommendation). +We'll take `doc/howto/usage/cluster/src/word2vec` as an example to introduce distributed training using PaddlePaddle v2 API. -[Scripts](https://github.com/baidu/Paddle/tree/develop/paddle/scripts/cluster_train) used in this article launch distributed jobs via SSH. They also work as a reference for users running more sophisticated cluster management systems like MPI and [Kubernetes](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/howto/usage/k8s). +## Command-line arguments -## Prerequisite +### Starting parameter server -1. Aforementioned scripts use a Python library [fabric](http://www.fabfile.org/) to run SSH commands. We can use `pip` to install fabric: +Type the below command to start a parameter server which will wait for trainers to connect: - ```bash - pip install fabric - ``` +```bash +$ paddle pserver --port=7164 --ports_num=1 --ports_num_for_sparse=1 --num_gradient_servers=1 +``` -1. We need to install PaddlePaddle on all nodes in the cluster. To enable GPUs, we need to install CUDA in `/usr/local/cuda`; otherwise Paddle would report errors at runtime. +If you wish to run parameter servers in background, and save a log file, you can type: +```bash +$ stdbuf -oL /usr/bin/nohup paddle pserver --port=7164 --ports_num=1 --ports_num_for_sparse=1 --num_gradient_servers=1 &> pserver.log +``` -1. Set the `ROOT_DIR` variable in [`cluster_train/conf.py`] on all nodes. For convenience, we often create a Unix user `paddle` on all nodes and set `ROOT_DIR=/home/paddle`. In this way, we can write public SSH keys into `/home/paddle/.ssh/authorized_keys` so that user `paddle` can SSH to all nodes without password. +| param | required | default | description | +| ------------- | ------------- | ------------- | ------------- | +| port | required | 7164 | port which parameter server will listen on. If ports_num greater than 1, parameter server will listen on multiple ports for more network throughput | +| ports_num | required | 1 | total number of ports will listen on | +| ports_num_for_sparse | required | 1 | number of ports which serves sparse parameter update | +| num_gradient_servers | required | 1 | total number of gradient servers | -## Prepare Job Workspace +### Starting trainer +Type the command below to start the trainer(name the file whatever you want, like "train.py") -We refer to the directory where we put dependent libraries, config files, etc., as *workspace*. +```bash +$ python train.py +``` -These `train/test` data should be prepared before launching cluster job. To satisfy the requirement that train/test data are placed in different directory from workspace, PADDLE refers train/test data according to index file named as `train.list/test.list` which are used in model config file. So the train/test data also contains train.list/test.list two list file. All local training demo already provides scripts to help you create these two files, and all nodes in cluster job will handle files with same logical code in normal condition. +Trainers' network need to be connected with parameter servers' network to finish the job. Trainers need to know port and IPs to locate parameter servers. You can pass arguments to trainers through [environment variables](https://en.wikipedia.org/wiki/Environment_variable) or pass to `paddle.init()` function. Arguments passed to the `paddle.init()` function will overwrite environment variables. -Generally, you can use same model file from local training for cluster training. What you should have in mind that, the `batch_size` set in `setting` function in model file means batch size in `each` node of cluster job instead of total batch size if synchronization SGD was used. +Use environment viriables: -Following steps are based on [demo/recommendation](https://github.com/PaddlePaddle/Paddle/tree/develop/demo/recommendation) demo in demo directory. +```bash +export PADDLE_INIT_USE_GPU=False +export PADDLE_INIT_TRAINER_COUNT=1 +export PADDLE_INIT_PORT=7164 +export PADDLE_INIT_PORTS_NUM=1 +export PADDLE_INIT_PORTS_NUM_FOR_SPARSE=1 +export PADDLE_INIT_NUM_GRADIENT_SERVERS=1 +export PADDLE_INIT_TRAINER_ID=0 +export PADDLE_INIT_PSERVERS=127.0.0.1 +python train.py +``` -You just go through demo/recommendation tutorial doc until `Train` section, and at last you will get train/test data and model configuration file. Finaly, just use demo/recommendation as workspace for cluster training. +Pass arguments: -At last your workspace should look like as follow: +```python +paddle.init( + use_gpu=False, + trainer_count=1, + port=7164, + ports_num=1, + ports_num_for_sparse=1, + num_gradient_servers=1, + trainer_id=0, + pservers="127.0.0.1") ``` -. -|-- common_utils.py -|-- data -| |-- config.json -| |-- config_generator.py -| |-- meta.bin -| |-- meta_config.json -| |-- meta_generator.py -| |-- ml-1m -| |-- ml_data.sh -| |-- ratings.dat.test -| |-- ratings.dat.train -| |-- split.py -| |-- test.list -| `-- train.list -|-- dataprovider.py -|-- evaluate.sh -|-- prediction.py -|-- preprocess.sh -|-- requirements.txt -|-- run.sh -`-- trainer_config.py + +| param | required | default | description | +| ------------- | ------------- | ------------- | ------------- | +| use_gpu | optional | False | set to "True" to enable GPU training | +| trainer_count | required | 1 | total count of trainers in the training job | +| port | required | 7164 | port to connect to parameter server | +| ports_num | required | 1 | number of ports for communication | +| ports_num_for_sparse | required | 1 | number of ports for sparse type caculation | +| num_gradient_servers | required | 1 | total number of gradient server | +| trainer_id | required | 0 | ID for every trainer, start from 0 | +| pservers | required | 127.0.0.1 | list of IPs of parameter servers, separated by "," | + +### Prepare Training Dataset + +Here's some example code [prepare.py](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/howto/usage/cluster/src/word2vec/prepare.py), it will download public `imikolov` dataset and split it into multiple files according to job parallelism(trainers count). Modify `SPLIT_COUNT` at the begining of `prepare.py` to change the count of output files. + +In the real world, we often use `MapReduce` job's output as training data, so there will be lots of files. You can use `mod` to assign training file to trainers: + +```python +import os +train_list = [] +flist = os.listdir("/train_data/") +for f in flist: + suffix = int(f.split("-")[1]) + if suffix % TRAINER_COUNT == TRAINER_ID: + train_list.append(f) ``` -Not all of these files are needed for cluster training, but it's not necessary to remove useless files. -`trainer_config.py` -Indicates the model config file. +Example code `prepare.py` will split training data and testing data into 3 files with digital suffix like `-00000`, `-00001` and`-00002`: -`train.list` and `test.list` -File index. It stores all relative or absolute file paths of all train/test data at current node. +``` +train.txt +train.txt-00000 +train.txt-00001 +train.txt-00002 +test.txt +test.txt-00000 +test.txt-00001 +test.txt-00002 +``` -`dataprovider.py` -used to read train/test samples. It's same as local training. +When job started, every trainer needs to get it's own part of data. In some distributed systems a storage service will be provided, so the date under that path can be accessed by all the trainer nodes. Without the storage service, you must copy the training data to each trainer node. -`data` -all files in data directory are refered by train.list/test.list which are refered by data provider. +Different training jobs may have different data format and `reader()` function, developers may need to write different data prepare scripts and `reader()` functions for their job. +### Prepare Training program -## Prepare Cluster Job Configuration +We'll create a *workspace* directory on each node, storing your training program, dependencies, mounted or downloaded dataset directory. -The options below must be carefully set in cluster_train/conf.py -`HOSTS` all nodes hostname or ip that will run cluster job. You can also append user and ssh port with hostname, such as root@192.168.100.17:9090. +Your workspace may looks like: +``` +. +|-- my_lib.py +|-- word_dict.pickle +|-- train.py +|-- train_data_dir/ +| |-- train.txt-00000 +| |-- train.txt-00001 +| |-- train.txt-00002 +`-- test_data_dir/ + |-- test.txt-00000 + |-- test.txt-00001 + `-- test.txt-00002 +``` -`ROOT_DIR` workspace ROOT directory for placing JOB workspace directory +- `my_lib.py`: user defined libraries, like PIL libs. This is optional. +- `word_dict.pickle`: dict file for training word embeding. +- `train.py`: training program. Sample code: [api_train_v2_cluster.py](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/howto/usage/cluster/src/word2vec/prepare.py). ***NOTE:*** You may need to modify the head part of `train.py` when using different cluster platform to retrive configuration environment variables: -`PADDLE_NIC` the NIC(Network Interface Card) interface name for cluster communication channel, such as eth0 for ethternet, ib0 for infiniband. + ```python + cluster_train_file = "./train_data_dir/train/train.txt" + cluster_test_file = "./test_data_dir/test/test.txt" + node_id = os.getenv("OMPI_COMM_WORLD_RANK") + if not node_id: + raise EnvironmentError("must provied OMPI_COMM_WORLD_RANK") + ``` -`PADDLE_PORT` port number for cluster commnunication channel +- `train_data_dir`: containing training data. Mount from storage service or copy trainning data to here. +- `test_data_dir`: containing testing data. -`PADDLE_PORTS_NUM` the number of port used for cluster communication channle. if the number of cluster nodes is small(less than 5~6nodes), recommend you set it to larger, such as 2 ~ 8, for better network performance. +## Use cluster platforms or cluster management tools -`PADDLE_PORTS_NUM_FOR_SPARSE` the number of port used for sparse updater cluster commnunication channel. if sparse remote update is used, set it like `PADDLE_PORTS_NUM` +PaddlePaddle supports running jobs on several platforms including: +- [Kubernetes](http://kubernetes.io) open-source system for automating deployment, scaling, and management of containerized applications from Google. +- [OpenMPI](https://www.open-mpi.org) Mature high performance parallel computing framework. +- [Fabric](http://www.fabfile.org) A cluster management tool. Write scripts to submit jobs or manage the cluster. -`LD_LIBRARY_PATH` set addtional LD_LIBRARY_PATH for cluster job. You can use it to set CUDA libraries path. +We'll introduce cluster job management on these platforms. The examples can be found under [cluster_train_v2](https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/scripts/cluster_train_v2). -Default Configuration as follow: +These cluster platforms provide API or environment variables for training processes, when the job is dispatched to different nodes. Like node ID, IP or total number of nodes etc. -```python -HOSTS = [ - "root@192.168.100.17", - "root@192.168.100.18", - ] - -''' -workspace configuration -''' - -#root dir for workspace -ROOT_DIR = "/home/paddle" - -''' -network configuration -''' -#pserver nics -PADDLE_NIC = "eth0" -#pserver port -PADDLE_PORT = 7164 -#pserver ports num -PADDLE_PORTS_NUM = 2 -#pserver sparse ports num -PADDLE_PORTS_NUM_FOR_SPARSE = 2 - -#environments setting for all processes in cluster job -LD_LIBRARY_PATH="/usr/local/cuda/lib64:/usr/lib64" -``` +### Cluster Training Using Fabric -### Launching Cluster Job -`paddle.py` provides automatical scripts to start all PaddlePaddle cluster processes in different nodes. By default, all command line options can set as `paddle.py` command options and `paddle.py` will transparently and automatically set these options to PaddlePaddle lower level processes. +#### Prepare a Linux cluster + +Run `kubectl -f ssh_servers.yaml` under the directory: `paddle/scripts/cluster_train_v2/fabric/docker_cluster` will launch a demo cluster. Run `kubectl get po -o wide` to get IP addresses of these nodes. + +#### Launching Cluster Job +`paddle.py` provides automatical scripts to start all PaddlePaddle cluster processes in different nodes. By default, all command line options can be set as `paddle.py` command options and `paddle.py` will transparently and automatically set these options to PaddlePaddle lower level processes. `paddle.py`provides two distinguished command option for easy job launching. -`job_dispatch_package` set it with local `workspace`directory, it will be dispatched to all nodes set in conf.py. It could be helpful for frequent hacking workspace files, otherwise frequent mulit-nodes workspace deployment could make your crazy. -`job_workspace` set it with already deployed workspace directory, `paddle.py` will skip dispatch stage to directly launch cluster job with all nodes. It could help to reduce heavy +- `job_dispatch_package` set it with local `workspace` directory, it will be dispatched to all nodes which is set in `conf.py`. It could be helpful for frequently manipulating workspace files. otherwise, frequent multi-nodes workspace deployment is very annoying. +- `job_workspace` set it with already deployed workspace directory, `paddle.py` will skip dispatch stage to directly launch cluster job with all nodes. It could help to reduce heavy dispatch latency. `cluster_train/run.sh` provides command line sample to run `demo/recommendation` cluster job, just modify `job_dispatch_package` and `job_workspace` with your defined directory, then: @@ -133,24 +224,70 @@ sh run.sh The cluster Job will start in several seconds. -### Kill Cluster Job -`paddle.py` can capture `Ctrl + C` SIGINT signal to automatically kill all processes launched by it. So just stop `paddle.py` to kill cluster job. You should mannally kill job if program crashed. +#### Kill Cluster Job +`paddle.py` can capture `Ctrl + C` SIGINT signal to automatically kill all processes launched by it. So just stop `paddle.py` to kill cluster job. You should manually kill the job if the program crashed. -### Check Cluster Training Result +#### Check Cluster Training Result Check log in $workspace/log for details, each node owns same log structure. `paddle_trainer.INFO` -It provides almost all interal output log for training, same as local training. Check runtime model convergence here. +It provides almost all internal output log for training, same as local training. Check runtime model convergence here. `paddle_pserver2.INFO` -It provides pserver running log, which could help to diagnose distributed error. +It provides parameter server running log, which could help to diagnose distributed error. `server.log` -It provides stderr and stdout of pserver process. Check error log if training crashs. +It provides stderr and stdout of parameter server process. Check error log if training crashes. `train.log` -It provides stderr and stdout of trainer process. Check error log if training crashs. +It provides stderr and stdout of trainer process. Check error log if training crashes. -### Check Model Output -After one pass finished, model files will be writed in `output` directory in node 0. +#### Check Model Output +After one pass finished, model files will be written in `output` directory in node 0. `nodefile` in workspace indicates the node id of current cluster job. + +### Cluster Training Using OpenMPI + +#### Prepare an OpenMPI cluster + +Run the following command to start a 3-node MPI cluster and one "head" node. + +```bash +cd paddle/scripts/cluster_train_v2/openmpi/docker_cluster +kubectl create -f head.yaml +kubectl create -f mpi-nodes.yaml +``` + +Then you can log in to every OpenMPI node using ssh without input any passwords. + +#### Launching Cluster Job + +Follow the steps to launch a PaddlePaddle training job in OpenMPI cluster:\ + +```bash +# find out node IP addresses +kubectl get po -o wide +# generate a "machines" file containing node IP addresses +kubectl get po -o wide | grep nodes | awk '{print $6}' > machines +# copy necessary files onto "head" node +scp -i ssh/id_rsa.mpi.pub machines prepare.py train.py start_mpi_train.sh tutorial@[headIP]:~ +# login to head node using ssh +ssh -i ssh/id_rsa.mpi.pub tutorial@[headIP] +# --------------- in head node --------------- +# prepare training data +python prepare.py +# copy training data and dict file to MPI nodes +cat machines | xargs -i scp word_dict.pickle train.py start_mpi_train.sh machines {}:/home/tutorial +# creat a directory for storing log files +mpirun -hostfile machines -n 3 mkdir /home/tutorial/logs +# copy training data to every node +scp train.txt-00000 test.txt-00000 [node1IP]:/home/tutorial +scp train.txt-00001 test.txt-00001 [node2IP]:/home/tutorial +scp train.txt-00002 test.txt-00002 [node3IP]:/home/tutorial +# start the job +mpirun -hostfile machines -n 3 /home/tutorial/start_mpi_train.sh +``` + +### Cluster Training Using Kubernetes + +The details can be found [here](../k8s/k8s_cn.md) diff --git a/doc/_sources/index_en.rst.txt b/doc/_sources/index_en.rst.txt index 168c7667c61da677905585d6c4b5037ce80b3765..23b64b6cadf776d44c4d0aa5a550ffe24be13b18 100644 --- a/doc/_sources/index_en.rst.txt +++ b/doc/_sources/index_en.rst.txt @@ -7,4 +7,4 @@ PaddlePaddle Documentation getstarted/index_en.rst howto/index_en.rst api/index_en.rst - about/index_en.rst + mobile/index_en.rst diff --git a/doc/_sources/mobile/cross_compiling_for_android_en.md.txt b/doc/_sources/mobile/cross_compiling_for_android_en.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..26858581fc1d77a9391520ac0dfd80fbd98f508c --- /dev/null +++ b/doc/_sources/mobile/cross_compiling_for_android_en.md.txt @@ -0,0 +1,175 @@ +# Build PaddlePaddle for Android + +There are two approaches to build PaddlePaddle for Android: using Docker and on Linux without Docker. + +## Cross-Compiling Using Docker + +Docker-based cross-compiling is the recommended approach because Docker runs on all major operating systems, including Linux, Mac OS X, and Windows. + +### Build the Docker Image + +The following steps pack all the tools that we need to build PaddlePaddle into a Docker image. + +```bash +$ git clone https://github.com/PaddlePaddle/Paddle.git +$ cd Paddle +$ docker build -t paddle:dev-android . -f Dockerfile.android +``` + +### Build the Inference Library + +We can run the Docker image we just created to build the inference library of PaddlePaddle for Android using the command below: + +```bash +$ docker run -it --rm -v $PWD:/paddle -e "ANDROID_ABI=armeabi-v7a" -e "ANDROID_API=21" paddle:dev-android +``` + +The Docker image accepts two arguments `ANDROID_ABI` and `ANDROID_API`: + + ++ + + + + + + + + + + + + + + + + + + + + + + +
ArgumentOptional ValuesDefault
ANDROID_ABIarmeabi-v7a, arm64-v8aarmeabi-v7a
ANDROID_API>= 2121
+ +The ARM-64 architecture (`arm64-v8a`) requires at least level 21 of Android API. + +The default entry-point of the Docker image, [`paddle/scripts/docker/build_android.sh`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build_android.sh) generates the [Android cross-compiling standalone toolchain](https://developer.android.com/ndk/guides/standalone_toolchain.html) based on the argument: `ANDROID_ABI` or `ANDROID_API`. For information about other configuration arguments, please continue reading. + +The above command generates and outputs the inference library in `$PWD/install_android` and puts third-party libraries in `$PWD/install_android/third_party`. + +## Cross-Compiling on Linux + +The Linux-base approach to cross-compile is to run steps in `Dockerfile.android` manually on a Linux x64 computer. + +### Setup the Environment + +To build for Android's, we need [Android NDK]( +https://developer.android.com/ndk/downloads/index.html): + +```bash +wget -q https://dl.google.com/android/repository/android-ndk-r14b-linux-x86_64.zip +unzip -q android-ndk-r14b-linux-x86_64.zip +``` + +Android NDK includes everything we need to build the [*standalone toolchain*](https://developer.android.com/ndk/guides/standalone_toolchain.html), which in then used to build PaddlePaddle for Android. (We plan to remove the intermediate stage of building the standalone toolchain in the near future.) + +- To build the standalone toolchain for `armeabi-v7a` and Android API level 21: + + ```bash + your/path/to/android-ndk-r14b-linux-x86_64/build/tools/make-standalone-toolchain.sh \ + --arch=arm --platform=android-21 --install-dir=your/path/to/arm_standalone_toolchain + ``` + + The generated standalone toolchain will be in `your/path/to/arm_standalone_toolchain`. + +- To build the standalone toolchain for `arm64-v8a` and Android API level 21: + + ```bash + your/path/to/android-ndk-r14b-linux-x86_64/build/tools/make-standalone-toolchain.sh \ + --arch=arm64 --platform=android-21 --install-dir=your/path/to/arm64_standalone_toolchain + ``` + + The generated standalone toolchain will be in `your/path/to/arm64_standalone_toolchain`. + +**Please be aware that the minimum level of Android API required by PaddlePaddle is 21.** + +### Cross-Compiling Arguments + +CMake supports [choosing the toolchain](https://cmake.org/cmake/help/v3.0/manual/cmake-toolchains.7.html#cross-compiling). PaddlePaddle provides [`android.cmake`](https://github.com/PaddlePaddle/Paddle/blob/develop/cmake/cross_compiling/android.cmake), which configures the Android cross-compiling toolchain for CMake. `android.cmake` is not required for CMake >= 3.7, which support Android cross-compiling. PaddlePaddle detects the CMake version, for those newer than 3.7, it uses [the official version](https://cmake.org/cmake/help/v3.7/manual/cmake-toolchains.7.html#cross-compiling). + +Some other CMake arguments you need to know: + +- `CMAKE_SYSTEM_NAME` must be `Android`. This tells PaddlePaddle's CMake system to cross-compile third-party dependencies. This also changes some other CMake arguments like `WITH_GPU=OFF`, `WITH_AVX=OFF`, `WITH_PYTHON=OFF`, and `WITH_RDMA=OFF`. +- `WITH_C_API` must be `ON`, to build the C-based inference library for Android. +- `WITH_SWIG_PY` must be `OFF` because the Android platform doesn't support SWIG-based API. + +Some Android-specific arguments: + +- `ANDROID_STANDALONE_TOOLCHAIN`: the absolute path of the Android standalone toolchain, or the path relative to the CMake build directory. PaddlePaddle's CMake extensions would derive the cross-compiler, sysroot and Android API level from this argument. +- `ANDROID_TOOLCHAIN`: could be `gcc` or `clang`. The default value is `clang`. + - For CMake >= 3.7, it should anyway be `clang`. For older versions, it could be `gcc`. + - Android's official `clang` requires `glibc` >= 2.15. +- `ANDROID_ABI`: could be `armeabi-v7a` or `arm64-v8a`. The default value is `armeabi-v7a`. +- `ANDROID_NATIVE_API_LEVEL`: could be derived from the value of `ANDROID_STANDALONE_TOOLCHAIN`. +- `ANROID_ARM_MODE`: + - could be `ON` or `OFF`, and defaults to `ON`, when `ANDROID_ABI=armeabi-v7a`; + - no need to specify when `ANDROID_ABI=arm64-v8a`. +- `ANDROID_ARM_NEON`: indicates if to use NEON instructions. + - could be `ON` or `OFF`, and defaults to `ON`, when `ANDROID_ABI=armeabi-v7a`; + - no need to specify when `ANDROID_ABI=arm64-v8a`. + +Other useful arguments: + +- `USE_EIGEN_FOR_BLAS`: indicates if using Eigen. Could be `ON` or `OFF`, defaults to `OFF`. +- `HOST_C/CXX_COMPILER`: specifies the host compiler, which is used to build the host-specific protoc and target-specific OpenBLAS. It defaults to the value of the environment variable `CC`, or `cc`. + +Some frequent configurations for your reference: + +```bash +cmake -DCMAKE_SYSTEM_NAME=Android \ + -DANDROID_STANDALONE_TOOLCHAIN=your/path/to/arm_standalone_toolchain \ + -DANDROID_ABI=armeabi-v7a \ + -DANDROID_ARM_NEON=ON \ + -DANDROID_ARM_MODE=ON \ + -DUSE_EIGEN_FOR_BLAS=ON \ + -DCMAKE_INSTALL_PREFIX=your/path/to/install \ + -DWITH_C_API=ON \ + -DWITH_SWIG_PY=OFF \ + .. +``` + +``` +cmake -DCMAKE_SYSTEM_NAME=Android \ + -DANDROID_STANDALONE_TOOLCHAIN=your/path/to/arm64_standalone_toolchain \ + -DANDROID_ABI=arm64-v8a \ + -DUSE_EIGEN_FOR_BLAS=OFF \ + -DCMAKE_INSTALL_PREFIX=your/path/to/install \ + -DWITH_C_API=ON \ + -DWITH_SWIG_PY=OFF \ + .. +``` + + +There are some other arguments you might want to configure. + +- `CMAKE_BUILD_TYPE=MinSizeRel` minimizes the size of library. +- `CMAKE_BUILD_TYPE-Release` optimizes the runtime performance. + +Our own tip for performance optimization to use clang and Eigen or OpenBLAS: +- `CMAKE_BUILD_TYPE=Release` +- `ANDROID_TOOLCHAIN=clang` +- `USE_EIGEN_BLAS=ON` for `armeabi-v7a`, or `USE_EIGEN_FOR_BLAS=OFF` for `arm64-v8a`. + +### Build and Install + +After running `cmake`, we can run `make; make install` to build and install. + +Before building, you might want to remove the `third_party` and `build` directories including pre-built libraries for other architectures. + +After building，in the directory `CMAKE_INSTALL_PREFIX`, you will find three sub-directories: + +- `include`: the header file of the inference library, +- `lib`: the inference library built for various Android ABIs, +- `third_party`: dependent third-party libraries built for Android. diff --git a/doc/_sources/mobile/cross_compiling_for_raspberry_en.md.txt b/doc/_sources/mobile/cross_compiling_for_raspberry_en.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..3c1a5950ff9553bb725d5a96e3fdf2e5e9f6f95c --- /dev/null +++ b/doc/_sources/mobile/cross_compiling_for_raspberry_en.md.txt @@ -0,0 +1,62 @@ +# Build PaddlePaddle for Raspberry Pi + +You may use any of the following two approaches to build the inference library of PaddlePaddle for Raspberry Pi: + +1. Build using SSH: Log in to a Raspberry Pi using SSH and build the library. The required development tools and third-party dependencies are listed in here: [`/Dockerfile`](https://github.com/PaddlePaddle/Paddle/blob/develop/Dockerfile). + +1. Cross-compile: We talk about how to cross-compile PaddlePaddle for Raspberry Pi on a Linux/x64 machine, in more detail in this article. + +## The Cross-Compiling Toolchain + +Step 1. Clone the Github repo by running the following command. + +```bash +git clone https://github.com/raspberrypi/tools.git +``` + +Step 2. Use the pre-built cross-compiler found in `./tools/tree/master/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64`. To run it on a Linux computer, glibc version >= 2.14 is needed. + +## CMake Arguments + +CMake supports [cross-compiling](https://cmake.org/cmake/help/v3.0/manual/cmake-toolchains.7.html#cross-compiling). All CMake configuration arguments required for the cross-compilation for Raspberry Pi can be found in [`cmake/cross_compiling/raspberry_pi.cmake`](https://github.com/PaddlePaddle/Paddle/blob/develop/cmake/cross_compiling/raspberry_pi.cmake). + +Some important arguments that need to be set: + +- `CMAKE_SYSTEM_NAME`: The target platform. Must be `RPi`. + +- `RPI_TOOLCHAIN`: The absolute path of the cross-compiling toolchain. + +- `RPI_ARM_NEON`: Use ARM NEON Intrinsics. This is a required argument and set default to `ON`. + +- `HOST_C/CXX_COMPILER`: The C/C++ compiler for the host. It is used to build building tools running on the host, for example, protoc. + +A commonly-used CMake configuration is as follows: + +``` +cmake -DCMAKE_SYSTEM_NAME=RPi \ + -DRPI_TOOLCHAIN=your/path/to/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64 \ + -DRPI_ARM_NEON=ON \ + -DCMAKE_INSTALL_PREFIX=your/path/to/install \ + -DWITH_GPU=OFF \ + -DWITH_C_API=ON \ + -DWITH_PYTHON=OFF \ + -DWITH_SWIG_PY=OFF \ + .. +``` + +To build the inference library, please set the argument WITH\_C\_API to ON: `WITH_C_API=ON`. + +You can add more arguments. For example, to minimize the size of the generated inference library, you may use `CMAKE_BUILD_TYPE=MinSizeRel`. For performance optimization, you may use `CMAKE_BUILD_TYPE=Release`. + +## Build and Install + +The following commands build the inference library of PaddlePaddle for Raspberry Pi and third-party dependencies. + +```bash +make +make install +``` + + The intermediate files will be stored in `build`. Third-party libraries will be located in `build/third_party`. If you have already built it for other platforms like Android or iOS, you may want to clear these directories by running the command: `rm -rf build`. + +The infernece library will be in `your/path/to/install/lib`, with related header files in `your/path/to/install/include`. diff --git a/doc/_sources/mobile/index_en.rst.txt b/doc/_sources/mobile/index_en.rst.txt new file mode 100644 index 0000000000000000000000000000000000000000..3c08d736717cfe8d5fdf449dc58015086befbe60 --- /dev/null +++ b/doc/_sources/mobile/index_en.rst.txt @@ -0,0 +1,8 @@ +MOBILE +====== + +.. toctree:: + :maxdepth: 1 + + cross_compiling_for_android_en.md + cross_compiling_for_raspberry_en.md diff --git a/doc/_sources/survey/cluster_bootstrapping_tools.md.txt b/doc/_sources/survey/cluster_bootstrapping_tools.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..1cd9962700bb49866f1ed6987abc28b27888a23f --- /dev/null +++ b/doc/_sources/survey/cluster_bootstrapping_tools.md.txt @@ -0,0 +1,71 @@ +# Cluster bootstrapping tool survey +## Abstract +In order to bring up a cluster from bare metal machine to a fully functional kubernetes cluster for Paddlepaddle to run, we need to utilize some tools. Here we are going to compare [Sextant](https://github.com/k8sp/sextant) and [Tectonic installer](https://github.com/coreos/tectonic-installer) + +## Basic assumptions +Here are some basic assumptions before we move on to details +1. You are an administrator of a bare metal machine cluster, which means: + * you have full control to each of the machines. + * you have full control to the network which machines are connected to. +2. Machines can be booted from network with PEX or iPXE +3. You understand the [general procedure to bring up a cluster](#appendix-general-procedure-to-bring-up-a-cluster) + +if your cluster is able to mark above items with checkmarks, then keep reading. + +## Comparing Sextant and Tectonic installer +### Sextant +Sextant is an end2end solution to bring up a bare metal cluster to a fully functional k8s cluster, it integrates DHCP, name service, PEX, cloud-config-service, docker registry services altogether. + +#### Pros +1. End2End: basically all admin need to do is to config the cluster.yaml and power on the cluster. +2. Offline cluster configuration: Sextant has 2 phases during working with it, config time and deploy time. when admin is configuring, it requires admin's machine has internet connectivity, which will download some images, etc. But in deploy time, it's completely OK to go offline since all dependencies are ready during config time. +3. docker registry integrated. +4. GPU machine took care of. + +### Cons +1. k8s API server is not deployed with high availability in considering by default. +2. No grouping support. +3. No API interface, a one-off service. + + +### Tectonic installer +First of all, Tectonic is not free, it requires coreos.com account as a step of installation, and free user can only create less than 10 nodes. + +Tectonic is a suite of software which wraps around k8s and providing more utility regarding dev ops, ie, +Tectonic installer as it's named, it installs Tectonic to a bare metal cluster which means it's not totally an equivalent of Sextant. At the "booting a cluster" part, it mostly utilizes [Matchbox](https://github.com/coreos/matchbox), which is a general cluster bootstrapper. + +Matchbox's Approach is similar to Sexstant. + +### Pros +1. supports grouping machines. +2. supports running provisioning service in rtk. (not a big deal though). +3. supports http/gRPC API interface. +4. supports multi-template. + +### Cons +1. Not an e2e solution to bring up a cluster, need a lot of extra work and other software. +2. [Not fully supporting](https://github.com/coreos/matchbox/issues/550) centOS deployment yet. + +## Conclusion +Sextant is a better solution overall for paddle cloud deploying to a bare metal cluster. It would be great if Sextant can also 1) deploy k8s api server with high availability by default; 2) not designed as a one-off service. + + + +## Appendix: General procedure to bring up a cluster +It's physically impossible for a cluster admin to manually install OS and applications into cluster nodes one by one, here is what an admin would do in cloud industry: +1. setup a bootstrap machine with static IP in the cluster, which has following services: + * DHCP: assigns ip address for rest of the nodes. + * name service: to map node name to a IP + * PXE related services: the booting related info will be delivered to newly booted machines as their IP is assigned via DHCP service, PXE service will provide further booting and installing info and image with TFTP and http protocol. + * cluster config service: this is for providing cluster node with OS config via http + * optional docker registry: a built-in docker registry makes the whole cluster independent from connecting internet, and speeds up software distribution. +2. New node powers on, it will + * broadcast the request for an IP address + * DHCP server assigns the IP address, and deliver the PXE booting related info to the node. + * cluster node will request config files with booting info delivered with DHCP via the TFTP service, and in most of the cases, the config file will point to a http service for the booting image. + * Since PXE is configured with initrd, it will utilize the cloud config service and do further installations like coreOS or K8s installations. + * then restart the node. + +For further understanding, following 2 links from Matchbox are some good readings: +* [Machine lifecycle](https://github.com/coreos/matchbox/blob/master/Documentation/machine-lifecycle.md) +* [PXE booting](https://github.com/coreos/matchbox/blob/master/Documentation/network-booting.md) diff --git a/doc/_sources/tutorials/image_classification/index_en.md.txt b/doc/_sources/tutorials/image_classification/index_en.md.txt deleted file mode 100644 index 60c81a6a539944634773f38ec4c9a59709dd4afc..0000000000000000000000000000000000000000 --- a/doc/_sources/tutorials/image_classification/index_en.md.txt +++ /dev/null @@ -1,221 +0,0 @@ -Image Classification Tutorial -============================== - -This tutorial will guide you through training a convolutional neural network to classify objects using the CIFAR-10 image classification dataset. -As shown in the following figure, the convolutional neural network can recognize the main object in images, and output the classification result. - -
![Image Classification](./image_classification.png)
- -## Data Preparation -First, download CIFAR-10 dataset. CIFAR-10 dataset can be downloaded from its official website. - - - -We have prepared a script to download and process CIFAR-10 dataset. The script will download CIFAR-10 dataset from the official dataset. -It will convert it to jpeg images and organize them into a directory with the required structure for the tutorial. Make sure that you have installed pillow and its dependents. -Consider the following commands: - -1. install pillow dependents - -```bash -sudo apt-get install libjpeg-dev -pip install pillow -``` - -2. download data and preparation - -```bash -cd demo/image_classification/data/ -sh download_cifar.sh -``` - -The CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. - -Here are the classes in the dataset, as well as 10 random images from each: -
![Image Classification](./cifar.png)
- - -After downloading and converting, we should find a directory (cifar-out) containing the dataset in the following format: - -``` -train ----airplane ----automobile ----bird ----cat ----deer ----dog ----frog ----horse ----ship ----truck -test ----airplane ----automobile ----bird ----cat ----deer ----dog ----frog ----horse ----ship ----truck -``` - -It has two directories:`train` and `test`. These two directories contain training data and testing data of CIFAR-10, respectively. Each of these two folders contains 10 sub-folders, ranging from `airplane` to `truck`. Each sub-folder contains images with the corresponding label. After the images are organized into this structure, we are ready to train an image classification model. - -## Preprocess -After the data has been downloaded, it needs to be pre-processed into the Paddle format. We can run the following command for preprocessing. - -``` -cd demo/image_classification/ -sh preprocess.sh -``` - -`preprocess.sh` calls `./demo/image_classification/preprocess.py` to preprocess image data. -```sh -export PYTHONPATH=$PYTHONPATH:../../ -data_dir=./data/cifar-out -python preprocess.py -i $data_dir -s 32 -c 1 -``` - -`./demo/image_classification/preprocess.py` has the following arguments - -- `-i` or `--input` specifes the input data directory. -- `-s` or `--size` specifies the processed size of images. -- `-c` or `--color` specifes whether images are color images or gray images. - - -## Model Training -We need to create a model config file before training the model. An example of the config file (vgg_16_cifar.py) is listed below. **Note**, it is slightly different from the `vgg_16_cifar.py` which also applies to the prediction. - -```python -from paddle.trainer_config_helpers import * -data_dir='data/cifar-out/batches/' -meta_path=data_dir+'batches.meta' -args = {'meta':meta_path, 'mean_img_size': 32, - 'img_size': 32, 'num_classes': 10, - 'use_jpeg': 1, 'color': "color"} -define_py_data_sources2(train_list=data_dir+"train.list", - test_list=data_dir+'test.list', - module='image_provider', - obj='processData', - args=args) -settings( - batch_size = 128, - learning_rate = 0.1 / 128.0, - learning_method = MomentumOptimizer(0.9), - regularization = L2Regularization(0.0005 * 128)) - -img = data_layer(name='image', size=3*32*32) -lbl = data_layer(name="label", size=10) -# small_vgg is predined in trainer_config_helpers.network -predict = small_vgg(input_image=img, num_channels=3) -outputs(classification_cost(input=predict, label=lbl)) -``` - -The first line imports python functions for defining networks. -```python -from paddle.trainer_config_helpers import * -``` - -Then define an `define_py_data_sources2` which use python data provider -interface. The arguments in `args` are used in `image_provider.py` which -yeilds image data and transform them to Paddle. - - `meta`: the mean value of training set. - - `mean_img_size`: the size of mean feature map. - - `img_size`: the height and width of input image. - - `num_classes`: the number of classes. - - `use_jpeg`: the data storage type when preprocessing. - - `color`: specify color image. - -`settings` specifies the training algorithm. In the following example, -it specifies learning rate as 0.1, but divided by batch size, and the weight decay -is 0.0005 and multiplied by batch size. -```python -settings( - batch_size = 128, - learning_rate = 0.1 / 128.0, - learning_method = MomentumOptimizer(0.9), - regularization = L2Regularization(0.0005 * 128) -) -``` - -The `small_vgg` specifies the network. We use a small version of VGG convolutional network as our network -for classification. A description of VGG network can be found here [http://www.robots.ox.ac.uk/~vgg/research/very_deep/](http://www.robots.ox.ac.uk/~vgg/research/very_deep/). -```python -# small_vgg is predined in trainer_config_helpers.network -predict = small_vgg(input_image=img, num_channels=3) -``` -After writing the config, we can train the model by running the script train.sh. - -```bash -config=vgg_16_cifar.py -output=./cifar_vgg_model -log=train.log - -paddle train \ ---config=$config \ ---dot_period=10 \ ---log_period=100 \ ---test_all_data_in_one_period=1 \ ---use_gpu=1 \ ---save_dir=$output \ -2>&1 | tee $log - -python -m paddle.utils.plotcurve -i $log > plot.png -``` - -- Here we use GPU mode to train. If you have no gpu environment, just set `use_gpu=0`. - -- `./demo/image_classification/vgg_16_cifar.py` is the network and data configuration file. The meaning of the other flags can be found in the documentation of the command line flags. - -- The script `plotcurve.py` requires the python module of `matplotlib`, so if it fails, maybe you need to install `matplotlib`. - - -After training finishes, the training and testing error curves will be saved to `plot.png` using `plotcurve.py` script. An example of the plot is shown below: - -
![Training and testing curves.](./plot.png)
- - -## Prediction -After we train the model, the model file as well as the model parameters are stored in path `./cifar_vgg_model/pass-%05d`. For example, the model of the 300-th pass is stored at `./cifar_vgg_model/pass-00299`. - -To make a prediction for an image, one can run `predict.sh` as follows. The script will output the label of the classfiication. - -``` -sh predict.sh -``` - -predict.sh: -``` -model=cifar_vgg_model/pass-00299/ -image=data/cifar-out/test/airplane/seaplane_s_000978.png -use_gpu=1 -python prediction.py $model $image $use_gpu -``` - -## Exercise -Train a image classification of birds using VGG model and CUB-200 dataset. The birds dataset can be downloaded here. It contains an image dataset with photos of 200 bird species (mostly North American). - - - - - - -## Delve into Details -### Convolutional Neural Network -A Convolutional Neural Network is a feedforward neural network that uses convolution layers. It is very suitable for building neural networks that process and understand images. A standard convolutional neural network is shown below: - -![Convolutional Neural Network](./lenet.png) - -Convolutional Neural Network contains the following layers: - -- Convolutional layer: It uses convolution operation to extract features from an image or a feature map. -- Pooling layer: It uses max-pooling to downsample feature maps. -- Fully Connected layer: It uses fully connected connections to transform features. - -Convolutional Neural Network achieves amazing performance for image classification because it exploits two important characteristics of images: *local correlation* and *spatial invariance*. By iteratively applying convolution and max-pooing operations, convolutional neural network can well represent these two characteristics of images. - - -For more details of how to define layers and their connections, please refer to the documentation of layers. diff --git a/doc/_sources/tutorials/index_en.md.txt b/doc/_sources/tutorials/index_en.md.txt deleted file mode 100644 index 77331a703b6f0fdf92921ebcc476325b7327e976..0000000000000000000000000000000000000000 --- a/doc/_sources/tutorials/index_en.md.txt +++ /dev/null @@ -1,14 +0,0 @@ -# TUTORIALS -There are several examples and demos here. - -* [Quick Start](quick_start/index_en.md) -* [MovieLens Regression](rec/ml_regression_en.rst) -* [Image Classification](image_classification/index_en.md) -* [Sentiment Analysis](sentiment_analysis/index_en.md) -* [Semantic Role Labeling](semantic_role_labeling/index_en.md) -* [Text Generation](text_generation/index_en.md) -* [Image Auto-Generation](gan/index_en.md) - -## Model Zoo -* [ImageNet: ResNet](imagenet_model/resnet_model_en.md) -* [Embedding: Chinese Word](embedding_model/index_en.md) diff --git a/doc/_sources/tutorials/rec/ml_dataset_en.md.txt b/doc/_sources/tutorials/rec/ml_dataset_en.md.txt deleted file mode 100644 index 25dea5c4afbf1ce1c1ac6195cbd245b116459e2e..0000000000000000000000000000000000000000 --- a/doc/_sources/tutorials/rec/ml_dataset_en.md.txt +++ /dev/null @@ -1,111 +0,0 @@ -```eval_rst -.. _demo_ml_dataset: -``` - -# MovieLens Dataset - -The [MovieLens Dataset](http://grouplens.org/datasets/movielens/) was collected by GroupLens Research. -The data set contains some user information, movie information, and many movie ratings from \[1-5\]. -The data sets have many version depending on the size of set. -We use [MovieLens 1M Dataset](http://files.grouplens.org/datasets/movielens/ml-1m.zip) as a demo dataset, which contains -1 million ratings from 6000 users on 4000 movies. Released 2/2003. - -## Dataset Features - -In [ml-1m Dataset](http://files.grouplens.org/datasets/movielens/ml-1m.zip), there are many features in these dataset. -The data files (which have ".dat" extension) in [ml-1m Dataset](http://files.grouplens.org/datasets/movielens/ml-1m.zip) -is basically CSV file that delimiter is "::". The description in README we quote here. - -### RATINGS FILE DESCRIPTION(ratings.dat) - - -All ratings are contained in the file "ratings.dat" and are in the -following format: - -UserID::MovieID::Rating::Timestamp - -- UserIDs range between 1 and 6040 -- MovieIDs range between 1 and 3952 -- Ratings are made on a 5-star scale (whole-star ratings only) -- Timestamp is represented in seconds since the epoch as returned by time(2) -- Each user has at least 20 ratings - -### USERS FILE DESCRIPTION(users.dat) - -User information is in the file "users.dat" and is in the following -format: - -UserID::Gender::Age::Occupation::Zip-code - -All demographic information is provided voluntarily by the users and is -not checked for accuracy. Only users who have provided some demographic -information are included in this data set. - -- Gender is denoted by a "M" for male and "F" for female -- Age is chosen from the following ranges: - - * 1: "Under 18" - * 18: "18-24" - * 25: "25-34" - * 35: "35-44" - * 45: "45-49" - * 50: "50-55" - * 56: "56+" - -- Occupation is chosen from the following choices: - - * 0: "other" or not specified - * 1: "academic/educator" - * 2: "artist" - * 3: "clerical/admin" - * 4: "college/grad student" - * 5: "customer service" - * 6: "doctor/health care" - * 7: "executive/managerial" - * 8: "farmer" - * 9: "homemaker" - * 10: "K-12 student" - * 11: "lawyer" - * 12: "programmer" - * 13: "retired" - * 14: "sales/marketing" - * 15: "scientist" - * 16: "self-employed" - * 17: "technician/engineer" - * 18: "tradesman/craftsman" - * 19: "unemployed" - * 20: "writer" - -### MOVIES FILE DESCRIPTION(movies.dat) - -Movie information is in the file "movies.dat" and is in the following -format: - -MovieID::Title::Genres - -- Titles are identical to titles provided by the IMDB (including -year of release) -- Genres are pipe-separated and are selected from the following genres: - - * Action - * Adventure - * Animation - * Children's - * Comedy - * Crime - * Documentary - * Drama - * Fantasy - * Film-Noir - * Horror - * Musical - * Mystery - * Romance - * Sci-Fi - * Thriller - * War - * Western - -- Some MovieIDs do not correspond to a movie due to accidental duplicate -entries and/or test entries -- Movies are mostly entered by hand, so errors and inconsistencies may exist diff --git a/doc/_sources/tutorials/rec/ml_regression_en.rst.txt b/doc/_sources/tutorials/rec/ml_regression_en.rst.txt deleted file mode 100644 index 993b9a516f134ff8b59e8755b721f76c8f32f0fd..0000000000000000000000000000000000000000 --- a/doc/_sources/tutorials/rec/ml_regression_en.rst.txt +++ /dev/null @@ -1,348 +0,0 @@ -Regression MovieLens Ratting -============================ - -Here we demonstrate a **Cosine Similarity Regression** job in movie lens dataset. -This demo will show how paddle does (word) embedding job, -handles the similarity regression, -the character-level convolutional networks for text, and how does paddle handle -multiple types of inputs. -Note that the model structure is not fine-tuned and just a demo to show how paddle works. - - -YOU ARE WELCOME TO BUILD A BETTER DEMO -BY USING PADDLEPADDLE, AND LET US KNOW TO MAKE THIS DEMO BETTER. - -Data Preparation -```````````````` -Download and extract dataset -'''''''''''''''''''''''''''' -We use :ref:`demo_ml_dataset` here. -To download and unzip the dataset, simply run the following commands. - -.. code-block:: bash - - cd demo/recommendation/data - ./ml_data.sh - -And the directory structure of :code:`demo/recommendation/data/ml-1m` is: - -.. code-block:: text - - +--ml-1m - +--- movies.dat # movie features - +--- ratings.dat # ratings - +--- users.dat # user features - +--- README # dataset description - -Field config file -''''''''''''''''' -**Field config file** is used to specify the fields of the dataset and the file format, -i.e, specific **WHAT** type it is in each feature file. - -The field config file of ml-1m shows in :code:`demo/recommendation/data/config.json`. -It specifics the field types and file names: 1) there are four types of field for user file\: id, gender, age and occupation; -2) the filename is "users.dat", and the delimiter of file is "::". - -.. include:: ../../../demo/recommendation/data/config.json - :code: json - :literal: - -Preprocess Data -``````````````` -You need to install python 3rd party libraries. -IT IS HIGHLY RECOMMEND TO USE VIRTUALENV MAKE A CLEAN PYTHON ENVIRONMENT. - -.. code-block:: bash - - pip install -r requirements.txt - -The general command for preprocessing the dataset is: - -.. code-block:: bash - - cd demo/recommendation - ./preprocess.sh - -And the detail steps are introduced as follows. - -Extract Movie/User features to python object -''''''''''''''''''''''''''''''''''''''''''''' - -There are many features in movie or user in movielens 1m dataset. -Each line of rating file just provides a Movie/User id to refer each movie or user. -We process the movie/user feature file first, and pickle the feature (**Meta**) object as a file. - -Meta config file -................ - -**Meta config file** is used to specific **HOW** to parse each field in dataset. -It could be translated from field config file, or written by hand. -Its file format could be either json or yaml syntax file. Parser will automatically choose the file format by extension name. - -To convert Field config file to meta config file, just run: - -.. code-block:: bash - - cd demo/recommendation/data - python config_generator.py config.json > meta_config.json - -The meta config file shows below: - -.. include:: ../../../demo/recommendation/data/meta_config.json - :code: json - :literal: - -There are two kinds of features in meta\: movie and user. - -* in movie file, whose name is movies.dat - * we just split each line by "::" - * pos 0 is id. - * pos 1 feature: - * name is title. - * it uses regex to parse this feature. - * it is a char based word embedding feature. - * it is a sequence. - * pos 2 feature: - * name is genres. - * type is one hot dense vector. - * dictionary is auto generated by parsing, each key is split by '|' -* in user file, whose name is users.dat - * we just split each line by "::" - * pos 0 is id. - * pos 1 feature: - * name is gender - * just simple char based embedding. - * pos 2 feature: - * name is age - * just whole word embedding. - * embedding id will be sort by word. - * pos 3 feature: - * name is occupation. - * just simple whole word embedding. - - -Meta file -''''''''' - -After having meta config file, we can generate **Meta file**, a python pickle object which stores movie/user information. -The following commands could be run to generate it. - -.. code-block:: bash - - python meta_generator.py ml-1m meta.bin --config=meta_config.json - -And the structure of the meta file :code:`meta.bin` is: - -.. code-block:: text - - +--+ movie - | +--+ __meta__ - | | +--+ raw_meta # each feature meta config. list - | | | + - | | | | # ID Field, we use id as key - | | | +--+ {'count': 3883, 'max': 3952, 'is_key': True, 'type': 'id', 'min': 1} - | | | | - | | | | # Titile field, the dictionary list of embedding. - | | | +--+ {'dict': [ ... ], 'type': 'embedding', 'name': 'title', 'seq': 'sequence'} - | | | | - | | | | # Genres field, the genres dictionary - | | | +--+ {'dict': [ ... ], 'type': 'one_hot_dense', 'name': 'genres'} - | | | - | | +--+ feature_map [1, 2] # a list for raw_meta index for feature field. - | | # it means there are 2 features for each key. - | | # * 0 offset of feature is raw_meta[1], Title. - | | # * 1 offset of feature is raw_meta[2], Genres. - | | - | +--+ 1 # movie 1 features - | | + - | | +---+ [[...], [...]] # title ids, genres dense vector - | | - | +--+ 2 - | | - | +--+ ... - | - +--- user - +--+ __meta__ - | + - | +--+ raw_meta - | | + - | | +--+ id field as user - | | | - | | +--+ {'dict': ['F', 'M'], 'type': 'embedding', 'name': 'gender', 'seq': 'no_sequence'} - | | | - | | +--+ {'dict': ['1', '18', '25', '35', '45', '50', '56'], 'type': 'embedding', 'name': 'age', 'seq': 'no_sequence'} - | | | - | | +--+ {'dict': [...], 'type': 'embedding', 'name': 'occupation', 'seq': 'no_sequence'} - | | - | +--+ feature_map [1, 2, 3] - | - +--+ 1 # user 1 features - | - +--+ 2 - +--+ ... - - -Split Training/Testing files -'''''''''''''''''''''''''''' - -We split :code:`ml-1m/ratings.dat` into a training and testing file. The way to split file is for each user, we split the -rating by two parts. So each user in testing file will have some rating information in training file. - -Use :code:`separate.py` to separate the training and testing file. - -.. code-block:: bash - - python split.py ml-1m/ratings.dat --delimiter="::" --test_ratio=0.1 - -Then two files will be generated\: :code:`ml-1m/ratings.dat.train` and :code:`ml-1m/rating.data.test`. -Move them to workspace :code:`data`, shuffle the train file, and prepare the file list for paddle train. - -.. code-block:: bash - - shuf ml-1m/ratings.dat.train > ratings.dat.train - cp ml-1m/ratings.dat.test . - echo "./data/ratings.dat.train" > train.list - echo "./data/ratings.dat.test" > test.list - - -Neural Network Configuration -```````````````````````````` - -Trainer Config File -''''''''''''''''''' - -The network structure shows below. - -.. image:: rec_regression_network.png - :align: center - :alt: rec_regression_network - -The demo's neural network config file :code:`trainer_config.py` show as below. - -.. literalinclude:: ../../../demo/recommendation/trainer_config.py - :language: python - :lines: 15- - -In this :code:`trainer_config.py`, we just map each feature type to -a feature vector, following shows how to map each feature to a vector shows below. - -* :code:`id`\: Just simple embedding, and then add to fully connected layer. -* :code:`embedding`\: - - if is_sequence, get the embedding and do a text convolutional operation, - get the average pooling result. - - if not sequence, get the embedding and add to fully connected layer. -* :code:`one_host_dense`\: - - just two fully connected layer. - -Then we combine each features of movie into one movie feature by a -:code:`fc_layer` with multiple inputs, and do the same thing to user features, -get one user feature. Then we calculate the cosine similarity of these two -features. - -In these networks, we use several APIs in :ref:`api_trainer_config` . There are - -* Data Layer, :ref:`api_trainer_config_helpers_layers_data_layer` -* Fully Connected Layer, :ref:`api_trainer_config_helpers_layers_fc_layer` -* Embedding Layer, :ref:`api_trainer_config_helpers_layers_embedding_layer` -* Context Projection Layer, :ref:`api_trainer_config_helpers_layers_context_projection` -* Pooling Layer, :ref:`api_trainer_config_helpers_layers_pooling_layer` -* Cosine Similarity Layer, :ref:`api_trainer_config_helpers_layers_cos_sim` -* Text Convolution Pooling Layer, :ref:`api_trainer_config_helpers_network_text_conv_pool` -* Declare Python Data Sources :ref:`api_trainer_config_helpers_data_sources`. - -Data Provider -''''''''''''' - -.. literalinclude:: ../../../demo/recommendation/dataprovider.py - :language: python - :lines: 15- - -The data provider just read the meta.bin and rating file, yield each sample for training. -In this :code:`dataprovider.py`, we should set\: - -* obj.slots\: The feature types and dimension. -* use_seq\: Whether this :code:`dataprovider.py` in sequence mode or not. -* process\: Return each sample of data to :code:`paddle`. - -The data provider details document see :ref:`api_pydataprovider2`. - -Train -````` - -After prepare data, config network, writting data provider, now we can run paddle training. - -The :code:`run.sh` is shown as follow: - -.. literalinclude:: ../../../demo/recommendation/run.sh - :language: bash - :lines: 16- - -It just start a paddle training process, write the log to :code:`log.txt`, -then print it on screen. - -Each command line argument in :code:`run.sh`, please refer to the :ref:`cmd_line_index` page. The short description of these arguments is shown as follow. - -* config\: Tell paddle which file is neural network configuration. -* save_dir\: Tell paddle save model into :code:`./output`. -* use_gpu\: Use gpu or not. Default is false. -* trainer_count\: The compute thread in one machine. -* test_all_data_in_one_period\: Test All Data during one test period. Otherwise, - will test a :code:`batch_size` data in one test period. -* log_period\: Print log after train :code:`log_period` batches. -* dot_period\: Print a :code:`.` after train :code:`dot_period` batches. -* num_passes\: Train at most :code:`num_passes`. - -If training process starts successfully, the output likes follow: - -.. code-block:: text - - I0601 08:07:22.832059 10549 TrainerInternal.cpp:157] Batch=100 samples=160000 AvgCost=4.13494 CurrentCost=4.13494 Eval: CurrentEval: - - I0601 08:07:50.672627 10549 TrainerInternal.cpp:157] Batch=200 samples=320000 AvgCost=3.80957 CurrentCost=3.48421 Eval: CurrentEval: - - I0601 08:08:18.877369 10549 TrainerInternal.cpp:157] Batch=300 samples=480000 AvgCost=3.68145 CurrentCost=3.42519 Eval: CurrentEval: - - I0601 08:08:46.863963 10549 TrainerInternal.cpp:157] Batch=400 samples=640000 AvgCost=3.6007 CurrentCost=3.35847 Eval: CurrentEval: - - I0601 08:09:15.413025 10549 TrainerInternal.cpp:157] Batch=500 samples=800000 AvgCost=3.54811 CurrentCost=3.33773 Eval: CurrentEval: - I0601 08:09:36.058670 10549 TrainerInternal.cpp:181] Pass=0 Batch=565 samples=902826 AvgCost=3.52368 Eval: - I0601 08:09:46.215489 10549 Tester.cpp:101] Test samples=97383 cost=3.32155 Eval: - I0601 08:09:46.215966 10549 GradientMachine.cpp:132] Saving parameters to ./output/model/pass-00000 - I0601 08:09:46.233397 10549 ParamUtil.cpp:99] save dir ./output/model/pass-00000 - I0601 08:09:46.233438 10549 Util.cpp:209] copy trainer_config.py to ./output/model/pass-00000 - I0601 08:09:46.233541 10549 ParamUtil.cpp:147] fileName trainer_config.py - -The model is saved in :code:`output/` directory. You can use :code:`Ctrl-C` to stop training whenever you want. - -Evaluate and Predict -```````````````````` - -After training several passes, you can evaluate them and get the best pass. Just run - -.. code-block:: bash - - ./evaluate.sh - -You will see messages like this: - -.. code-block:: text - - Best pass is 00009, error is 3.06949, which means predict get error as 0.875998002281 - evaluating from pass output/pass-00009 - -Then, you can predict what any user will rate a movie. Just run - -.. code-block:: bash - - python prediction.py 'output/pass-00009/' - -Predictor will read user input, and predict scores. It has a command-line user interface as follows: - -.. code-block:: text - - Input movie_id: 9 - Input user_id: 4 - Prediction Score is 2.56 - Input movie_id: 8 - Input user_id: 2 - Prediction Score is 3.13 diff --git a/doc/_sources/tutorials/semantic_role_labeling/index_en.md.txt b/doc/_sources/tutorials/semantic_role_labeling/index_en.md.txt deleted file mode 100644 index 92d7c634832119c718711a57c16f69492d405f28..0000000000000000000000000000000000000000 --- a/doc/_sources/tutorials/semantic_role_labeling/index_en.md.txt +++ /dev/null @@ -1,204 +0,0 @@ -```eval_rst -.. _semantic_role_labeling: -``` - -# Semantic Role labeling Tutorial # - -Semantic role labeling (SRL) is a form of shallow semantic parsing whose goal is to discover the predicate-argument structure of each predicate in a given input sentence. SRL is useful as an intermediate step in a wide range of natural language processing tasks, such as information extraction. automatic document categorization and question answering. An instance is as following [1]: - - [ A0 He ] [ AM-MOD would ][ AM-NEG n’t ] [ V accept] [ A1 anything of value ] from [A2 those he was writing about ]. - -- V: verb -- A0: acceptor -- A1: thing accepted -- A2: accepted-from -- A3: Attribute -- AM-MOD: modal -- AM-NEG: negation - -Given the verb "accept", the chunks in sentence would play certain semantic roles. Here, the label scheme is from Penn Proposition Bank. - -To this date, most of the successful SRL systems are built on top of some form of parsing results where pre-defined feature templates over the syntactic structure are used. This tutorial will present an end-to-end system using deep bidirectional long short-term memory (DB-LSTM)[2] for solving the SRL task, which largely outperforms the previous state-of-the-art systems. The system regards SRL task as the sequence labelling problem. - -## Data Description -The relevant paper[2] takes the data set in CoNLL-2005&2012 Shared Task for training and testing. Accordingto data license, the demo adopts the test data set of CoNLL-2005, which can be reached on website. - -To download and process the original data, user just need to execute the following command: - -```bash -cd data -./get_data.sh -``` -Several new files appear in the `data `directory as follows. -```bash -conll05st-release：the test data set of CoNll-2005 shared task -test.wsj.words：the Wall Street Journal data sentences -test.wsj.props: the propositional arguments -feature: the extracted features from data set -``` - -## Training -### DB-LSTM -Please refer to the Sentiment Analysis demo to learn more about the long short-term memory unit. - -Unlike Bidirectional-LSTM that used in Sentiment Analysis demo, the DB-LSTM adopts another way to stack LSTM layer. First a standard LSTM processes the sequence in forward direction. The input and output of this LSTM layer are taken by the next LSTM layer as input, processed in reversed direction. These two standard LSTM layers compose a pair of LSTM. Then we stack LSTM layers pair after pair to obtain the deep LSTM model. - -The following figure shows a temporal expanded 2-layer DB-LSTM network. -
-![pic](./src/network_arch.png) -
- -### Features -Two input features play an essential role in this pipeline: predicate (pred) and argument (argu). Two other features: predicate context (ctx-p) and region mark (mr) are also adopted. Because a single predicate word can not exactly describe the predicate information, especially when the same words appear more than one times in a sentence. With the predicate context, the ambiguity can be largely eliminated. Similarly, we use region mark mr = 1 to denote the argument position if it locates in the predicate context region, or mr = 0 if does not. These four simple features are all we need for our SRL system. Features of one sample with context size set to 1 is showed as following[2]: -
-![pic](./src/feature.jpg) -
- -In this sample, the coresponding labelled sentence is: - -[ A1 A record date ] has [ AM-NEG n't ] been [ V set ] . - -In the demo, we adopt the feature template as above, consists of : `argument`, `predicate`, `ctx-p (p=-1,0,1)`, `mark` and use `B/I/O` scheme to label each argument. These features and labels are stored in `feature` file, and separated by `\t`. - -### Data Provider - -`dataprovider.py` is the python file to wrap data. `hook()` function is to define the data slots for network. The Six features and label are all IndexSlots. -``` -def hook(settings, word_dict, label_dict, **kwargs): - settings.word_dict = word_dict - settings.label_dict = label_dict - #all inputs are integral and sequential type - settings.slots = [ - integer_value_sequence(len(word_dict)), - integer_value_sequence(len(predicate_dict)), - integer_value_sequence(len(word_dict)), - integer_value_sequence(len(word_dict)), - integer_value_sequence(len(word_dict)), - integer_value_sequence(len(word_dict)), - integer_value_sequence(len(word_dict)), - integer_value_sequence(2), - integer_value_sequence(len(label_dict))] -``` -The corresponding data iterator is as following: -``` -@provider(init_hook=hook, should_shuffle=True, calc_batch_size=get_batch_size, - can_over_batch_size=False, cache=CacheType.CACHE_PASS_IN_MEM) -def process(settings, file_name): - with open(file_name, 'r') as fdata: - for line in fdata: - sentence, predicate, ctx_n2, ctx_n1, ctx_0, ctx_p1, ctx_p2, mark, label = \ - line.strip().split('\t') - - words = sentence.split() - sen_len = len(words) - word_slot = [settings.word_dict.get(w, UNK_IDX) for w in words] - - predicate_slot = [settings.predicate_dict.get(predicate)] * sen_len - ctx_n2_slot = [settings.word_dict.get(ctx_n2, UNK_IDX)] * sen_len - ctx_n1_slot = [settings.word_dict.get(ctx_n1, UNK_IDX)] * sen_len - ctx_0_slot = [settings.word_dict.get(ctx_0, UNK_IDX)] * sen_len - ctx_p1_slot = [settings.word_dict.get(ctx_p1, UNK_IDX)] * sen_len - ctx_p2_slot = [settings.word_dict.get(ctx_p2, UNK_IDX)] * sen_len - - marks = mark.split() - mark_slot = [int(w) for w in marks] - - label_list = label.split() - label_slot = [settings.label_dict.get(w) for w in label_list] - yield word_slot, predicate_slot, ctx_n2_slot, ctx_n1_slot, \ - ctx_0_slot, ctx_p1_slot, ctx_p2_slot, mark_slot, label_slot -``` -The `process`function yield 9 lists which are 8 features and label. - -### Neural Network Config -`db_lstm.py` is the neural network config file to load the dictionaries and define the data provider module and network architecture during the training procedure. - -Nine `data_layer` load instances from data provider. Eight features are transformed into embedddings respectively, and mixed by `mixed_layer` . Deep bidirectional LSTM layers extract features for the softmax layer. The objective function is cross entropy of labels. - -### Run Training -The script for training is `train.sh`, user just need to execute: -```bash - ./train.sh -``` -The content in `train.sh`: -``` -paddle train \ - --config=./db_lstm.py \ - --use_gpu=0 \ - --log_period=5000 \ - --trainer_count=1 \ - --show_parameter_stats_period=5000 \ - --save_dir=./output \ - --num_passes=10000 \ - --average_test_period=10000000 \ - --init_model_path=./data \ - --load_missing_parameter_strategy=rand \ - --test_all_data_in_one_period=1 \ -2>&1 | tee 'train.log' -``` - -- \--config=./db_lstm.py : network config file. -- \--use_gpu=false: use CPU to train, set true, if you install GPU version of PaddlePaddle and want to use GPU to train, until now crf_layer do not support GPU -- \--log_period=500: print log every 20 batches. -- \--trainer_count=1: set thread number (or GPU count). -- \--show_parameter_stats_period=5000: show parameter statistic every 100 batches. -- \--save_dir=./output: output path to save models. -- \--num_passes=10000: set pass number, one pass in PaddlePaddle means training all samples in dataset one time. -- \--average_test_period=10000000: do test on average parameter every average_test_period batches -- \--init_model_path=./data: parameter initialization path -- \--load_missing_parameter_strategy=rand: random initialization unexisted parameters -- \--test_all_data_in_one_period=1: test all data in one period - - -After training, the models will be saved in directory `output`. Our training curve is as following: -
-![pic](./src/curve.jpg) -
- -### Run testing -The script for testing is `test.sh`, user just need to execute: -```bash - ./test.sh -``` -The main part in `tesh.sh` -``` -paddle train \ - --config=./db_lstm.py \ - --model_list=$model_list \ - --job=test \ - --config_args=is_test=1 \ -``` - - - \--config=./db_lstm.py: network config file - - \--model_list=$model_list.list: model list file - - \--job=test: indicate the test job - - \--config_args=is_test=1: flag to indicate test - - \--test_all_data_in_one_period=1: test all data in 1 period - - -### Run prediction -The script for prediction is `predict.sh`, user just need to execute: -```bash - ./predict.sh - -``` -In `predict.sh`, user should offer the network config file, model path, label file, word dictionary file, feature file -``` -python predict.py - -c $config_file \ - -w $best_model_path \ - -l $label_file \ - -p $predicate_dict_file \ - -d $dict_file \ - -i $input_file \ - -o $output_file -``` - -`predict.py` is the main executable python script, which includes functions: load model, load data, data prediction. The network model will output the probability distribution of labels. In the demo, we take the label with maximum probability as result. User can also implement the beam search or viterbi decoding upon the probability distribution matrix. - -After prediction, the result is saved in `predict.res`. - -## Reference -[1] Martha Palmer, Dan Gildea, and Paul Kingsbury. The Proposition Bank: An Annotated Corpus of Semantic Roles , Computational Linguistics, 31(1), 2005. - -[2] Zhou, Jie, and Wei Xu. "End-to-end learning of semantic role labeling using recurrent neural networks." Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015. diff --git a/doc/_sources/tutorials/sentiment_analysis/index_en.md.txt b/doc/_sources/tutorials/sentiment_analysis/index_en.md.txt deleted file mode 100644 index bb7681db44ca6f286ad6935ddfecb9becb429192..0000000000000000000000000000000000000000 --- a/doc/_sources/tutorials/sentiment_analysis/index_en.md.txt +++ /dev/null @@ -1,328 +0,0 @@ -# Sentiment Analysis Tutorial - -Sentiment analysis has many applications. A basic task in sentiment analysis is classifying the polarity of a given text at the document, sentence or feature/aspect level. One simple example is to classify the customer reviews in a shopping website, a tourism website, and group buying websites like Amazon, TaoBao, Tmall etc. - -Sentiment analysis is also used to monitor social media based on large amount of reviews or blogs. For example, the researchers analyzed several surveys on consumer confidence and political opinion, found they correlate to sentiment word frequencies in contemporaneous Twitter messages [1]. Another example is to forecast stock movements through analyzing the text content of a daily Twitter blog [2]. - -On the other hand, grabbing the user comments of products and analyzing their sentiment are useful to understand user preferences for companies, products, even competing products. - -This tutorial will guide you through the process of training a Long Short Term Memory (LSTM) Network to classify the sentiment of sentences from [Large Movie Review Dataset](http://ai.stanford.edu/~amaas/data/sentiment/), sometimes known as the Internet Movie Database (IMDB). This dataset contains movie reviews along with their associated binary sentiment polarity labels, namely positive and negative. So randomly guessing yields 50% accuracy. - -## Data Preparation - -### IMDB Data Introduction - -Before training models, we need to preprocess the data and build a dictionary. First, you can use following script to download IMDB dataset and [Moses](http://www.statmt.org/moses/) tool, which is a statistical machine translation system. We provide a data preprocessing script, which is capable of handling not only IMDB data, but also other user-defined data. In order to use the pre-written script, it needs to move labeled train and test samples to another path, which has been done in `get_imdb.sh`. - -``` -cd demo/sentiment/data -./get_imdb.sh -``` -If the data is obtained successfuly, you will see the following files at ```./demo/sentiment/data```: - -``` -aclImdb get_imdb.sh imdb mosesdecoder-master -``` - -* aclImdb: raw dataset downloaded from website. -* imdb: only contains train and test data. -* mosesdecoder-master: Moses tool. - -IMDB dataset contains 25,000 highly polar movie reviews for training, and 25,000 for testing. A negative review has a score ≤ 4 out of 10, and a positive review has a score ≥ 7 out of 10. After running `./get_imdb.sh`, we can find the dataset has the following structure in `aclImdb`. - -``` -imdbEr.txt imdb.vocab README test train -``` -* train: train sets. -* test : test sets. -* imdb.vocab: dictionary. -* imdbEr.txt: expected rating for each token in imdb.vocab. -* README: data documentation. - -The file in train set directory is as follows. The test set also contains them except `unsup` and `urls_unsup.txt`. - -``` -labeledBow.feat neg pos unsup unsupBow.feat urls_neg.txt urls_pos.txt urls_unsup.txt -``` - -* pos: positive samples, contains 12,500 txt files, each file is one movie review. -* neg: negative samples, contains 12,500 txt files, each file is one movie review. -* unsup: unlabeled samples, contains 50,000 txt files. -* urls_xx.txt: urls of each reviews. -* xxBow.feat: already-tokenized bag of words (BoW) features. - -### IMDB Data Preparation - -In this demo, we only use labled train and test set and not use imdb.vocab as dictionary. By default, dictionary is builded on train set. Train set is shuffled and test set is not. `tokenizer.perl` in Moses tool is used to tokenize the words and punctuation. Simply execute the following command to preprcess data. - -``` -cd demo/sentiment/ -./preprocess.sh -``` -preprocess.sh: - -``` -data_dir="./data/imdb" -python preprocess.py -i data_dir -``` - -* data_dir: input data directory. -* preprocess.py: preprocess script. - -If running successfully, you will see `demo/sentiment/data/pre-imdb` directory as follows: - -``` -dict.txt labels.list test.list test_part_000 train.list train_part_000 -``` -* test\_part\_000 and train\_part\_000: all labeled test and train sets. Train sets have be shuffled. -* train.list and test.list: train and test file lists. -* dict.txt: dictionary generated on train sets by default. -* labels.txt: neg 0, pos 1, means label 0 is negative review, label 1 is positive review. - -### User-defined Data Preparation - -If you perform other sentiment classifcation task, you can prepare data as follows. We have provided the scripts to build dictionary and preprocess data. So just organize data as follows. - -``` -dataset -|----train -| |----class1 -| | |----text_files -| |----class2 -| | |----text_files -| | ... -|----test -| |----class1 -| | |----text_files -| |----class2 -| | |----text_files -| | ... -``` -* dataset: 1st directory. -* train, test: 2nd directory. -* class1,class2,...: 3rd directory. -* text_files: samples with text file format. - -All samples with text files format under the same folder are same category. Each text file contains one or more samples and each line is one sample. In order to shuffle fully, the preprocessing is a little different for data with multiple lines in one text file, which needs to set `-m True` in `preprocess.sh`. And tokenizer.perl is used by default. If you don't need it, only set `-t False` in `preprocess.sh'. - -## Training - -In this task, we use Recurrent Neural Network (RNN) of LSTM architecure to train sentiment analysis model. LSTM model was introduced primarily in order to overcome the problem of vanishing gradients. LSTM network resembles a standard recurrent neural network with a hidden layer, but each ordinary node in the hidden layer is replaced by a memory cell. Each memory cell contains four main elements: an input gate, a neuron with a self-recurrent connection, a forget gate and an output gate. More details can be found in the literature [4]. The biggest advantage of the LSTM architecture is that it learns to memorize information over long time intervals without the loss of short time memory. At each time step with a new coming word, historical information stored in the memory block is updated to iteratively learn the sequence representation. - -
![LSTM](./lstm.png)
-
Figure 1. LSTM [3]
- -Sentiment analysis is among the most typical problems in natural language understanding. It aims at predicting the attitude expressed in a sequence. Usually, only some key words, like adjectives and adverbs words, play a major role in predicting the sentiment of sequences or paragraphs. However, some review or comment contexts are very long, such as IMDB dataset. We use LSTM to perform this task for its improved design with the gate mechanism. First, it is able to summarize the representation from word level to context level with variable context length which is adapted by the gate values. Second, it can utilize the expanded context at the sentence level, while most methods are good at utilizing n-gram level knowledge. Third, it learns the paragraph representation directly rather than combining the context level information. This results in this end-to-end framework. - -In this demo we provide two network, namely bidirectional-LSTM and three layers of stacked-LSTM. - -#### Bidirectional-LSTM - -One is a bidirectional LSTM network, connected by fully connected layer and softmax, as shown in Figure 2. - -
![BiLSTM](./bi_lstm.jpg)
-
Figure 2. Bidirectional-LSTM
- -#### Stacked-LSTM -Another is three-layer LSTM structure in Figure 3. The bottom of the figure is word embedding. Next, three LSTM-Hidden layers are connected and the second LSTM is reversed. Then extract the maximum hidden vectors of all time step of hidden and LSTM layer as the representation for the entire sequence. Finally, a fully connected feed forward layer with softmax activation is used to perform the classification task. This network is refered to paper [5]. - -
![StackedLSTM](./stacked_lstm.jpg)
-
Figure 3. Stacked-LSTM for sentiment analysis
- -**Config** - -Switch into `demo/sentiment` directory, `trainer_config.py` file is an example of the config, containing algorithm and newtork configure. The first line imports predefined networks from `sentiment_net.py`. - -trainer_config.py: - -```python -from sentiment_net import * - -data_dir = "./data/pre-imdb" -# whether this config is used for test -is_test = get_config_arg('is_test', bool, False) -# whether this config is used for prediction -is_predict = get_config_arg('is_predict', bool, False) -dict_dim, class_dim = sentiment_data(data_dir, is_test, is_predict) - -################## Algorithm Config ##################### - -settings( - batch_size=128, - learning_rate=2e-3, - learning_method=AdamOptimizer(), - average_window=0.5, - regularization=L2Regularization(8e-4), - gradient_clipping_threshold=25 -) - -#################### Network Config ###################### -stacked_lstm_net(dict_dim, class_dim=class_dim, - stacked_num=3, is_predict=is_predict) -#bidirectional_lstm_net(dict_dim, class_dim=class_dim, is_predict=is_predict) -``` - -* **Data Definition**: - * get\_config\_arg(): get arguments setted by `--config_args=xx` in commandline argument. - * Define data provider, here using Python interface to load data. For details, you can refer to the document of PyDataProvider2. - -* **Algorithm Configuration**: - * set batch size of 128. - * set global learning rate. - * use adam optimization. - * set average sgd window. - * set L2 regularization. - * set gradient clipping threshold. -* **Network Configuration**: - * dict_dim: dictionary dimension. - * class_dim: category number, IMDB has two label, namely positive and negative label. - * `stacked_lstm_net`: predefined network as shown in Figure 3, use this network by default. - * `bidirectional_lstm_net`: predefined network as shown in Figure 2. - -**Training** - -Install PaddlePaddle first if necessary. Then you can use script `train.sh` as follows to launch local training. - -``` -cd demo/sentiment/ -./train.sh -``` - -train.sh: - -``` -config=trainer_config.py -output=./model_output -paddle train --config=$config \ - --save_dir=$output \ - --job=train \ - --use_gpu=false \ - --trainer_count=4 \ - --num_passes=10 \ - --log_period=20 \ - --dot_period=20 \ - --show_parameter_stats_period=100 \ - --test_all_data_in_one_period=1 \ - 2>&1 | tee 'train.log' -``` - -* \--config=$config: set network config. -* \--save\_dir=$output: set output path to save models. -* \--job=train: set job mode to train. -* \--use\_gpu=false: use CPU to train, set true, if you install GPU version of PaddlePaddle and want to use GPU to train. -* \--trainer\_count=4: set thread number (or GPU count). -* \--num\_passes=15: set pass number, one pass in PaddlePaddle means training all samples in dataset one time. -* \--log\_period=20: print log every 20 batches. -* \--show\_parameter\_stats\_period=100: show parameter statistic every 100 batches. -* \--test\_all_data\_in\_one\_period=1: test all data every testing. - -If the run succeeds, the output log is saved in path of `demo/sentiment/train.log` and model is saved in path of `demo/sentiment/model_output/`. The output log is explained as follows. - -``` -Batch=20 samples=2560 AvgCost=0.681644 CurrentCost=0.681644 Eval: classification_error_evaluator=0.36875 CurrentEval: classification_error_evaluator=0.36875 -... -Pass=0 Batch=196 samples=25000 AvgCost=0.418964 Eval: classification_error_evaluator=0.1922 -Test samples=24999 cost=0.39297 Eval: classification_error_evaluator=0.149406 -``` -- Batch=xx: means passing xx batches. -- samples=xx: means passing xx samples. -- AvgCost=xx: averaged cost from 0-th batch to current batch. -- CurrentCost=xx: current cost of latest log_period batches. -- Eval: classification\_error\_evaluator=xx: means classfication error from 0-th batch ro current batch. -- CurrentEval: classification\_error\_evaluator: current classfication error of the lates log_period batches. -- Pass=0: Going through all training set one time is called one pass. 0 means going through training set first time. - -By default, we use the `stacked_lstm_net` network, which converges at a faster rate than `bidirectional_lstm_net` when passing same sample number. If you want to use bidirectional LSTM, just remove comment in the last line and comment `stacked_lstm_net`. - -## Testing - -Testing means evaluating the labeled validation set using trained model. - -``` -cd demo/sentiment -./test.sh -``` - -test.sh: - -```bash -function get_best_pass() { - cat $1 | grep -Pzo 'Test .*\n.*pass-.*' | \ - sed -r 'N;s/Test.* error=([0-9]+\.[0-9]+).*\n.*pass-([0-9]+)/\1 \2/g' | \ - sort | head -n 1 -} - -log=train.log -LOG=`get_best_pass $log` -LOG=(${LOG}) -evaluate_pass="model_output/pass-${LOG[1]}" - -echo 'evaluating from pass '$evaluate_pass - -model_list=./model.list -touch $model_list | echo $evaluate_pass > $model_list -net_conf=trainer_config.py -paddle train --config=$net_conf \ - --model_list=$model_list \ - --job=test \ - --use_gpu=false \ - --trainer_count=4 \ - --config_args=is_test=1 \ - 2>&1 | tee 'test.log' -``` - -The function `get_best_pass` gets the best model by classification error rate for testing. In this example, We use test dataset of IMDB as validation by default. Unlike training, it needs to specify `--job=test` and model path, namely `--model_list=$model_list` here. If running successfully, the log is saved in path of `demo/sentiment/test.log`. For example, in our test, the best model is `model_output/pass-00002`, the classification error is 0.115645 as follows. - -``` -Pass=0 samples=24999 AvgCost=0.280471 Eval: classification_error_evaluator=0.115645 -``` - -## Prediction - -`predict.py` provides a predicting interface. You should install python api of PaddlePaddle before using it. One example to predict unlabeled review of IMDB is as follows. Simply running: - -``` -cd demo/sentiment -./predict.sh -``` -predict.sh: - -``` -#Note the default model is pass-00002, you shold make sure the model path -#exists or change the mode path. -model=model_output/pass-00002/ -config=trainer_config.py -label=data/pre-imdb/labels.list -cat ./data/aclImdb/test/pos/10007_10.txt | python predict.py \ - --tconf=$config\ - --model=$model \ - --label=$label \ - --dict=./data/pre-imdb/dict.txt \ - --batch_size=1 -``` - -* `cat ./data/aclImdb/test/pos/10007_10.txt` : the input sample. -* `predict.py` : predicting interface. -* `--tconf=$config` : set network configure. -* ` --model=$model` : set model path. -* `--label=$label` : set dictionary about corresponding relation between integer label and string label. -* `--dict=data/pre-imdb/dict.txt` : set dictionary. -* `--batch_size=1` : set batch size. - -Note you should make sure the default model path `model_output/pass-00002` -exists or change the model path. - -Predicting result of this example: - -``` -Loading parameters from model_output/pass-00002/ -./data/aclImdb/test/pos/10014_7.txt: predicting label is pos -``` -We sincerely appreciate your interest and welcome your contributions. - -## Reference -[1] Brendan O'Connor, Ramnath Balasubramanyan, Bryan R. Routledge, and Noah A. Smith. 2010. [From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series](http://homes.cs.washington.edu/~nasmith/papers/oconnor+balasubramanyan+routledge+smith.icwsm10.pdf). In ICWSM-2010.
-[2] Johan Bollen, Huina Mao, Xiaojun Zeng. 2011. [Twitter mood predicts the stock market](http://arxiv.org/abs/1010.3003), Journal of Computational Science.
-[3] Alex Graves, Marcus Liwicki, Santiago Fernan- dez, Roman Bertolami, Horst Bunke, and Ju ̈rgen Schmidhuber. 2009. [A novel connectionist system for unconstrained handwriting recognition. IEEE Transactions on Pattern Analysis and Machine In- telligence](http://www.cs.toronto.edu/~graves/tpami_2009.pdf), 31(5):855–868.
-[4] Zachary C. Lipton, [A Critical Review of Recurrent Neural Networks for Sequence Learning](http://arxiv.org/abs/1506.00019v1), arXiv:1506.00019.
-[5] Jie Zhou and Wei Xu; [End-to-end Learning of Semantic Role Labeling Using Recurrent Neural Networks](http://www.aclweb.org/anthology/P/P15/P15-1109.pdf); ACL-IJCNLP 2015.
diff --git a/doc/_sources/tutorials/text_generation/index_en.md.txt b/doc/_sources/tutorials/text_generation/index_en.md.txt deleted file mode 100644 index 5d8e667c20bd1fda64a6e11a88517d52112b72fa..0000000000000000000000000000000000000000 --- a/doc/_sources/tutorials/text_generation/index_en.md.txt +++ /dev/null @@ -1,338 +0,0 @@ -# Text generation Tutorial # - -Sequence to sequence has been proven to be a powerful model for language generation. It can be used for machine translation, query rewriting, image captioning, etc. - -This tutorial guides you through training a sequence to sequence model for neural machine translation (NMT) network that translates French to English. - -We follow the paper [Neural Machine Translation by Jointly Learning to Align and Translate](http://arxiv.org/abs/1409.0473) , which details the model architecture and training procedure for good performance on WMT-14 dataset. This tutorial reproduces this result in PaddlePaddle. - -We thank @caoying for the pull request that defines the model architecture and solver configurations. - -## Data Preparation ## -### Download and Extract ### -Download the WMT-14 dataset from [http://www-lium.univ-lemans.fr/~schwenk/cslm\_joint\_paper/](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/), extract it, and divide Develop and Test data into separate folder. - -- **Train data**: [bitexts (after selection)](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/bitexts.tgz) -- **Develop and Test data**: [dev+test data](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/dev+test.tgz) - -To do this, simply run the following commands in linux, otherwise, you need to download, extract, divide, and rename the file suffix respectively. - -```bash -cd demo/seqToseq/data -./wmt14_data.sh -``` - -We should find that the dataset `wmt14` has three folders as shown in the following table. - ------ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
folder nameFrench-English parallel corpora filenumber of total filesize
train_dataccb2_pc30.src, ccb2_pc30.trg, etctwelve3.55G
test_datantst1213.src, ntst1213.trgtwo1636k
gen_datantst14.src, ntst14.trgtwo864k
-
- -- Each folder has French-English parallel corpora -- **XXX.src** are source French files; **XXX.trg** are target English files. -- The number of lines of **XXX.src** and **XXX.trg** should be the same. -- Each line is a French/English sentence. -- There is a one-to-one correspondence between the sentence at the i-th line of **XXX.src** and **XXX.trg**. - -### User Defined Dataset ### - -If you need to do other sequence-to-sequence tasks, such as Paraphrasing, you only need to organize the data as follows, and place them in `demo/seqToseq/data`: - - dataset - train - file1.src file1.trg - file2.src file2.trg - ...... - test - file1.src file1.trg - file2.src file2.trg - ...... - gen - file1.src file1.trg - file2.src file2.trg - ...... -- 1st directory: dataset folder name -- 2nd directory: folder of train, test, and gen. The names of these three folders are fixed. -- 3rd file: Source-Target parallel corpora files. - - **XXX.src** are source files, **XXX.trg** are target files. - - Each line of the file must be a sequence. - - There should be a one-to-one correspondence between the i-th sequence of **XXX.src** and **XXX.trg**. - -## Data Preprocess ## -### Preprocessing Workflow ### -- Concat each Source-Target parallel corpora to be one file: - - concat each **XXX.src** and **XXX.trg** to be **XXX**. - - the i-th line of **XXX** = the i-th line of **XXX.src** + '\t' + the i-th line of **XXX.trg** -- Build source and target dictionary of train data, each dictionary has DICTSIZE words: - - the most frequent (DICTSIZE-3) words - - 3 special token: - - ``: the start of a sequence - - ``: the end of a sequence - - ``: a word not included in dictionary - -### Preprocessing Command and Result -The general command for preprocessing the dataset is: - -```python -cd demo/seqToseq/ -python preprocess.py -i INPUT [-d DICTSIZE] [-m] -``` - -- `-i INPUT`: the path of input original dataset -- `-d DICTSIZE`: the specified word count of dictionary, if not set, dictionary will contain all the words in input dataset -- `-m --mergeDict`: merge source and target dictionary, thus, two dictionaries have the same context - -And you will see messages like this: - - concat parallel corpora for dataset - build source dictionary for train data - build target dictionary for train data - dictionary size is XXX - -Here, you can simply run the command: - -```python -python preprocess.py -i data/wmt14 -d 30000 -``` - -It will take several minutes, and store the preprocessed dataset in `demo/seqToseq/data/pre-wmt14`, the directory has following structure. - - train test gen train.list test.list gen.list src.dict trg.dict - -- **train, test, gen**: folder contains French-English parallel corpora of train data, test data and gen data respectively. Each line of file in folder contains two parts, the former is a French sequence, and the latter is a corresponding English sequence. -- **train.list, test.list, gen.list**: text contains a file list in train folder, test folder and gen folder respectively -- **src.dict, trg.dict**: source (French) / target (English) dictionary, each dictionary has 30000 words: the most frequent 29997 words and 3 special token - -## Model Training ## -### Introduction ### - -Neural machine translation (NMT) aims at building a single neural network that can be jointly tuned to maximize translation performance. Recently proposed NMT models often belong to a family of encoder–decoder models. Encoder-Decoder models encode a source sentence into a fixed-length vector from which a decoder generates a target sentence. - -In this task, we use an extension to the encoder–decoder model which learns to align and translate jointly. Each time the model generates a word in a translation, it searches for a set of positions in the source sentence for the most relevant information. The decoder predicts a target word based on the context vectors associated with these source positions and all the previous generated target words. For more detailed explanation, readers can refer to paper [Neural Machine Translation by Jointly Learning to Align and Translate](http://arxiv.org/abs/1409.0473). - -The most distinguishing feature of this model is that it doesn't encode an input sentence into a single fixed-length vector. Instead, it encodes the input sentence into a sequence of vectors, where one vector corresponds to an input element. A subset of these vectors is chosen adaptively while decoding the translated sentence. This frees a NMT model from having to squash all the information of a source sentence, regardless of its length, into a fixed-length vector. The improvement of this model is more apparent for longer sentences, but the improvement can be observed for sentences of any length. -
![](./encoder-decoder-attention-model.png)
-
Figure 1. Encoder-Decoder-Attention-Model
- -### Training Model in PaddlePaddle ### -We need to create a model config file before training. Here is an example `demo/seqToseq/translation/train.conf`. The first three lines import python function for defining network, and define the job_mode and attention_mode. - -```python -from seqToseq_net import * -is_generating = False - -### Data Definiation -train_conf = seq_to_seq_data(data_dir = "./data/pre-wmt14", - is_generating = is_generating) - -### Algorithm Configuration -settings( - learning_method = AdamOptimizer(), - batch_size = 50, - learning_rate = 5e-4) - -### Network Architecture -gru_encoder_decoder(train_conf, is_generating) -``` - -1. **Data Definiation**: We define a SeqToSeq train and test data in our example. It returns train_conf as the configuration, following is its input arguments: - - data_dir: directory of train data and test data - - is\_generating: whether this config is used for generating, here is false -2. **Algorithm Configuration**: We use the SGD training algorithm (default), ADAM learning method in our example, specify batch_size as 50, and learning rate as 5e-4. -3. **Network Architecture**: We use an attention version of GRU Encoder-Decoder network in our example. It consists a bidirectional GRU as an encoder and a decoder that emulates searching through a source sentence during decoding a translation. - -### Training Command and Result### -After writing the model config, we can train the model by running the command: - -```bash -cd demo/seqToseq/translation -./train.sh -``` - -The `train.sh` is shown as follows: - -```bash -paddle train \ ---config='translation/train.conf' \ ---save_dir='translation/model' \ ---use_gpu=false \ ---num_passes=16 \ ---show_parameter_stats_period=100 \ ---trainer_count=4 \ ---log_period=10 \ ---dot_period=5 \ -2>&1 | tee 'translation/train.log' -``` -- config: set config of neural network -- save_dir: set output path to save models -- use_gpu: whether to use GPU to train, here use CPU -- num_passes: set number of passes. One pass in paddle means training all samples in dataset one time -- show_parameter_stats_period: here show parameter statistic every 100 batches -- trainer_count: set number of CPU threads or GPU devices -- log_period: here print log every 10 batches -- dot_period: here print '.' every 5 batches - -The training loss function is printed every 10 batch by default, and you will see messages like this: - - I0719 19:16:45.952062 15563 TrainerInternal.cpp:160] Batch=10 samples=500 AvgCost=198.475 CurrentCost=198.475 Eval: classification_error_evaluator=0.737155 CurrentEval: classification_error_evaluator=0.737155 - I0719 19:17:56.707319 15563 TrainerInternal.cpp:160] Batch=20 samples=1000 AvgCost=157.479 CurrentCost=116.483 Eval: classification_error_evaluator=0.698392 CurrentEval: classification_error_evaluator=0.659065 - ..... -- AvgCost: Average Cost from 0th batch to current batch -- CurrentCost: Cost in current batch -- classification\_error\_evaluator(Eval): False prediction rate for each word from 0th evaluation to current evaluation -- classification\_error\_evaluator(CurrentEval): False prediction rate for each word in current evaluation - -And when the classification\_error\_evaluator is less than 0.35, the model is trained sucessfully. - -## Text Generation ## -### Introduction ### - -Generally speaking, the NMT model is conditioned on the encodings of the source sentence, and then to predict the next target word by given the current target word. In the training process, the current word is always knowns as the ground truth, by contrast. In the generating process, the current word is the output of the decoder in last time step, which is accessed to from a memory in PaddlePaddle. - -Besides, we use Beam Search to generate sequences. Beam search uses breadth-first search to build its search tree. At each level of the tree, it generates all successors of the states at the current level, sorting them in increasing order of heuristic cost. However, it only stores a predetermined number of best states at each level (called the beam size). - -### Pretrained model ### -We trained the model on a cluster with 50 nodes, each node has two 6-core CPUs. We trained 16 passes in 5 days, where each pass takes 7 hours. The model_dir has 16 sub-folder, each of which contains the whole model parameters with 202MB size. And we find pass-00012 model has the highest BLEU 27.77 (see paper [BLEU: a Method for Automatic Evaluation of Machine Translation](http://www.aclweb.org/anthology/P02-1040.pdf)). To download and extract this model, simply run the following commands in linux. - -```bash -cd demo/seqToseq/data -./wmt14_model.sh -``` - -### Generating Model in PaddlePaddle ### -We need to create a model config file before translating French sequence. Here is an example `demo/seqToseq/translation/gen.conf`, the first three lines import python function for defining network, and define the job\_mode and attention\_mode. - -```python -from seqToseq_net import * -is_generating = True - -################## Data Definiation ##################### -gen_conf = seq_to_seq_data(data_dir = "./data/pre-wmt14", - is_generating = is_generating, - gen_result = "./translation/gen_result") - -############## Algorithm Configuration ################## -settings( - learning_method = AdamOptimizer(), - batch_size = 1, - learning_rate = 0) - -################# Network configure ##################### -gru_encoder_decoder(gen_conf, is_generating) -``` - -1. **Data Definiation**: We defines an SeqToSeq gen data in our example. It returns gen_conf as the configuration, following is its input arguments: - - data\_dir: directory of gen data -   - is\_generating: whether this config is used for generating, here is true -   - gen\_result: file to store the generation result -2. **Algorithm Configuration**: We use SGD traing algorithm in generation, and specify batch_size as 1 (each time generate one sequence), and learning rate as 0. -3. **Network Architecture**: Essentially the same as the training model. - -### Generating Command and Result ### -After writing the model config, we can do text translation from French to English by running the command: - -```bash -cd demo/seqToseq/translation -./gen.sh -``` - -The `gen.sh` is shown as follows, unlike training, there are some different arguments to specify: - -```bash -paddle train \ ---job=test \ ---config='translation/gen.conf' \ ---save_dir='data/wmt14_model' \ ---use_gpu=true \ ---num_passes=13 \ ---test_pass=12 \ ---trainer_count=1 \ -2>&1 | tee 'translation/gen.log' -``` -- job: set job mode to test -- save_dir: the path of saved models -- num_passes and test_pass: loading model parameters from test_pass to (num_passes - 1), here only loads `data/wmt14_model/pass-00012` - -You will see messages like this: - - I0706 14:48:31.178915 31441 GradientMachine.cpp:143] Loading parameters from data/wmt14_model/pass-00012 - I0706 14:48:40.012039 31441 Tester.cpp:125] Batch=100 samples=100 AvgCost=0 - I0706 14:48:48.898632 31441 Tester.cpp:125] Batch=200 samples=200 AvgCost=0 - ... - -And the generating result in `demo/seqToseq/translation/gen_result` likes: - - 0 - 0 -11.1314 The about the width of the seats while large controls are at stake - 1 -11.1519 The on the width of the seats while large controls are at stake - 2 -11.5988 The about the width of the seats while large controls are at stake . - - 1 - 0 -24.4149 The dispute is between the major aircraft manufacturers about the width of the tourist seats on the flights , paving the way for a confrontation during the month of the Dubai . - 1 -26.9524 The dispute is between the major aircraft manufacturers about the width of the tourist seats on the flights , paving the way for a confrontation during the month of Dubai ' s . - 2 -27.9574 The dispute is between the major aircraft manufacturers about the width of the tourist seats on the flights , paving the way for a confrontation during the month of Dubai ' s Dubai . - ... - -- This is the beam search result, where beam size is 3 -- '0' in 1st-line and '1' in 6th-line mean the sequence-id in gen data -- Other six lines list the beam search results - - The 2nd-column is the score of beam search (from large to small) - - The 3rd-colunm is the generating English sequence -- There is 2 special tokens: - - ``: the end of a sequence - - ``: a word not included in dictionary - -### Bleu Evalutaion ### -Human evaluations of machine translation are extensive but expensive. Paper [BLEU: a Method for Automatic Evaluation of Machine Translation](http://www.aclweb.org/anthology/P02-1040.pdf) presents a method as an automated understudy to skilled human judges which substitutes for them when there is need for quick or frequent evaluations. [Moses](http://www.statmt.org/moses/) is a statistical machine translation system, and we use [multi-bleu.perl](https://github.com/moses-smt/mosesdecoder/blob/master/scripts/generic/multi-bleu.perl) of it to do Bleu Evalution. To download this script, simply run the following command: - -```bash -cd demo/seqToseq/translation -./moses_bleu.sh -``` - -Since the standard translation is alrealy downloaded as `data/wmt14/gen/ntst14.trg`, we can do Bleu Evalution by running the command: - -```bash -cd demo/seqToseq/translation -./eval_bleu.sh FILE BEAMSIZE -``` - -- FILE: the generation result file -- BEAMSIZE: expand width in beam search diff --git a/doc/_sources/v1_api_tutorials/README.md.txt b/doc/_sources/v1_api_tutorials/README.md.txt new file mode 100644 index 0000000000000000000000000000000000000000..071b8da61fbcab3e88819273008b4526546202ad --- /dev/null +++ b/doc/_sources/v1_api_tutorials/README.md.txt @@ -0,0 +1,5 @@ +The tutorials in v1_api_tutorials are using v1_api currently, and will be upgraded to v2_api later. +Thus, v1_api_tutorials is a temporary directory. We decide not to maintain it and will delete it in future. + +Please go to [PaddlePaddle/book](https://github.com/PaddlePaddle/book) and +[PaddlePaddle/models](https://github.com/PaddlePaddle/models) to learn PaddlePaddle. diff --git a/doc/_sources/tutorials/embedding_model/index_en.md.txt b/doc/_sources/v1_api_tutorials/embedding_model/index_en.md.txt similarity index 100% rename from doc/_sources/tutorials/embedding_model/index_en.md.txt rename to doc/_sources/v1_api_tutorials/embedding_model/index_en.md.txt diff --git a/doc/_sources/tutorials/gan/index_en.md.txt b/doc/_sources/v1_api_tutorials/gan/index_en.md.txt similarity index 100% rename from doc/_sources/tutorials/gan/index_en.md.txt rename to doc/_sources/v1_api_tutorials/gan/index_en.md.txt diff --git a/doc/_sources/tutorials/imagenet_model/resnet_model_en.md.txt b/doc/_sources/v1_api_tutorials/imagenet_model/resnet_model_en.md.txt similarity index 100% rename from doc/_sources/tutorials/imagenet_model/resnet_model_en.md.txt rename to doc/_sources/v1_api_tutorials/imagenet_model/resnet_model_en.md.txt diff --git a/doc/_sources/tutorials/quick_start/index_en.md.txt b/doc/_sources/v1_api_tutorials/quick_start/index_en.md.txt similarity index 100% rename from doc/_sources/tutorials/quick_start/index_en.md.txt rename to doc/_sources/v1_api_tutorials/quick_start/index_en.md.txt diff --git a/doc/api/index_en.html b/doc/api/index_en.html index 1738caea7d0edbecdd781f0bb75c576d78bbdd28..61396b4c43ea688893ce79ca5e1924fdb7e3765c 100644 --- a/doc/api/index_en.html +++ b/doc/api/index_en.html @@ -64,7 +64,7 @@
- Folk me on Github + Fork me on Github
English @@ -86,7 +86,7 @@
  • GET STARTED
  • HOW TO
  • API
    • -
  • ABOUT
    • +
  • MOBILE
    • @@ -110,9 +110,10 @@ -
  • Run Distributed Training
    • +
  • PaddlePaddle Distributed Training
  • Paddle On Kubernetes
  • Distributed PaddlePaddle Training on AWS with Kubernetes
  • Write New Layers
  • Contribute Code
    • -
  • RNN Models
    • +
  • Contribute Documentation
    • +
  • RNN Models +
  • Tune GPU Performance
    • @@ -137,17 +142,40 @@
  • Model Configuration
    • -
  • Data Reader Interface and DataSets
    • +
  • Data Reader Interface and DataSets +
  • Training and Inference
    • +
  • Fluid +
    • + + +
  • MOBILE
    • -
  • ABOUT
    • @@ -184,6 +212,7 @@
  • Model Configuration
  • Data Reader Interface and DataSets
  • Training and Inference
    • +
  • Fluid
    • diff --git a/doc/api/v1/data_provider/dataprovider_en.html b/doc/api/v1/data_provider/dataprovider_en.html index ee1b046476565a6e4bc330b101fa5683d1688632..94ad741071534283ff33a3980d6fe3c330379a0f 100644 --- a/doc/api/v1/data_provider/dataprovider_en.html +++ b/doc/api/v1/data_provider/dataprovider_en.html @@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -84,7 +84,7 @@
  • GET STARTED
  • HOW TO
  • API
    • -
  • ABOUT
    • +
  • MOBILE
    • @@ -108,9 +108,10 @@ -
  • Run Distributed Training
    • +
  • PaddlePaddle Distributed Training
  • Paddle On Kubernetes
  • Distributed PaddlePaddle Training on AWS with Kubernetes
  • Write New Layers
  • Contribute Code
    • -
  • RNN Models
    • +
  • Contribute Documentation
    • +
  • RNN Models +
  • Tune GPU Performance
    • @@ -135,17 +140,40 @@
  • Model Configuration
    • -
  • Data Reader Interface and DataSets
    • +
  • Data Reader Interface and DataSets +
  • Training and Inference
    • +
  • Fluid +
    • + + +
  • MOBILE
    • -
  • ABOUT
    • diff --git a/doc/api/v1/data_provider/pydataprovider2_en.html b/doc/api/v1/data_provider/pydataprovider2_en.html index ed4eacccd7cefb618e2b67af82a25ee6b7171edc..2bd66ed2e65ab0413dee28066d4a87da6da2cd91 100644 --- a/doc/api/v1/data_provider/pydataprovider2_en.html +++ b/doc/api/v1/data_provider/pydataprovider2_en.html @@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -84,7 +84,7 @@
  • GET STARTED
  • HOW TO
  • API
    • -
  • ABOUT
    • +
  • MOBILE
    • @@ -108,9 +108,10 @@ -
  • Run Distributed Training
    • +
  • PaddlePaddle Distributed Training
  • Paddle On Kubernetes
  • Distributed PaddlePaddle Training on AWS with Kubernetes
  • Write New Layers
  • Contribute Code
    • -
  • RNN Models
    • +
  • Contribute Documentation
    • +
  • RNN Models +
  • Tune GPU Performance
    • @@ -135,17 +140,40 @@
  • Model Configuration
    • -
  • Data Reader Interface and DataSets
    • +
  • Data Reader Interface and DataSets +
  • Training and Inference
    • +
  • Fluid +
    • + + +
  • MOBILE
    • -
  • ABOUT
    • diff --git a/doc/api/v1/index_en.html b/doc/api/v1/index_en.html index d000bcdc566750d8d0d947264bf8c6f18b348c8f..597e50a33137992a26e74688ea4a6fc8a81f0e2d 100644 --- a/doc/api/v1/index_en.html +++ b/doc/api/v1/index_en.html @@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -84,7 +84,7 @@
  • GET STARTED
  • HOW TO
  • API
    • -
  • ABOUT
    • +
  • MOBILE
    • @@ -108,9 +108,10 @@ -
  • Run Distributed Training
    • +
  • PaddlePaddle Distributed Training
  • Paddle On Kubernetes
  • Distributed PaddlePaddle Training on AWS with Kubernetes
  • Write New Layers
  • Contribute Code
    • -
  • RNN Models
    • +
  • Contribute Documentation
    • +
  • RNN Models +
  • Tune GPU Performance
    • @@ -135,17 +140,40 @@
  • Model Configuration
    • -
  • Data Reader Interface and DataSets
    • +
  • Data Reader Interface and DataSets +
  • Training and Inference
    • +
  • Fluid +
    • + + +
  • MOBILE
    • -
  • ABOUT
    • @@ -189,16 +217,6 @@
    diff --git a/doc/api/v1/predict/swig_py_paddle_en.html b/doc/api/v1/predict/swig_py_paddle_en.html index 02c77b18fd08d47872c0c3e4d71e9e946fd922f9..fc8c93f7b9f75c9846e74aefc7eaf27eeef50e01 100644 --- a/doc/api/v1/predict/swig_py_paddle_en.html +++ b/doc/api/v1/predict/swig_py_paddle_en.html @@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -84,7 +84,7 @@
  • GET STARTED
  • HOW TO
  • API
    • -
  • ABOUT
    • +
  • MOBILE
    • @@ -108,9 +108,10 @@ -
  • Run Distributed Training
    • +
  • PaddlePaddle Distributed Training
  • Paddle On Kubernetes
  • Distributed PaddlePaddle Training on AWS with Kubernetes
  • Write New Layers
  • Contribute Code
    • -
  • RNN Models
    • +
  • Contribute Documentation
    • +
  • RNN Models +
  • Tune GPU Performance
    • @@ -135,17 +140,40 @@
  • Model Configuration
    • -
  • Data Reader Interface and DataSets
    • +
  • Data Reader Interface and DataSets +
  • Training and Inference
    • +
  • Fluid +
    • + + +
  • MOBILE
    • -
  • ABOUT
    • diff --git a/doc/api/v1/trainer_config_helpers/activations.html b/doc/api/v1/trainer_config_helpers/activations.html deleted file mode 100644 index a8af1617c0fac758c03074bbf026f733b2281bed..0000000000000000000000000000000000000000 --- a/doc/api/v1/trainer_config_helpers/activations.html +++ /dev/null @@ -1,373 +0,0 @@ - - - - - - - - - - - Activations — PaddlePaddle documentation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    -
      - -
    • Activations
      • -
    -
    - -
    -
    -
    -
    - -
    -

    Activations

    -
    -

    BaseActivation

    -
    -
    -paddle.trainer_config_helpers.activations.BaseActivation
    -

    alias of Base

    -
    - -
    -
    -

    AbsActivation

    -
    -
    -paddle.trainer_config_helpers.activations.AbsActivation
    -

    alias of Abs

    -
    - -
    -
    -

    ExpActivation

    -
    -
    -paddle.trainer_config_helpers.activations.ExpActivation
    -

    alias of Exp

    -
    - -
    -
    -

    IdentityActivation

    -
    -
    -paddle.trainer_config_helpers.activations.IdentityActivation
    -

    alias of Linear

    -
    - -
    -
    -

    LinearActivation

    -
    -
    -paddle.trainer_config_helpers.activations.LinearActivation
    -

    alias of Linear

    -
    - -
    -
    -

    LogActivation

    -
    -
    -paddle.trainer_config_helpers.activations.LogActivation
    -

    alias of Log

    -
    - -
    -
    -

    SquareActivation

    -
    -
    -paddle.trainer_config_helpers.activations.SquareActivation
    -

    alias of Square

    -
    - -
    -
    -

    SigmoidActivation

    -
    -
    -paddle.trainer_config_helpers.activations.SigmoidActivation
    -

    alias of Sigmoid

    -
    - -
    -
    -

    SoftmaxActivation

    -
    -
    -paddle.trainer_config_helpers.activations.SoftmaxActivation
    -

    alias of Softmax

    -
    - -
    -
    -

    SequenceSoftmaxActivation

    -
    -
    -paddle.trainer_config_helpers.activations.SequenceSoftmaxActivation
    -

    alias of SequenceSoftmax

    -
    - -
    -
    -

    ReluActivation

    -
    -
    -paddle.trainer_config_helpers.activations.ReluActivation
    -

    alias of Relu

    -
    - -
    -
    -

    BReluActivation

    -
    -
    -paddle.trainer_config_helpers.activations.BReluActivation
    -

    alias of BRelu

    -
    - -
    -
    -

    SoftReluActivation

    -
    -
    -paddle.trainer_config_helpers.activations.SoftReluActivation
    -

    alias of SoftRelu

    -
    - -
    -
    -

    TanhActivation

    -
    -
    -paddle.trainer_config_helpers.activations.TanhActivation
    -

    alias of Tanh

    -
    - -
    -
    -

    STanhActivation

    -
    -
    -paddle.trainer_config_helpers.activations.STanhActivation
    -

    alias of STanh

    -
    - -
    -
    - - -
    -
    -
    - - -
    - -
    -

    - © Copyright 2016, PaddlePaddle developers. - -

    -
    - Built with Sphinx using a theme provided by Read the Docs. - -
    - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc/api/v1/trainer_config_helpers/layers.html b/doc/api/v1/trainer_config_helpers/layers.html deleted file mode 100644 index c356ab5209bf528514491cc5b66cecbf58048d56..0000000000000000000000000000000000000000 --- a/doc/api/v1/trainer_config_helpers/layers.html +++ /dev/null @@ -1,3515 +0,0 @@ - - - - - - - - - - - Layers — PaddlePaddle documentation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    -
      - -
    • Layers
      • -
    -
    - -
    -
    -
    -
    - -
    -

    Layers

    -
    -

    Base

    -
    -

    LayerType

    -
    -
    -class paddle.trainer_config_helpers.layers.LayerType
    -

    Layer type enumerations.

    -
    -
    -static is_layer_type(type_name)
    -

    If type_name is a layer type.

    - --- - - - - - - - -
    Parameters:type_name (basestring) – layer type name. Because layer type enumerations are -strings.
    Returns:True if is a layer_type
    Return type:bool
    -
    - -
    - -
    -
    -

    LayerOutput

    -
    -
    -class paddle.trainer_config_helpers.layers.LayerOutput(name, layer_type, parents=None, activation=None, num_filters=None, img_norm_type=None, size=None, outputs=None, reverse=None)
    -

    LayerOutput is output for layer function. It is used internally by several -reasons.

    -
      -

    • Check layer connection make sense.

      -
      -
        -
      • FC(Softmax) => Cost(MSE Error) is not good for example.
        • -
      -
      -
      • -

    • Tracking layer connection.

      -
      • -

    • Pass to layer methods as input.

      -
      • -
    - --- - - - -
    Parameters:
      -
    • name (basestring) – Layer output name.
      • -
    • layer_type (basestring) – Current Layer Type. One of LayerType enumeration.
      • -
    • activation (BaseActivation.) – Layer Activation.
      • -
    • parents (list|tuple|collections.Sequence) – Layer’s parents.
      • -
    -
    -
    -
    -set_input(input)
    -

    Set the input for a memory layer. Can only be used for memory layer

    -
    - -
    - -
    -
    -
    -

    Data layer

    -
    -

    data_layer

    -
    -
    -paddle.trainer_config_helpers.layers.data_layer(*args, **kwargs)
    -

    Define DataLayer For NeuralNetwork.

    -

    The example usage is:

    -
    data = data_layer(name="input", size=1000)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – Name of this data layer.
      • -
    • size (int) – Size of this data layer.
      • -
    • height (int|None) – Height of this data layer, used for image
      • -
    • width (int|None) – Width of this data layer, used for image
      • -
    • layer_attr (ExtraLayerAttribute.) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Fully Connected Layers

    -
    -

    fc_layer

    -
    -
    -paddle.trainer_config_helpers.layers.fc_layer(*args, **kwargs)
    -

    Helper for declare fully connected layer.

    -

    The example usage is:

    -
    fc = fc_layer(input=layer,
-              size=1024,
-              act=LinearActivation(),
-              bias_attr=False)
-
    -
    -

    which is equal to:

    -
    with mixed_layer(size=1024) as fc:
-    fc += full_matrix_projection(input=layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – The Layer Name.
      • -
    • input (LayerOutput|list|tuple) – The input layer. Could be a list/tuple of input layer.
      • -
    • size (int) – The layer dimension.
      • -
    • act (BaseActivation) – Activation Type. Default is tanh.
      • -
    • param_attr (ParameterAttribute) – The Parameter Attribute|list.
      • -
    • bias_attr (ParameterAttribute|None|Any) – The Bias Attribute. If no bias, then pass False or -something not type of ParameterAttribute. None will get a -default Bias.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    selective_fc_layer

    -
    -
    -paddle.trainer_config_helpers.layers.selective_fc_layer(*args, **kwargs)
    -

    Selectived fully connected layer. Different from fc_layer, the output -of this layer maybe sparse. It requires an additional input to indicate -several selected columns for output. If the selected columns is not -specified, selective_fc_layer acts exactly like fc_layer.

    -

    The simple usage is:

    -
    sel_fc = selective_fc_layer(input=input, size=128, act=TanhActivation())
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – The Layer Name.
      • -
    • input (LayerOutput|list|tuple) – The input layer.
      • -
    • select (LayerOutput) – The select layer. The output of select layer should be a -sparse binary matrix, and treat as the mask of selective fc. -If is None, acts exactly like fc_layer.
      • -
    • size (int) – The layer dimension.
      • -
    • act (BaseActivation) – Activation Type. Default is tanh.
      • -
    • param_attr (ParameterAttribute) – The Parameter Attribute.
      • -
    • bias_attr (ParameterAttribute|None|Any) – The Bias Attribute. If no bias, then pass False or -something not type of ParameterAttribute. None will get a -default Bias.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Conv Layers

    -
    -

    conv_operator

    -
    -
    -paddle.trainer_config_helpers.layers.conv_operator(img, filter, filter_size, num_filters, num_channels=None, stride=1, padding=0, filter_size_y=None, stride_y=None, padding_y=None, trans=False)
    -

    Different from img_conv_layer, conv_op is an Operator, which can be used -in mixed_layer. And conv_op takes two inputs to perform convolution. -The first input is the image and the second is filter kernel. It only -support GPU mode.

    -

    The example usage is:

    -
    op = conv_operator(img=input1,
-                   filter=input2,
-                   filter_size=3,
-                   num_filters=64,
-                   num_channels=64)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • img (LayerOutput) – input image
      • -
    • filter (LayerOutput) – input filter
      • -
    • filter_size (int) – The x dimension of a filter kernel.
      • -
    • filter_size_y (int) – The y dimension of a filter kernel. Since -PaddlePaddle now supports rectangular filters, -the filter’s shape can be (filter_size, filter_size_y).
      • -
    • num_filters (int) – channel of output data.
      • -
    • num_channels (int) – channel of input data.
      • -
    • stride (int) – The x dimension of the stride.
      • -
    • stride_y (int) – The y dimension of the stride.
      • -
    • padding (int) – The x dimension of padding.
      • -
    • padding_y (int) – The y dimension of padding.
      • -
    -
    Returns:

    A ConvOperator Object.

    -
    Return type:

    ConvOperator

    -
    -
    - -
    -
    -

    conv_projection

    -
    -
    -paddle.trainer_config_helpers.layers.conv_projection(*args, **kwargs)
    -

    Different from img_conv_layer and conv_op, conv_projection is an Projection, -which can be used in mixed_layer and conat_layer. It use cudnn to implement -conv and only support GPU mode.

    -

    The example usage is:

    -
    proj = conv_projection(input=input1,
-                       filter_size=3,
-                       num_filters=64,
-                       num_channels=64)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – input layer
      • -
    • filter_size (int) – The x dimension of a filter kernel.
      • -
    • filter_size_y (int) – The y dimension of a filter kernel. Since -PaddlePaddle now supports rectangular filters, -the filter’s shape can be (filter_size, filter_size_y).
      • -
    • num_filters (int) – channel of output data.
      • -
    • num_channels (int) – channel of input data.
      • -
    • stride (int) – The x dimension of the stride.
      • -
    • stride_y (int) – The y dimension of the stride.
      • -
    • padding (int) – The x dimension of padding.
      • -
    • padding_y (int) – The y dimension of padding.
      • -
    • groups (int) – The group number.
      • -
    • param_attr (ParameterAttribute) – Convolution param attribute. None means default attribute
      • -
    • trans (boolean) – whether it is convTrans or conv
      • -
    -
    Returns:

    A DotMulProjection Object.

    -
    Return type:

    DotMulProjection

    -
    -
    - -
    -
    -

    conv_shift_layer

    -
    -
    -paddle.trainer_config_helpers.layers.conv_shift_layer(*args, **kwargs)
    -
    -
    This layer performs cyclic convolution for two input. For example:
    -
      -
    • a[in]: contains M elements.
      • -
    • b[in]: contains N elements (N should be odd).
      • -
    • c[out]: contains M elements.
      • -
    -
    -
    -
    -\[c[i] = \sum_{j=-(N-1)/2}^{(N-1)/2}a_{i+j} * b_{j}\]
    -
    -
    In this formular:
    -
      -
    • a’s index is computed modulo M. When it is negative, then get item from -the right side (which is the end of array) to the left.
      • -
    • b’s index is computed modulo N. When it is negative, then get item from -the right size (which is the end of array) to the left.
      • -
    -
    -
    -

    The example usage is:

    -
    conv_shift = conv_shift_layer(a=layer1, b=layer2)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – layer name
      • -
    • a (LayerOutput) – Input layer a.
      • -
    • b (LayerOutput) – input layer b.
      • -
    • layer_attr (ExtraLayerAttribute) – layer’s extra attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    img_conv_layer

    -
    -
    -paddle.trainer_config_helpers.layers.img_conv_layer(*args, **kwargs)
    -

    Convolution layer for image. Paddle can support both square and non-square -input currently.

    -

    The details of convolution layer, please refer UFLDL’s convolution .

    -

    Convolution Transpose (deconv) layer for image. Paddle can support both square -and non-square input currently.

    -

    The details of convolution transpose layer, -please refer to the following explanation and references therein -<http://datascience.stackexchange.com/questions/6107/ -what-are-deconvolutional-layers/>`_ . -The num_channel means input image’s channel number. It may be 1 or 3 when -input is raw pixels of image(mono or RGB), or it may be the previous layer’s -num_filters * num_group.

    -

    There are several group of filter in PaddlePaddle implementation. -Each group will process some channel of the inputs. For example, if an input -num_channel = 256, group = 4, num_filter=32, the PaddlePaddle will create -32*4 = 128 filters to process inputs. The channels will be split into 4 -pieces. First 256/4 = 64 channels will process by first 32 filters. The -rest channels will be processed by rest group of filters.

    -

    The example usage is:

    -
    conv = img_conv_layer(input=data, filter_size=1, filter_size_y=1,
-                      num_channels=8,
-                      num_filters=16, stride=1,
-                      bias_attr=False,
-                      act=ReluActivation())
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – Layer name.
      • -
    • input (LayerOutput) – Layer Input.
      • -
    • filter_size (int|tuple|list) – The x dimension of a filter kernel. Or input a tuple for -two image dimension.
      • -
    • filter_size_y (int|None) – The y dimension of a filter kernel. Since PaddlePaddle -currently supports rectangular filters, the filter’s -shape will be (filter_size, filter_size_y).
      • -
    • num_filters – Each filter group’s number of filter
      • -
    • act (BaseActivation) – Activation type. Default is tanh
      • -
    • groups (int) – Group size of filters.
      • -
    • stride (int|tuple|list) – The x dimension of the stride. Or input a tuple for two image -dimension.
      • -
    • stride_y (int) – The y dimension of the stride.
      • -
    • padding (int|tuple|list) – The x dimension of the padding. Or input a tuple for two -image dimension
      • -
    • padding_y (int) – The y dimension of the padding.
      • -
    • bias_attr (ParameterAttribute|False) – Convolution bias attribute. None means default bias. -False means no bias.
      • -
    • num_channels (int) – number of input channels. If None will be set -automatically from previous output.
      • -
    • param_attr (ParameterAttribute) – Convolution param attribute. None means default attribute
      • -
    • shared_biases (bool) – Is biases will be shared between filters or not.
      • -
    • layer_attr (ExtraLayerAttribute) – Layer Extra Attribute.
      • -
    • trans (bool) – true if it is a convTransLayer, false if it is a convLayer
      • -
    • layer_type (String) – specify the layer_type, default is None. If trans=True, -layer_type has to be “exconvt” or “cudnn_convt”, -otherwise layer_type has to be either “exconv” or -“cudnn_conv”
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    context_projection

    -
    -
    -paddle.trainer_config_helpers.layers.context_projection(*args, **kwargs)
    -

    Context Projection.

    -

    It just simply reorganizes input sequence, combines “context_len” sequence -to one context from context_start. “context_start” will be set to --(context_len - 1) / 2 by default. If context position out of sequence -length, padding will be filled as zero if padding_attr = False, otherwise -it is trainable.

    -

    For example, origin sequence is [A B C D E F G], context len is 3, then -after context projection and not set padding_attr, sequence will -be [ 0AB ABC BCD CDE DEF EFG FG0 ].

    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input Sequence.
      • -
    • context_len (int) – context length.
      • -
    • context_start (int) – context start position. Default is --(context_len - 1)/2
      • -
    • padding_attr (bool|ParameterAttribute) – Padding Parameter Attribute. If false, it means padding -always be zero. Otherwise Padding is learnable, and -parameter attribute is set by this parameter.
      • -
    -
    Returns:

    Projection

    -
    Return type:

    Projection

    -
    -
    - -
    -
    -
    -

    Image Pooling Layer

    -
    -

    img_pool_layer

    -
    -
    -paddle.trainer_config_helpers.layers.img_pool_layer(*args, **kwargs)
    -

    Image pooling Layer.

    -

    The details of pooling layer, please refer ufldl’s pooling .

    -
      -
    • ceil_mode=True:
      • -
    -
    -\[w = 1 + int(ceil(input\_width + 2 * padding - pool\_size) / float(stride)) -h = 1 + int(ceil(input\_height + 2 * padding\_y - pool\_size\_y) / float(stride\_y))\]
    -
      -
    • ceil_mode=False:
      • -
    -
    -\[w = 1 + int(floor(input\_width + 2 * padding - pool\_size) / float(stride)) -h = 1 + int(floor(input\_height + 2 * padding\_y - pool\_size\_y) / float(stride\_y))\]
    -

    The example usage is:

    -
    maxpool = img_pool_layer(input=conv,
-                         pool_size=3,
-                         pool_size_y=5,
-                         num_channels=8,
-                         stride=1,
-                         stride_y=2,
-                         padding=1,
-                         padding_y=2,
-                         pool_type=MaxPooling())
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • padding (int) – pooling padding width.
      • -
    • padding_y (int|None) – pooling padding height. It’s equal to padding by default.
      • -
    • name (basestring.) – name of pooling layer
      • -
    • input (LayerOutput) – layer’s input
      • -
    • pool_size (int) – pooling window width
      • -
    • pool_size_y (int|None) – pooling window height. It’s eaqual to pool_size by default.
      • -
    • num_channels (int) – number of input channel.
      • -
    • pool_type (BasePoolingType) – pooling type. MaxPooling or AvgPooling. Default is -MaxPooling.
      • -
    • stride (int) – stride width of pooling.
      • -
    • stride_y (int|None) – stride height of pooling. It is equal to stride by default.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer attribute.
      • -
    • ceil_mode (bool) – Wether to use ceil mode to calculate output height and with. -Defalut is True. If set false, Otherwise use floor.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    spp_layer

    -
    -
    -paddle.trainer_config_helpers.layers.spp_layer(*args, **kwargs)
    -

    Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. -The details please refer to -Kaiming He’s paper.

    -

    The example usage is:

    -
    spp = spp_layer(input=data,
-                pyramid_height=2,
-                num_channels=16,
-                pool_type=MaxPooling())
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – layer name.
      • -
    • input (LayerOutput) – layer’s input.
      • -
    • num_channels (int) – number of input channel.
      • -
    • pool_type – Pooling type. MaxPooling or AveragePooling. Default is MaxPooling.
      • -
    • pyramid_height (int) – pyramid height.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    maxout_layer

    -
    -
    -paddle.trainer_config_helpers.layers.maxout_layer(*args, **kwargs)
    -
    -
    A layer to do max out on conv layer output.
    -
      -
    • Input: output of a conv layer.
      • -
    • Output: feature map size same as input. Channel is (input channel) / groups.
      • -
    -
    -
    -

    So groups should be larger than 1, and the num of channels should be able -to devided by groups.

    -
    -
    Please refer to Paper:
    -
    -
    -
    -

    The simple usage is:

    -
    maxout = maxout_layer(input,
-                      num_channels=128,
-                      groups=4)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – The input layer.
      • -
    • num_channels (int|None) – The channel number of input layer. If None will be set -automatically from previous output.
      • -
    • groups (int) – The group number of input layer.
      • -
    • name (None|basestring.) – The name of this layer, which can not specify.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Norm Layer

    -
    -

    img_cmrnorm_layer

    -
    -
    -paddle.trainer_config_helpers.layers.img_cmrnorm_layer(*args, **kwargs)
    -

    Response normalization across feature maps. -The details please refer to -Alex’s paper.

    -

    The example usage is:

    -
    norm = img_cmrnorm_layer(input=net, size=5)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (None|basestring) – layer name.
      • -
    • input (LayerOutput) – layer’s input.
      • -
    • size (int) – Normalize in number of \(size\) feature maps.
      • -
    • scale (float) – The hyper-parameter.
      • -
    • power (float) – The hyper-parameter.
      • -
    • num_channels – input layer’s filers number or channels. If -num_channels is None, it will be set automatically.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    batch_norm_layer

    -
    -
    -paddle.trainer_config_helpers.layers.batch_norm_layer(*args, **kwargs)
    -

    Batch Normalization Layer. The notation of this layer as follow.

    -

    \(x\) is the input features over a mini-batch.

    -
    -\[\begin{split}\mu_{\beta} &\gets \frac{1}{m} \sum_{i=1}^{m} x_i \qquad &//\ -\ mini-batch\ mean \\ -\sigma_{\beta}^{2} &\gets \frac{1}{m} \sum_{i=1}^{m}(x_i - \ -\mu_{\beta})^2 \qquad &//\ mini-batch\ variance \\ -\hat{x_i} &\gets \frac{x_i - \mu_\beta} {\sqrt{\ -\sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\ -y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift\end{split}\]
    -

    The details of batch normalization please refer to this -paper.

    -

    The example usage is:

    -
    norm = batch_norm_layer(input=net, act=ReluActivation())
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – layer name.
      • -
    • input (LayerOutput) – batch normalization input. Better be linear activation. -Because there is an activation inside batch_normalization.
      • -
    • batch_norm_type (None|string, None or "batch_norm" or "cudnn_batch_norm") – We have batch_norm and cudnn_batch_norm. batch_norm -supports both CPU and GPU. cudnn_batch_norm requires -cuDNN version greater or equal to v4 (>=v4). But -cudnn_batch_norm is faster and needs less memory -than batch_norm. By default (None), we will -automaticly select cudnn_batch_norm for GPU and -batch_norm for CPU. Otherwise, select batch norm -type based on the specified type. If you use cudnn_batch_norm, -we suggested you use latest version, such as v5.1.
      • -
    • act (BaseActivation) – Activation Type. Better be relu. Because batch -normalization will normalize input near zero.
      • -
    • num_channels (int) – num of image channels or previous layer’s number of -filters. None will automatically get from layer’s -input.
      • -
    • bias_attr (ParameterAttribute) – \(\beta\), better be zero when initialize. So the -initial_std=0, initial_mean=1 is best practice.
      • -
    • param_attr (ParameterAttribute) – \(\gamma\), better be one when initialize. So the -initial_std=0, initial_mean=1 is best practice.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    • use_global_stats (bool|None.) – whether use moving mean/variance statistics -during testing peroid. If None or True, -it will use moving mean/variance statistics during -testing. If False, it will use the mean -and variance of current batch of test data for -testing.
      • -
    • moving_average_fraction (float.) – Factor used in the moving average -computation, referred to as facotr, -\(runningMean = newMean*(1-factor) -+ runningMean*factor\)
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    sum_to_one_norm_layer

    -
    -
    -paddle.trainer_config_helpers.layers.sum_to_one_norm_layer(*args, **kwargs)
    -

    A layer for sum-to-one normalization, -which is used in NEURAL TURING MACHINE.

    -
    -\[out[i] = \frac {in[i]} {\sum_{k=1}^N in[k]}\]
    -

    where \(in\) is a (batchSize x dataDim) input vector, -and \(out\) is a (batchSize x dataDim) output vector.

    -

    The example usage is:

    -
    sum_to_one_norm = sum_to_one_norm_layer(input=layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Recurrent Layers

    -
    -

    recurrent_layer

    -
    -
    -paddle.trainer_config_helpers.layers.recurrent_layer(*args, **kwargs)
    -

    Simple recurrent unit layer. It is just a fully connect layer through both -time and neural network.

    -

    For each sequence [start, end] it performs the following computation:

    -
    -\[\begin{split}out_{i} = act(in_{i}) \ \ \text{for} \ i = start \\ -out_{i} = act(in_{i} + out_{i-1} * W) \ \ \text{for} \ start < i <= end\end{split}\]
    -

    If reversed is true, the order is reversed:

    -
    -\[\begin{split}out_{i} = act(in_{i}) \ \ \text{for} \ i = end \\ -out_{i} = act(in_{i} + out_{i+1} * W) \ \ \text{for} \ start <= i < end\end{split}\]
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input Layer
      • -
    • act (BaseActivation) – activation.
      • -
    • bias_attr (ParameterAttribute) – bias attribute.
      • -
    • param_attr (ParameterAttribute) – parameter attribute.
      • -
    • name (basestring) – name of the layer
      • -
    • layer_attr (ExtraLayerAttribute) – Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    lstmemory

    -
    -
    -paddle.trainer_config_helpers.layers.lstmemory(*args, **kwargs)
    -

    Long Short-term Memory Cell.

    -

    The memory cell was implemented as follow equations.

    -
    -\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    -

    NOTE: In PaddlePaddle’s implementation, the multiplications -\(W_{xi}x_{t}\) , \(W_{xf}x_{t}\), -\(W_{xc}x_t\), \(W_{xo}x_{t}\) are not done in the lstmemory layer, -so an additional mixed_layer with full_matrix_projection or a fc_layer must -be included in the configuration file to complete the input-to-hidden -mappings before lstmemory is called.

    -

    NOTE: This is a low level user interface. You can use network.simple_lstm -to config a simple plain lstm layer.

    -

    Please refer to Generating Sequences With Recurrent Neural Networks for -more details about LSTM.

    -

    Link goes as below.

    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – The lstmemory layer name.
      • -
    • input (LayerOutput) – input layer name.
      • -
    • reverse (bool) – is sequence process reversed or not.
      • -
    • act (BaseActivation) – activation type, TanhActivation by default. \(h_t\)
      • -
    • gate_act (BaseActivation) – gate activation type, SigmoidActivation by default.
      • -
    • state_act (BaseActivation) – state activation type, TanhActivation by default.
      • -
    • bias_attr (ParameterAttribute|None|False) – Bias attribute. None means default bias. False means no -bias.
      • -
    • param_attr (ParameterAttribute|None|False) – Parameter Attribute.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer attribute
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    grumemory

    -
    -
    -paddle.trainer_config_helpers.layers.grumemory(*args, **kwargs)
    -

    Gate Recurrent Unit Layer.

    -

    The memory cell was implemented as follow equations.

    -

    1. update gate \(z\): defines how much of the previous memory to -keep around or the unit updates its activations. The update gate -is computed by:

    -
    -\[z_t = \sigma(W_{z}x_{t} + U_{z}h_{t-1} + b_z)\]
    -

    2. reset gate \(r\): determines how to combine the new input with the -previous memory. The reset gate is computed similarly to the update gate:

    -
    -\[r_t = \sigma(W_{r}x_{t} + U_{r}h_{t-1} + b_r)\]
    -

    3. The candidate activation \(\tilde{h_t}\) is computed similarly to -that of the traditional recurrent unit:

    -
    -\[{\tilde{h_t}} = tanh(W x_{t} + U (r_{t} \odot h_{t-1}) + b)\]
    -

    4. The hidden activation \(h_t\) of the GRU at time t is a linear -interpolation between the previous activation \(h_{t-1}\) and the -candidate activation \(\tilde{h_t}\):

    -
    -\[h_t = (1 - z_t) h_{t-1} + z_t {\tilde{h_t}}\]
    -

    NOTE: In PaddlePaddle’s implementation, the multiplication operations -\(W_{r}x_{t}\), \(W_{z}x_{t}\) and \(W x_t\) are not computed in -gate_recurrent layer. Consequently, an additional mixed_layer with -full_matrix_projection or a fc_layer must be included before grumemory -is called.

    -

    More details can be found by referring to Empirical Evaluation of Gated -Recurrent Neural Networks on Sequence Modeling.

    -

    The simple usage is:

    -
    gru = grumemory(input)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (None|basestring) – The gru layer name.
      • -
    • input (LayerOutput.) – input layer.
      • -
    • reverse (bool) – Whether sequence process is reversed or not.
      • -
    • act (BaseActivation) – activation type, TanhActivation by default. This activation -affects the \({\tilde{h_t}}\).
      • -
    • gate_act (BaseActivation) – gate activation type, SigmoidActivation by default. -This activation affects the \(z_t\) and \(r_t\). It is the -\(\sigma\) in the above formula.
      • -
    • bias_attr (ParameterAttribute|None|False) – Bias attribute. None means default bias. False means no -bias.
      • -
    • param_attr (ParameterAttribute|None|False) – Parameter Attribute.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer attribute
      • -
    • size (None) – Stub parameter of size, but actually not used. If set this size -will get a warning.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Recurrent Layer Group

    -
    -

    memory

    -
    -
    -paddle.trainer_config_helpers.layers.memory(*args, **kwargs)
    -

    The memory layers is a layer cross each time step. Reference this output -as previous time step layer name ‘s output.

    -

    The default memory is zero in first time step, previous time step’s -output in the rest time steps.

    -

    If boot_bias, the first time step value is this bias and -with activation.

    -

    If boot_with_const_id, then the first time stop is a IndexSlot, the -Arguments.ids()[0] is this cost_id.

    -

    If boot_layer is not null, the memory is just the boot_layer’s output. -Set is_seq is true boot layer is sequence.

    -

    The same name layer in recurrent group will set memory on each time -step.

    -
    mem = memory(size=256, name='state')
-state = fc_layer(input=mem, size=256, name='state')
-
    -
    -

    If you do not want to specify the name, you can equivalently use set_input() -to specify the layer needs to be remembered as the following:

    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – the name of the layer which this memory remembers. -If name is None, user should call set_input() to specify the -name of the layer which this memory remembers.
      • -
    • size (int) – size of memory.
      • -
    • memory_name (basestring) – the name of the memory. -It is ignored when name is provided.
      • -
    • is_seq (bool) – is sequence for boot_layer
      • -
    • boot_layer (LayerOutput|None) – boot layer of memory.
      • -
    • boot_bias (ParameterAttribute|None) – boot layer’s bias
      • -
    • boot_bias_active_type (BaseActivation) – boot layer’s active type.
      • -
    • boot_with_const_id (int) – boot layer’s id.
      • -
    -
    Returns:

    LayerOutput object which is a memory.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    recurrent_group

    -
    -
    -paddle.trainer_config_helpers.layers.recurrent_group(*args, **kwargs)
    -

    Recurrent layer group is an extremely flexible recurrent unit in -PaddlePaddle. As long as the user defines the calculation done within a -time step, PaddlePaddle will iterate such a recurrent calculation over -sequence input. This is extremely usefull for attention based model, or -Neural Turning Machine like models.

    -

    The basic usage (time steps) is:

    -
    def step(input):
-    output = fc_layer(input=layer,
-                      size=1024,
-                      act=LinearActivation(),
-                      bias_attr=False)
-    return output
-
-group = recurrent_group(input=layer,
-                        step=step)
-
    -
    -

    You can see following configs for further usages:

    -
      -
    • time steps: lstmemory_group, paddle/gserver/tests/sequence_layer_group.conf, demo/seqToseq/seqToseq_net.py
      • -
    • sequence steps: paddle/gserver/tests/sequence_nest_layer_group.conf
      • -
    - --- - - - -
    Parameters:
      -
    • step (callable) –

      recurrent one time step function.The input of this function is -input of the group. The return of this function will be -recurrent group’s return value.

      -

      The recurrent group scatter a sequence into time steps. And -for each time step, will invoke step function, and return -a time step result. Then gather each time step of output into -layer group’s output.

      -
      • -
    • name (basestring) – recurrent_group’s name.
      • -
    • input (LayerOutput|StaticInput|SubsequenceInput|list|tuple) –

      Input links array.

      -

      LayerOutput will be scattered into time steps. -SubsequenceInput will be scattered into sequence steps. -StaticInput will be imported to each time step, and doesn’t change -through time. It’s a mechanism to access layer outside step function.

      -
      • -
    • reverse (bool) – If reverse is set true, the recurrent unit will process the -input sequence in a reverse order.
      • -
    • targetInlink (LayerOutput|SubsequenceInput) –

      the input layer which share info with layer group’s output

      -

      Param input specifies multiple input layers. For -SubsequenceInput inputs, config should assign one input -layer that share info(the number of sentences and the number -of words in each sentence) with all layer group’s outputs. -targetInlink should be one of the layer group’s input.

      -
      • -
    • is_generating – If is generating, none of input type should be LayerOutput; -else, for training or testing, one of the input type must -be LayerOutput.
      • -
    -
    -

    : type is_generating: bool

    - --- - - - - - -
    Returns:LayerOutput object.
    Return type:LayerOutput
    -
    - -
    -
    -

    lstm_step_layer

    -
    -
    -paddle.trainer_config_helpers.layers.lstm_step_layer(*args, **kwargs)
    -

    LSTM Step Layer. It used in recurrent_group. The lstm equations are shown -as follow.

    -
    -\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    -

    The input of lstm step is \(Wx_t + Wh_{t-1}\), and user should use -mixed_layer and full_matrix_projection to calculate these -input vector.

    -

    The state of lstm step is \(c_{t-1}\). And lstm step layer will do

    -
    -\[ \begin{align}\begin{aligned}i_t = \sigma(input + W_{ci}c_{t-1} + b_i)\\...\end{aligned}\end{align} \]
    -

    This layer contains two outputs. Default output is \(h_t\). The other -output is \(o_t\), which name is ‘state’ and can use -get_output_layer to extract this output.

    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – Layer’s name.
      • -
    • size (int) – Layer’s size. NOTE: lstm layer’s size, should be equal as -input.size/4, and should be equal as -state.size.
      • -
    • input (LayerOutput) – input layer. \(Wx_t + Wh_{t-1}\)
      • -
    • state (LayerOutput) – State Layer. \(c_{t-1}\)
      • -
    • act (BaseActivation) – Activation type. Default is tanh
      • -
    • gate_act (BaseActivation) – Gate Activation Type. Default is sigmoid, and should -be sigmoid only.
      • -
    • state_act (BaseActivation) – State Activation Type. Default is sigmoid, and should -be sigmoid only.
      • -
    • bias_attr (ParameterAttribute) – Bias Attribute.
      • -
    • layer_attr (ExtraLayerAttribute) – layer’s extra attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    gru_step_layer

    -
    -
    -paddle.trainer_config_helpers.layers.gru_step_layer(*args, **kwargs)
    -
    --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) –
      • -
    • output_mem –
      • -
    • size –
      • -
    • act –
      • -
    • name –
      • -
    • gate_act –
      • -
    • bias_attr –
      • -
    • param_attr – the parameter_attribute for transforming the output_mem -from previous step.
      • -
    • layer_attr –
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    - -
    -

    get_output_layer

    -
    -
    -paddle.trainer_config_helpers.layers.get_output_layer(*args, **kwargs)
    -

    Get layer’s output by name. In PaddlePaddle, a layer might return multiple -values, but returns one layer’s output. If the user wants to use another -output besides the default one, please use get_output_layer first to get -the output from input.

    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – Layer’s name.
      • -
    • input (LayerOutput) – get output layer’s input. And this layer should contains -multiple outputs.
      • -
    • arg_name (basestring) – Output name from input.
      • -
    • layer_attr – Layer’s extra attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Mixed Layer

    -
    -

    mixed_layer

    -
    -
    -paddle.trainer_config_helpers.layers.mixed_layer(*args, **kwargs)
    -

    Mixed Layer. A mixed layer will add all inputs together, then activate. -Each inputs is a projection or operator.

    -

    There are two styles of usages.

    -
      -
    1. When not set inputs parameter, use mixed_layer like this:
      2. -
    -
    with mixed_layer(size=256) as m:
-    m += full_matrix_projection(input=layer1)
-    m += identity_projection(input=layer2)
-
    -
    -
      -
    1. You can also set all inputs when invoke mixed_layer as follows:
      2. -
    -
    m = mixed_layer(size=256,
-                input=[full_matrix_projection(input=layer1),
-                       full_matrix_projection(input=layer2)])
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – mixed layer name. Can be referenced by other layer.
      • -
    • size (int) – layer size.
      • -
    • input – inputs layer. It is an optional parameter. If set, -then this function will just return layer’s name.
      • -
    • act (BaseActivation) – Activation Type.
      • -
    • bias_attr (ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or -something not type of ParameterAttribute. None will get a -default Bias.
      • -
    • layer_attr (ExtraLayerAttribute) – The extra layer config. Default is None.
      • -
    -
    Returns:

    MixedLayerType object can add inputs or layer name.

    -
    Return type:

    MixedLayerType

    -
    -
    - -
    -
    -

    embedding_layer

    -
    -
    -paddle.trainer_config_helpers.layers.embedding_layer(*args, **kwargs)
    -

    Define a embedding Layer.

    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – Name of this embedding layer.
      • -
    • input (LayerOutput) – The input layer for this embedding. NOTE: must be Index Data.
      • -
    • size (int) – The embedding dimension.
      • -
    • param_attr (ParameterAttribute|None) – The embedding parameter attribute. See ParameterAttribute -for details.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra layer Config. Default is None.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    scaling_projection

    -
    -
    -paddle.trainer_config_helpers.layers.scaling_projection(*args, **kwargs)
    -

    scaling_projection multiplies the input with a scalar parameter and add to -the output.

    -
    -\[out += w * in\]
    -

    The example usage is:

    -
    proj = scaling_projection(input=layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input Layer.
      • -
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • -
    -
    Returns:

    A ScalingProjection object

    -
    Return type:

    ScalingProjection

    -
    -
    - -
    -
    -

    dotmul_projection

    -
    -
    -paddle.trainer_config_helpers.layers.dotmul_projection(*args, **kwargs)
    -

    DotMulProjection with a layer as input. -It performs element-wise multiplication with weight.

    -
    -\[out.row[i] += in.row[i] .* weight\]
    -

    where \(.*\) means element-wise multiplication.

    -

    The example usage is:

    -
    proj = dotmul_projection(input=layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input layer.
      • -
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • -
    -
    Returns:

    A DotMulProjection Object.

    -
    Return type:

    DotMulProjection

    -
    -
    - -
    -
    -

    dotmul_operator

    -
    -
    -paddle.trainer_config_helpers.layers.dotmul_operator(a=None, b=None, scale=1, **kwargs)
    -

    DotMulOperator takes two inputs and performs element-wise multiplication:

    -
    -\[out.row[i] += scale * (a.row[i] .* b.row[i])\]
    -

    where \(.*\) means element-wise multiplication, and -scale is a config scalar, its default value is one.

    -

    The example usage is:

    -
    op = dotmul_operator(a=layer1, b=layer2, scale=0.5)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • a (LayerOutput) – Input layer1
      • -
    • b (LayerOutput) – Input layer2
      • -
    • scale (float) – config scalar, default value is one.
      • -
    -
    Returns:

    A DotMulOperator Object.

    -
    Return type:

    DotMulOperator

    -
    -
    - -
    -
    -

    full_matrix_projection

    -
    -
    -paddle.trainer_config_helpers.layers.full_matrix_projection(*args, **kwargs)
    -

    Full Matrix Projection. It performs full matrix multiplication.

    -
    -\[out.row[i] += in.row[i] * weight\]
    -

    There are two styles of usage.

    -
      -
    1. When used in mixed_layer like this, you can only set the input:
      2. -
    -
    with mixed_layer(size=100) as m:
-    m += full_matrix_projection(input=layer)
-
    -
    -
      -
    1. When used as an independant object like this, you must set the size:
      2. -
    -
    proj = full_matrix_projection(input=layer,
-                              size=100,
-                              param_attr=ParamAttr(name='_proj'))
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – input layer
      • -
    • size (int) – The parameter size. Means the width of parameter.
      • -
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • -
    -
    Returns:

    A FullMatrixProjection Object.

    -
    Return type:

    FullMatrixProjection

    -
    -
    - -
    -
    -

    identity_projection

    -
    -
    -paddle.trainer_config_helpers.layers.identity_projection(input, offset=None)
    -
      -
    1. IdentityProjection if offset=None. It performs:
      2. -
    -
    -\[out.row[i] += in.row[i]\]
    -

    The example usage is:

    -
    proj = identity_projection(input=layer)
-
    -
    -

    2. IdentityOffsetProjection if offset!=None. It likes IdentityProjection, -but layer size may be smaller than input size. -It select dimesions [offset, offset+layer_size) from input:

    -
    -\[out.row[i] += in.row[i + \textrm{offset}]\]
    -

    The example usage is:

    -
    proj = identity_projection(input=layer,
-                           offset=10)
-
    -
    -

    Note that both of two projections should not have any parameter.

    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input Layer.
      • -
    • offset (int) – Offset, None if use default.
      • -
    -
    Returns:

    A IdentityProjection or IdentityOffsetProjection object

    -
    Return type:

    IdentityProjection or IdentityOffsetProjection

    -
    -
    - -
    -
    -

    table_projection

    -
    -
    -paddle.trainer_config_helpers.layers.table_projection(*args, **kwargs)
    -

    Table Projection. It selects rows from parameter where row_id -is in input_ids.

    -
    -\[out.row[i] += table.row[ids[i]]\]
    -

    where \(out\) is output, \(table\) is parameter, \(ids\) is input_ids, -and \(i\) is row_id.

    -

    There are two styles of usage.

    -
      -
    1. When used in mixed_layer like this, you can only set the input:
      2. -
    -
    with mixed_layer(size=100) as m:
-    m += table_projection(input=layer)
-
    -
    -
      -
    1. When used as an independant object like this, you must set the size:
      2. -
    -
    proj = table_projection(input=layer,
-                        size=100,
-                        param_attr=ParamAttr(name='_proj'))
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input layer, which must contains id fields.
      • -
    • size (int) – The parameter size. Means the width of parameter.
      • -
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • -
    -
    Returns:

    A TableProjection Object.

    -
    Return type:

    TableProjection

    -
    -
    - -
    -
    -

    trans_full_matrix_projection

    -
    -
    -paddle.trainer_config_helpers.layers.trans_full_matrix_projection(*args, **kwargs)
    -

    Different from full_matrix_projection, this projection performs matrix -multiplication, using transpose of weight.

    -
    -\[out.row[i] += in.row[i] * w^\mathrm{T}\]
    -

    \(w^\mathrm{T}\) means transpose of weight. -The simply usage is:

    -
    proj = trans_full_matrix_projection(input=layer,
-                                    size=100,
-                                    param_attr=ParamAttr(
-                                         name='_proj',
-                                         initial_mean=0.0,
-                                         initial_std=0.01))
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – input layer
      • -
    • size (int) – The parameter size. Means the width of parameter.
      • -
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • -
    -
    Returns:

    A TransposedFullMatrixProjection Object.

    -
    Return type:

    TransposedFullMatrixProjection

    -
    -
    - -
    -
    -
    -

    Aggregate Layers

    -
    -

    pooling_layer

    -
    -
    -paddle.trainer_config_helpers.layers.pooling_layer(*args, **kwargs)
    -

    Pooling layer for sequence inputs, not used for Image.

    -

    The example usage is:

    -
    seq_pool = pooling_layer(input=layer,
-                         pooling_type=AvgPooling(),
-                         agg_level=AggregateLevel.EACH_SEQUENCE)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • agg_level (AggregateLevel) – AggregateLevel.EACH_TIMESTEP or -AggregateLevel.EACH_SEQUENCE
      • -
    • name (basestring) – layer name.
      • -
    • input (LayerOutput) – input layer name.
      • -
    • pooling_type (BasePoolingType|None) – Type of pooling, MaxPooling(default), AvgPooling, -SumPooling, SquareRootNPooling.
      • -
    • bias_attr (ParameterAttribute|None|False) – Bias parameter attribute. False if no bias.
      • -
    • layer_attr (ExtraLayerAttribute|None) – The Extra Attributes for layer, such as dropout.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    last_seq

    -
    -
    -paddle.trainer_config_helpers.layers.last_seq(*args, **kwargs)
    -

    Get Last Timestamp Activation of a sequence.

    -

    If stride > 0, this layer slides a window whose size is determined by stride, -and return the last value of the window as the output. Thus, a long sequence -will be shorten. Note that for sequence with sub-sequence, the default value -of stride is -1.

    -

    The simple usage is:

    -
    seq = last_seq(input=layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • agg_level – Aggregated level
      • -
    • name (basestring) – Layer name.
      • -
    • input (LayerOutput) – Input layer name.
      • -
    • stride (Int) – window size.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    first_seq

    -
    -
    -paddle.trainer_config_helpers.layers.first_seq(*args, **kwargs)
    -

    Get First Timestamp Activation of a sequence.

    -

    If stride > 0, this layer slides a window whose size is determined by stride, -and return the first value of the window as the output. Thus, a long sequence -will be shorten. Note that for sequence with sub-sequence, the default value -of stride is -1.

    -

    The simple usage is:

    -
    seq = first_seq(input=layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • agg_level – aggregation level
      • -
    • name (basestring) – Layer name.
      • -
    • input (LayerOutput) – Input layer name.
      • -
    • stride (Int) – window size.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    concat_layer

    -
    -
    -paddle.trainer_config_helpers.layers.concat_layer(*args, **kwargs)
    -

    Concat all input vector into one huge vector. -Inputs can be list of LayerOutput or list of projection.

    -

    The example usage is:

    -
    concat = concat_layer(input=[layer1, layer2])
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – Layer name.
      • -
    • input (list|tuple|collections.Sequence) – input layers or projections
      • -
    • act (BaseActivation) – Activation type.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    seq_concat_layer

    -
    -
    -paddle.trainer_config_helpers.layers.seq_concat_layer(*args, **kwargs)
    -

    Concat sequence a with sequence b.

    -
    -
    Inputs:
    -
      -
    • a = [a1, a2, ..., an]
      • -
    • b = [b1, b2, ..., bn]
      • -
    • Note that the length of a and b should be the same.
      • -
    -
    -
    -

    Output: [a1, b1, a2, b2, ..., an, bn]

    -

    The example usage is:

    -
    concat = seq_concat_layer(a=layer1, b=layer2)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – Layer name.
      • -
    • a (LayerOutput) – input sequence layer
      • -
    • b (LayerOutput) – input sequence layer
      • -
    • act (BaseActivation) – Activation type.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    • bias_attr (ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or -something not type of ParameterAttribute. None will get a -default Bias.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Reshaping Layers

    -
    -

    block_expand_layer

    -
    -
    -paddle.trainer_config_helpers.layers.block_expand_layer(*args, **kwargs)
    -
    -
    Expand feature map to minibatch matrix.
    -
      -
    • matrix width is: block_y * block_x * num_channels
      • -
    • matirx height is: outputH * outputW
      • -
    -
    -
    -
    -\[ \begin{align}\begin{aligned}outputH = 1 + (2 * padding_y + imgSizeH - block_y + stride_y - 1) / stride_y\\outputW = 1 + (2 * padding_x + imgSizeW - block_x + stride_x - 1) / stride_x\end{aligned}\end{align} \]
    -

    The expand method is the same with ExpandConvLayer, but saved the transposed -value. After expanding, output.sequenceStartPositions will store timeline. -The number of time steps are outputH * outputW and the dimension of each -time step is block_y * block_x * num_channels. This layer can be used after -convolution neural network, and before recurrent neural network.

    -

    The simple usage is:

    -
    block_expand = block_expand_layer(input=layer,
-                                  num_channels=128,
-                                  stride_x=1,
-                                  stride_y=1,
-                                  block_x=1,
-                                  block_x=3)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – The input layer.
      • -
    • num_channels (int|None) – The channel number of input layer.
      • -
    • block_x (int) – The width of sub block.
      • -
    • block_y (int) – The width of sub block.
      • -
    • stride_x (int) – The stride size in horizontal direction.
      • -
    • stride_y (int) – The stride size in vertical direction.
      • -
    • padding_x (int) – The padding size in horizontal direction.
      • -
    • padding_y (int) – The padding size in vertical direction.
      • -
    • name (None|basestring.) – The name of this layer, which can not specify.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    expand_layer

    -
    -
    -paddle.trainer_config_helpers.layers.expand_layer(*args, **kwargs)
    -

    A layer for “Expand Dense data or (sequence data where the length of each -sequence is one) to sequence data.”

    -

    The example usage is:

    -
    expand = expand_layer(input=layer1,
-                      expand_as=layer2,
-                      expand_level=ExpandLevel.FROM_TIMESTEP)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input layer
      • -
    • expand_as (LayerOutput) – Expand as this layer’s sequence info.
      • -
    • name (basestring) – Layer name.
      • -
    • bias_attr (ParameterAttribute|None|False) – Bias attribute. None means default bias. False means no -bias.
      • -
    • expand_level (ExpandLevel) – whether input layer is timestep(default) or sequence.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    repeat_layer

    -
    -
    -paddle.trainer_config_helpers.layers.repeat_layer(*args, **kwargs)
    -

    A layer for repeating the input for num_repeats times. This is equivalent -to apply concat_layer() with num_repeats same input.

    -
    -\[y = [x, x, \cdots, x]\]
    -

    The example usage is:

    -
    expand = repeat_layer(input=layer, num_repeats=4)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input layer
      • -
    • num_repeats (int) – Repeat the input so many times
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    rotate_layer

    -
    -
    -paddle.trainer_config_helpers.layers.rotate_layer(*args, **kwargs)
    -

    A layer for rotating 90 degrees (clock-wise) for each feature channel, -usually used when the input sample is some image or feature map.

    -
    -\[y(j,i,:) = x(M-i-1,j,:)\]
    -

    where \(x\) is (M x N x C) input, and \(y\) is (N x M x C) output.

    -

    The example usage is:

    -
    rot = rotate_layer(input=layer,
-                   height=100,
-                   width=100)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input layer.
      • -
    • height (int) – The height of the sample matrix
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    seq_reshape_layer

    -
    -
    -paddle.trainer_config_helpers.layers.seq_reshape_layer(*args, **kwargs)
    -

    A layer for reshaping the sequence. Assume the input sequence has T instances, -the dimension of each instance is M, and the input reshape_size is N, then the -output sequence has T*M/N instances, the dimension of each instance is N.

    -

    Note that T*M/N must be an integer.

    -

    The example usage is:

    -
    reshape = seq_reshape_layer(input=layer, reshape_size=4)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input layer.
      • -
    • reshape_size (int) – the size of reshaped sequence.
      • -
    • name (basestring) – Layer name.
      • -
    • act (BaseActivation) – Activation type.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    • bias_attr (ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or -something not type of ParameterAttribute. None will get a -default Bias.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Math Layers

    -
    -

    addto_layer

    -
    -
    -paddle.trainer_config_helpers.layers.addto_layer(*args, **kwargs)
    -

    AddtoLayer.

    -
    -\[y = f(\sum_{i} x_i + b)\]
    -

    where \(y\) is output, \(x\) is input, \(b\) is bias, -and \(f\) is activation function.

    -

    The example usage is:

    -
    addto = addto_layer(input=[layer1, layer2],
-                    act=ReluActivation(),
-                    bias_attr=False)
-
    -
    -

    This layer just simply add all input layers together, then activate the sum -inputs. Each input of this layer should be the same size, which is also the -output size of this layer.

    -

    There is no weight matrix for each input, because it just a simple add -operation. If you want a complicated operation before add, please use -mixed_layer.

    -

    It is a very good way to set dropout outside the layers. Since not all -PaddlePaddle layer support dropout, you can add an add_to layer, set -dropout here. -Please refer to dropout_layer for details.

    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – Layer name.
      • -
    • input (LayerOutput|list|tuple) – Input layers. It could be a LayerOutput or list/tuple of -LayerOutput.
      • -
    • act (BaseActivation) – Activation Type, default is tanh.
      • -
    • bias_attr (ParameterAttribute|bool) – Bias attribute. If False, means no bias. None is default -bias.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    linear_comb_layer

    -
    -
    -paddle.trainer_config_helpers.layers.linear_comb_layer(*args, **kwargs)
    -
    -
    A layer for weighted sum of vectors takes two inputs.
    -
      -
    • -
      Input: size of weights is M
      -
      size of vectors is M*N
      -
      -
      • -
    • Output: a vector of size=N
      • -
    -
    -
    -
    -\[z(i) = \sum_{j=0}^{M-1} x(j) y(i+Nj)\]
    -

    where \(0 \le i \le N-1\)

    -

    Or in the matrix notation:

    -
    -\[z = x^\mathrm{T} Y\]
    -
    -
    In this formular:
    -
      -
    • \(x\): weights
      • -
    • \(y\): vectors.
      • -
    • \(z\): the output.
      • -
    -
    -
    -

    Note that the above computation is for one sample. Multiple samples are -processed in one batch.

    -

    The simple usage is:

    -
    linear_comb = linear_comb_layer(weights=weight, vectors=vectors,
-                                size=elem_dim)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • weights (LayerOutput) – The weight layer.
      • -
    • vectors (LayerOutput) – The vector layer.
      • -
    • size (int) – the dimension of this layer.
      • -
    • name (basestring) – The Layer Name.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    interpolation_layer

    -
    -
    -paddle.trainer_config_helpers.layers.interpolation_layer(*args, **kwargs)
    -

    This layer is for linear interpolation with two inputs, -which is used in NEURAL TURING MACHINE.

    -
    -\[y.row[i] = w[i] * x_1.row[i] + (1 - w[i]) * x_2.row[i]\]
    -

    where \(x_1\) and \(x_2\) are two (batchSize x dataDim) inputs, -\(w\) is (batchSize x 1) weight vector, and \(y\) is -(batchSize x dataDim) output.

    -

    The example usage is:

    -
    interpolation = interpolation_layer(input=[layer1, layer2], weight=layer3)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (list|tuple) – Input layer.
      • -
    • weight (LayerOutput) – Weight layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    bilinear_interp_layer

    -
    -
    -paddle.trainer_config_helpers.layers.bilinear_interp_layer(*args, **kwargs)
    -

    This layer is to implement bilinear interpolation on conv layer output.

    -

    Please refer to Wikipedia: https://en.wikipedia.org/wiki/Bilinear_interpolation

    -

    The simple usage is:

    -
    bilinear = bilinear_interp_layer(input=layer1, out_size_x=64, out_size_y=64)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput.) – A input layer.
      • -
    • out_size_x (int|None) – bilinear interpolation output width.
      • -
    • out_size_y (int|None) – bilinear interpolation output height.
      • -
    • name (None|basestring) – The layer’s name, which cna not be specified.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    power_layer

    -
    -
    -paddle.trainer_config_helpers.layers.power_layer(*args, **kwargs)
    -

    This layer applies a power function to a vector element-wise, -which is used in NEURAL TURING MACHINE.

    -
    -\[y = x^w\]
    -

    where \(x\) is a input vector, \(w\) is scalar weight, -and \(y\) is a output vector.

    -

    The example usage is:

    -
    power = power_layer(input=layer1, weight=layer2)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input layer.
      • -
    • weight (LayerOutput) – Weight layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    scaling_layer

    -
    -
    -paddle.trainer_config_helpers.layers.scaling_layer(*args, **kwargs)
    -

    A layer for multiplying input vector by weight scalar.

    -
    -\[y = w x\]
    -

    where \(x\) is size=dataDim input, \(w\) is size=1 weight, -and \(y\) is size=dataDim output.

    -

    Note that the above computation is for one sample. Multiple samples are -processed in one batch.

    -

    The example usage is:

    -
    scale = scaling_layer(input=layer1, weight=layer2)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input layer.
      • -
    • weight (LayerOutput) – Weight layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    slope_intercept_layer

    -
    -
    -paddle.trainer_config_helpers.layers.slope_intercept_layer(*args, **kwargs)
    -

    This layer for applying a slope and an intercept to the input -element-wise. There is no activation and weight.

    -
    -\[y = slope * x + intercept\]
    -

    The simple usage is:

    -
    scale = slope_intercept_layer(input=input, slope=-1.0, intercept=1.0)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – The input layer.
      • -
    • name (basestring) – The Layer Name.
      • -
    • slope (float.) – the scale factor.
      • -
    • intercept (float.) – the offset.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    tensor_layer

    -
    -
    -paddle.trainer_config_helpers.layers.tensor_layer(*args, **kwargs)
    -

    This layer performs tensor operation for two input. -For example, each sample:

    -
    -\[y_{i} = a * W_{i} * {b^\mathrm{T}}, i=0,1,...,K-1\]
    -
    -
    In this formular:
    -
      -
    • \(a\): the first input contains M elements.
      • -
    • \(b\): the second input contains N elements.
      • -
    • \(y_{i}\): the i-th element of y.
      • -
    • \(W_{i}\): the i-th learned weight, shape if [M, N]
      • -
    • \(b^\mathrm{T}\): the transpose of \(b_{2}\).
      • -
    -
    -
    -

    The simple usage is:

    -
    tensor = tensor_layer(a=layer1, b=layer2, size=1000)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – layer name
      • -
    • a (LayerOutput) – Input layer a.
      • -
    • b (LayerOutput) – input layer b.
      • -
    • size (int.) – the layer dimension.
      • -
    • act (BaseActivation) – Activation Type. Default is tanh.
      • -
    • param_attr (ParameterAttribute) – The Parameter Attribute.
      • -
    • bias_attr (ParameterAttribute|None|Any) – The Bias Attribute. If no bias, then pass False or -something not type of ParameterAttribute. None will get a -default Bias.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    cos_sim

    -
    -
    -paddle.trainer_config_helpers.layers.cos_sim(*args, **kwargs)
    -

    Cosine Similarity Layer. The cosine similarity equation is here.

    -
    -\[similarity = cos(\theta) = {\mathbf{a} \cdot \mathbf{b} -\over \|\mathbf{a}\| \|\mathbf{b}\|}\]
    -

    The size of a is M, size of b is M*N, -Similarity will be calculated N times by step M. The output size is -N. The scale will be multiplied to similarity.

    -

    Note that the above computation is for one sample. Multiple samples are -processed in one batch.

    -

    The example usage is:

    -
    cos = cos_sim(a=layer1, b=layer2, size=3)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – layer name
      • -
    • a (LayerOutput) – input layer a
      • -
    • b (LayerOutput) – input layer b
      • -
    • scale (float) – scale for cosine value. default is 5.
      • -
    • size (int) – layer size. NOTE size_a * size should equal size_b.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    trans_layer

    -
    -
    -paddle.trainer_config_helpers.layers.trans_layer(*args, **kwargs)
    -

    A layer for transposing a minibatch matrix.

    -
    -\[y = x^\mathrm{T}\]
    -

    where \(x\) is (M x N) input, and \(y\) is (N x M) output.

    -

    The example usage is:

    -
    trans = trans_layer(input=layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Sampling Layers

    -
    -

    maxid_layer

    -
    -
    -paddle.trainer_config_helpers.layers.maxid_layer(*args, **kwargs)
    -

    A layer for finding the id which has the maximal value for each sample. -The result is stored in output.ids.

    -

    The example usage is:

    -
    maxid = maxid_layer(input=layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Input layer name.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    sampling_id_layer

    -
    -
    -paddle.trainer_config_helpers.layers.sampling_id_layer(*args, **kwargs)
    -

    A layer for sampling id from multinomial distribution from the input layer. -Sampling one id for one sample.

    -

    The simple usage is:

    -
    samping_id = sampling_id_layer(input=input)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – The input layer.
      • -
    • name (basestring) – The Layer Name.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Slicing and Joining Layers

    -
    -

    pad_layer

    -
    -
    -paddle.trainer_config_helpers.layers.pad_layer(*args, **kwargs)
    -

    This operation pads zeros to the input data according to pad_c,pad_h -and pad_w. pad_c, pad_h, pad_w specifies the which dimension and size -of padding. And the input data shape is NCHW.

    -

    For example, pad_c=[2,3] means padding 2 zeros before the -input data and 3 zeros after the input data in channel dimension. -pad_h means padding zeros in height dimension. pad_w means padding zeros -in width dimension.

    -

    For example,

    -
    input(2,2,2,3)  = [
-                    [ [[1,2,3], [3,4,5]],
-                      [[2,3,5], [1,6,7]] ],
-                    [ [[4,3,1], [1,8,7]],
-                      [[3,8,9], [2,3,5]] ]
-                  ]
-
-pad_c=[1,1], pad_h=[0,0], pad_w=[0,0]
-
-output(2,4,2,3) = [
-                    [ [[0,0,0], [0,0,0]],
-                      [[1,2,3], [3,4,5]],
-                      [[2,3,5], [1,6,7]],
-                      [[0,0,0], [0,0,0]] ],
-                    [ [[0,0,0], [0,0,0]],
-                      [[4,3,1], [1,8,7]],
-                      [[3,8,9], [2,3,5]],
-                      [[0,0,0], [0,0,0]] ]
-                  ]
-
    -
    -

    The simply usage is:

    -
    pad = pad_layer(input=ipt,
-                pad_c=[4,4],
-                pad_h=[0,0],
-                pad_w=[2,2])
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – layer’s input.
      • -
    • pad_c (list|None) – padding size in channel dimension.
      • -
    • pad_h (list|None) – padding size in height dimension.
      • -
    • pad_w (list|None) – padding size in width dimension.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    • name (basestring) – layer name.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Cost Layers

    -
    -

    cross_entropy

    -
    -
    -paddle.trainer_config_helpers.layers.cross_entropy(*args, **kwargs)
    -

    A loss layer for multi class entropy.

    -
    cost = cross_entropy(input=input_layer,
-                     label=label_layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput.) – The first input layer.
      • -
    • label – The input label.
      • -
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • -
    • coeff (float.) – The cost is multiplied with coeff. -The coefficient affects the gradient in the backward.
      • -
    • weight (LayerOutout) – The cost of each sample is multiplied with each weight. -The weight should be a layer with size=1. Note that gradient -will not be calculated for weight.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput.

    -
    -
    - -
    -
    -

    cross_entropy_with_selfnorm

    -
    -
    -paddle.trainer_config_helpers.layers.cross_entropy_with_selfnorm(*args, **kwargs)
    -

    A loss layer for multi class entropy with selfnorm. -Input should be a vector of positive numbers, without normalization.

    -
    cost = cross_entropy_with_selfnorm(input=input_layer,
-                                   label=label_layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput.) – The first input layer.
      • -
    • label – The input label.
      • -
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • -
    • coeff (float.) – The coefficient affects the gradient in the backward.
      • -
    • softmax_selfnorm_alpha (float.) – The scale factor affects the cost.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput.

    -
    -
    - -
    -
    -

    multi_binary_label_cross_entropy

    -
    -
    -paddle.trainer_config_helpers.layers.multi_binary_label_cross_entropy(*args, **kwargs)
    -

    A loss layer for multi binary label cross entropy.

    -
    cost = multi_binary_label_cross_entropy(input=input_layer,
-                                        label=label_layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – The first input layer.
      • -
    • label – The input label.
      • -
    • type (basestring) – The type of cost.
      • -
    • name (None|basestring) – The name of this layers. It is not necessary.
      • -
    • coeff (float) – The coefficient affects the gradient in the backward.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    mse_cost

    -
    -
    -paddle.trainer_config_helpers.layers.mse_cost(*args, **kwargs)
    -
    -

    mean squared error cost:

    -
    -\[\]
    -
    -

    rac{1}{N}sum_{i=1}^N(t_i-y_i)^2

    -
    -
    --- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    param name:layer name.
    type name:basestring
    param input:Network prediction.
    type input:LayerOutput
    param label:Data label.
    type label:LayerOutput
    param weight:The weight affects the cost, namely the scale of cost. -It is an optional argument.
    type weight:LayerOutput
    param layer_attr:
     layer’s extra attribute.
    type layer_attr:
     ExtraLayerAttribute
    return:LayerOutput object.
    rtype:LayerOutput
    -
    -
    - -
    -
    -

    huber_cost

    -
    -
    -paddle.trainer_config_helpers.layers.huber_cost(*args, **kwargs)
    -

    A loss layer for huber loss.

    -
    cost = huber_cost(input=input_layer,
-                  label=label_layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput.) – The first input layer.
      • -
    • label – The input label.
      • -
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • -
    • coeff (float.) – The coefficient affects the gradient in the backward.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput.

    -
    -
    - -
    -
    -

    lambda_cost

    -
    -
    -paddle.trainer_config_helpers.layers.lambda_cost(*args, **kwargs)
    -

    lambdaCost for lambdaRank LTR approach.

    -

    The simple usage:

    -
    cost = lambda_cost(input=input,
-                   score=score,
-                   NDCG_num=8,
-                   max_sort_size=-1)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – Samples of the same query should be loaded as sequence.
      • -
    • score – The 2nd input. Score of each sample.
      • -
    • NDCG_num (int) – The size of NDCG (Normalized Discounted Cumulative Gain), -e.g., 5 for NDCG@5. It must be less than for equal to the -minimum size of lists.
      • -
    • max_sort_size (int) – The size of partial sorting in calculating gradient. -If max_sort_size = -1, then for each list, the -algorithm will sort the entire list to get gradient. -In other cases, max_sort_size must be greater than or -equal to NDCG_num. And if max_sort_size is greater -than the size of a list, the algorithm will sort the -entire list of get gradient.
      • -
    • name (None|basestring) – The name of this layers. It is not necessary.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    rank_cost

    -
    -
    -paddle.trainer_config_helpers.layers.rank_cost(*args, **kwargs)
    -

    A cost Layer for learning to rank using gradient descent. Details can refer -to papers. -This layer contains at least three inputs. The weight is an optional -argument, which affects the cost.

    -
    -\[ \begin{align}\begin{aligned}C_{i,j} & = -\tilde{P_{ij}} * o_{i,j} + log(1 + e^{o_{i,j}})\\o_{i,j} & = o_i - o_j\\\tilde{P_{i,j}} & = \{0, 0.5, 1\} \ or \ \{0, 1\}\end{aligned}\end{align} \]
    -
    -
    In this formula:
    -
      -
    • \(C_{i,j}\) is the cross entropy cost.
      • -
    • \(\tilde{P_{i,j}}\) is the label. 1 means positive order -and 0 means reverse order.
      • -
    • \(o_i\) and \(o_j\): the left output and right output. -Their dimension is one.
      • -
    -
    -
    -

    The simple usage:

    -
    cost = rank_cost(left=out_left,
-                 right=out_right,
-                 label=label)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • left (LayerOutput) – The first input, the size of this layer is 1.
      • -
    • right (LayerOutput) – The right input, the size of this layer is 1.
      • -
    • label (LayerOutput) – Label is 1 or 0, means positive order and reverse order.
      • -
    • weight (LayerOutput) – The weight affects the cost, namely the scale of cost. -It is an optional argument.
      • -
    • name (None|basestring) – The name of this layers. It is not necessary.
      • -
    • coeff (float) – The coefficient affects the gradient in the backward.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    sum_cost

    -
    -
    -paddle.trainer_config_helpers.layers.sum_cost(*args, **kwargs)
    -

    A loss layer which calculate the sum of the input as loss

    -
    cost = sum_cost(input=input_layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput.) – The first input layer.
      • -
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput.

    -
    -
    - -
    -
    -

    crf_layer

    -
    -
    -paddle.trainer_config_helpers.layers.crf_layer(*args, **kwargs)
    -

    A layer for calculating the cost of sequential conditional random -field model.

    -

    The simple usage:

    -
    crf = crf_layer(input=input,
-                label=label,
-                size=label_dim)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – The first input layer is the feature.
      • -
    • label (LayerOutput) – The second input layer is label.
      • -
    • size (int) – The category number.
      • -
    • weight (LayerOutput) – The third layer is “weight” of each sample, which is an -optional argument.
      • -
    • param_attr (ParameterAttribute) – Parameter attribute. None means default attribute
      • -
    • name (None|basestring) – The name of this layers. It is not necessary.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    crf_decoding_layer

    -
    -
    -paddle.trainer_config_helpers.layers.crf_decoding_layer(*args, **kwargs)
    -

    A layer for calculating the decoding sequence of sequential conditional -random field model. The decoding sequence is stored in output.ids. -If a second input is provided, it is treated as the ground-truth label, and -this layer will also calculate error. output.value[i] is 1 for incorrect -decoding or 0 for correct decoding.

    -

    The simple usage:

    -
    crf_decoding = crf_decoding_layer(input=input,
-                                  size=label_dim)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – The first input layer.
      • -
    • size (int) – size of this layer.
      • -
    • label (LayerOutput or None) – None or ground-truth label.
      • -
    • param_attr (ParameterAttribute) – Parameter attribute. None means default attribute
      • -
    • name (None|basestring) – The name of this layers. It is not necessary.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    ctc_layer

    -
    -
    -paddle.trainer_config_helpers.layers.ctc_layer(*args, **kwargs)
    -

    Connectionist Temporal Classification (CTC) is designed for temporal -classication task. That is, for sequence labeling problems where the -alignment between the inputs and the target labels is unknown.

    -

    More details can be found by referring to Connectionist Temporal -Classification: Labelling Unsegmented Sequence Data with Recurrent -Neural Networks

    -
    -

    Note

    -

    Considering the ‘blank’ label needed by CTC, you need to use -(num_classes + 1) as the input size. num_classes is the category number. -And the ‘blank’ is the last category index. So the size of ‘input’ layer, such as -fc_layer with softmax activation, should be num_classes + 1. The size of ctc_layer -should also be num_classes + 1.

    -
    -

    The simple usage:

    -
    ctc = ctc_layer(input=input,
-                label=label,
-                size=9055,
-                norm_by_times=True)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – The input layer.
      • -
    • label (LayerOutput) – The data layer of label with variable length.
      • -
    • size (int) – category numbers + 1.
      • -
    • name (basestring|None) – The name of this layer
      • -
    • norm_by_times (bool) – Whether to normalization by times. False by default.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    warp_ctc_layer

    -
    -
    -paddle.trainer_config_helpers.layers.warp_ctc_layer(*args, **kwargs)
    -

    A layer intergrating the open-source warp-ctc -<https://github.com/baidu-research/warp-ctc> library, which is used in -Deep Speech 2: End-toEnd Speech Recognition in English and Mandarin -<https://arxiv.org/pdf/1512.02595v1.pdf>, to compute Connectionist Temporal -Classification (CTC) loss.

    -

    More details of CTC can be found by referring to Connectionist Temporal -Classification: Labelling Unsegmented Sequence Data with Recurrent -Neural Networks

    -
    -

    Note

    -
      -
    • Let num_classes represent the category number. Considering the ‘blank’ -label needed by CTC, you need to use (num_classes + 1) as the input -size. Thus, the size of both warp_ctc_layer and ‘input’ layer should -be set to num_classes + 1.
      • -
    • You can set ‘blank’ to any value ranged in [0, num_classes], which -should be consistent as that used in your labels.
      • -
    • As a native ‘softmax’ activation is interated to the warp-ctc library, -‘linear’ activation is expected instead in the ‘input’ layer.
      • -
    -
    -

    The simple usage:

    -
    ctc = warp_ctc_layer(input=input,
-                     label=label,
-                     size=1001,
-                     blank=1000,
-                     norm_by_times=False)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – The input layer.
      • -
    • label (LayerOutput) – The data layer of label with variable length.
      • -
    • size (int) – category numbers + 1.
      • -
    • name (basestring|None) – The name of this layer, which can not specify.
      • -
    • blank (int) – the ‘blank’ label used in ctc
      • -
    • norm_by_times (bool) – Whether to normalization by times. False by default.
      • -
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    nce_layer

    -
    -
    -paddle.trainer_config_helpers.layers.nce_layer(*args, **kwargs)
    -

    Noise-contrastive estimation. -Implements the method in the following paper: -A fast and simple algorithm for training neural probabilistic language models.

    -

    The example usage is:

    -
    cost = nce_layer(input=layer1, label=layer2, weight=layer3,
-                 num_classes=3, neg_distribution=[0.1,0.3,0.6])
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – layer name
      • -
    • input (LayerOutput|list|tuple|collections.Sequence) – input layers. It could be a LayerOutput of list/tuple of LayerOutput.
      • -
    • label (LayerOutput) – label layer
      • -
    • weight (LayerOutput) – weight layer, can be None(default)
      • -
    • num_classes (int) – number of classes.
      • -
    • act (BaseActivation) – Activation, default is Sigmoid.
      • -
    • num_neg_samples (int) – number of negative samples. Default is 10.
      • -
    • neg_distribution (list|tuple|collections.Sequence|None) – The distribution for generating the random negative labels. -A uniform distribution will be used if not provided. -If not None, its length must be equal to num_classes.
      • -
    • bias_attr (ParameterAttribute|None|False) – Bias parameter attribute. True if no bias.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    layer name.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    hsigmoid

    -
    -
    -paddle.trainer_config_helpers.layers.hsigmoid(*args, **kwargs)
    -

    Organize the classes into a binary tree. At each node, a sigmoid function -is used to calculate the probability of belonging to the right branch. -This idea is from “F. Morin, Y. Bengio (AISTATS 05): -Hierarchical Probabilistic Neural Network Language Model.”

    -

    The example usage is:

    -
    cost = hsigmoid(input=[layer1, layer2],
-                label=data_layer)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput|list|tuple) – Input layers. It could be a LayerOutput or list/tuple of -LayerOutput.
      • -
    • label (LayerOutput) – Label layer.
      • -
    • num_classes (int|None) – number of classes.
      • -
    • name (basestring) – layer name
      • -
    • bias_attr (ParameterAttribute|False) – Bias attribute. None means default bias. -False means no bias.
      • -
    • param_attr (ParameterAttribute|None) – Parameter Attribute. None means default parameter.
      • -
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Check Layer

    -
    -

    eos_layer

    -
    -
    -paddle.trainer_config_helpers.layers.eos_layer(*args, **kwargs)
    -

    A layer for checking EOS for each sample: -- output_id = (input_id == conf.eos_id)

    -

    The result is stored in output_.ids. -It is used by recurrent layer group.

    -

    The example usage is:

    -
    eos = eos_layer(input=layer, eos_id=id)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – Layer name.
      • -
    • input (LayerOutput) – Input layer name.
      • -
    • eos_id (int) – end id of sequence
      • -
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • -
    -
    Returns:

    LayerOutput object.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    - - -
    -
    -
    - - -
    - -
    -

    - © Copyright 2016, PaddlePaddle developers. - -

    -
    - Built with Sphinx using a theme provided by Read the Docs. - -
    - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc/api/v1/trainer_config_helpers/networks.html b/doc/api/v1/trainer_config_helpers/networks.html deleted file mode 100644 index 9769b3b4bf8364610f1a032536c14b7135e9a152..0000000000000000000000000000000000000000 --- a/doc/api/v1/trainer_config_helpers/networks.html +++ /dev/null @@ -1,949 +0,0 @@ - - - - - - - - - - - Networks — PaddlePaddle documentation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    -
      - -
    • Networks
      • -
    -
    - -
    -
    -
    -
    - -
    -

    Networks

    -

    The networks module contains pieces of neural network that combine multiple layers.

    -
    -

    NLP

    -
    -

    sequence_conv_pool

    -
    -
    -paddle.trainer_config_helpers.networks.sequence_conv_pool(*args, **kwargs)
    -

    Text convolution pooling layers helper.

    -

    Text input => Context Projection => FC Layer => Pooling => Output.

    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – name of output layer(pooling layer name)
      • -
    • input (LayerOutput) – name of input layer
      • -
    • context_len (int) – context projection length. See -context_projection’s document.
      • -
    • hidden_size (int) – FC Layer size.
      • -
    • context_start (int or None) – context projection length. See -context_projection’s context_start.
      • -
    • pool_type (BasePoolingType.) – pooling layer type. See pooling_layer’s document.
      • -
    • context_proj_layer_name (basestring) – context projection layer name. -None if user don’t care.
      • -
    • context_proj_param_attr (ParameterAttribute or None.) – context projection parameter attribute. -None if user don’t care.
      • -
    • fc_layer_name (basestring) – fc layer name. None if user don’t care.
      • -
    • fc_param_attr (ParameterAttribute or None) – fc layer parameter attribute. None if user don’t care.
      • -
    • fc_bias_attr (ParameterAttribute or None) – fc bias parameter attribute. False if no bias, -None if user don’t care.
      • -
    • fc_act (BaseActivation) – fc layer activation type. None means tanh
      • -
    • pool_bias_attr (ParameterAttribute or None.) – pooling layer bias attr. None if don’t care. -False if no bias.
      • -
    • fc_attr (ExtraLayerAttribute) – fc layer extra attribute.
      • -
    • context_attr (ExtraLayerAttribute) – context projection layer extra attribute.
      • -
    • pool_attr (ExtraLayerAttribute) – pooling layer extra attribute.
      • -
    -
    Returns:

    output layer name.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    text_conv_pool

    -
    -
    -paddle.trainer_config_helpers.networks.text_conv_pool(*args, **kwargs)
    -

    Text convolution pooling layers helper.

    -

    Text input => Context Projection => FC Layer => Pooling => Output.

    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – name of output layer(pooling layer name)
      • -
    • input (LayerOutput) – name of input layer
      • -
    • context_len (int) – context projection length. See -context_projection’s document.
      • -
    • hidden_size (int) – FC Layer size.
      • -
    • context_start (int or None) – context projection length. See -context_projection’s context_start.
      • -
    • pool_type (BasePoolingType.) – pooling layer type. See pooling_layer’s document.
      • -
    • context_proj_layer_name (basestring) – context projection layer name. -None if user don’t care.
      • -
    • context_proj_param_attr (ParameterAttribute or None.) – context projection parameter attribute. -None if user don’t care.
      • -
    • fc_layer_name (basestring) – fc layer name. None if user don’t care.
      • -
    • fc_param_attr (ParameterAttribute or None) – fc layer parameter attribute. None if user don’t care.
      • -
    • fc_bias_attr (ParameterAttribute or None) – fc bias parameter attribute. False if no bias, -None if user don’t care.
      • -
    • fc_act (BaseActivation) – fc layer activation type. None means tanh
      • -
    • pool_bias_attr (ParameterAttribute or None.) – pooling layer bias attr. None if don’t care. -False if no bias.
      • -
    • fc_attr (ExtraLayerAttribute) – fc layer extra attribute.
      • -
    • context_attr (ExtraLayerAttribute) – context projection layer extra attribute.
      • -
    • pool_attr (ExtraLayerAttribute) – pooling layer extra attribute.
      • -
    -
    Returns:

    output layer name.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    Images

    -
    -

    img_conv_bn_pool

    -
    -
    -paddle.trainer_config_helpers.networks.img_conv_bn_pool(*args, **kwargs)
    -

    Convolution, batch normalization, pooling group.

    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – group name
      • -
    • input (LayerOutput) – layer’s input
      • -
    • filter_size (int) – see img_conv_layer’s document
      • -
    • num_filters (int) – see img_conv_layer’s document
      • -
    • pool_size (int) – see img_pool_layer’s document.
      • -
    • pool_type (BasePoolingType) – see img_pool_layer’s document.
      • -
    • act (BaseActivation) – see batch_norm_layer’s document.
      • -
    • groups (int) – see img_conv_layer’s document
      • -
    • conv_stride (int) – see img_conv_layer’s document.
      • -
    • conv_padding (int) – see img_conv_layer’s document.
      • -
    • conv_bias_attr (ParameterAttribute) – see img_conv_layer’s document.
      • -
    • num_channel (int) – see img_conv_layer’s document.
      • -
    • conv_param_attr (ParameterAttribute) – see img_conv_layer’s document.
      • -
    • shared_bias (bool) – see img_conv_layer’s document.
      • -
    • conv_layer_attr (ExtraLayerOutput) – see img_conv_layer’s document.
      • -
    • bn_param_attr (ParameterAttribute.) – see batch_norm_layer’s document.
      • -
    • bn_bias_attr – see batch_norm_layer’s document.
      • -
    • bn_layer_attr – ParameterAttribute.
      • -
    • pool_stride (int) – see img_pool_layer’s document.
      • -
    • pool_padding (int) – see img_pool_layer’s document.
      • -
    • pool_layer_attr (ExtraLayerAttribute) – see img_pool_layer’s document.
      • -
    -
    Returns:

    Layer groups output

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    img_conv_group

    -
    -
    -paddle.trainer_config_helpers.networks.img_conv_group(*args, **kwargs)
    -

    Image Convolution Group, Used for vgg net.

    -

    TODO(yuyang18): Complete docs

    - --- - - - - - -
    Parameters:
      -
    • conv_batchnorm_drop_rate –
      • -
    • input –
      • -
    • conv_num_filter –
      • -
    • pool_size –
      • -
    • num_channels –
      • -
    • conv_padding –
      • -
    • conv_filter_size –
      • -
    • conv_act –
      • -
    • conv_with_batchnorm –
      • -
    • pool_stride –
      • -
    • pool_type –
      • -
    -
    Returns:

    -
    -
    - -
    -
    -

    simple_img_conv_pool

    -
    -
    -paddle.trainer_config_helpers.networks.simple_img_conv_pool(*args, **kwargs)
    -

    Simple image convolution and pooling group.

    -

    Input => conv => pooling

    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – group name
      • -
    • input (LayerOutput) – input layer name.
      • -
    • filter_size (int) – see img_conv_layer for details
      • -
    • num_filters (int) – see img_conv_layer for details
      • -
    • pool_size (int) – see img_pool_layer for details
      • -
    • pool_type (BasePoolingType) – see img_pool_layer for details
      • -
    • act (BaseActivation) – see img_conv_layer for details
      • -
    • groups (int) – see img_conv_layer for details
      • -
    • conv_stride (int) – see img_conv_layer for details
      • -
    • conv_padding (int) – see img_conv_layer for details
      • -
    • bias_attr (ParameterAttribute) – see img_conv_layer for details
      • -
    • num_channel (int) – see img_conv_layer for details
      • -
    • param_attr (ParameterAttribute) – see img_conv_layer for details
      • -
    • shared_bias (bool) – see img_conv_layer for details
      • -
    • conv_layer_attr (ExtraLayerAttribute) – see img_conv_layer for details
      • -
    • pool_stride (int) – see img_pool_layer for details
      • -
    • pool_padding (int) – see img_pool_layer for details
      • -
    • pool_layer_attr (ExtraLayerAttribute) – see img_pool_layer for details
      • -
    -
    Returns:

    Layer’s output

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    vgg_16_network

    -
    -
    -paddle.trainer_config_helpers.networks.vgg_16_network(input_image, num_channels, num_classes=1000)
    -

    Same model from https://gist.github.com/ksimonyan/211839e770f7b538e2d8

    - --- - - - - - -
    Parameters:
      -
    • num_classes –
      • -
    • input_image (LayerOutput) –
      • -
    • num_channels (int) –
      • -
    -
    Returns:

    -
    -
    - -
    -
    -
    -

    Recurrent

    -
    -

    LSTM

    -
    -

    lstmemory_unit

    -
    -
    -paddle.trainer_config_helpers.networks.lstmemory_unit(*args, **kwargs)
    -

    Define calculations that a LSTM unit performs in a single time step. -This function itself is not a recurrent layer, so that it can not be -directly applied to sequence input. This function is always used in -recurrent_group (see layers.py for more details) to implement attention -mechanism.

    -

    Please refer to Generating Sequences With Recurrent Neural Networks -for more details about LSTM. The link goes as follows: -.. _Link: https://arxiv.org/abs/1308.0850

    -
    -\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    -

    The example usage is:

    -
    lstm_step = lstmemory_unit(input=[layer1],
-                           size=256,
-                           act=TanhActivation(),
-                           gate_act=SigmoidActivation(),
-                           state_act=TanhActivation())
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – input layer name.
      • -
    • name (basestring) – lstmemory unit name.
      • -
    • size (int) – lstmemory unit size.
      • -
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • -
    • act (BaseActivation) – lstm final activiation type
      • -
    • gate_act (BaseActivation) – lstm gate activiation type
      • -
    • state_act (BaseActivation) – lstm state activiation type.
      • -
    • mixed_bias_attr (ParameterAttribute|False) – bias parameter attribute of mixed layer. -False means no bias, None means default bias.
      • -
    • lstm_bias_attr (ParameterAttribute|False) – bias parameter attribute of lstm layer. -False means no bias, None means default bias.
      • -
    • mixed_layer_attr (ExtraLayerAttribute) – mixed layer’s extra attribute.
      • -
    • lstm_layer_attr (ExtraLayerAttribute) – lstm layer’s extra attribute.
      • -
    • get_output_layer_attr (ExtraLayerAttribute) – get output layer’s extra attribute.
      • -
    -
    Returns:

    lstmemory unit name.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    lstmemory_group

    -
    -
    -paddle.trainer_config_helpers.networks.lstmemory_group(*args, **kwargs)
    -

    lstm_group is a recurrent layer group version of Long Short Term Memory. It -does exactly the same calculation as the lstmemory layer (see lstmemory in -layers.py for the maths) does. A promising benefit is that LSTM memory -cell states, or hidden states in every time step are accessible to the -user. This is especially useful in attention model. If you do not need to -access the internal states of the lstm, but merely use its outputs, -it is recommended to use the lstmemory, which is relatively faster than -lstmemory_group.

    -

    NOTE: In PaddlePaddle’s implementation, the following input-to-hidden -multiplications: -\(W_{xi}x_{t}\) , \(W_{xf}x_{t}\), -\(W_{xc}x_t\), \(W_{xo}x_{t}\) are not done in lstmemory_unit to -speed up the calculations. Consequently, an additional mixed_layer with -full_matrix_projection must be included before lstmemory_unit is called.

    -

    The example usage is:

    -
    lstm_step = lstmemory_group(input=[layer1],
-                            size=256,
-                            act=TanhActivation(),
-                            gate_act=SigmoidActivation(),
-                            state_act=TanhActivation())
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – input layer name.
      • -
    • name (basestring) – lstmemory group name.
      • -
    • size (int) – lstmemory group size.
      • -
    • reverse (bool) – is lstm reversed
      • -
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • -
    • act (BaseActivation) – lstm final activiation type
      • -
    • gate_act (BaseActivation) – lstm gate activiation type
      • -
    • state_act (BaseActivation) – lstm state activiation type.
      • -
    • mixed_bias_attr (ParameterAttribute|False) – bias parameter attribute of mixed layer. -False means no bias, None means default bias.
      • -
    • lstm_bias_attr (ParameterAttribute|False) – bias parameter attribute of lstm layer. -False means no bias, None means default bias.
      • -
    • mixed_layer_attr (ExtraLayerAttribute) – mixed layer’s extra attribute.
      • -
    • lstm_layer_attr (ExtraLayerAttribute) – lstm layer’s extra attribute.
      • -
    • get_output_layer_attr (ExtraLayerAttribute) – get output layer’s extra attribute.
      • -
    -
    Returns:

    the lstmemory group.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    simple_lstm

    -
    -
    -paddle.trainer_config_helpers.networks.simple_lstm(*args, **kwargs)
    -

    Simple LSTM Cell.

    -

    It just combine a mixed layer with fully_matrix_projection and a lstmemory -layer. The simple lstm cell was implemented as follow equations.

    -
    -\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    -

    Please refer Generating Sequences With Recurrent Neural Networks if you -want to know what lstm is. Link is here.

    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – lstm layer name.
      • -
    • input (LayerOutput) – input layer name.
      • -
    • size (int) – lstm layer size.
      • -
    • reverse (bool) – whether to process the input data in a reverse order
      • -
    • mat_param_attr (ParameterAttribute) – mixed layer’s matrix projection parameter attribute.
      • -
    • bias_param_attr (ParameterAttribute|False) – bias parameter attribute. False means no bias, None -means default bias.
      • -
    • inner_param_attr (ParameterAttribute) – lstm cell parameter attribute.
      • -
    • act (BaseActivation) – lstm final activiation type
      • -
    • gate_act (BaseActivation) – lstm gate activiation type
      • -
    • state_act (BaseActivation) – lstm state activiation type.
      • -
    • mixed_layer_attr (ExtraLayerAttribute) – mixed layer’s extra attribute.
      • -
    • lstm_cell_attr (ExtraLayerAttribute) – lstm layer’s extra attribute.
      • -
    -
    Returns:

    lstm layer name.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    bidirectional_lstm

    -
    -
    -paddle.trainer_config_helpers.networks.bidirectional_lstm(*args, **kwargs)
    -

    A bidirectional_lstm is a recurrent unit that iterates over the input -sequence both in forward and bardward orders, and then concatenate two -outputs form a final output. However, concatenation of two outputs -is not the only way to form the final output, you can also, for example, -just add them together.

    -

    Please refer to Neural Machine Translation by Jointly Learning to Align -and Translate for more details about the bidirectional lstm. -The link goes as follows: -.. _Link: https://arxiv.org/pdf/1409.0473v3.pdf

    -

    The example usage is:

    -
    bi_lstm = bidirectional_lstm(input=[input1], size=512)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • name (basestring) – bidirectional lstm layer name.
      • -
    • input (LayerOutput) – input layer.
      • -
    • size (int) – lstm layer size.
      • -
    • return_seq (bool) – If set False, outputs of the last time step are -concatenated and returned. -If set True, the entire output sequences that are -processed in forward and backward directions are -concatenated and returned.
      • -
    -
    Returns:

    LayerOutput object accroding to the return_seq.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    GRU

    -
    -

    gru_unit

    -
    -
    -paddle.trainer_config_helpers.networks.gru_unit(*args, **kwargs)
    -

    Define calculations that a gated recurrent unit performs in a single time -step. This function itself is not a recurrent layer, so that it can not be -directly applied to sequence input. This function is almost always used in -the recurrent_group (see layers.py for more details) to implement attention -mechanism.

    -

    Please see grumemory in layers.py for the details about the maths.

    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – input layer name.
      • -
    • name (basestring) – name of the gru group.
      • -
    • size (int) – hidden size of the gru.
      • -
    • act (BaseActivation) – type of the activation
      • -
    • gate_act (BaseActivation) – type of the gate activation
      • -
    • gru_layer_attr (ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • -
    -
    Returns:

    the gru output layer.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    gru_group

    -
    -
    -paddle.trainer_config_helpers.networks.gru_group(*args, **kwargs)
    -

    gru_group is a recurrent layer group version of Gated Recurrent Unit. It -does exactly the same calculation as the grumemory layer does. A promising -benefit is that gru hidden states are accessible to the user. This is -especially useful in attention model. If you do not need to access -any internal state, but merely use the outputs of a GRU, it is recommended -to use the grumemory, which is relatively faster.

    -

    Please see grumemory in layers.py for more detail about the maths.

    -

    The example usage is:

    -
    gru = gur_group(input=[layer1],
-                size=256,
-                act=TanhActivation(),
-                gate_act=SigmoidActivation())
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – input layer name.
      • -
    • name (basestring) – name of the gru group.
      • -
    • size (int) – hidden size of the gru.
      • -
    • reverse (bool) – whether to process the input data in a reverse order
      • -
    • act (BaseActivation) – type of the activiation
      • -
    • gate_act (BaseActivation) – type of the gate activiation
      • -
    • gru_bias_attr (ParameterAttribute|False) – bias. False means no bias, None means default bias.
      • -
    • gru_layer_attr (ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • -
    -
    Returns:

    the gru group.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -

    simple_gru

    -
    -
    -paddle.trainer_config_helpers.networks.simple_gru(*args, **kwargs)
    -

    You maybe see gru_step_layer, grumemory in layers.py, gru_unit, gru_group, -simple_gru in network.py. The reason why there are so many interfaces is -that we have two ways to implement recurrent neural network. One way is to -use one complete layer to implement rnn (including simple rnn, gru and lstm) -with multiple time steps, such as recurrent_layer, lstmemory, grumemory. But, -the multiplication operation \(W x_t\) is not computed in these layers. -See details in their interfaces in layers.py. -The other implementation is to use an recurrent group which can ensemble a -series of layers to compute rnn step by step. This way is flexible for -attenion mechanism or other complex connections.

    -
      -
    • gru_step_layer: only compute rnn by one step. It needs an memory as input -and can be used in recurrent group.
      • -
    • gru_unit: a wrapper of gru_step_layer with memory.
      • -
    • gru_group: a GRU cell implemented by a combination of multiple layers in -recurrent group. -But \(W x_t\) is not done in group.
      • -
    • gru_memory: a GRU cell implemented by one layer, which does same calculation -with gru_group and is faster than gru_group.
      • -
    • simple_gru: a complete GRU implementation inlcuding \(W x_t\) and -gru_group. \(W\) contains \(W_r\), \(W_z\) and \(W\), see -formula in grumemory.
      • -
    -

    The computational speed is that, grumemory is relatively better than -gru_group, and gru_group is relatively better than simple_gru.

    -

    The example usage is:

    -
    gru = simple_gru(input=[layer1], size=256)
-
    -
    - --- - - - - - - - -
    Parameters:
      -
    • input (LayerOutput) – input layer name.
      • -
    • name (basestring) – name of the gru group.
      • -
    • size (int) – hidden size of the gru.
      • -
    • reverse (bool) – whether to process the input data in a reverse order
      • -
    • act (BaseActivation) – type of the activiation
      • -
    • gate_act (BaseActivation) – type of the gate activiation
      • -
    • gru_bias_attr (ParameterAttribute|False) – bias. False means no bias, None means default bias.
      • -
    • gru_layer_attr (ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • -
    -
    Returns:

    the gru group.

    -
    Return type:

    LayerOutput

    -
    -
    - -
    -
    -
    -

    simple_attention

    -
    -
    -paddle.trainer_config_helpers.networks.simple_attention(*args, **kwargs)
    -

    Calculate and then return a context vector by attention machanism. -Size of the context vector equals to size of the encoded_sequence.

    -
    -\[ \begin{align}\begin{aligned}a(s_{i-1},h_{j}) & = v_{a}f(W_{a}s_{t-1} + U_{a}h_{j})\\e_{i,j} & = a(s_{i-1}, h_{j})\\a_{i,j} & = \frac{exp(e_{i,j})}{\sum_{k=1}^{T_x}{exp(e_{i,k})}}\\c_{i} & = \sum_{j=1}^{T_{x}}a_{i,j}h_{j}\end{aligned}\end{align} \]
    -

    where \(h_{j}\) is the jth element of encoded_sequence, -\(U_{a}h_{j}\) is the jth element of encoded_proj -\(s_{i-1}\) is decoder_state -\(f\) is weight_act, and is set to tanh by default.

    -

    Please refer to Neural Machine Translation by Jointly Learning to -Align and Translate for more details. The link is as follows: -https://arxiv.org/abs/1409.0473.

    -

    The example usage is:

    -
    context = simple_attention(encoded_sequence=enc_seq,
-                           encoded_proj=enc_proj,
-                           decoder_state=decoder_prev,)
-
    -
    - --- - - - - - -
    Parameters:
      -
    • name (basestring) – name of the attention model.
      • -
    • softmax_param_attr (ParameterAttribute) – parameter attribute of sequence softmax -that is used to produce attention weight
      • -
    • weight_act (Activation) – activation of the attention model
      • -
    • encoded_sequence (LayerOutput) – output of the encoder
      • -
    • encoded_proj (LayerOutput) – attention weight is computed by a feed forward neural -network which has two inputs : decoder’s hidden state -of previous time step and encoder’s output. -encoded_proj is output of the feed-forward network for -encoder’s output. Here we pre-compute it outside -simple_attention for speed consideration.
      • -
    • decoder_state (LayerOutput) – hidden state of decoder in previous time step
      • -
    • transform_param_attr (ParameterAttribute) – parameter attribute of the feed-forward -network that takes decoder_state as inputs to -compute attention weight.
      • -
    -
    Returns:

    a context vector

    -
    -
    - -
    -
    -
    -

    Miscs

    -
    -

    dropout_layer

    -
    -
    -paddle.trainer_config_helpers.networks.dropout_layer(*args, **kwargs)
    -

    @TODO(yuyang18): Add comments.

    - --- - - - - - -
    Parameters:
      -
    • name –
      • -
    • input –
      • -
    • dropout_rate –
      • -
    -
    Returns:

    -
    -
    - -
    -
    -

    outputs

    -
    -
    -paddle.trainer_config_helpers.networks.outputs(layers, *args)
    -

    Declare the outputs of network. If user have not defined the inputs of -network, this method will calculate the input order by dfs travel.

    - --- - - - - - -
    Parameters:layers (list|tuple|LayerOutput) – Output layers.
    Returns:
    -
    - -
    -
    -
    - - -
    -
    -
    - - -
    - -
    -

    - © Copyright 2016, PaddlePaddle developers. - -

    -
    - Built with Sphinx using a theme provided by Read the Docs. - -
    - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc/api/v1/trainer_config_helpers/optimizers.html b/doc/api/v1/trainer_config_helpers/optimizers.html deleted file mode 100644 index c8f15d6a847f12fa581c6861b80a01af66015c9f..0000000000000000000000000000000000000000 --- a/doc/api/v1/trainer_config_helpers/optimizers.html +++ /dev/null @@ -1,474 +0,0 @@ - - - - - - - - - - - Optimizers — PaddlePaddle documentation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    -
      - -
    • Optimizers
      • -
    -
    - -
    -
    -
    -
    - -
    -

    Optimizers

    -
    -

    BaseSGDOptimizer

    -
    -
    -class paddle.trainer_config_helpers.optimizers.BaseSGDOptimizer
    -

    SGD Optimizer.

    -

    SGD is an optimization method, trying to find a neural network that -minimize the “cost/error” of it by iteration. In paddle’s implementation -SGD Optimizer is synchronized, which means all gradients will be wait to -calculate and reduced into one gradient, then do optimize operation.

    -

    The neural network consider the learning problem of minimizing an objective -function, that has the form of a sum

    -
    -\[Q(w) = \sum_{i}^{n} Q_i(w)\]
    -

    The value of function Q sometimes is the cost of neural network (Mean -Square Error between prediction and label for example). The function Q is -parametrised by w, the weight/bias of neural network. And weights is what to -be learned. The i is the i-th observation in (trainning) data.

    -

    So, the SGD method will optimize the weight by

    -
    -\[w = w - \eta \nabla Q(w) = w - \eta \sum_{i}^{n} \nabla Q_i(w)\]
    -

    where \(\eta\) is learning rate. And \(n\) is batch size.

    -
    - -
    -
    -

    MomentumOptimizer

    -
    -
    -class paddle.trainer_config_helpers.optimizers.MomentumOptimizer(momentum=None, sparse=False)
    -

    MomentumOptimizer.

    -

    When sparse=True, the update scheme:

    -
    -\[\begin{split}\alpha_t &= \alpha_{t-1} / k \\ -\beta_t &= \beta_{t-1} / (1 + \lambda \gamma_t) \\ -u_t &= u_{t-1} - \alpha_t \gamma_t g_t \\ -v_t &= v_{t-1} + \tau_{t-1} \alpha_t \gamma_t g_t \\ -\tau_t &= \tau_{t-1} + \beta_t / \alpha_t\end{split}\]
    -

    where \(k\) is momentum, \(\lambda\) is decay rate, -\(\gamma_t\) is learning rate at the t’th step.

    - --- - - - -
    Parameters:sparse (bool) – with sparse support or not.
    -
    - -
    -
    -

    AdamOptimizer

    -
    -
    -class paddle.trainer_config_helpers.optimizers.AdamOptimizer(beta1=0.9, beta2=0.999, epsilon=1e-08)
    -

    Adam optimizer. -The details of please refer Adam: A Method for Stochastic Optimization

    -
    -\[\begin{split}m(w, t) & = \beta_1 m(w, t-1) + (1 - \beta_1) \nabla Q_i(w) \\ -v(w, t) & = \beta_2 v(w, t-1) + (1 - \beta_2)(\nabla Q_i(w)) ^2 \\ -w & = w - \frac{\eta}{\sqrt{v(w,t) + \epsilon}}\end{split}\]
    - --- - - - -
    Parameters:
      -
    • beta1 (float) – the \(\beta_1\) in equation.
      • -
    • beta2 (float) – the \(\beta_2\) in equation.
      • -
    • epsilon (float) – the \(\epsilon\) in equation. It is used to prevent -divided by zero.
      • -
    -
    -
    - -
    -
    -

    AdamaxOptimizer

    -
    -
    -class paddle.trainer_config_helpers.optimizers.AdamaxOptimizer(beta1, beta2)
    -

    Adamax optimizer.

    -

    The details of please refer this Adam: A Method for Stochastic Optimization

    -
    -\[\begin{split}m_t & = \beta_1 * m_{t-1} + (1-\beta_1)* \nabla Q_i(w) \\ -u_t & = max(\beta_2*u_{t-1}, abs(\nabla Q_i(w))) \\ -w_t & = w_{t-1} - (\eta/(1-\beta_1^t))*m_t/u_t\end{split}\]
    - --- - - - -
    Parameters:
      -
    • beta1 (float) – the \(\beta_1\) in the equation.
      • -
    • beta2 (float) – the \(\beta_2\) in the equation.
      • -
    -
    -
    - -
    -
    -

    AdaGradOptimizer

    -
    -
    -class paddle.trainer_config_helpers.optimizers.AdaGradOptimizer
    -

    Adagrad(for ADAptive GRAdient algorithm) optimizer.

    -

    For details please refer this Adaptive Subgradient Methods for -Online Learning and Stochastic Optimization.

    -
    -\[\begin{split}G &= \sum_{\tau=1}^{t} g_{\tau} g_{\tau}^T \\ -w & = w - \eta diag(G)^{-\frac{1}{2}} \circ g\end{split}\]
    -
    - -
    -
    -

    DecayedAdaGradOptimizer

    -
    -
    -class paddle.trainer_config_helpers.optimizers.DecayedAdaGradOptimizer(rho=0.95, epsilon=1e-06)
    -

    AdaGrad method with decayed sum gradients. The equations of this method -show as follow.

    -
    -\[\begin{split}E(g_t^2) &= \rho * E(g_{t-1}^2) + (1-\rho) * g^2 \\ -learning\_rate &= 1/sqrt( ( E(g_t^2) + \epsilon )\end{split}\]
    - --- - - - -
    Parameters:
      -
    • rho (float) – The \(\rho\) parameter in that equation
      • -
    • epsilon (float) – The \(\epsilon\) parameter in that equation.
      • -
    -
    -
    - -
    -
    -

    AdaDeltaOptimizer

    -
    -
    -class paddle.trainer_config_helpers.optimizers.AdaDeltaOptimizer(rho=0.95, epsilon=1e-06)
    -

    AdaDelta method. The details of adadelta please refer to this -ADADELTA: AN ADAPTIVE LEARNING RATE METHOD.

    -
    -\[\begin{split}E(g_t^2) &= \rho * E(g_{t-1}^2) + (1-\rho) * g^2 \\ -learning\_rate &= sqrt( ( E(dx_{t-1}^2) + \epsilon ) / ( \ - E(g_t^2) + \epsilon ) ) \\ -E(dx_t^2) &= \rho * E(dx_{t-1}^2) + (1-\rho) * (-g*learning\_rate)^2\end{split}\]
    - --- - - - -
    Parameters:
      -
    • rho (float) – \(\rho\) in equation
      • -
    • epsilon (float) – \(\rho\) in equation
      • -
    -
    -
    - -
    -
    -

    RMSPropOptimizer

    -
    -
    -class paddle.trainer_config_helpers.optimizers.RMSPropOptimizer(rho=0.95, epsilon=1e-06)
    -

    RMSProp(for Root Mean Square Propagation) optimizer. For details please -refer this slide.

    -

    The equations of this method as follows:

    -
    -\[\begin{split}v(w, t) & = \rho v(w, t-1) + (1 - \rho)(\nabla Q_{i}(w))^2 \\ -w & = w - \frac{\eta} {\sqrt{v(w,t) + \epsilon}} \nabla Q_{i}(w)\end{split}\]
    - --- - - - -
    Parameters:
      -
    • rho (float) – the \(\rho\) in the equation. The forgetting factor.
      • -
    • epsilon (float) – the \(\epsilon\) in the equation.
      • -
    -
    -
    - -
    -
    -

    settings

    -
    -
    -paddle.trainer_config_helpers.optimizers.settings(*args, **kwargs)
    -

    Set the optimization method, learning rate, batch size, and other training -settings. The currently supported algorithms are SGD and Async-SGD.

    -
    -

    Warning

    -

    Note that the ‘batch_size’ in PaddlePaddle is not equal to global -training batch size. It represents the single training process’s batch -size. If you use N processes to train one model, for example use three -GPU machines, the global batch size is N*’batch_size’.

    -
    - --- - - - -
    Parameters:
      -
    • batch_size (int) – batch size for one training process.
      • -
    • learning_rate (float) – learning rate for SGD
      • -
    • learning_method (BaseSGDOptimizer) – The extension optimization algorithms of gradient -descent, such as momentum, adagrad, rmsprop, etc. -Note that it should be instance with base type -BaseSGDOptimizer.
      • -
    • regularization (BaseRegularization) – The regularization method.
      • -
    • is_async (bool) – Is Async-SGD or not. Default value is False.
      • -
    • model_average (ModelAverage) – Model Average Settings.
      • -
    • gradient_clipping_threshold (float) – gradient clipping threshold. If gradient -value larger than some value, will be -clipped.
      • -
    -
    -
    - -
    -
    - - -
    -
    -
    - - -
    - -
    -

    - © Copyright 2016, PaddlePaddle developers. - -

    -
    - Built with Sphinx using a theme provided by Read the Docs. - -
    - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc/api/v2/config/activation.html b/doc/api/v2/config/activation.html index 570a39e2f32b7450d2b834ff64fe460d7911ef39..e1a3b5db292d22199af11f7fdc6be3dcd963f379 100644 --- a/doc/api/v2/config/activation.html +++ b/doc/api/v2/config/activation.html @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -87,7 +87,7 @@
  • GET STARTED
  • HOW TO
  • API
    • -
  • ABOUT
    • +
  • MOBILE
    • @@ -111,9 +111,10 @@ -
  • Run Distributed Training
    • +
  • PaddlePaddle Distributed Training
  • Paddle On Kubernetes
  • Distributed PaddlePaddle Training on AWS with Kubernetes
  • Write New Layers
  • Contribute Code
    • -
  • RNN Models
    • +
  • Contribute Documentation
    • +
  • RNN Models +
  • Tune GPU Performance
    • @@ -138,17 +143,40 @@
  • Model Configuration
    • -
  • Data Reader Interface and DataSets
    • +
  • Data Reader Interface and DataSets +
  • Training and Inference
    • +
  • Fluid +
    • + + +
  • MOBILE
    • -
  • ABOUT
    • @@ -346,6 +374,17 @@ \[f(z) = 1.7159 * tanh(2/3*z)\]
    +
    +
    +

    SoftSign

    +
    +
    +class paddle.v2.activation.SoftSign
    +

    SoftSign Activation.

    +
    +\[f(z)=\frac{z}{1 + |z|}\]
    +
    +
    diff --git a/doc/api/v2/config/attr.html b/doc/api/v2/config/attr.html index f6cbaf8e58ff66163840a5610a9a793d9e344f2d..2b368d6b528c2f52e11f80fc852e135f2bf13d04 100644 --- a/doc/api/v2/config/attr.html +++ b/doc/api/v2/config/attr.html @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -87,7 +87,7 @@
  • GET STARTED
  • HOW TO
  • API
    • -
  • ABOUT
    • +
  • MOBILE
    • @@ -111,9 +111,10 @@ -
  • Run Distributed Training
    • +
  • PaddlePaddle Distributed Training
  • Paddle On Kubernetes
  • Distributed PaddlePaddle Training on AWS with Kubernetes
  • Write New Layers
  • Contribute Code
    • -
  • RNN Models
    • +
  • Contribute Documentation
    • +
  • RNN Models +
  • Tune GPU Performance
    • @@ -138,17 +143,40 @@
  • Model Configuration
    • -
  • Data Reader Interface and DataSets
    • +
  • Data Reader Interface and DataSets +
  • Training and Inference
    • +
  • Fluid +
    • + + +
  • MOBILE
    • -
  • ABOUT
    • @@ -196,6 +224,18 @@

    alias of ExtraLayerAttribute

    +
    +
    +paddle.v2.attr.Hook
    +

    alias of HookAttribute

    +
    + +
    +
    +paddle.v2.attr.HookAttr
    +

    alias of HookAttribute

    +
    +
    paddle.v2.attr.ParamAttr
    @@ -210,7 +250,7 @@
    -class paddle.v2.attr.ParameterAttribute(name=None, is_static=False, initial_std=None, initial_mean=None, initial_max=None, initial_min=None, l1_rate=None, l2_rate=None, learning_rate=None, momentum=None, gradient_clipping_threshold=None, sparse_update=False)
    +class paddle.v2.attr.ParameterAttribute(name=None, is_static=False, initial_std=None, initial_mean=None, initial_max=None, initial_min=None, l1_rate=None, l2_rate=None, learning_rate=None, momentum=None, gradient_clipping_threshold=None, sparse_update=False, update_hooks=None, initializer=None)

    Parameter Attributes object. To fine-tuning network training process, user can set attribute to control training details, such as l1,l2 rate / learning rate / how to init param.

    @@ -239,6 +279,10 @@ value larger than some value, will be clipped.
  • sparse_update (bool) – Enable sparse update for this parameter. It will enable both local and remote sparse update.
    • +
  • initializer – If not None, it should be a callable object which accepts +a parameter name and returns numpy array for the initial +value of the parameter
    • +
  • initializer – callable object
    • @@ -276,7 +320,7 @@ layer that not support this attribute, paddle will print an error and core.

  • drop_rate (float) – Dropout rate. Dropout will create a mask on layer output. The dropout rate is the zero rate of this mask. The details of what dropout is please refer to here.
    • -
  • device (int) –

    device ID of layer. device=-1, use CPU. device>0, use GPU. +

  • device (int) –

    device ID of layer. device=-1, use CPU. device>=0, use GPU. The details allocation in parallel_nn please refer to here.

    • diff --git a/doc/api/v1/trainer_config_helpers/evaluators.html b/doc/api/v2/config/evaluators.html similarity index 53% rename from doc/api/v1/trainer_config_helpers/evaluators.html rename to doc/api/v2/config/evaluators.html index 2411ba063cc5b07271d2a3375a9224deef322e8c..dcf52c2250b8a8f6a38bb244d652051c46770709 100644 --- a/doc/api/v1/trainer_config_helpers/evaluators.html +++ b/doc/api/v2/config/evaluators.html @@ -32,7 +32,10 @@ - + + + + @@ -62,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -80,11 +83,11 @@
    -
      + @@ -105,12 +108,13 @@

    The input of lstm step is \(Wx_t + Wh_{t-1}\), and user should use mixed and full_matrix_projection to calculate these -input vector.

    +input vectors.

    The state of lstm step is \(c_{t-1}\). And lstm step layer will do

    \[ \begin{align}\begin{aligned}i_t = \sigma(input + W_{ci}c_{t-1} + b_i)\\...\end{aligned}\end{align} \]
    -

    This layer contains two outputs. Default output is \(h_t\). The other -output is \(o_t\), which name is ‘state’ and can use +

    This layer has two outputs. The default output is \(h_t\). The other +output is \(o_t\), whose name is ‘state’ and users can use get_output to extract this output.

    @@ -1272,22 +1501,28 @@ be sigmoid only.

    gru_step

    -class paddle.v2.layer.gru_step(*args, **kwargs)
    +class paddle.v2.layer.gru_step
    Parameters:
      -
    • name (basestring) – Layer’s name.
      • -
    • size (int) – Layer’s size. NOTE: lstm layer’s size, should be equal as -input.size/4, and should be equal as -state.size.
      • -
    • input (paddle.v2.config_base.Layer) – input layer. \(Wx_t + Wh_{t-1}\)
      • -
    • state (paddle.v2.config_base.Layer) – State Layer. \(c_{t-1}\)
      • -
    • act (paddle.v2.Activation.Base) – Activation type. Default is tanh
      • -
    • gate_act (paddle.v2.Activation.Base) – Gate Activation Type. Default is sigmoid, and should -be sigmoid only.
      • -
    • state_act (paddle.v2.Activation.Base) – State Activation Type. Default is sigmoid, and should -be sigmoid only.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute) – Bias Attribute.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – layer’s extra attribute.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • size (int) – The dimension of this layer’s output, which must be +equal to the dimension of the state.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • state (paddle.v2.config_base.Layer) – The state of the LSTM unit.
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Tanh is the default activation.
      • +
    • gate_act (paddle.v2.activation.Base) – Activation type of the gate. paddle.v2.activation.Sigmoid is the +default activation.
      • +
    • state_act (paddle.v2.activation.Base) – Activation type of the state. paddle.v2.activation.Tanh is the +default activation.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for details.
    @@ -1313,14 +1548,20 @@ to maintain tractability.

    The example usage is:

    def rnn_step(input):
     last_time_step_output = memory(name='rnn', size=512)
-    with mixed_layer(size=512, name='rnn') as simple_rnn:
+    with mixed(size=512, name='rnn') as simple_rnn:
         simple_rnn += full_matrix_projection(input)
         simple_rnn += last_time_step_output
     return simple_rnn
 
+generated_word_embedding = GeneratedInput(
+                       size=target_dictionary_dim,
+                       embedding_name="target_language_embedding",
+                       embedding_size=word_vector_dim)
+
 beam_gen = beam_search(name="decoder",
                        step=rnn_step,
-                       input=[StaticInput(encoder_last)],
+                       input=[StaticInput(encoder_last),
+                              generated_word_embedding],
                        bos_id=0,
                        eos_id=1,
                        beam_size=5)
@@ -1335,14 +1576,17 @@ to maintain tractability.
    - @@ -1380,7 +1624,7 @@ beam size.

    get_output

    -class paddle.v2.layer.get_output(*args, **kwargs)
    +class paddle.v2.layer.get_output

    Get layer’s output by name. In PaddlePaddle, a layer might return multiple values, but returns one layer’s output. If the user wants to use another output besides the default one, please use get_output first to get @@ -1390,11 +1634,12 @@ the output from input.

    @@ -1421,17 +1666,17 @@ multiple outputs. Each inputs is a projection or operator.

    There are two styles of usages.

      -
    1. When not set inputs parameter, use mixed_layer like this:
      2. +
    2. When not set inputs parameter, use mixed like this:
    -
    with mixed_layer(size=256) as m:
+
    with mixed(size=256) as m:
     m += full_matrix_projection(input=layer1)
     m += identity_projection(input=layer2)
 
    
 
    
 
      
-
    1. You can also set all inputs when invoke mixed_layer as follows:
      2. 
+
    2. You can also set all inputs when invoke mixed as follows:
      3. 
 
    
-
    m = mixed_layer(size=256,
+
    m = mixed(size=256,
                 input=[full_matrix_projection(input=layer1),
                        full_matrix_projection(input=layer2)])
 
    
@@ -1443,13 +1688,13 @@ Each inputs is a projection or operator.
    @@ -1468,19 +1713,19 @@ default Bias.

    embedding

    -class paddle.v2.layer.embedding(*args, **kwargs)
    +class paddle.v2.layer.embedding

    Define a embedding Layer.

    Parameters:
      -
    • input (paddle.v2.config_base.Layer) –
      • -
    • output_mem –
      • -
    • size –
      • -
    • act –
      • -
    • name –
      • -
    • gate_act –
      • -
    • bias_attr –
      • -
    • param_attr – the parameter_attribute for transforming the output_mem -from previous step.
      • -
    • layer_attr –
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer, whose dimension can be divided by 3.
      • +
    • output_mem (paddle.v2.config_base.Layer) – A memory which memorizes the output of this layer at previous +time step.
      • +
    • size (int) – The dimension of this layer’s output. If it is not set or set to None, +it will be set to one-third of the dimension of the input automatically.
      • +
    • act (paddle.v2.activation.Base) – Activation type of this layer’s output. paddle.v2.activation.Tanh +is the default activation.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • gate_act (paddle.v2.activation.Base) – Activation type of this layer’s two gates. paddle.v2.activation.Sigmoid is +the default activation.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The parameter attribute for bias. If this parameter is set to +False or an object whose type is not paddle.v2.attr.ParameterAttribute, no bias +is defined. If this parameter is set to True, +the bias is initialized to zero.
      • +
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for details.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for details.
    Parameters:
      -
    • name (base string) – Name of the recurrent unit that generates sequences.
      • +
    • name (basestring) – The name of the recurrent unit that is responsible for +generating sequences. It is optional.
    • step (callable) –

      A callable function that defines the calculation in a time step, and it is applied to sequences with arbitrary length by sharing a same set of weights.

      You can refer to the first parameter of recurrent_group, or demo/seqToseq/seqToseq_net.py for more details.

      • -
    • input (list) – Input data for the recurrent unit
      • +
    • input (list) – Input data for the recurrent unit, which should include the +previously generated words as a GeneratedInput object. +In beam_search, none of the input’s type should be paddle.v2.config_base.Layer.
    • bos_id (int) – Index of the start symbol in the dictionary. The start symbol is a special token for NLP task, which indicates the beginning of a sequence. In the generation task, the start @@ -1368,7 +1612,7 @@ beam size.
    Returns:

    The generated word index.
    Return type:

    LayerOutput

    +
    Return type:

    paddle.v2.config_base.Layer
    Parameters:
      -
    • name (basestring) – Layer’s name.
      • -
    • input (paddle.v2.config_base.Layer) – get output layer’s input. And this layer should contains +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input layer. And this layer should contain multiple outputs.
      • -
    • arg_name (basestring) – Output name from input.
      • -
    • layer_attr – Layer’s extra attribute.
      • +
    • arg_name (basestring) – The name of the output to be extracted from the input layer.
      • +
    • layer_attr – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    Parameters:
    • name (basestring) – mixed layer name. Can be referenced by other layer.
    • size (int) – layer size.
      • -
    • input – inputs layer. It is an optional parameter. If set, +
    • input – The input of this layer. It is an optional parameter. If set, then this function will just return layer’s name.
      • -
    • act (BaseActivation) – Activation Type.
      • -
    • bias_attr (ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or -something not type of ParameterAttribute. None will get a -default Bias.
      • -
    • layer_attr (ExtraLayerAttribute) – The extra layer config. Default is None.
      • +
    • act (paddle.v2.activation.Base) – Activation Type. paddle.v2.activation.Linear is the default activation.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer config. Default is None.
    @@ -1499,7 +1744,7 @@ for details.

    scaling_projection

    -class paddle.v2.layer.scaling_projection(**kwargs)
    +class paddle.v2.layer.scaling_projection

    scaling_projection multiplies the input with a scalar parameter and add to the output.

    @@ -1513,7 +1758,7 @@ the output.

    @@ -1533,7 +1778,7 @@ the output.

    dotmul_projection

    -class paddle.v2.layer.dotmul_projection(**kwargs)
    +class paddle.v2.layer.dotmul_projection

    DotMulProjection with a layer as input. It performs element-wise multiplication with weight.

    @@ -1548,7 +1793,7 @@ It performs element-wise multiplication with weight.

    @@ -1568,7 +1813,7 @@ It performs element-wise multiplication with weight.

    dotmul_operator

    -class paddle.v2.layer.dotmul_operator(**kwargs)
    +class paddle.v2.layer.dotmul_operator

    DotMulOperator takes two inputs and performs element-wise multiplication:

    \[out.row[i] += scale * (a.row[i] .* b.row[i])\]
    @@ -1604,7 +1849,7 @@ scale is a config scalar, its default value is one.

    full_matrix_projection

    -class paddle.v2.layer.full_matrix_projection(**kwargs)
    +class paddle.v2.layer.full_matrix_projection

    Full Matrix Projection. It performs full matrix multiplication.

    \[out.row[i] += in.row[i] * weight\]
    @@ -1629,7 +1874,7 @@ scale is a config scalar, its default value is one.

    @@ -1691,12 +1936,49 @@ It select dimesions [offset, offset+layer_size) from input:

    Parameters:
      -
    • name (basestring) – Name of this embedding layer.
      • -
    • input (paddle.v2.config_base.Layer) – The input layer for this embedding. NOTE: must be Index Data.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer, which must be Index Data.
    • size (int) – The embedding dimension.
      • -
    • param_attr (paddle.v2.attr.ParameterAttribute|None) – The embedding parameter attribute. See paddle.v2.attr.ParameterAttribute +
    • param_attr (paddle.v2.attr.ParameterAttribute | None) – The embedding parameter attribute. See paddle.v2.attr.ParameterAttribute for details.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra layer Config. Default is None.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute | None) – Extra layer Config. Default is None.
    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input Layer.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – input layer
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • size (int) – The parameter size. Means the width of parameter.
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
    @@ -1650,7 +1895,7 @@ scale is a config scalar, its default value is one.

    identity_projection

    -class paddle.v2.layer.identity_projection(**kwargs)
    +class paddle.v2.layer.identity_projection
    1. IdentityProjection if offset=None. It performs:
    @@ -1676,7 +1921,7 @@ It select dimesions [offset, offset+layer_size) from input:
    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input Layer.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • offset (int) – Offset, None if use default.
    +
    +
    +

    slice_projection

    +
    +
    +class paddle.v2.layer.slice_projection
    +

    slice_projection can slice the input value into multiple parts, +and then select some of them to merge into a new output.

    +
    +\[output = [input.slices()]\]
    +

    The example usage is:

    +
    proj = slice_projection(input=layer, slices=[(0, 10), (20, 30)])
+
    +
    +

    Note that slice_projection should not have any parameter.

    + +++ + + + + + + + +
    Parameters:
      +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • slices (pair of int) – An array of slice parameters. +Each slice contains the start and end offsets based +on the input.
      • +
    +
    Returns:

    A SliceProjection object

    +
    Return type:

    SliceProjection

    +
    +
    +

    table_projection

    -class paddle.v2.layer.table_projection(**kwargs)
    +class paddle.v2.layer.table_projection

    Table Projection. It selects rows from parameter where row_id is in input_ids.

    @@ -1724,7 +2006,7 @@ and \(i\) is row_id.

    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input layer, which must contains id fields.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer, which must contains id fields.
    • size (int) – The parameter size. Means the width of parameter.
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
    @@ -1745,7 +2027,7 @@ and \(i\) is row_id.

    trans_full_matrix_projection

    -class paddle.v2.layer.trans_full_matrix_projection(**kwargs)
    +class paddle.v2.layer.trans_full_matrix_projection

    Different from full_matrix_projection, this projection performs matrix multiplication, using transpose of weight.

    @@ -1765,7 +2047,7 @@ The simply usage is:

    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – input layer
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • size (int) – The parameter size. Means the width of parameter.
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
    @@ -1785,16 +2067,46 @@ The simply usage is:

    Aggregate Layers

    +
    +

    AggregateLevel

    +
    +
    +class paddle.v2.layer.AggregateLevel
    +

    PaddlePaddle supports three sequence types:

    +
      +
    • SequenceType.NO_SEQUENCE means the sample is not a sequence.
      • +
    • SequenceType.SEQUENCE means the sample is a sequence.
      • +
    • SequenceType.SUB_SEQUENCE means the sample is a nested sequence, +each timestep of which is also a sequence.
      • +
    +

    Accordingly, AggregateLevel supports two modes:

    +
      +
    • AggregateLevel.TO_NO_SEQUENCE means the aggregation acts on each +timestep of a sequence, both SUB_SEQUENCE and SEQUENCE will +be aggregated to NO_SEQUENCE.
      • +
    • AggregateLevel.TO_SEQUENCE means the aggregation acts on each +sequence of a nested sequence, SUB_SEQUENCE will be aggregated to +SEQUENCE.
      • +
    +
    + +

    pooling

    -class paddle.v2.layer.pooling(*args, **kwargs)
    +class paddle.v2.layer.pooling

    Pooling layer for sequence inputs, not used for Image.

    +

    If stride > 0, this layer slides a window whose size is determined by stride, +and return the pooling value of the window as the output. Thus, a long sequence +will be shorten.

    +

    The parameter stride specifies the intervals at which to apply the pooling +operation. Note that for sequence with sub-sequence, the default value +of stride is -1.

    The example usage is:

    seq_pool = pooling(input=layer,
                          pooling_type=AvgPooling(),
-                         agg_level=AggregateLevel.EACH_SEQUENCE)
+                         agg_level=AggregateLevel.TO_NO_SEQUENCE)
    @@ -1802,14 +2114,17 @@ The simply usage is:

    @@ -1828,7 +2143,7 @@ SumPooling, SquareRootNPooling.

    last_seq

    -class paddle.v2.layer.last_seq(*args, **kwargs)
    +class paddle.v2.layer.last_seq

    Get Last Timestamp Activation of a sequence.

    If stride > 0, this layer slides a window whose size is determined by stride, and return the last value of the window as the output. Thus, a long sequence @@ -1844,9 +2159,9 @@ of stride is -1.

    @@ -1866,7 +2181,7 @@ of stride is -1.

    first_seq

    -class paddle.v2.layer.first_seq(*args, **kwargs)
    +class paddle.v2.layer.first_seq

    Get First Timestamp Activation of a sequence.

    If stride > 0, this layer slides a window whose size is determined by stride, and return the first value of the window as the output. Thus, a long sequence @@ -1882,9 +2197,9 @@ of stride is -1.

    @@ -1904,9 +2219,9 @@ of stride is -1.

    concat

    -class paddle.v2.layer.concat(*args, **kwargs)
    -

    Concat all input vector into one huge vector. -Inputs can be list of paddle.v2.config_base.Layer or list of projection.

    +class paddle.v2.layer.concat +

    Concatenate all input vectors to one vector. +Inputs can be a list of paddle.v2.config_base.Layer or a list of projection.

    The example usage is:

    concat = concat(input=[layer1, layer2])
    @@ -1916,10 +2231,11 @@ Inputs can be list of paddle.v2.config_base.Layer or list of projection.

    @@ -1938,18 +2254,19 @@ Inputs can be list of paddle.v2.config_base.Layer or list of projection.

    seq_concat

    -class paddle.v2.layer.seq_concat(*args, **kwargs)
    -

    Concat sequence a with sequence b.

    +class paddle.v2.layer.seq_concat +

    Concatenate sequence a and sequence b.

    Inputs:
      -
    • a = [a1, a2, ..., an]
      • +
    • a = [a1, a2, ..., am]
    • b = [b1, b2, ..., bn]
      • -
    • Note that the length of a and b should be the same.
    -

    Output: [a1, b1, a2, b2, ..., an, bn]

    +

    Output: [a1, ..., am, b1, ..., bn]

    +

    Note that the above computation is for one sample. Multiple samples are +processed in one batch.

    The example usage is:

    concat = seq_concat(a=layer1, b=layer2)
    @@ -1959,14 +2276,102 @@ Inputs can be list of paddle.v2.config_base.Layer or list of projection.

    + + + + + + +
    Parameters:
      -
    • agg_level (AggregateLevel) – AggregateLevel.EACH_TIMESTEP or -AggregateLevel.EACH_SEQUENCE
      • -
    • name (basestring) – layer name.
      • -
    • input (paddle.v2.config_base.Layer) – input layer name.
      • -
    • pooling_type (BasePoolingType|None) – Type of pooling, MaxPooling(default), AvgPooling, +
    • agg_level (AggregateLevel) – AggregateLevel.TO_NO_SEQUENCE or +AggregateLevel.TO_SEQUENCE
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • pooling_type (BasePoolingType | None) – Type of pooling, MaxPooling(default), AvgPooling, SumPooling, SquareRootNPooling.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute|None|False) – Bias parameter attribute. False if no bias.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – The Extra Attributes for layer, such as dropout.
      • +
    • stride (Int) – The step size between successive pooling regions.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute | None) – The Extra Attributes for layer, such as dropout.
    Parameters:
    • agg_level – Aggregated level
      • -
    • name (basestring) – Layer name.
      • -
    • input (paddle.v2.config_base.Layer) – Input layer name.
      • -
    • stride (Int) – window size.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • stride (Int) – The step size between successive pooling regions.
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
    Parameters:
    • agg_level – aggregation level
      • -
    • name (basestring) – Layer name.
      • -
    • input (paddle.v2.config_base.Layer) – Input layer name.
      • -
    • stride (Int) – window size.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • stride (Int) – The step size between successive pooling regions.
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
    Parameters:
      -
    • name (basestring) – Layer name.
      • -
    • input (list|tuple|collections.Sequence) – input layers or projections
      • -
    • act (paddle.v2.Activation.Base) – Activation type.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (list | tuple | collections.Sequence) – The input layers or projections
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Identity is the default activation.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    Parameters:
      -
    • name (basestring) – Layer name.
      • -
    • a (paddle.v2.config_base.Layer) – input sequence layer
      • -
    • b (paddle.v2.config_base.Layer) – input sequence layer
      • -
    • act (paddle.v2.Activation.Base) – Activation type.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or -something not type of paddle.v2.attr.ParameterAttribute. None will get a -default Bias.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • a (paddle.v2.config_base.Layer) – The first input sequence layer
      • +
    • b (paddle.v2.config_base.Layer) – The second input sequence layer
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Identity is the default activation.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
      • +
    +
    Returns:

    paddle.v2.config_base.Layer object.

    +
    Return type:

    paddle.v2.config_base.Layer

    +
    +
    + +
    +
    +

    seq_slice

    +
    +
    +class paddle.v2.layer.seq_slice
    +

    seq_slice will return one or several sub-sequences from the +input sequence layer given start and end indices.

    +
    +
      +
    • If only start indices are given, and end indices are set to None, +this layer slices the input sequence from the given start indices +to its end.
      • +
    • If only end indices are given, and start indices are set to None, +this layer slices the input sequence from its beginning to the +given end indices.
      • +
    • If start and end indices are both given, they should have the same +number of elements.
      • +
    +
    +

    If start or end indices contains more than one elements, the input sequence +will be sliced for multiple times.

    +
    seq_silce = seq_slice(input=input_seq,
+                            starts=start_pos, ends=end_pos)
+
    +
    + +++ + + + + + + + +
    Parameters:
      +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer, which should be a sequence.
      • +
    • starts (paddle.v2.config_base.Layer | None) – The start indices to slice the input sequence.
      • +
    • ends (paddle.v2.config_base.Layer | None) – The end indices to slice the input sequence.
      • +
    +
    Returns:

    paddle.v2.config_base.Layer object.

    +
    Return type:

    paddle.v2.config_base.Layer

    +
    +
    + +
    +
    +

    kmax_sequence_score

    +
    +
    +

    sub_nested_seq

    +
    +
    +class paddle.v2.layer.sub_nested_seq
    +

    The sub_nested_seq accepts two inputs: the first one is a nested +sequence; the second one is a set of selceted indices in the nested sequence.

    +

    Then sub_nest_seq trims the first nested sequence input according +to the selected indices to form a new output. This layer is useful in +beam training.

    +

    The example usage is:

    +
    sub_nest_seq = sub_nested_seq(input=data, selected_indices=selected_ids)
+
    +
    + +++ + @@ -1988,7 +2393,7 @@ default Bias.

    block_expand

    -class paddle.v2.layer.block_expand(*args, **kwargs)
    +class paddle.v2.layer.block_expand
    Expand feature map to minibatch matrix.
      @@ -1999,11 +2404,11 @@ default Bias.
    \[ \begin{align}\begin{aligned}outputH = 1 + (2 * padding_y + imgSizeH - block_y + stride_y - 1) / stride_y\\outputW = 1 + (2 * padding_x + imgSizeW - block_x + stride_x - 1) / stride_x\end{aligned}\end{align} \]
    -

    The expand method is the same with ExpandConvLayer, but saved the transposed +

    The expanding method is the same with ExpandConvLayer, but saved the transposed value. After expanding, output.sequenceStartPositions will store timeline. -The number of time steps are outputH * outputW and the dimension of each +The number of time steps is outputH * outputW and the dimension of each time step is block_y * block_x * num_channels. This layer can be used after -convolution neural network, and before recurrent neural network.

    +convolutional neural network, and before recurrent neural network.

    The simple usage is:

    block_expand = block_expand(input=layer,
                                   num_channels=128,
@@ -2018,16 +2423,19 @@ convolution neural network, and before recurrent neural network.
    @@ -2041,18 +2449,36 @@ convolution neural network, and before recurrent neural network.

    Parameters:
      +
    • input (paddle.v2.config_base.Layer) – The input of this layer. It is a nested sequence.
      • +
    • selected_indices – A set of sequence indices in the nested sequence.
      • +
    • name (basestring) – The name of this layer. It is optional.
    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – The input layer.
      • -
    • num_channels (int|None) – The channel number of input layer.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • num_channels (int) – The number of input channels. If the parameter is not set or +set to None, its actual value will be automatically set to +the channels number of the input.
    • block_x (int) – The width of sub block.
    • block_y (int) – The width of sub block.
    • stride_x (int) – The stride size in horizontal direction.
    • stride_y (int) – The stride size in vertical direction.
    • padding_x (int) – The padding size in horizontal direction.
    • padding_y (int) – The padding size in vertical direction.
      • -
    • name (None|basestring.) – The name of this layer, which can not specify.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • +
    • name (basestring.) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    +
    +
    +

    ExpandLevel

    +
    +
    +class paddle.v2.layer.ExpandLevel
    +

    Please refer to AggregateLevel first.

    +

    ExpandLevel supports two modes:

    +
      +
    • ExpandLevel.FROM_NO_SEQUENCE means the expansion acts on +NO_SEQUENCE, which will be expanded to +SEQUENCE or SUB_SEQUENCE.
      • +
    • ExpandLevel.FROM_SEQUENCE means the expansion acts on +SEQUENCE, which will be expanded to +SUB_SEQUENCE.
      • +
    +
    +
    -

    expand

    +

    expand

    -class paddle.v2.layer.expand(*args, **kwargs)
    +class paddle.v2.layer.expand

    A layer for “Expand Dense data or (sequence data where the length of each sequence is one) to sequence data.”

    The example usage is:

    expand = expand(input=layer1,
                       expand_as=layer2,
-                      expand_level=ExpandLevel.FROM_TIMESTEP)
+                      expand_level=ExpandLevel.FROM_NO_SEQUENCE)
    @@ -2060,11 +2486,12 @@ sequence is one) to sequence data.”

    @@ -2121,7 +2557,7 @@ to apply concat() with num_repeats same input.

    rotate

    -class paddle.v2.layer.rotate(*args, **kwargs)
    +class paddle.v2.layer.rotate

    A layer for rotating 90 degrees (clock-wise) for each feature channel, usually used when the input sample is some image or feature map.

    @@ -2138,10 +2574,12 @@ usually used when the input sample is some image or feature map.

    @@ -2160,7 +2598,7 @@ usually used when the input sample is some image or feature map.

    seq_reshape

    -class paddle.v2.layer.seq_reshape(*args, **kwargs)
    +class paddle.v2.layer.seq_reshape

    A layer for reshaping the sequence. Assume the input sequence has T instances, the dimension of each instance is M, and the input reshape_size is N, then the output sequence has T*M/N instances, the dimension of each instance is N.

    @@ -2174,14 +2612,15 @@ output sequence has T*M/N instances, the dimension of each instance is N.

    @@ -2203,7 +2642,7 @@ default Bias.

    addto

    -class paddle.v2.layer.addto(*args, **kwargs)
    +class paddle.v2.layer.addto

    AddtoLayer.

    \[y = f(\sum_{i} x_i + b)\]
    @@ -2211,32 +2650,30 @@ default Bias. and \(f\) is activation function.

    The example usage is:

    addto = addto(input=[layer1, layer2],
-                    act=paddle.v2.Activation.Relu(),
+                    act=paddle.v2.activation.Relu(),
                     bias_attr=False)
    -

    This layer just simply add all input layers together, then activate the sum -inputs. Each input of this layer should be the same size, which is also the -output size of this layer.

    +

    This layer just simply adds all input layers together, then activates the +sum. All inputs should share the same dimension, which is also the dimension +of this layer’s output.

    There is no weight matrix for each input, because it just a simple add operation. If you want a complicated operation before add, please use mixed.

    -

    It is a very good way to set dropout outside the layers. Since not all -PaddlePaddle layer support dropout, you can add an add_to layer, set -dropout here. -Please refer to dropout for details.

    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input layer
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • expand_as (paddle.v2.config_base.Layer) – Expand as this layer’s sequence info.
      • -
    • name (basestring) – Layer name.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute|None|False) – Bias attribute. None means default bias. False means no -bias.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
    • expand_level (ExpandLevel) – whether input layer is timestep(default) or sequence.
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
    @@ -2085,11 +2512,14 @@ bias.

    repeat

    -class paddle.v2.layer.repeat(*args, **kwargs)
    -

    A layer for repeating the input for num_repeats times. This is equivalent -to apply concat() with num_repeats same input.

    +class paddle.v2.layer.repeat +

    A layer for repeating the input for num_repeats times.

    +

    If as_row_vector:

    +
    +\[y = [x_1,\cdots, x_n, \cdots, x_1, \cdots, x_n]\]
    +

    If not as_row_vector:

    -\[y = [x, x, \cdots, x]\]
    +\[y = [x_1,\cdots, x_1, \cdots, x_n, \cdots, x_n]\]

    The example usage is:

    expand = repeat(input=layer, num_repeats=4)
    @@ -2099,10 +2529,16 @@ to apply concat() with num_repeats same input.
    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input layer
      • -
    • num_repeats (int) – Repeat the input so many times
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • num_repeats (int) – The times of repeating the input.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • as_row_vector (bool) – Whether to treat the input as row vectors or not. If +the parameter is set to True, the repeating operation +will be performed in the column direction. Otherwise, +it will be performed in the row direction.
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Identity is the default activation.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • -
    • height (int) – The height of the sample matrix
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • height (int) – The height of the sample matrix.
      • +
    • width (int) – The width of the sample matrix.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • -
    • reshape_size (int) – the size of reshaped sequence.
      • -
    • name (basestring) – Layer name.
      • -
    • act (paddle.v2.Activation.Base) – Activation type.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or -something not type of paddle.v2.attr.ParameterAttribute. None will get a -default Bias.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • reshape_size (int) – The dimension of the reshaped sequence.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Identity is the default activation.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
    @@ -2255,7 +2692,7 @@ bias.

    linear_comb

    -class paddle.v2.layer.linear_comb(*args, **kwargs)
    +class paddle.v2.layer.linear_comb
    A layer for weighted sum of vectors takes two inputs.
      @@ -2297,9 +2734,10 @@ processed in one batch.

    @@ -2318,8 +2756,8 @@ processed in one batch.

    interpolation

    -class paddle.v2.layer.interpolation(*args, **kwargs)
    -

    This layer is for linear interpolation with two inputs, +class paddle.v2.layer.interpolation +

    This layer performs linear interpolation on two inputs, which is used in NEURAL TURING MACHINE.

    \[y.row[i] = w[i] * x_1.row[i] + (1 - w[i]) * x_2.row[i]\]
    @@ -2335,10 +2773,11 @@ which is used in NEURAL TURING MACHINE.

    @@ -2357,8 +2796,8 @@ which is used in NEURAL TURING MACHINE.

    bilinear_interp

    -class paddle.v2.layer.bilinear_interp(*args, **kwargs)
    -

    This layer is to implement bilinear interpolation on conv layer output.

    +class paddle.v2.layer.bilinear_interp +

    This layer implements bilinear interpolation on convolutional layer’s output.

    Please refer to Wikipedia: https://en.wikipedia.org/wiki/Bilinear_interpolation

    The simple usage is:

    bilinear = bilinear_interp(input=layer1, out_size_x=64, out_size_y=64)
@@ -2369,11 +2808,12 @@ which is used in NEURAL TURING MACHINE.
    @@ -2388,17 +2828,86 @@ which is used in NEURAL TURING MACHINE.

    -
    -

    power

    +
    +

    dot_prod

    -class paddle.v2.layer.power(*args, **kwargs)
    -

    This layer applies a power function to a vector element-wise, +class paddle.v2.layer.dot_prod +

    A layer for computing the dot product of two vectors.

    +

    The example usage is:

    +
    dot_prod = dot_prod(input1=vec1, input2=vec2)
+
    +
    +
    Parameters:
      -
    • name (basestring) – Layer name.
      • -
    • input (paddle.v2.config_base.Layer|list|tuple) – Input layers. It could be a paddle.v2.config_base.Layer or list/tuple of +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer | list | tuple) – The input layers. It could be a paddle.v2.config_base.Layer or list/tuple of paddle.v2.config_base.Layer.
      • -
    • act (paddle.v2.Activation.Base) – Activation Type, default is tanh.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute|bool) – Bias attribute. If False, means no bias. None is default -bias.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer attribute.
      • +
    • act (paddle.v2.activation.Base) – Activation Type. paddle.v2.activation.Linear is the default activation.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    Parameters:
    • weights (paddle.v2.config_base.Layer) – The weight layer.
    • vectors (paddle.v2.config_base.Layer) – The vector layer.
      • -
    • size (int) – the dimension of this layer.
      • -
    • name (basestring) – The Layer Name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • +
    • size (int) – The dimension of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    Parameters:
      -
    • input (list|tuple) – Input layer.
      • +
    • input (list | tuple) – The input of this layer.
    • weight (paddle.v2.config_base.Layer) – Weight layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    Parameters:
      -
    • input (paddle.v2.config_base.Layer.) – A input layer.
      • -
    • out_size_x (int|None) – bilinear interpolation output width.
      • -
    • out_size_y (int|None) – bilinear interpolation output height.
      • -
    • name (None|basestring) – The layer’s name, which cna not be specified.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer attribute.
      • +
    • input (paddle.v2.config_base.Layer.) – The input of this layer.
      • +
    • out_size_x (int) – The width of the output.
      • +
    • out_size_y (int) – The height of the output.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    +++ + + + + + + + +
    Parameters:
      +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input1 (paddle.v2.config_base.Layer) – The first input layer.
      • +
    • input2 (paddle.v2.config_base.Layer) – The second input layer.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    +
    Returns:

    paddle.v2.config_base.Layer object.

    +
    Return type:

    paddle.v2.config_base.Layer

    +
    +
    + +
    +
    +

    out_prod

    +
    +
    +class paddle.v2.layer.out_prod
    +

    A layer for computing the outer product of two vectors +The result is a matrix of size(input1) x size(input2)

    +

    The example usage is:

    +
    out_prod = out_prod(input1=vec1, input2=vec2)
+
    +
    + +++ + + + + + + + +
    Parameters:
      +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input1 – The first input layer.
      • +
    • input2 (paddle.v2.config_base.Layer) – The second input layer.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    +
    Returns:

    paddle.v2.config_base.Layer object.

    +
    Return type:

    paddle.v2.config_base.Layer

    +
    +
    + +
    +
    +

    power

    +
    +
    +class paddle.v2.layer.power
    +

    This layer applies a power function to a vector element-wise, which is used in NEURAL TURING MACHINE.

    \[y = x^w\]
    -

    where \(x\) is a input vector, \(w\) is scalar weight, -and \(y\) is a output vector.

    +

    where \(x\) is an input vector, \(w\) is a scalar exponent, +and \(y\) is an output vector.

    The example usage is:

    power = power(input=layer1, weight=layer2)
    @@ -2408,10 +2917,11 @@ and \(y\) is a output vector.

    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • -
    • weight (paddle.v2.config_base.Layer) – Weight layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • weight (paddle.v2.config_base.Layer) – The exponent of the power.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    @@ -2430,7 +2940,7 @@ and \(y\) is a output vector.

    scaling

    -class paddle.v2.layer.scaling(*args, **kwargs)
    +class paddle.v2.layer.scaling

    A layer for multiplying input vector by weight scalar.

    \[y = w x\]
    @@ -2447,10 +2957,11 @@ processed in one batch.

    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • -
    • weight (paddle.v2.config_base.Layer) – Weight layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • weight (paddle.v2.config_base.Layer) – The weight of each sample.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    @@ -2464,14 +2975,89 @@ processed in one batch.

    +
    +
    +

    clip

    +
    +
    +class paddle.v2.layer.clip
    +
    +

    A layer for clipping the input value by the threshold.

    +
    +\[out[i] = \min\left(\max\left(in[i],p_{1}\]
    +
    +

    ight),p_{2} +ight)

    +
    +
    clip = clip(input=input, min=-10, max=10)
+
    +
    + +++ + + + + + + + + + + + + + + + + + + + + + +
    param name:The name of this layer. It is optional.
    type name:basestring
    param input:The input of this layer.
    type input:paddle.v2.config_base.Layer.
    param min:The lower threshold for clipping.
    type min:float
    param max:The upper threshold for clipping.
    type max:float
    return:paddle.v2.config_base.Layer object.
    rtype:paddle.v2.config_base.Layer
    +
    +
    + +
    +
    +

    resize

    +
    +
    +class paddle.v2.layer.resize
    +

    The resize layer resizes the input matrix with a shape of [Height, Width] +into the output matrix with a shape of [Height x Width / size, size], +where size is the parameter of this layer indicating the output dimension.

    + +++ + + + + + + + +
    Parameters:
      +
    • input (paddle.v2.config_base.Layer.) – The input of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • size (int) – The resized output dimension of this layer.
      • +
    +
    Returns:

    A paddle.v2.config_base.Layer object.

    +
    Return type:

    paddle.v2.config_base.Layer

    +
    +
    +

    slope_intercept

    -class paddle.v2.layer.slope_intercept(*args, **kwargs)
    -

    This layer for applying a slope and an intercept to the input -element-wise. There is no activation and weight.

    +class paddle.v2.layer.slope_intercept +

    This layer for applying a slope and an intercept to the input.

    \[y = slope * x + intercept\]

    The simple usage is:

    @@ -2483,11 +3069,12 @@ element-wise. There is no activation and weight.

    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – The input layer.
      • -
    • name (basestring) – The Layer Name.
      • -
    • slope (float.) – the scale factor.
      • -
    • intercept (float.) – the offset.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • slope (float) – The scale factor.
      • +
    • intercept (float) – The offset.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    @@ -2506,9 +3093,9 @@ element-wise. There is no activation and weight.

    tensor

    -class paddle.v2.layer.tensor(*args, **kwargs)
    -

    This layer performs tensor operation for two input. -For example, each sample:

    +class paddle.v2.layer.tensor +

    This layer performs tensor operation on two inputs. +For example:

    \[y_{i} = a * W_{i} * {b^\mathrm{T}}, i=0,1,...,K-1\]
    @@ -2531,16 +3118,19 @@ For example, each sample:

    Parameters:
      -
    • name (basestring) – layer name
      • -
    • a (paddle.v2.config_base.Layer) – Input layer a.
      • -
    • b (paddle.v2.config_base.Layer) – input layer b.
      • -
    • size (int.) – the layer dimension.
      • -
    • act (paddle.v2.Activation.Base) – Activation Type. Default is tanh.
      • -
    • param_attr (paddle.v2.attr.ParameterAttribute) – The Parameter Attribute.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute|None|Any) – The Bias Attribute. If no bias, then pass False or -something not type of paddle.v2.attr.ParameterAttribute. None will get a -default Bias.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • a (paddle.v2.config_base.Layer) – The first input of this layer.
      • +
    • b (paddle.v2.config_base.Layer) – The second input of this layer.
      • +
    • size (int) – The dimension of this layer.
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Linear is the default activation.
      • +
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for +details.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The parameter attribute for bias. If this parameter is set to +False or an object whose type is not paddle.v2.attr.ParameterAttribute, +no bias is defined. If this parameter is set to True, +the bias is initialized to zero.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute | None) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    @@ -2559,7 +3149,7 @@ default Bias.

    cos_sim

    -class paddle.v2.layer.cos_sim(*args, **kwargs)
    +class paddle.v2.layer.cos_sim

    Cosine Similarity Layer. The cosine similarity equation is here.

    \[similarity = cos(\theta) = {\mathbf{a} \cdot \mathbf{b} @@ -2578,12 +3168,12 @@ processed in one batch.

    Parameters:
      -
    • name (basestring) – layer name
      • -
    • a (paddle.v2.config_base.Layer) – input layer a
      • -
    • b (paddle.v2.config_base.Layer) – input layer b
      • -
    • scale (float) – scale for cosine value. default is 5.
      • -
    • size (int) – layer size. NOTE size_a * size should equal size_b.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • a (paddle.v2.config_base.Layer) – The first input of this layer.
      • +
    • b (paddle.v2.config_base.Layer) – The second input of this layer.
      • +
    • scale (float) – The scale of the cosine similarity. 1 is the default value.
      • +
    • size (int) – The dimension of this layer. NOTE size_a * size should equal size_b.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for details.
    @@ -2597,12 +3187,55 @@ processed in one batch.

    +
    +
    +

    l2_distance

    +
    +
    +class paddle.v2.layer.l2_distance
    +

    This layer calculates and returns the Euclidean distance between two input +vectors x and y. The equation is as follows:

    +
    +\[l2_distance(\mathbf{x}, \mathbf{y}) = \sqrt{\sum_{i=1}^D(x_i - y_i)}\]
    +

    The output size of this layer is fixed to be 1. Note that the above +computation is for one sample. Multiple samples are processed in one batch.

    +

    The example usage is:

    +
    l2_sim = l2_distance(x=layer1, y=layer2)
+
    +
    + +++ + + + + + + + +
    Parameters:
      +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • x (paddle.v2.config_base.Layer) – The first input x for this layer, whose output is a matrix with +dimensionality N x D. N is the sample number in a mini-batch. +D is the dimensionality of x’s output.
      • +
    • y (paddle.v2.config_base.Layer) – The second input y for this layer, whose output is a matrix with +dimensionality N x D. N is the sample number in a mini-batch. +D is the dimensionality of y’s output.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attributes, for example, drop rate. +See paddle.v2.attr.ExtraAttribute for more details.
      • +
    +
    Returns:

    The returned paddle.v2.config_base.Layer object.

    +
    Return type:

    paddle.v2.config_base.Layer

    +
    +
    +

    trans

    -class paddle.v2.layer.trans(*args, **kwargs)
    +class paddle.v2.layer.trans

    A layer for transposing a minibatch matrix.

    \[y = x^\mathrm{T}\]
    @@ -2616,9 +3249,51 @@ processed in one batch.

    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    + + +Returns:

    paddle.v2.config_base.Layer object.

    + + +Return type:

    paddle.v2.config_base.Layer

    + + + + +
    + +
    +
    +

    scale_shift

    +
    +
    +class paddle.v2.layer.scale_shift
    +

    A layer applies a linear transformation to each element in each row of +the input matrix. For each element, the layer first re-scales it and then +adds a bias to it.

    +

    This layer is very like the SlopeInterceptLayer, except the scale and +bias are trainable.

    +
    +\[y = w * x + b\]
    +
    scale_shift = scale_shift(input=input, bias_attr=False)
+
    +
    + +++ + @@ -2640,7 +3315,7 @@ processed in one batch.

    maxid

    -class paddle.v2.layer.max_id(*args, **kwargs)
    +class paddle.v2.layer.max_id

    A layer for finding the id which has the maximal value for each sample. The result is stored in output.ids.

    The example usage is:

    @@ -2652,9 +3327,10 @@ The result is stored in output.ids.

    @@ -2673,8 +3349,8 @@ The result is stored in output.ids.

    sampling_id

    -class paddle.v2.layer.sampling_id(*args, **kwargs)
    -

    A layer for sampling id from multinomial distribution from the input layer. +class paddle.v2.layer.sampling_id +

    A layer for sampling id from a multinomial distribution from the input layer. Sampling one id for one sample.

    The simple usage is:

    samping_id = sampling_id(input=input)
@@ -2685,9 +3361,10 @@ Sampling one id for one sample.
    @@ -2701,6 +3378,122 @@ Sampling one id for one sample.

    Parameters:
      +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute of scaling. See paddle.v2.attr.ParameterAttribute for +details.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – Input layer name.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    Parameters:
      -
    • input (paddle.v2.config_base.Layer) – The input layer.
      • -
    • name (basestring) – The Layer Name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    +
    +
    +

    multiplex

    +
    +
    +class paddle.v2.layer.multiplex
    +

    This layer multiplex multiple layers according to the indexes, +which are provided by the first input layer. +inputs[0]: the indexes of the layers to form the output of size batchSize. +inputs[1:N]; the candidate output data. +For each index i from 0 to batchSize - 1, the i-th row of the output is the +the same to the i-th row of the (index[i] + 1)-th layer.

    +

    For each i-th row of output: +.. math:

    +
    y[i][j] = x_{x_{0}[i] + 1}[i][j], j = 0,1, ... , (x_{1}.width - 1)
+
    +
    +

    where, y is output. \(x_{k}\) is the k-th input layer and +\(k = x_{0}[i] + 1\).

    +

    The example usage is:

    +
    maxid = multiplex(input=layers)
+
    +
    + +++ + + + + + + + +
    Parameters:
      +
    • input (list of paddle.v2.config_base.Layer) – Input layers.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    +
    Returns:

    paddle.v2.config_base.Layer object.

    +
    Return type:

    paddle.v2.config_base.Layer

    +
    +
    + +
    +
    +
    +

    Factorization Machine Layer

    +
    +

    factorization_machine

    +
    +
    +class paddle.v2.layer.factorization_machine
    +
    +

    The Factorization Machine models pairwise feature interactions as inner +product of the learned latent vectors corresponding to each input feature. +The Factorization Machine can effectively capture feature interactions +especially when the input is sparse.

    +

    This implementation only consider the 2-order feature interactions using +Factorization Machine with the formula:

    +
    +\[y = \sum_{i=1}^{n-1}\sum_{j=i+1}^n\langle v_i, v_j\]
    +
    +

    angle x_i x_j

    +
    +
    +
    Note:
    +
    X is the input vector with size n. V is the factor matrix. Each row of V +is the latent vector corresponding to each input dimesion. The size of +each latent vector is k.
    +
    +

    For details of Factorization Machine, please refer to the paper: +Factorization machines.

    + +++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    param input:The input layer. Supported input types: all input data types +on CPU, and only dense input types on GPU.
    type input:paddle.v2.config_base.Layer
    param factor_size:
     The hyperparameter that defines the dimensionality of +the latent vector size.
    type context_len:
     int
    param act:Activation Type. Default is linear activation.
    type act:paddle.v2.activation.Base
    param param_attr:
     The parameter attribute. See paddle.v2.attr.ParameterAttribute for +details.
    type param_attr:
     paddle.v2.attr.ParameterAttribute
    param layer_attr:
     Extra Layer config.
    type layer_attr:
     paddle.v2.attr.ExtraAttributeNone
    return:paddle.v2.config_base.Layer object.
    rtype:paddle.v2.config_base.Layer
    +
    +
    +
    @@ -2709,14 +3502,14 @@ Sampling one id for one sample.

    pad

    -class paddle.v2.layer.pad(*args, **kwargs)
    +class paddle.v2.layer.pad

    This operation pads zeros to the input data according to pad_c,pad_h -and pad_w. pad_c, pad_h, pad_w specifies the which dimension and size -of padding. And the input data shape is NCHW.

    -

    For example, pad_c=[2,3] means padding 2 zeros before the -input data and 3 zeros after the input data in channel dimension. -pad_h means padding zeros in height dimension. pad_w means padding zeros -in width dimension.

    +and pad_w. pad_c, pad_h, pad_w specify the size in the corresponding +dimension. And the input data shape is NCHW.

    +

    For example, pad_c=[2,3] means padding 2 zeros before the input data +and 3 zeros after the input data in the channel dimension. pad_h means +padding zeros in the height dimension. pad_w means padding zeros in the +width dimension.

    For example,

    input(2,2,2,3)  = [
                     [ [[1,2,3], [3,4,5]],
@@ -2751,12 +3544,13 @@ in width dimension.
    
 
 
 Parameters:
      
-
    • input (paddle.v2.config_base.Layer) – layer’s input.
      • 
-
    • pad_c (list|None) – padding size in channel dimension.
      • 
-
    • pad_h (list|None) – padding size in height dimension.
      • 
-
    • pad_w (list|None) – padding size in width dimension.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
-
    • name (basestring) – layer name.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • pad_c (list | None) – The padding size in the channel dimension.
      • 
+
    • pad_h (list | None) – The padding size in the height dimension.
      • 
+
    • pad_w (list | None) – The padding size in the width dimension.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
 
    
 
 
@@ -2778,8 +3572,9 @@ in width dimension.
    
 
    cross_entropy_cost
    
 
    
 
    
-class paddle.v2.layer.cross_entropy_cost(*args, **kwargs)
    
+class paddle.v2.layer.cross_entropy_cost
 
    A loss layer for multi class entropy.
    
+
    The example usage is:
    
 
    cost = cross_entropy(input=input,
                      label=label)
 
    
@@ -2789,22 +3584,22 @@ in width dimension.
    
 
 
 Parameters:
      
-
    • input (paddle.v2.config_base.Layer.) – The first input layer.
      • 
+
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
 
    • label – The input label.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float.) – The cost is multiplied with coeff.
-The coefficient affects the gradient in the backward.
      • 
-
    • weight (LayerOutout) – The cost of each sample is multiplied with each weight.
-The weight should be a layer with size=1. Note that gradient
-will not be calculated for weight.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • weight (LayerOutout) – The weight layer defines a weight for each sample in the
+mini-batch. It is optional.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
 Returns:
    paddle.v2.config_base.Layer object.
    
 
 
-Return type:
    paddle.v2.config_base.Layer.
    
+Return type:
    paddle.v2.config_base.Layer
    
 
 
 
@@ -2816,9 +3611,10 @@ will not be calculated for weight.
 
    cross_entropy_with_selfnorm_cost
    
 
    
 
    
-class paddle.v2.layer.cross_entropy_with_selfnorm_cost(*args, **kwargs)
    
+class paddle.v2.layer.cross_entropy_with_selfnorm_cost
 
    A loss layer for multi class entropy with selfnorm.
 Input should be a vector of positive numbers, without normalization.
    
+
    The example usage is:
    
 
    cost = cross_entropy_with_selfnorm(input=input,
                                    label=label)
 
    
@@ -2828,19 +3624,21 @@ Input should be a vector of positive numbers, without normalization.
    
 
 
 Parameters:
      
-
    • input (paddle.v2.config_base.Layer.) – The first input layer.
      • 
+
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
 
    • label – The input label.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float.) – The coefficient affects the gradient in the backward.
      • 
-
    • softmax_selfnorm_alpha (float.) – The scale factor affects the cost.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • softmax_selfnorm_alpha (float) – The scale factor affects the cost.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
 Returns:
    paddle.v2.config_base.Layer object.
    
 
 
-Return type:
    paddle.v2.config_base.Layer.
    
+Return type:
    paddle.v2.config_base.Layer
    
 
 
 
@@ -2852,8 +3650,9 @@ Input should be a vector of positive numbers, without normalization.
    
 
    multi_binary_label_cross_entropy_cost
    
 
    
 
    
-class paddle.v2.layer.multi_binary_label_cross_entropy_cost(*args, **kwargs)
    
+class paddle.v2.layer.multi_binary_label_cross_entropy_cost
 
    A loss layer for multi binary label cross entropy.
    
+
    The example usage is:
    
 
    cost = multi_binary_label_cross_entropy(input=input,
                                         label=label)
 
    
@@ -2865,10 +3664,11 @@ Input should be a vector of positive numbers, without normalization.
    
 Parameters:
      
 
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
 
    • label – The input label.
      • 
-
    • type (basestring) – The type of cost.
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float) – The coefficient affects the gradient in the backward.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -2883,37 +3683,123 @@ Input should be a vector of positive numbers, without normalization.
    
 
    
 
 
    
-
    
-
    huber_cost
    
+
    
+
    huber_regression_cost
    
 
    
 
    
-class paddle.v2.layer.huber_cost(*args, **kwargs)
    
-
    A loss layer for huber loss.
    
-
    cost = huber_cost(input=input,
-                  label=label)
+class paddle.v2.layer.huber_regression_cost
+
    
+
    In statistics, the Huber loss is a loss function used in robust regression,
+that is less sensitive to outliers in data than the squared error loss.
+Given a prediction f(x), a label y and \(\delta\), the loss function
+is defined as:
    
+
    ight )^2, left | y-f(x)
+ight |leq delta
    
+
    
+
    loss = delta left | y-f(x)
    
+
    ight |-0.5delta ^2, otherwise
    
+
    
+
    The example usage is:
    
+
    cost = huber_regression_cost(input=input, label=label)
 
    
 
    
 
-
+
-
+
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
    
 
 
 
    Parameters:
      
-
    • input (paddle.v2.config_base.Layer.) – The first input layer.
      • 
-
    • label – The input label.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float.) – The coefficient affects the gradient in the backward.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
-
    
-
    param input:The first input layer.
 
    Returns:
    paddle.v2.config_base.Layer object.
    
-
    type input:paddle.v2.config_base.Layer
 
    Return type:
    paddle.v2.config_base.Layer.
    
-
    param label:The input label.
    type input:paddle.v2.config_base.Layer
    param name:The name of this layer. It is optional.
    type name:basestring
    param delta:The difference between the observed and predicted values.
    type delta:float
    param coeff:The weight of the gradient in the back propagation.
+1.0 is the default value.
    type coeff:float
    param layer_attr:
     The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
    type layer_attr:
     paddle.v2.attr.ExtraAttribute
    return:paddle.v2.config_base.Layer object.
    rtype:paddle.v2.config_base.Layer.
 
    
     
 
    
+
    
+
    
+
+
    
+
    
+
    huber_classification_cost
    
+
    
+
    
+class paddle.v2.layer.huber_classification_cost
    
+
    
+
    For classification purposes, a variant of the Huber loss called modified Huber
+is sometimes used. Given a prediction f(x) (a real-valued classifier score) and
+a true binary class label :math:`yin left {-1, 1
    
+
    
+
    ight }`, the modified Huber
    
+
    loss is defined as:
    
+
    ight )^2, yf(x)geq 1
    
+
    
+
    loss = -4yf(x),  ext{otherwise}
    
+
    The example usage is:
    
+
    cost = huber_classification_cost(input=input, label=label)
+
    
+
    
+
+++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
    


    param input:The first input layer.
    type input:paddle.v2.config_base.Layer
    param label:The input label.
    type input:paddle.v2.config_base.Layer
    param name:The name of this layer. It is optional.
    type name:basestring
    param coeff:The weight of the gradient in the back propagation.
+1.0 is the default value.
    type coeff:float
    param layer_attr:
     The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
    type layer_attr:
     paddle.v2.attr.ExtraAttribute
    return:paddle.v2.config_base.Layer object.
    rtype:paddle.v2.config_base.Layer
    
+
    
+
    
 
    
 
 
    
@@ -2921,9 +3807,9 @@ Input should be a vector of positive numbers, without normalization.
    
 
    lambda_cost
    
 
    
 
    
-class paddle.v2.layer.lambda_cost(*args, **kwargs)
    
+class paddle.v2.layer.lambda_cost
 
    lambdaCost for lambdaRank LTR approach.
    
-
    The simple usage:
    
+
    The example usage is:
    
 
    cost = lambda_cost(input=input,
                    score=score,
                    NDCG_num=8,
@@ -2935,20 +3821,20 @@ Input should be a vector of positive numbers, without normalization.
    
 
 
 Parameters:
      
-
    • input (paddle.v2.config_base.Layer) – Samples of the same query should be loaded as sequence.
      • 
-
    • score – The 2nd input. Score of each sample.
      • 
+
    • input (paddle.v2.config_base.Layer) – The first input of this layer, which is often a document
+samples list of the same query and whose type must be sequence.
      • 
+
    • score – The scores of the samples.
      • 
 
    • NDCG_num (int) – The size of NDCG (Normalized Discounted Cumulative Gain),
-e.g., 5 for NDCG@5. It must be less than for equal to the
-minimum size of lists.
      • 
-
    • max_sort_size (int) – The size of partial sorting in calculating gradient.
-If max_sort_size = -1, then for each list, the
-algorithm will sort the entire list to get gradient.
-In other cases, max_sort_size must be greater than or
-equal to NDCG_num. And if max_sort_size is greater
-than the size of a list, the algorithm will sort the
-entire list of get gradient.
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+e.g., 5 for NDCG@5. It must be less than or equal to the
+minimum size of the list.
+
    • max_sort_size (int) – The size of partial sorting in calculating gradient. If
+max_sort_size is equal to -1 or greater than the number
+of the samples in the list, then the algorithm will sort
+the entire list to compute the gradient. In other cases,
+max_sort_size must be greater than or equal to NDCG_num.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -2963,52 +3849,39 @@ entire list of get gradient.
 
    
 
 
    
-
    
-
    mse_cost
    
+
    
+
    square_error_cost
    
 
    
 
    
-class paddle.v2.layer.mse_cost(*args, **kwargs)
    
-
    
-
    mean squared error cost:
    
+class paddle.v2.layer.square_error_cost
+
    sum of square error cost:
    
 
    
-\[\]
    
-
    
-
    rac{1}{N}sum_{i=1}^N(t_i-y_i)^2
    
-
    
-
    
+\[cost = \sum_{i=1}^N(t_i-y_i)^2\]
+
    
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
+
-
+
    
 
 
 
    param name:layer name.
    type name:basestring
    param input:Network prediction.
    type input:paddle.v2.config_base.Layer
    param label:Data label.
    type label:paddle.v2.config_base.Layer
    param weight:The weight affects the cost, namely the scale of cost.
-It is an optional argument.
    type weight:paddle.v2.config_base.Layer
    param layer_attr:
     layer’s extra attribute.
    type layer_attr:
     paddle.v2.attr.ExtraAttribute
    Parameters:
      
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • weight (paddle.v2.config_base.Layer) – The weight layer defines a weight for each sample in the
+mini-batch. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    
+    		
 
    return:paddle.v2.config_base.Layer object.
    Returns:
    paddle.v2.config_base.Layer object.
    
+    		
 
    rtype:paddle.v2.config_base.Layer
    Return type:
    paddle.v2.config_base.Layer
    
+    		
 
    
     
 
    
-
    
 
    
 
 
    
@@ -3016,11 +3889,13 @@ It is an optional argument.
 
    rank_cost
    
 
    
 
    
-class paddle.v2.layer.rank_cost(*args, **kwargs)
    
-
    A cost Layer for learning to rank using gradient descent. Details can refer
-to papers.
-This layer contains at least three inputs. The weight is an optional
-argument, which affects the cost.
    
+class paddle.v2.layer.rank_cost
+
    A cost Layer for learning to rank using gradient descent.
    
+
    
+
    Reference:
    
+
    `Learning to Rank using Gradient Descent
+http://research.microsoft.com/en-us/um/people/cburges/papers/ICML_ranking.pdf`_
    
+
    
 
    
 \[ \begin{align}\begin{aligned}C_{i,j} & = -\tilde{P_{ij}} * o_{i,j} + log(1 + e^{o_{i,j}})\\o_{i,j} & =  o_i - o_j\\\tilde{P_{i,j}} & = \{0, 0.5, 1\} \ or \ \{0, 1\}\end{aligned}\end{align} \]
    
 
    
@@ -3034,7 +3909,7 @@ Their dimension is one.
 
 
    
 
    
-
    The simple usage:
    
+
    The example usage is:
    
 
    cost = rank_cost(left=out_left,
                  right=out_right,
                  label=label)
@@ -3048,11 +3923,13 @@ Their dimension is one.
 
  • left (paddle.v2.config_base.Layer) – The first input, the size of this layer is 1.
    • 
 
  • right (paddle.v2.config_base.Layer) – The right input, the size of this layer is 1.
    • 
 
  • label (paddle.v2.config_base.Layer) – Label is 1 or 0, means positive order and reverse order.
    • 
-
  • weight (paddle.v2.config_base.Layer) – The weight affects the cost, namely the scale of cost.
-It is an optional argument.
    • 
-
  • name (None|basestring) – The name of this layers. It is not necessary.
    • 
-
  • coeff (float) – The coefficient affects the gradient in the backward.
    • 
-
  • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
    • 
+
  • weight (paddle.v2.config_base.Layer) – The weight layer defines a weight for each sample in the
+mini-batch. It is optional.
    • 
+
  • name (basestring) – The name of this layer. It is optional.
    • 
+
  • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
    • 
+
  • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
    • 
 
 
 
@@ -3071,8 +3948,9 @@ It is an optional argument.
 
    sum_cost
    
 
    
 
    
-class paddle.v2.layer.sum_cost(*args, **kwargs)
    
-
    A loss layer which calculate the sum of the input as loss
    
+class paddle.v2.layer.sum_cost
+
    A loss layer which calculates the sum of the input as loss.
    
+
    The example usage is:
    
 
    cost = sum_cost(input=input)
 
    
 
    
@@ -3081,9 +3959,10 @@ It is an optional argument.
 
 
 Parameters:
      
-
    • input (paddle.v2.config_base.Layer.) – The first input layer.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -3102,10 +3981,10 @@ It is an optional argument.
 
    crf
    
 
    
 
    
-class paddle.v2.layer.crf(*args, **kwargs)
    
+class paddle.v2.layer.crf
 
    A layer for calculating the cost of sequential conditional random
 field model.
    
-
    The simple usage:
    
+
    The example usage is:
    
 
    crf = crf(input=input,
                 label=label,
                 size=label_dim)
@@ -3116,14 +3995,18 @@ field model.
    
 
 
 Parameters:
      
-
    • input (paddle.v2.config_base.Layer) – The first input layer is the feature.
      • 
-
    • label (paddle.v2.config_base.Layer) – The second input layer is label.
      • 
+
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
 
    • size (int) – The category number.
      • 
-
    • weight (paddle.v2.config_base.Layer) – The third layer is “weight” of each sample, which is an
-optional argument.
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter attribute. None means default attribute
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • 
+
    • weight (paddle.v2.config_base.Layer) – The weight layer defines a weight for each sample in the
+mini-batch. It is optional.
      • 
+
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for
+details.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -3142,13 +4025,13 @@ optional argument.
 
    crf_decoding
    
 
    
 
    
-class paddle.v2.layer.crf_decoding(*args, **kwargs)
    
+class paddle.v2.layer.crf_decoding
 
    A layer for calculating the decoding sequence of sequential conditional
 random field model. The decoding sequence is stored in output.ids.
-If a second input is provided, it is treated as the ground-truth label, and
-this layer will also calculate error. output.value[i] is 1 for incorrect
-decoding or 0 for correct decoding.
    
-
    The simple usage:
    
+If the input ‘label’ is provided, it is treated as the ground-truth label, and
+this layer will also calculate error. output.value[i] is 1 for an incorrect
+decoding and 0 for the correct.
    
+
    The example usage is:
    
 
    crf_decoding = crf_decoding(input=input,
                                   size=label_dim)
 
    
@@ -3159,11 +4042,13 @@ decoding or 0 for correct decoding.
    
 
 Parameters:
      
 
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
-
    • size (int) – size of this layer.
      • 
-
    • label (paddle.v2.config_base.Layer or None) – None or ground-truth label.
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter attribute. None means default attribute
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • 
+
    • size (int) – The dimension of this layer.
      • 
+
    • label (paddle.v2.config_base.Layer | None) – The input label.
      • 
+
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for
+details.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -3182,22 +4067,25 @@ decoding or 0 for correct decoding.
    
 
    ctc
    
 
    
 
    
-class paddle.v2.layer.ctc(*args, **kwargs)
    
+class paddle.v2.layer.ctc
 
    Connectionist Temporal Classification (CTC) is designed for temporal
-classication task. That is, for sequence labeling problems where the
+classication task. e.g. sequence labeling problems where the
 alignment between the inputs and the target labels is unknown.
    
-
    More details can be found by referring to Connectionist Temporal
-Classification: Labelling Unsegmented Sequence Data with Recurrent
-Neural Networks
    
+
    
+
    Reference:
    
+
    `Connectionist Temporal Classification: Labelling Unsegmented Sequence Data
+with Recurrent Neural Networks
+http://machinelearning.wustl.edu/mlpapers/paper_files/icml2006_GravesFGS06.pdf`_
    
+
    
 
    
 
    Note
    
-
    Considering the ‘blank’ label needed by CTC, you need to use
-(num_classes + 1) as the input size. num_classes is the category number.
-And the ‘blank’ is the last category index. So the size of ‘input’ layer, such as
-fc with softmax activation, should be num_classes + 1. The size of ctc
-should also be num_classes + 1.
    
+
    Considering the ‘blank’ label needed by CTC, you need to use (num_classes + 1)
+as the size of the input, where num_classes is the category number.
+And the ‘blank’ is the last category index. So the size of ‘input’ layer (e.g.
+fc with softmax activation) should be (num_classes + 1). The size of
+ctc should also be (num_classes + 1).
    
 
    
-
    The simple usage:
    
+
    The example usage is:
    
 
    ctc = ctc(input=input,
                 label=label,
                 size=9055,
@@ -3209,12 +4097,13 @@ should also be num_classes + 1.
    
 
 
 Parameters:
      
-
    • input (paddle.v2.config_base.Layer) – The input layer.
      • 
-
    • label (paddle.v2.config_base.Layer) – The data layer of label with variable length.
      • 
-
    • size (int) – category numbers + 1.
      • 
-
    • name (basestring|None) – The name of this layer
      • 
-
    • norm_by_times (bool) – Whether to normalization by times. False by default.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • size (int) – The dimension of this layer, which must be equal to (category number + 1).
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • norm_by_times (bool) – Whether to do normalization by times. False is the default.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -3233,29 +4122,32 @@ should also be num_classes + 1.
    
 
    warp_ctc
    
 
    
 
    
-class paddle.v2.layer.warp_ctc(*args, **kwargs)
    
-
    A layer intergrating the open-source warp-ctc
-<https://github.com/baidu-research/warp-ctc> library, which is used in
-Deep Speech 2: End-toEnd Speech Recognition in English and Mandarin
-<https://arxiv.org/pdf/1512.02595v1.pdf>, to compute Connectionist Temporal
-Classification (CTC) loss.
    
-
    More details of CTC can be found by referring to Connectionist Temporal
-Classification: Labelling Unsegmented Sequence Data with Recurrent
-Neural Networks
    
+class paddle.v2.layer.warp_ctc
+
    A layer intergrating the open-source warp-ctc library, which is used in
+Deep Speech 2: End-toEnd Speech Recognition in English and Mandarin, to compute Connectionist Temporal
+Classification (CTC) loss. Besides, another warp-ctc repository, which is forked from
+the official one, is maintained to enable more compiling options. During the
+building process, PaddlePaddle will clone the source codes, build and
+install it to third_party/install/warpctc directory.
    
+
    
+
    Reference:
    
+
    `Connectionist Temporal Classification: Labelling Unsegmented Sequence Data
+with Recurrent Neural Networks
+http://machinelearning.wustl.edu/mlpapers/paper_files/icml2006_GravesFGS06.pdf`_
    
+
    
 
    
 
    Note
    
 
      
-
    • Let num_classes represent the category number. Considering the ‘blank’
-label needed by CTC, you need to use (num_classes + 1) as the input
-size. Thus, the size of both warp_ctc and ‘input’ layer should
-be set to num_classes + 1.
      • 
+
    • Let num_classes represents the category number. Considering the ‘blank’
+label needed by CTC, you need to use (num_classes + 1) as the size of
+warp_ctc layer.
      • 
 
    • You can set ‘blank’ to any value ranged in [0, num_classes], which
-should be consistent as that used in your labels.
      • 
+should be consistent with those used in your labels.
 
    • As a native ‘softmax’ activation is interated to the warp-ctc library,
-‘linear’ activation is expected instead in the ‘input’ layer.
      • 
+‘linear’ activation is expected to be used instead in the ‘input’ layer.
 
    
 
    
-
    The simple usage:
    
+
    The example usage is:
    
 
    ctc = warp_ctc(input=input,
                      label=label,
                      size=1001,
@@ -3268,13 +4160,14 @@ should be consistent as that used in your labels.
 
 
 Parameters:
      
-
    • input (paddle.v2.config_base.Layer) – The input layer.
      • 
-
    • label (paddle.v2.config_base.Layer) – The data layer of label with variable length.
      • 
-
    • size (int) – category numbers + 1.
      • 
-
    • name (basestring|None) – The name of this layer, which can not specify.
      • 
-
    • blank (int) – the ‘blank’ label used in ctc
      • 
-
    • norm_by_times (bool) – Whether to normalization by times. False by default.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • size (int) – The dimension of this layer, which must be equal to (category number + 1).
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • blank (int) – The ‘blank’ label used in ctc.
      • 
+
    • norm_by_times (bool) – Whether to do normalization by times. False is the default.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -3293,12 +4186,16 @@ should be consistent as that used in your labels.
 
    nce
    
 
    
 
    
-class paddle.v2.layer.nce(*args, **kwargs)
    
-
    Noise-contrastive estimation.
-Implements the method in the following paper:
-A fast and simple algorithm for training neural probabilistic language models.
    
+class paddle.v2.layer.nce
+
    Noise-contrastive estimation.
    
+
    
+
    Reference:
    
+
    `A fast and simple algorithm for training neural probabilistic language
+models. https://www.cs.toronto.edu/~amnih/papers/ncelm.pdf`_
    
+
    
 
    The example usage is:
    
-
    cost = nce(input=layer1, label=layer2, weight=layer3,
+
    cost = nce(input=[layer1, layer2], label=layer2,
+                 param_attr=[attr1, attr2], weight=layer3,
                  num_classes=3, neg_distribution=[0.1,0.3,0.6])
 
    
 
    
@@ -3307,22 +4204,34 @@ A fast and simple algorithm for training neural probabilistic language models.
 
 Parameters:
      
-
    • name (basestring) – layer name
      • 
-
    • input (paddle.v2.config_base.Layer|list|tuple|collections.Sequence) – input layers. It could be a paddle.v2.config_base.Layer of list/tuple of paddle.v2.config_base.Layer.
      • 
-
    • label (paddle.v2.config_base.Layer) – label layer
      • 
-
    • weight (paddle.v2.config_base.Layer) – weight layer, can be None(default)
      • 
-
    • num_classes (int) – number of classes.
      • 
-
    • act (paddle.v2.Activation.Base) – Activation, default is Sigmoid.
      • 
-
    • num_neg_samples (int) – number of negative samples. Default is 10.
      • 
-
    • neg_distribution (list|tuple|collections.Sequence|None) – The distribution for generating the random negative labels.
-A uniform distribution will be used if not provided.
-If not None, its length must be equal to num_classes.
      • 
-
    • bias_attr (paddle.v2.attr.ParameterAttribute|None|False) – Bias parameter attribute. True if no bias.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input (paddle.v2.config_base.Layer | list | tuple | collections.Sequence) – The first input of this layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • weight (paddle.v2.config_base.Layer) – The weight layer defines a weight for each sample in the
+mini-batch. It is optional.
      • 
+
    • num_classes (int) – The number of classes.
      • 
+
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Sigmoid is the default activation.
      • 
+
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for
+details.
      • 
+
    • num_neg_samples (int) – The number of sampled negative labels. 10 is the
+default value.
      • 
+
    • neg_distribution (list | tuple | collections.Sequence | None) – The discrete noisy distribution over the output
+space from which num_neg_samples negative labels
+are sampled. If this parameter is not set, a
+uniform distribution will be used. A user-defined
+distribution is a list whose length must be equal
+to the num_classes. Each member of the list defines
+the probability of a class given input x.
      • 
+
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The parameter attribute for bias. If this parameter is set to
+False or an object whose type is not paddle.v2.attr.ParameterAttribute,
+no bias is defined. If this parameter is set to True,
+the bias is initialized to zero.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
-Returns:
    layer name.
    
+Returns:
    paddle.v2.config_base.Layer object.
    
 
 
 Return type:
    paddle.v2.config_base.Layer
    
@@ -3337,11 +4246,13 @@ If not None, its length must be equal to num_classes.
 
    hsigmoid
    
 
    
 
    
-class paddle.v2.layer.hsigmoid(*args, **kwargs)
    
+class paddle.v2.layer.hsigmoid
 
    Organize the classes into a binary tree. At each node, a sigmoid function
-is used to calculate the probability of belonging to the right branch.
-This idea is from “F. Morin, Y. Bengio (AISTATS 05):
-Hierarchical Probabilistic Neural Network Language Model.”
    
+is used to calculate the probability of belonging to the right branch.
    
+
    
+
    Reference:
    
+
    Hierarchical Probabilistic Neural Network Language Model
    
+
    
 
    The example usage is:
    
 
    cost = hsigmoid(input=[layer1, layer2],
                 label=data)
@@ -3352,15 +4263,65 @@ Hierarchical Probabilistic Neural Network Language Model.”
    
 
 
 Parameters:
      
-
    • input (paddle.v2.config_base.Layer|list|tuple) – Input layers. It could be a paddle.v2.config_base.Layer or list/tuple of
-paddle.v2.config_base.Layer.
      • 
-
    • label (paddle.v2.config_base.Layer) – Label layer.
      • 
-
    • num_classes (int|None) – number of classes.
      • 
-
    • name (basestring) – layer name
      • 
-
    • bias_attr (paddle.v2.attr.ParameterAttribute|False) – Bias attribute. None means default bias.
-False means no bias.
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute|None) – Parameter Attribute. None means default parameter.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • input (paddle.v2.config_base.Layer | list | tuple) – The input of this layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • num_classes (int) – The number of classes. And it should be larger than 2. If the parameter
+is not set or set to None, its actual value will be automatically set to
+the number of labels.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object
+whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the
+parameter is set to True, the bias is initialized to zero.
      • 
+
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for details.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for details.
      • 
+
    
+
+
+Returns:
    paddle.v2.config_base.Layer object.
    
+
+
+Return type:
    paddle.v2.config_base.Layer
    
+
+
+
+
+
    
+
+
    
+
    
+
    smooth_l1_cost
    
+
    
+
    
+class paddle.v2.layer.smooth_l1_cost
    
+
    This is a L1 loss but more smooth. It requires that the
+sizes of input and label are equal. The formula is as follows,
    
+
    
+\[L = \sum_{i} smooth_{L1}(input_i - label_i)\]
    
+
    in which
    
+
    
+\[\begin{split}smooth_{L1}(x) = \begin{cases} 0.5x^2& \text{if}  \ |x| < 1 \\ |x|-0.5& \text{otherwise} \end{cases}\end{split}\]
    
+
    
+
    Reference:
    
+
    `Fast R-CNN
+https://arxiv.org/pdf/1504.08083v2.pdf`_
    
+
    
+
    The example usage is:
    
+
    cost = smooth_l1_cost(input=input,
+                      label=label)
+
    
+
    
+
+++
+
@@ -3374,6 +4335,38 @@ False means no bias.
    


    Parameters:
      
+
    • input (paddle.v2.config_base.Layer) – The input layer.
      • 
+
    • label – The input label.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
     		
 
    
 
    
 
    
 
+
    
+
    
+
    multibox_loss
    
+
    
+
    
+class paddle.v2.layer.multibox_loss
    
+
    Compute the location loss and the confidence loss for ssd.
    
+
+++
+
+
+
+
+
+
    


    Parameters:
      
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input_loc (paddle.v2.config_base.Layer | List of paddle.v2.config_base.Layer) – The input predict locations.
      • 
+
    • input_conf (paddle.v2.config_base.Layer | List of paddle.v2.config_base.Layer) – The input priorbox confidence.
      • 
+
    • priorbox (paddle.v2.config_base.Layer) – The input priorbox location and the variance.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • num_classes (int) – The number of the classification.
      • 
+
    • overlap_threshold (float) – The threshold of the overlap.
      • 
+
    • neg_pos_ratio (float) – The ratio of the negative bbox to the positive bbox.
      • 
+
    • neg_overlap (float) – The negative bbox overlap threshold.
      • 
+
    • background_id (int) – The background class index.
      • 
+
    
+
    Returns:
    paddle.v2.config_base.Layer
    
+
    
+
    
+
 
    
 
    
 
    
@@ -3382,7 +4375,7 @@ False means no bias.
 
    eos
    
 
    
 
    
-class paddle.v2.layer.eos(*args, **kwargs)
    
+class paddle.v2.layer.eos
 
    A layer for checking EOS for each sample:
 - output_id = (input_id == conf.eos_id)
    
 
    The result is stored in output_.ids.
@@ -3396,10 +4389,11 @@ It is used by recurrent layer group.
    
 
 
 Parameters:
      
-
    • name (basestring) – Layer name.
      • 
-
    • input (paddle.v2.config_base.Layer) – Input layer name.
      • 
-
    • eos_id (int) – end id of sequence
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • eos_id (int) – End id of sequence
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -3413,6 +4407,193 @@ It is used by recurrent layer group.
    
 
 
    
 
+
    
+
    
+
    
+
    Miscs
    
+
    
+
    dropout
    
+
    
+
    
+class paddle.v2.layer.dropout
    
+
    The example usage is:
    
+
    dropout = dropout(input=input, dropout_rate=0.5)
+
    
+
    
+
+++
+
+
+
+
+
+
+
+
    


    Parameters:
      
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • dropout_rate (float) – The probability of dropout.
      • 
+
    
+
    Returns:
    paddle.v2.config_base.Layer object.
    
+
    Return type:
    paddle.v2.config_base.Layer
    
+
    
+
    
+
+
    
+
    
+
    
+
    Activation with learnable parameter
    
+
    
+
    prelu
    
+
    
+
    
+class paddle.v2.layer.prelu
    
+
    The Parametric Relu activation that actives outputs with a learnable weight.
    
+
    
+
    Reference:
    
+
    `Delving Deep into Rectifiers: Surpassing Human-Level Performance on
+ImageNet Classification http://arxiv.org/pdf/1502.01852v1.pdf`_
    
+
    
+
    
+\[\begin{split}z_i &\quad if \quad z_i > 0 \\
+a_i * z_i  &\quad \mathrm{otherwise}\end{split}\]
    
+
    The example usage is:
    
+
    prelu = prelu(input=layers, partial_sum=1)
+
    
+
    
+
+++
+
+
+
+
+
+
+
+
    


    Parameters:
      
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • partial_sum (int) – 
      this parameter makes a group of inputs share the same weight.
      
+
        
+
      • partial_sum = 1, indicates the element-wise activation: each element has a weight.
        • 
+
      • partial_sum = number of elements in one channel, indicates the channel-wise activation, elements in a channel share the same weight.
        • 
+
      • partial_sum = number of outputs, indicates all elements share the same weight.
        • 
+
      
+
      • 
+
    • channel_shared (bool) – 
      whether or not the parameter are shared across channels.
      
+
        
+
      • channel_shared = True, we set the partial_sum to the number of outputs.
        • 
+
      • channel_shared = False, we set the partial_sum to the number of elements in one channel.
        • 
+
      
+
      • 
+
    • num_channels (int) – number of input channel.
      • 
+
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for details.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute | None) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    
+
    Returns:
    paddle.v2.config_base.Layer object.
    
+
    Return type:
    paddle.v2.config_base.Layer
    
+
    
+
    
+
+
    
+
    
+
    gated_unit
    
+
    
+
    
+class paddle.v2.layer.gated_unit
    
+
    The gated unit layer implements a simple gating mechanism over the input.
+The input \(X\) is first projected into a new space \(X'\), and
+it is also used to produce a gate weight \(\sigma\). Element-wise
+product between :match:`X’` and \(\sigma\) is finally returned.
    
+
    
+
    Reference:
    
+
    `Language Modeling with Gated Convolutional Networks
+https://arxiv.org/abs/1612.08083`_
    
+
    
+
    
+\[y=\text{act}(X \cdot W + b)\otimes \sigma(X \cdot V + c)\]
    
+
    The example usage is:
    
+
+++
+
+
+
+
+
+
+
+
    


    Parameters:
      
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • size (int) – The dimension of this layer’s output.
      • 
+
    • act (paddle.v2.activation.Base) – Activation type of the projection. paddle.v2.activation.Linear is the default
+activation.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • gate_attr (paddle.v2.attr.ExtraAttribute | None) – The extra layer attribute of the gate. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    • gate_param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute of the gate. See paddle.v2.attr.ParameterAttribute
+for details.
      • 
+
    • gate_bias_attr (paddle.v2.attr.ParameterAttribute | bool | None | Any) – The bias attribute of the gate. If this parameter is set to False or
+an object whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined.
+If this parameter is set to True, the bias is initialized to zero.
      • 
+
    • inproj_attr (paddle.v2.attr.ExtraAttribute | None) – Extra layer attributes of the projection. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    • inproj_param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute of the projection. See paddle.v2.attr.ParameterAttribute
+for details.
      • 
+
    • inproj_bias_attr (paddle.v2.attr.ParameterAttribute | bool | None | Any) – The bias attribute of the projection. If this parameter is set to False
+or an object whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined.
+If this parameter is set to True, the bias is initialized to zero.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute | None) – Extra layer attribute of the product. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    
+
    Returns:
    paddle.v2.config_base.Layer object.
    
+
    Return type:
    paddle.v2.config_base.Layer
    
+
    
+
    
+
+
    
+
    
+
    
+
    Detection output Layer
    
+
    
+
    detection_output
    
+
    
+
    
+class paddle.v2.layer.detection_output
    
+
    Apply the NMS to the output of network and compute the predict bounding
+box location. The output’s shape of this layer could be zero if there is
+no valid bounding box.
    
+
+++
+
+
+
+
+
+
    


    Parameters:
      
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input_loc (paddle.v2.config_base.Layer | List of paddle.v2.config_base.Layer.) – The input predict locations.
      • 
+
    • input_conf (paddle.v2.config_base.Layer | List of paddle.v2.config_base.Layer.) – The input priorbox confidence.
      • 
+
    • priorbox (paddle.v2.config_base.Layer) – The input priorbox location and the variance.
      • 
+
    • num_classes (int) – The number of the classification.
      • 
+
    • nms_threshold (float) – The Non-maximum suppression threshold.
      • 
+
    • nms_top_k (int) – The bbox number kept of the NMS’s output
      • 
+
    • keep_top_k (int) – The bbox number kept of the layer’s output
      • 
+
    • confidence_threshold (float) – The classification confidence threshold
      • 
+
    • background_id (int) – The background class index.
      • 
+
    
+
    Returns:
    paddle.v2.config_base.Layer
    
+
    
+
    
+
 
    
 
    
 
    
@@ -3424,7 +4605,7 @@ It is used by recurrent layer group.
    
   
     
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -189,45 +217,45 @@
 
    NLP
    
 
    
 
    sequence_conv_pool
    
-
    
+
    
 
    
-class paddle.v2.networks.sequence_conv_pool(*args, **kwargs)
    
-
    Text convolution pooling layers helper.
    
+paddle.v2.networks.sequence_conv_pool(*args, **kwargs)
+
    Text convolution pooling group.
    
 
    Text input => Context Projection => FC Layer => Pooling => Output.
    
 
-
-
@@ -237,45 +265,45 @@ False if no bias.
    
 
    text_conv_pool
    
-
    
+
    
 
    
-class paddle.v2.networks.text_conv_pool(*args, **kwargs)
    
-
    Text convolution pooling layers helper.
    
+paddle.v2.networks.text_conv_pool(*args, **kwargs)
+
    Text convolution pooling group.
    
 
    Text input => Context Projection => FC Layer => Pooling => Output.
    
 
    
 
 
 
 
    Parameters:
      
-
    • name (basestring) – name of output layer(pooling layer name)
      • 
-
    • input (paddle.v2.config_base.Layer) – name of input layer
      • 
+
    • name (basestring) – group name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
 
    • context_len (int) – context projection length. See
 context_projection’s document.
      • 
 
    • hidden_size (int) – FC Layer size.
      • 
-
    • context_start (int or None) – context projection length. See
+
    • context_start (int|None) – context start position. See
 context_projection’s context_start.
      • 
-
    • pool_type (BasePoolingType.) – pooling layer type. See pooling’s document.
      • 
-
    • context_proj_name (basestring) – context projection layer name.
+
    • pool_type (BasePoolingType) – pooling layer type. See pooling_layer’s document.
      • 
+
    • context_proj_layer_name (basestring) – context projection layer name.
 None if user don’t care.
      • 
-
    • context_proj_param_attr (paddle.v2.attr.ParameterAttribute or None.) – context projection parameter attribute.
+
    • context_proj_param_attr (ParameterAttribute|None) – padding parameter attribute of context projection layer.
+If false, it means padding always be zero.
      • 
+
    • fc_layer_name (basestring) – fc layer name. None if user don’t care.
      • 
+
    • fc_param_attr (ParameterAttribute|None) – fc layer parameter attribute. None if user don’t care.
      • 
+
    • fc_bias_attr (ParameterAttribute|False|None) – fc bias parameter attribute. False if no bias,
 None if user don’t care.
      • 
-
    • fc_name (basestring) – fc layer name. None if user don’t care.
      • 
-
    • fc_param_attr (paddle.v2.attr.ParameterAttribute or None) – fc layer parameter attribute. None if user don’t care.
      • 
-
    • fc_bias_attr (paddle.v2.attr.ParameterAttribute or None) – fc bias parameter attribute. False if no bias,
+
    • fc_act (BaseActivation) – fc layer activation type. None means tanh.
      • 
+
    • pool_bias_attr (ParameterAttribute|False|None) – pooling layer bias attr. False if no bias.
 None if user don’t care.
      • 
-
    • fc_act (paddle.v2.Activation.Base) – fc layer activation type. None means tanh
      • 
-
    • pool_bias_attr (paddle.v2.attr.ParameterAttribute or None.) – pooling layer bias attr. None if don’t care.
-False if no bias.
      • 
-
    • fc_attr (paddle.v2.attr.ExtraAttribute) – fc layer extra attribute.
      • 
-
    • context_attr (paddle.v2.attr.ExtraAttribute) – context projection layer extra attribute.
      • 
-
    • pool_attr (paddle.v2.attr.ExtraAttribute) – pooling layer extra attribute.
      • 
+
    • fc_attr (ExtraLayerAttribute) – fc layer extra attribute.
      • 
+
    • context_attr (ExtraLayerAttribute) – context projection layer extra attribute.
      • 
+
    • pool_attr (ExtraLayerAttribute) – pooling layer extra attribute.
      • 
 
    
     		
 
    Returns:
    output layer name.
    
+
    Returns:
    layer’s output.
    
     		
 
    Return type:
    paddle.v2.config_base.Layer
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
    
-
-
@@ -288,43 +316,44 @@ False if no bias.
    Images
    
 
    img_conv_bn_pool
    
-
    
+
    
 
    
-class paddle.v2.networks.img_conv_bn_pool(*args, **kwargs)
    
+paddle.v2.networks.img_conv_bn_pool(*args, **kwargs)
 
    Convolution, batch normalization, pooling group.
    
+
    Img input => Conv => BN => Pooling => Output.
    
 
    
 
 
 
 
    Parameters:
      
-
    • name (basestring) – name of output layer(pooling layer name)
      • 
-
    • input (paddle.v2.config_base.Layer) – name of input layer
      • 
+
    • name (basestring) – group name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
 
    • context_len (int) – context projection length. See
 context_projection’s document.
      • 
 
    • hidden_size (int) – FC Layer size.
      • 
-
    • context_start (int or None) – context projection length. See
+
    • context_start (int|None) – context start position. See
 context_projection’s context_start.
      • 
-
    • pool_type (BasePoolingType.) – pooling layer type. See pooling’s document.
      • 
-
    • context_proj_name (basestring) – context projection layer name.
+
    • pool_type (BasePoolingType) – pooling layer type. See pooling_layer’s document.
      • 
+
    • context_proj_layer_name (basestring) – context projection layer name.
 None if user don’t care.
      • 
-
    • context_proj_param_attr (paddle.v2.attr.ParameterAttribute or None.) – context projection parameter attribute.
+
    • context_proj_param_attr (ParameterAttribute|None) – padding parameter attribute of context projection layer.
+If false, it means padding always be zero.
      • 
+
    • fc_layer_name (basestring) – fc layer name. None if user don’t care.
      • 
+
    • fc_param_attr (ParameterAttribute|None) – fc layer parameter attribute. None if user don’t care.
      • 
+
    • fc_bias_attr (ParameterAttribute|False|None) – fc bias parameter attribute. False if no bias,
 None if user don’t care.
      • 
-
    • fc_name (basestring) – fc layer name. None if user don’t care.
      • 
-
    • fc_param_attr (paddle.v2.attr.ParameterAttribute or None) – fc layer parameter attribute. None if user don’t care.
      • 
-
    • fc_bias_attr (paddle.v2.attr.ParameterAttribute or None) – fc bias parameter attribute. False if no bias,
+
    • fc_act (BaseActivation) – fc layer activation type. None means tanh.
      • 
+
    • pool_bias_attr (ParameterAttribute|False|None) – pooling layer bias attr. False if no bias.
 None if user don’t care.
      • 
-
    • fc_act (paddle.v2.Activation.Base) – fc layer activation type. None means tanh
      • 
-
    • pool_bias_attr (paddle.v2.attr.ParameterAttribute or None.) – pooling layer bias attr. None if don’t care.
-False if no bias.
      • 
-
    • fc_attr (paddle.v2.attr.ExtraAttribute) – fc layer extra attribute.
      • 
-
    • context_attr (paddle.v2.attr.ExtraAttribute) – context projection layer extra attribute.
      • 
-
    • pool_attr (paddle.v2.attr.ExtraAttribute) – pooling layer extra attribute.
      • 
+
    • fc_attr (ExtraLayerAttribute) – fc layer extra attribute.
      • 
+
    • context_attr (ExtraLayerAttribute) – context projection layer extra attribute.
      • 
+
    • pool_attr (ExtraLayerAttribute) – pooling layer extra attribute.
      • 
 
    
     		
 
    Returns:
    output layer name.
    
+
    Returns:
    layer’s output.
    
     		
 
    Return type:
    paddle.v2.config_base.Layer
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
    
-
-
@@ -334,31 +363,37 @@ False if no bias.
    
 
    img_conv_group
    
-
    
+
    
 
    
-class paddle.v2.networks.img_conv_group(**kwargs)
    
+paddle.v2.networks.img_conv_group(*args, **kwargs)
 
    Image Convolution Group, Used for vgg net.
    
-
    TODO(yuyang18): Complete docs
    
 
    
 
 
 
 
    Parameters:
      
-
    • name (basestring) – group name
      • 
-
    • input (paddle.v2.config_base.Layer) – layer’s input
      • 
-
    • filter_size (int) – see img_conv’s document
      • 
-
    • num_filters (int) – see img_conv’s document
      • 
-
    • pool_size (int) – see img_pool’s document.
      • 
-
    • pool_type (BasePoolingType) – see img_pool’s document.
      • 
-
    • act (paddle.v2.Activation.Base) – see batch_norm’s document.
      • 
-
    • groups (int) – see img_conv’s document
      • 
-
    • conv_stride (int) – see img_conv’s document.
      • 
-
    • conv_padding (int) – see img_conv’s document.
      • 
-
    • conv_bias_attr (paddle.v2.attr.ParameterAttribute) – see img_conv’s document.
      • 
-
    • num_channel (int) – see img_conv’s document.
      • 
-
    • conv_param_attr (paddle.v2.attr.ParameterAttribute) – see img_conv’s document.
      • 
-
    • shared_bias (bool) – see img_conv’s document.
      • 
-
    • conv_attr (Extrapaddle.v2.config_base.Layer) – see img_conv’s document.
      • 
-
    • bn_param_attr (paddle.v2.attr.ParameterAttribute.) – see batch_norm’s document.
      • 
-
    • bn_bias_attr – see batch_norm’s document.
      • 
-
    • bn_attr – paddle.v2.attr.ParameterAttribute.
      • 
-
    • pool_stride (int) – see img_pool’s document.
      • 
-
    • pool_padding (int) – see img_pool’s document.
      • 
-
    • pool_attr (paddle.v2.attr.ExtraAttribute) – see img_pool’s document.
      • 
+
    • name (basestring) – group name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • filter_size (int) – see img_conv_layer for details.
      • 
+
    • num_filters (int) – see img_conv_layer for details.
      • 
+
    • pool_size (int) – see img_pool_layer for details.
      • 
+
    • pool_type (BasePoolingType) – see img_pool_layer for details.
      • 
+
    • act (BaseActivation) – see batch_norm_layer for details.
      • 
+
    • groups (int) – see img_conv_layer for details.
      • 
+
    • conv_stride (int) – see img_conv_layer for details.
      • 
+
    • conv_padding (int) – see img_conv_layer for details.
      • 
+
    • conv_bias_attr (ParameterAttribute) – see img_conv_layer for details.
      • 
+
    • num_channel (int) – see img_conv_layer for details.
      • 
+
    • conv_param_attr (ParameterAttribute) – see img_conv_layer for details.
      • 
+
    • shared_bias (bool) – see img_conv_layer for details.
      • 
+
    • conv_layer_attr (ExtraLayerOutput) – see img_conv_layer for details.
      • 
+
    • bn_param_attr (ParameterAttribute) – see batch_norm_layer for details.
      • 
+
    • bn_bias_attr (ParameterAttribute) – see batch_norm_layer for details.
      • 
+
    • bn_layer_attr (ExtraLayerAttribute) – see batch_norm_layer for details.
      • 
+
    • pool_stride (int) – see img_pool_layer for details.
      • 
+
    • pool_padding (int) – see img_pool_layer for details.
      • 
+
    • pool_layer_attr (ExtraLayerAttribute) – see img_pool_layer for details.
      • 
 
    
     		
 
    Returns:
    Layer groups output
    
+
    Returns:
    layer’s output
    
     		
 
    Return type:
    paddle.v2.config_base.Layer
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
    
-
+
+
@@ -368,66 +403,72 @@ False if no bias.
    
 
    simple_img_conv_pool
    
-
    
+
    
 
    
-class paddle.v2.networks.simple_img_conv_pool(*args, **kwargs)
    
+paddle.v2.networks.simple_img_conv_pool(*args, **kwargs)
 
    Simple image convolution and pooling group.
    
-
    Input => conv => pooling
    
+
    Img input => Conv => Pooling => Output.
    
 
    
 
 
 
 
    Parameters:
      
-
    • conv_batchnorm_drop_rate – 
      • 
-
    • input – 
      • 
-
    • conv_num_filter – 
      • 
-
    • pool_size – 
      • 
-
    • num_channels – 
      • 
-
    • conv_padding – 
      • 
-
    • conv_filter_size – 
      • 
-
    • conv_act – 
      • 
-
    • conv_with_batchnorm – 
      • 
-
    • pool_stride – 
      • 
-
    • pool_type – 
      • 
+
    • conv_batchnorm_drop_rate (list) – if conv_with_batchnorm[i] is true,
+conv_batchnorm_drop_rate[i] represents the drop rate of each batch norm.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • conv_num_filter (list|tuple) – list of output channels num.
      • 
+
    • pool_size (int) – pooling filter size.
      • 
+
    • num_channels (int) – input channels num.
      • 
+
    • conv_padding (int) – convolution padding size.
      • 
+
    • conv_filter_size (int) – convolution filter size.
      • 
+
    • conv_act (BaseActivation) – activation funciton after convolution.
      • 
+
    • conv_with_batchnorm (list) – if conv_with_batchnorm[i] is true,
+there is a batch normalization operation after each convolution.
      • 
+
    • pool_stride (int) – pooling stride size.
      • 
+
    • pool_type (BasePoolingType) – pooling type.
      • 
+
    • param_attr (ParameterAttribute) – param attribute of convolution layer,
+None means default attribute.
      • 
 
    
     		
 
    Returns:
    
+
    Returns:
    layer’s output
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
    
-
-
    
 
 
 
 
    Parameters:
      
-
    • name (basestring) – group name
      • 
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
-
    • filter_size (int) – see img_conv for details
      • 
-
    • num_filters (int) – see img_conv for details
      • 
-
    • pool_size (int) – see img_pool for details
      • 
-
    • pool_type (BasePoolingType) – see img_pool for details
      • 
-
    • act (paddle.v2.Activation.Base) – see img_conv for details
      • 
-
    • groups (int) – see img_conv for details
      • 
-
    • conv_stride (int) – see img_conv for details
      • 
-
    • conv_padding (int) – see img_conv for details
      • 
-
    • bias_attr (paddle.v2.attr.ParameterAttribute) – see img_conv for details
      • 
-
    • num_channel (int) – see img_conv for details
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute) – see img_conv for details
      • 
-
    • shared_bias (bool) – see img_conv for details
      • 
-
    • conv_attr (paddle.v2.attr.ExtraAttribute) – see img_conv for details
      • 
-
    • pool_stride (int) – see img_pool for details
      • 
-
    • pool_padding (int) – see img_pool for details
      • 
-
    • pool_attr (paddle.v2.attr.ExtraAttribute) – see img_pool for details
      • 
+
    • name (basestring) – group name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • filter_size (int) – see img_conv_layer for details.
      • 
+
    • num_filters (int) – see img_conv_layer for details.
      • 
+
    • pool_size (int) – see img_pool_layer for details.
      • 
+
    • pool_type (BasePoolingType) – see img_pool_layer for details.
      • 
+
    • act (BaseActivation) – see img_conv_layer for details.
      • 
+
    • groups (int) – see img_conv_layer for details.
      • 
+
    • conv_stride (int) – see img_conv_layer for details.
      • 
+
    • conv_padding (int) – see img_conv_layer for details.
      • 
+
    • bias_attr (ParameterAttribute) – see img_conv_layer for details.
      • 
+
    • num_channel (int) – see img_conv_layer for details.
      • 
+
    • param_attr (ParameterAttribute) – see img_conv_layer for details.
      • 
+
    • shared_bias (bool) – see img_conv_layer for details.
      • 
+
    • conv_layer_attr (ExtraLayerAttribute) – see img_conv_layer for details.
      • 
+
    • pool_stride (int) – see img_pool_layer for details.
      • 
+
    • pool_padding (int) – see img_pool_layer for details.
      • 
+
    • pool_layer_attr (ExtraLayerAttribute) – see img_pool_layer for details.
      • 
 
    
     		
 
    Returns:
    Layer’s output
    
+
    Returns:
    layer’s output
    
     		
 
    Return type:
    paddle.v2.config_base.Layer
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
    
 
    
 
+
    
+
    
+
    small_vgg
    
 
    
 
    
 
    vgg_16_network
    
-
    
+
    
 
    
-class paddle.v2.networks.vgg_16_network(**kwargs)
    
+paddle.v2.networks.vgg_16_network(input_image, num_channels, num_classes=1000)
 
    Same model from https://gist.github.com/ksimonyan/211839e770f7b538e2d8
    
 
-
+
+
@@ -442,25 +483,25 @@ False if no bias.
    LSTM
    
 
    lstmemory_unit
    
-
    
+
    
 
    
-class paddle.v2.networks.lstmemory_unit(*args, **kwargs)
    
-
    Define calculations that a LSTM unit performs in a single time step.
-This function itself is not a recurrent layer, so that it can not be
-directly applied to sequence input. This function is always used in
+paddle.v2.networks.lstmemory_unit(*args, **kwargs)
+
    lstmemory_unit defines the caculation process of a LSTM unit during a
+single time step. This function is not a recurrent layer, so it can not be
+directly used to process sequence input. This function is always used in
 recurrent_group (see layers.py for more details) to implement attention
 mechanism.
    
 
    Please refer to  Generating Sequences With Recurrent Neural Networks
 for more details about LSTM. The link goes as follows:
 .. _Link: https://arxiv.org/abs/1308.0850
    
 
    
-\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    
+\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{x_i}x_{t} + W_{h_i}h_{t-1} + W_{c_i}c_{t-1} + b_i)\\f_t & = \sigma(W_{x_f}x_{t} + W_{h_f}h_{t-1} + W_{c_f}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{x_c}x_t+W_{h_c}h_{t-1} + b_c)\\o_t & = \sigma(W_{x_o}x_{t} + W_{h_o}h_{t-1} + W_{c_o}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    The example usage is:
    lstm_step = lstmemory_unit(input=[layer1],
                            size=256,
-                           act=paddle.v2.Activation.Tanh(),
-                           gate_act=paddle.v2.Activation.Sigmoid(),
-                           state_act=paddle.v2.Activation.Tanh())
+                           act=TanhActivation(),
+                           gate_act=SigmoidActivation(),
+                           state_act=TanhActivation())
 
    
 
    
 
 
 
 
    Parameters:
      
-
    • num_classes – 
      • 
-
    • input_image (paddle.v2.config_base.Layer) – 
      • 
-
    • num_channels (int) – 
      • 
+
    • num_classes (int) – number of class.
      • 
+
    • input_image (LayerOutput) – input layer.
      • 
+
    • num_channels (int) – input channels num.
      • 
 
    
     		
 
    Returns:
    
+
    Returns:
    layer’s output
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
 
 
 
    
@@ -468,27 +509,36 @@ for more details about LSTM. The link goes as follows:
 
-
-
@@ -498,29 +548,29 @@ False means no bias, None means default bias.
    
 
    lstmemory_group
    
-
    
+
    
 
    
-class paddle.v2.networks.lstmemory_group(*args, **kwargs)
    
-
    lstm_group is a recurrent layer group version of Long Short Term Memory. It
+paddle.v2.networks.lstmemory_group(*args, **kwargs)
+
    lstm_group is a recurrent_group version of Long Short Term Memory. It
 does exactly the same calculation as the lstmemory layer (see lstmemory in
 layers.py for the maths) does. A promising benefit is that LSTM memory
-cell states, or hidden states in every time step are accessible to the
+cell states(or hidden states) in every time step are accessible to the
 user. This is especially useful in attention model. If you do not need to
-access the internal states of the lstm, but merely use its outputs,
+access the internal states of the lstm and merely use its outputs,
 it is recommended to use the lstmemory, which is relatively faster than
 lstmemory_group.
    
 
    NOTE: In PaddlePaddle’s implementation, the following input-to-hidden
 multiplications:
-\(W_{xi}x_{t}\) , \(W_{xf}x_{t}\),
-\(W_{xc}x_t\), \(W_{xo}x_{t}\) are not done in lstmemory_unit to
-speed up the calculations. Consequently, an additional mixed with
+\(W_{x_i}x_{t}\) , \(W_{x_f}x_{t}\),
+\(W_{x_c}x_t\), \(W_{x_o}x_{t}\) are not done in lstmemory_unit to
+speed up the calculations. Consequently, an additional mixed_layer with
 full_matrix_projection must be included before lstmemory_unit is called.
    
 
    The example usage is:
    
 
    lstm_step = lstmemory_group(input=[layer1],
                             size=256,
-                            act=paddle.v2.Activation.Tanh(),
-                            gate_act=paddle.v2.Activation.Sigmoid(),
-                            state_act=paddle.v2.Activation.Tanh())
+                            act=TanhActivation(),
+                            gate_act=SigmoidActivation(),
+                            state_act=TanhActivation())
 
    
 
    
 
    
 
 
    Parameters:
      
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
-
    • name (basestring) – lstmemory unit name.
      • 
-
    • size (int) – lstmemory unit size.
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
      • 
-
    • act (paddle.v2.Activation.Base) – lstm final activiation type
      • 
-
    • gate_act (paddle.v2.Activation.Base) – lstm gate activiation type
      • 
-
    • state_act (paddle.v2.Activation.Base) – lstm state activiation type.
      • 
-
    • mixed_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias parameter attribute of mixed layer.
-False means no bias, None means default bias.
      • 
-
    • lstm_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias parameter attribute of lstm layer.
-False means no bias, None means default bias.
      • 
-
    • mixed_attr (paddle.v2.attr.ExtraAttribute) – mixed layer’s extra attribute.
      • 
-
    • lstm_attr (paddle.v2.attr.ExtraAttribute) – lstm layer’s extra attribute.
      • 
-
    • get_output_attr (paddle.v2.attr.ExtraAttribute) – get output layer’s extra attribute.
      • 
+
    • input (LayerOutput) – Input layer.
      • 
+
    • out_memory (LayerOutput | None) – The output of previous time step.
      • 
+
    • name (basestring) – The lstmemory unit name.
      • 
+
    • size (int) – The lstmemory unit size.
      • 
+
    • param_attr (ParameterAttribute) – The parameter attribute for the weights in
+input to hidden projection.
+None means default attribute.
      • 
+
    • act (BaseActivation) – The last activiation type of lstm.
      • 
+
    • gate_act (BaseActivation) – The gate activiation type of lstm.
      • 
+
    • state_act (BaseActivation) – The state activiation type of lstm.
      • 
+
    • input_proj_bias_attr (ParameterAttribute|bool|None) – The parameter attribute for the bias in
+input to hidden projection.
+False or None means no bias.
+If this parameter is set to True,
+the bias is initialized to zero.
      • 
+
    • input_proj_layer_attr (ExtraLayerAttribute) – The extra layer attribute for
+input to hidden projection of the LSTM unit,
+such as dropout, error clipping.
      • 
+
    • lstm_bias_attr (ParameterAttribute|True|None) – The parameter attribute for the bias in lstm layer.
+False or None means no bias.
+If this parameter is set to True,
+the bias is initialized to zero.
      • 
+
    • lstm_layer_attr (ExtraLayerAttribute) – The extra attribute of lstm layer.
      • 
 
    
     		
 
    Returns:
    lstmemory unit name.
    
+
    Returns:
    The lstmemory unit name.
    
     		
 
    Return type:
    paddle.v2.config_base.Layer
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
    
@@ -528,28 +578,37 @@ full_matrix_projection must be included before lstmemory_unit is called.
    
-
@@ -559,41 +618,41 @@ False means no bias, None means default bias.
    
 
    simple_lstm
    
-
    
+
    
 
    
-class paddle.v2.networks.simple_lstm(*args, **kwargs)
    
+paddle.v2.networks.simple_lstm(*args, **kwargs)
 
    Simple LSTM Cell.
    
-
    It just combine a mixed layer with fully_matrix_projection and a lstmemory
-layer. The simple lstm cell was implemented as follow equations.
    
+
    It just combines a mixed layer with fully_matrix_projection and a lstmemory
+layer. The simple lstm cell was implemented with follow equations.
    
 
    
 \[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    
-
    Please refer Generating Sequences With Recurrent Neural Networks if you
-want to know what lstm is. Link is here.
    
+
    Please refer to Generating Sequences With Recurrent Neural Networks for more
+details about lstm. Link is here.
    
 
    
 
 
 
    Parameters:
      
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
-
    • name (basestring) – lstmemory group name.
      • 
-
    • size (int) – lstmemory group size.
      • 
-
    • reverse (bool) – is lstm reversed
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
      • 
-
    • act (paddle.v2.Activation.Base) – lstm final activiation type
      • 
-
    • gate_act (paddle.v2.Activation.Base) – lstm gate activiation type
      • 
-
    • state_act (paddle.v2.Activation.Base) – lstm state activiation type.
      • 
-
    • mixed_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias parameter attribute of mixed layer.
-False means no bias, None means default bias.
      • 
-
    • lstm_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias parameter attribute of lstm layer.
-False means no bias, None means default bias.
      • 
-
    • mixed_attr (paddle.v2.attr.ExtraAttribute) – mixed layer’s extra attribute.
      • 
-
    • lstm_attr (paddle.v2.attr.ExtraAttribute) – lstm layer’s extra attribute.
      • 
-
    • get_output_attr (paddle.v2.attr.ExtraAttribute) – get output layer’s extra attribute.
      • 
+
    • input (LayerOutput) – Input layer.
      • 
+
    • size (int) – The lstmemory group size.
      • 
+
    • name (basestring) – The name of lstmemory group.
      • 
+
    • out_memory (LayerOutput | None) – The output of previous time step.
      • 
+
    • reverse (bool) – Process the input in a reverse order or not.
      • 
+
    • param_attr (ParameterAttribute) – The parameter attribute for the weights in
+input to hidden projection.
+None means default attribute.
      • 
+
    • act (BaseActivation) – The last activiation type of lstm.
      • 
+
    • gate_act (BaseActivation) – The gate activiation type of lstm.
      • 
+
    • state_act (BaseActivation) – The state activiation type of lstm.
      • 
+
    • input_proj_bias_attr (ParameterAttribute|bool|None) – The parameter attribute for the bias in
+input to hidden projection.
+False or None means no bias.
+If this parameter is set to True,
+the bias is initialized to zero.
      • 
+
    • input_proj_layer_attr (ExtraLayerAttribute) – The extra layer attribute for
+input to hidden projection of the LSTM unit,
+such as dropout, error clipping.
      • 
+
    • lstm_bias_attr (ParameterAttribute|True|None) – The parameter attribute for the bias in lstm layer.
+False or None means no bias.
+If this parameter is set to True,
+the bias is initialized to zero.
      • 
+
    • lstm_layer_attr (ExtraLayerAttribute) – The extra attribute of lstm layer.
      • 
 
    
     		
 
    
 
    Returns:
    the lstmemory group.
    
     		
 
    Return type:
    paddle.v2.config_base.Layer
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
    
-
-
@@ -603,12 +662,12 @@ means default bias.
    
 
    bidirectional_lstm
    
-
    
+
    
 
    
-class paddle.v2.networks.bidirectional_lstm(*args, **kwargs)
    
+paddle.v2.networks.bidirectional_lstm(*args, **kwargs)
 
    A bidirectional_lstm is a recurrent unit that iterates over the input
-sequence both in forward and bardward orders, and then concatenate two
-outputs form a final output. However, concatenation of two outputs
+sequence both in forward and backward orders, and then concatenate two
+outputs to form a final output. However, concatenation of two outputs
 is not the only way to form the final output, you can also, for example,
 just add them together.
    
 
    Please refer to  Neural Machine Translation by Jointly Learning to Align
@@ -625,20 +684,19 @@ The link goes as follows:
 
    
-
-
@@ -651,12 +709,12 @@ concatenated and returned.
 
    GRU
    
 
    
 
    gru_unit
    
-
    
+
    
 
    
-class paddle.v2.networks.gru_unit(*args, **kwargs)
    
-
    Define calculations that a gated recurrent unit performs in a single time
-step. This function itself is not a recurrent layer, so that it can not be
-directly applied to sequence input. This function is almost always used in
+paddle.v2.networks.gru_unit(*args, **kwargs)
+
    gru_unit defines the calculation process of a gated recurrent unit during a single
+time step. This function is not a recurrent layer, so it can not be
+directly used to process sequence input. This function is always used in
 the recurrent_group (see layers.py for more details) to implement attention
 mechanism.
    
 
    Please see grumemory in layers.py for the details about the maths.
    
@@ -665,19 +723,20 @@ mechanism.
    
 
    
-
@@ -687,21 +746,21 @@ mechanism.
    
 
 
    
 
    gru_group
    
-
    
+
    
 
    
-class paddle.v2.networks.gru_group(*args, **kwargs)
    
-
    gru_group is a recurrent layer group version of Gated Recurrent Unit. It
+paddle.v2.networks.gru_group(*args, **kwargs)
+
    gru_group is a recurrent_group version of Gated Recurrent Unit. It
 does exactly the same calculation as the grumemory layer does. A promising
 benefit is that gru hidden states are accessible to the user. This is
 especially useful in attention model. If you do not need to access
-any internal state, but merely use the outputs of a GRU, it is recommended
+any internal state and merely use the outputs of a GRU, it is recommended
 to use the grumemory, which is relatively faster.
    
 
    Please see grumemory in layers.py for more detail about the maths.
    
 
    The example usage is:
    
-
    gru = gur_group(input=[layer1],
+
    gru = gru_group(input=[layer1],
                 size=256,
-                act=paddle.v2.Activation.Tanh(),
-                gate_act=paddle.v2.Activation.Sigmoid())
+                act=TanhActivation(),
+                gate_act=SigmoidActivation())
 
    
 
    
 
    
 
 
 
 
    Parameters:
      
 
    • name (basestring) – lstm layer name.
      • 
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
+
    • input (LayerOutput) – layer’s input.
      • 
 
    • size (int) – lstm layer size.
      • 
-
    • reverse (bool) – whether to process the input data in a reverse order
      • 
-
    • mat_param_attr (paddle.v2.attr.ParameterAttribute) – mixed layer’s matrix projection parameter attribute.
      • 
-
    • bias_param_attr (paddle.v2.attr.ParameterAttribute|False) – bias parameter attribute. False means no bias, None
+
    • reverse (bool) – process the input in a reverse order or not.
      • 
+
    • mat_param_attr (ParameterAttribute) – parameter attribute of matrix projection in mixed layer.
      • 
+
    • bias_param_attr (ParameterAttribute|False) – bias parameter attribute. False means no bias, None
 means default bias.
      • 
-
    • inner_param_attr (paddle.v2.attr.ParameterAttribute) – lstm cell parameter attribute.
      • 
-
    • act (paddle.v2.Activation.Base) – lstm final activiation type
      • 
-
    • gate_act (paddle.v2.Activation.Base) – lstm gate activiation type
      • 
-
    • state_act (paddle.v2.Activation.Base) – lstm state activiation type.
      • 
-
    • mixed_attr (paddle.v2.attr.ExtraAttribute) – mixed layer’s extra attribute.
      • 
-
    • lstm_cell_attr (paddle.v2.attr.ExtraAttribute) – lstm layer’s extra attribute.
      • 
+
    • inner_param_attr (ParameterAttribute) – parameter attribute of lstm cell.
      • 
+
    • act (BaseActivation) – last activiation type of lstm.
      • 
+
    • gate_act (BaseActivation) – gate activiation type of lstm.
      • 
+
    • state_act (BaseActivation) – state activiation type of lstm.
      • 
+
    • mixed_layer_attr (ExtraLayerAttribute) – extra attribute of mixed layer.
      • 
+
    • lstm_cell_attr (ExtraLayerAttribute) – extra attribute of lstm.
      • 
 
    
     		
 
    Returns:
    lstm layer name.
    
+
    Returns:
    layer’s output.
    
     		
 
    Return type:
    paddle.v2.config_base.Layer
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
 
    Parameters:
      
 
    • name (basestring) – bidirectional lstm layer name.
      • 
-
    • input (paddle.v2.config_base.Layer) – input layer.
      • 
+
    • input (LayerOutput) – input layer.
      • 
 
    • size (int) – lstm layer size.
      • 
-
    • return_seq (bool) – If set False, outputs of the last time step are
+
    • return_seq (bool) – If set False, the last time step of output are
 concatenated and returned.
-If set True, the entire output sequences that are
-processed in forward and backward directions are
-concatenated and returned.
      • 
+If set True, the entire output sequences in forward
+and backward directions are concatenated and returned.
 
    
     		
 
    Returns:
    paddle.v2.config_base.Layer object accroding to the return_seq.
    
+
    Returns:
    LayerOutput object.
    
     		
 
    Return type:
    paddle.v2.config_base.Layer
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
    Parameters:
      
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • memory_boot (LayerOutput | None) – the initialization state of the LSTM cell.
      • 
 
    • name (basestring) – name of the gru group.
      • 
 
    • size (int) – hidden size of the gru.
      • 
-
    • act (paddle.v2.Activation.Base) – type of the activation
      • 
-
    • gate_act (paddle.v2.Activation.Base) – type of the gate activation
      • 
-
    • gru_attr (paddle.v2.attr.ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • 
+
    • act (BaseActivation) – activation type of gru
      • 
+
    • gate_act (BaseActivation) – gate activation type or gru
      • 
+
    • gru_layer_attr (ExtraLayerAttribute) – Extra attribute of the gru layer.
      • 
 
    
     		
 
    
 
    Returns:
    the gru output layer.
    
     		
 
    Return type:
    paddle.v2.config_base.Layer
    
+
    Return type:
    LayerOutput
    
     		
 
    
 
    
@@ -709,21 +768,23 @@ to use the grumemory, which is relatively faster.
    
     
-
@@ -733,23 +794,23 @@ to use the grumemory, which is relatively faster.
    
 
 
    
 
    simple_gru
    
-
    
+
    
 
    
-class paddle.v2.networks.simple_gru(*args, **kwargs)
    
-
    You maybe see gru_step, grumemory in layers.py, gru_unit, gru_group,
+paddle.v2.networks.simple_gru(*args, **kwargs)
+
    You may see gru_step_layer, grumemory in layers.py, gru_unit, gru_group,
 simple_gru in network.py. The reason why there are so many interfaces is
 that we have two ways to implement recurrent neural network. One way is to
 use one complete layer to implement rnn (including simple rnn, gru and lstm)
-with multiple time steps, such as recurrent, lstmemory, grumemory. But,
+with multiple time steps, such as recurrent_layer, lstmemory, grumemory. But
 the multiplication operation \(W x_t\) is not computed in these layers.
 See details in their interfaces in layers.py.
 The other implementation is to use an recurrent group which can ensemble a
 series of layers to compute rnn step by step. This way is flexible for
 attenion mechanism or other complex connections.
    
 
      
-
    • gru_step: only compute rnn by one step. It needs an memory as input
+
    • gru_step_layer: only compute rnn by one step. It needs an memory as input
 and can be used in recurrent group.
      • 
-
    • gru_unit: a wrapper of gru_step with memory.
      • 
+
    • gru_unit: a wrapper of gru_step_layer with memory.
      • 
 
    • gru_group: a GRU cell implemented by a combination of multiple layers in
 recurrent group.
 But \(W x_t\) is not done in group.
      • 
@@ -770,21 +831,102 @@ gru_group, and gru_group is relatively better than simple_gru.
      
 
    
-
+
+
+
    
 
 
    Parameters:
      
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • memory_boot (LayerOutput | None) – the initialization state of the LSTM cell.
      • 
 
    • name (basestring) – name of the gru group.
      • 
 
    • size (int) – hidden size of the gru.
      • 
-
    • reverse (bool) – whether to process the input data in a reverse order
      • 
-
    • act (paddle.v2.Activation.Base) – type of the activiation
      • 
-
    • gate_act (paddle.v2.Activation.Base) – type of the gate activiation
      • 
-
    • gru_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias. False means no bias, None means default bias.
      • 
-
    • gru_attr (paddle.v2.attr.ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • 
+
    • reverse (bool) – process the input in a reverse order or not.
      • 
+
    • act (BaseActivation) – activiation type of gru
      • 
+
    • gate_act (BaseActivation) – gate activiation type of gru
      • 
+
    • gru_bias_attr (ParameterAttribute|False|None) – bias parameter attribute of gru layer,
+False means no bias, None means default bias.
      • 
+
    • gru_layer_attr (ExtraLayerAttribute) – Extra attribute of the gru layer.
      • 
 
    
     		
 
    
 
    Returns:
    the gru group.
    
     		
 
    Return type:
    paddle.v2.config_base.Layer
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
    Parameters:
      
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
 
    • name (basestring) – name of the gru group.
      • 
 
    • size (int) – hidden size of the gru.
      • 
-
    • reverse (bool) – whether to process the input data in a reverse order
      • 
-
    • act (paddle.v2.Activation.Base) – type of the activiation
      • 
-
    • gate_act (paddle.v2.Activation.Base) – type of the gate activiation
      • 
-
    • gru_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias. False means no bias, None means default bias.
      • 
-
    • gru_attr (paddle.v2.attr.ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • 
+
    • reverse (bool) – process the input in a reverse order or not.
      • 
+
    • act (BaseActivation) – activiation type of gru
      • 
+
    • gate_act (BaseActivation) – gate activiation type of gru
      • 
+
    • gru_bias_attr (ParameterAttribute|False|None) – bias parameter attribute of gru layer,
+False means no bias, None means default bias.
      • 
+
    • gru_layer_attr (ExtraLayerAttribute) – Extra attribute of the gru layer.
      • 
 
    
     		
 
    
 
    Returns:
    the gru group.
    
     		
 
    Return type:
    paddle.v2.config_base.Layer
    
+
    Return type:
    LayerOutput
    
+
    
+
    
+
+
    
+
    
+
    simple_gru2
    
+
    
+
    
+paddle.v2.networks.simple_gru2(*args, **kwargs)
    
+
    simple_gru2 is the same with simple_gru, but using grumemory instead.
+Please refer to grumemory in layers.py for more detail about the math.
+simple_gru2 is faster than simple_gru.
    
+
    The example usage is:
    
+
    gru = simple_gru2(input=[layer1], size=256)
+
    
+
    
+
+++
+
+
+
+
+
+
+
+
    


    Parameters:
      
+
    • input (LayerOutput) – input layer.
      • 
+
    • name (basestring) – name of the gru group.
      • 
+
    • size (int) – hidden size of the gru.
      • 
+
    • reverse (bool) – process the input in a reverse order or not.
      • 
+
    • act (BaseActivation) – activiation type of gru
      • 
+
    • gate_act (BaseActivation) – gate activiation type of gru
      • 
+
    • gru_bias_attr (ParameterAttribute|False|None) – bias parameter attribute of gru layer,
+False means no bias, None means default bias.
      • 
+
    • gru_layer_attr (ExtraLayerAttribute) – Extra attribute of the gru layer.
      • 
+
    
+
    Returns:
    the gru group.
    
+
    Return type:
    LayerOutput
    
+
    
+
    
+
+
    
+
    
+
    bidirectional_gru
    
+
    
+
    
+paddle.v2.networks.bidirectional_gru(*args, **kwargs)
    
+
    A bidirectional_gru is a recurrent unit that iterates over the input
+sequence both in forward and backward orders, and then concatenate two
+outputs to form a final output. However, concatenation of two outputs
+is not the only way to form the final output, you can also, for example,
+just add them together.
    
+
    The example usage is:
    
+
    bi_gru = bidirectional_gru(input=[input1], size=512)
+
    
+
    
+
+++
+
+
+
+
+
@@ -795,10 +937,10 @@ gru_group, and gru_group is relatively better than simple_gru.
    
 
    simple_attention
    
-
    
+
    
 
    
-class paddle.v2.networks.simple_attention(*args, **kwargs)
    
-
    Calculate and then return a context vector by attention machanism.
+paddle.v2.networks.simple_attention(*args, **kwargs)
+
    Calculate and return a context vector with attention mechanism.
 Size of the context vector equals to size of the encoded_sequence.
    
 
    
 \[ \begin{align}\begin{aligned}a(s_{i-1},h_{j}) & = v_{a}f(W_{a}s_{t-1} + U_{a}h_{j})\\e_{i,j} & = a(s_{i-1}, h_{j})\\a_{i,j} & = \frac{exp(e_{i,j})}{\sum_{k=1}^{T_x}{exp(e_{i,k})}}\\c_{i} & = \sum_{j=1}^{T_{x}}a_{i,j}h_{j}\end{aligned}\end{align} \]
    
@@ -821,24 +963,27 @@ Align and Translate for more details. The link is as follows:
 
    
-
+
+
@@ -846,27 +991,51 @@ compute attention weight.
-
-
    
-
    Miscs
    
-
    
-
    dropout_layer
    
-
    
+
    
+
    dot_product_attention
    
+
    
 
    
-class paddle.v2.networks.dropout_layer(*args, **kwargs)
    
-
    @TODO(yuyang18): Add comments.
    
+paddle.v2.networks.dot_product_attention(*args, **kwargs)
+
    Calculate and return a context vector with dot-product attention mechanism.
+The dimension of the context vector equals to that of the attended_sequence.
    
+
    
+\[ \begin{align}\begin{aligned}a(s_{i-1},h_{j}) & = s_{i-1}^\mathrm{T} h_{j}\\e_{i,j} & = a(s_{i-1}, h_{j})\\a_{i,j} & = \frac{exp(e_{i,j})}{\sum_{k=1}^{T_x}{exp(e_{i,k})}}\\c_{i} & = \sum_{j=1}^{T_{x}}a_{i,j}z_{j}\end{aligned}\end{align} \]
    
+
    where \(h_{j}\) is the jth element of encoded_sequence,
+\(z_{j}\) is the jth element of attended_sequence,
+\(s_{i-1}\) is transformed_state.
    
+
    The example usage is:
    
+
    context = dot_product_attention(encoded_sequence=enc_seq,
+                                attended_sequence=att_seq,
+                                transformed_state=state,)
+
    
+
    
 
    


    Parameters:
      
+
    • name (basestring) – bidirectional gru layer name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • size (int) – gru layer size.
      • 
+
    • return_seq (bool) – If set False, the last time step of output are
+concatenated and returned.
+If set True, the entire output sequences in forward
+and backward directions are concatenated and returned.
      • 
+
    
+
    Returns:
    LayerOutput object.
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
 
    Parameters:
      
 
    • name (basestring) – name of the attention model.
      • 
-
    • softmax_param_attr (paddle.v2.attr.ParameterAttribute) – parameter attribute of sequence softmax
-that is used to produce attention weight
      • 
-
    • weight_act (Activation) – activation of the attention model
      • 
-
    • encoded_sequence (paddle.v2.config_base.Layer) – output of the encoder
      • 
-
    • encoded_proj (paddle.v2.config_base.Layer) – attention weight is computed by a feed forward neural
+
    • softmax_param_attr (ParameterAttribute) – parameter attribute of sequence softmax
+that is used to produce attention weight.
      • 
+
    • weight_act (BaseActivation) – activation of the attention model.
      • 
+
    • encoded_sequence (LayerOutput) – output of the encoder
      • 
+
    • encoded_proj (LayerOutput) – attention weight is computed by a feed forward neural
 network which has two inputs : decoder’s hidden state
 of previous time step and encoder’s output.
 encoded_proj is output of the feed-forward network for
 encoder’s output. Here we pre-compute it outside
 simple_attention for speed consideration.
      • 
-
    • decoder_state (paddle.v2.config_base.Layer) – hidden state of decoder in previous time step
      • 
-
    • transform_param_attr (paddle.v2.attr.ParameterAttribute) – parameter attribute of the feed-forward
+
    • decoder_state (LayerOutput) – hidden state of decoder in previous time step
      • 
+
    • transform_param_attr (ParameterAttribute) – parameter attribute of the feed-forward
 network that takes decoder_state as inputs to
 compute attention weight.
      • 
 
    
     		
 
    Returns:
    a context vector
    
+
    Returns:
    a context vector
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
 
 
 
    
-
+
+
diff --git a/doc/api/v2/config/optimizer.html b/doc/api/v2/config/optimizer.html
index 01071e7410085be37ca272ced1e34a0a9f78c986..9c3cce064cd60b492521009215f0626b312948ae 100644
--- a/doc/api/v2/config/optimizer.html
+++ b/doc/api/v2/config/optimizer.html
@@ -35,7 +35,7 @@
     
-         
+        
@@ -65,7 +65,7 @@
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -189,23 +217,36 @@
 
    
 
    
 class paddle.v2.optimizer.Momentum(momentum=None, sparse=False, **kwargs)
    
-
    SGD Optimizer.
    
-
    SGD is an optimization method, trying to find a neural network that
-minimize the “cost/error” of it by iteration. In paddle’s implementation
-SGD Optimizer is synchronized, which means all gradients will be wait to
-calculate and reduced into one gradient, then do optimize operation.
    
-
    The neural network consider the learning problem of minimizing an objective
-function, that has the form of a sum
    
+
    Momentum Optimizer.
    
+
    When sparse=False, the momentum update formula is as follows:
    
 
    
-\[Q(w) = \sum_{i}^{n} Q_i(w)\]
    
-
    The value of function Q sometimes is the cost of neural network (Mean
-Square Error between prediction and label for example). The function Q is
-parametrised by w, the weight/bias of neural network. And weights is what to
-be learned. The i is the i-th observation in (trainning) data.
    
-
    So, the SGD method will optimize the weight by
    
+\[\begin{split}v_{t} &= k * v_{t-1} - \gamma_t (g_{t} + \lambda w_{t-1}) \\
+w_{t} &= w_{t-1} + v_{t} \\\end{split}\]
    
+
    where, \(k\) is momentum, \(\lambda\) is decay rate,
+\(\gamma_t\) is learning rate at the t’th iteration.
+\(w_{t}\) is the weight as the t’th iteration.
+And the \(v_{t}\) is the history momentum variable.
    
+
    When sparse=True, the update scheme:
    
 
    
-\[w = w - \eta \nabla Q(w) = w - \eta \sum_{i}^{n} \nabla Q_i(w)\]
    
-
    where \(\eta\) is learning rate. And \(n\) is batch size.
    
+\[\begin{split}\alpha_t &= \alpha_{t-1} / k \\
+\beta_t &= \beta_{t-1} / (1 + \lambda \gamma_t) \\
+u_t &= u_{t-1} - \alpha_t \gamma_t g_t \\
+v_t &= v_{t-1} + \tau_{t-1} \alpha_t \gamma_t g_t \\
+\tau_t &= \tau_{t-1} + \beta_t / \alpha_t\end{split}\]
    
+
    where \(k\) is momentum, \(\lambda\) is decay rate,
+\(\gamma_t\) is learning rate at the t’th iteration.
    
+
    
 
 
 
 
    Parameters:
      
-
    • name – 
      • 
-
    • input – 
      • 
-
    • dropout_rate – 
      • 
+
    • name (basestring) – A prefix attached to the name of each layer that defined inside
+the dot_product_attention.
      • 
+
    • softmax_param_attr (ParameterAttribute) – The parameter attribute of sequence softmax
+that is used to produce attention weight.
      • 
+
    • encoded_sequence (LayerOutput) – The output hidden vectors of the encoder.
      • 
+
    • attended_sequence (LayerOutput) – The attention weight is computed by a feed forward neural
+network which has two inputs : decoder’s transformed hidden
+state of previous time step and encoder’s output.
+attended_sequence is the sequence to be attended.
      • 
+
    • transformed_state (LayerOutput) – The transformed hidden state of decoder in previous time step.
+Since the dot-product operation will be performed on it and the
+encoded_sequence, their dimensions must be equal. For flexibility,
+we suppose transformations of the decoder’s hidden state have been
+done outside dot_product_attention and no more will be performed
+inside. Then users can use either the original or transformed one.
      • 
 
    
     		
 
    Returns:
    
+
    Returns:
    The context vector.
    
+
    Return type:
    LayerOutput
    
     		
 
    
     
         
          
 
   
   
     
    
+++
+
+
+
+
    


    Parameters:
      
+
    • momentum (float) – the momentum factor.
      • 
+
    • sparse (bool) – with sparse support or not, False by default.
      • 
+
    
+
    
 
    
 
 
    
@@ -219,7 +260,7 @@ The details of please refer 
 \[\begin{split}m(w, t) & = \beta_1 m(w, t-1) + (1 - \beta_1) \nabla Q_i(w) \\
 v(w, t) & = \beta_2 v(w, t-1) + (1 - \beta_2)(\nabla Q_i(w)) ^2 \\
-w & = w - \frac{\eta}{\sqrt{v(w,t) + \epsilon}}\end{split}\]
    
+w & = w - \frac{\eta m(w, t)}{\sqrt{v(w,t) + \epsilon}}\end{split}\]
    
 @@ -367,7 +408,7 @@ w & = w - \frac{\eta} {\sqrt{v(w,t) + \epsilon}} \nabla Q_{i}(w)\end{split}\
         Next 
       
       
-         Previous
+         Previous
   
diff --git a/doc/api/v2/config/pooling.html b/doc/api/v2/config/pooling.html
index 7871dfcc7d6bc1f90a5fde9b08f1866efbb45f3d..c33e11043b33acd7abc649007f0c5e39a7fa0730 100644
--- a/doc/api/v2/config/pooling.html
+++ b/doc/api/v2/config/pooling.html
@@ -65,7 +65,7 @@
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
diff --git a/doc/api/v2/data.html b/doc/api/v2/data.html
index bf80cfb18e575dc272f90545cfa7111fd81f54bc..6c4e6b8378f51a8e223fc52bb1345b56bcdbc07c 100644
--- a/doc/api/v2/data.html
+++ b/doc/api/v2/data.html
@@ -34,7 +34,7 @@
         
     
         
-        
+        
          
 
   
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -182,1076 +210,12 @@
             
   
    
 
    Data Reader Interface and DataSets
    
-
    
-
    DataTypes
    
-
    
-
    
-paddle.v2.data_type.dense_vector(dim, seq_type=0)
    
-
    Dense Vector. It means the input feature is dense float vector. For example,
-if the input is an image with 28*28 pixels, the input of Paddle neural
-network should be a dense vector with dimension 784.
    
-
    
 
 

       
     
     
    
---
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • dim (int) – dimension of this vector.
      • 
-
    • seq_type (int) – sequence type of input.
      • 
-
    
-
    Returns:
    An input type object.
    
-
    Return type:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.dense_vector_sequence(dim)
    
-
    Data type of a sequence of dense vector.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:dim (int) – dimension of dense vector.
    Returns:An input type object
    Return type:InputType
    
-
    
-
-
    
-
    
-paddle.v2.data_type.integer_value(value_range, seq_type=0)
    
-
    Data type of integer.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • seq_type (int) – sequence type of this input.
      • 
-
    • value_range (int) – range of this integer.
      • 
-
    
-
    Returns:
    An input type object
    
-
    Return type:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.integer_value_sequence(value_range)
    
-
    Data type of a sequence of integer.
    
-
---
-
-
-
-
    


    Parameters:value_range (int) – range of each element.
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_binary_vector(dim, seq_type=0)
    
-
    Sparse binary vector. It means the input feature is a sparse vector and the
-every element in this vector is either zero or one.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • dim (int) – dimension of this vector.
      • 
-
    • seq_type (int) – sequence type of this input.
      • 
-
    
-
    Returns:
    An input type object.
    
-
    Return type:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_binary_vector_sequence(dim)
    
-
    
-
    Data type of a sequence of sparse vector, which every element is either zero
    
-
    or one.
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:dim (int) – dimension of sparse vector.
    Returns:An input type object
    Return type:InputType
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_non_value_slot(dim, seq_type=0)
    
-
    Sparse binary vector. It means the input feature is a sparse vector and the
-every element in this vector is either zero or one.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • dim (int) – dimension of this vector.
      • 
-
    • seq_type (int) – sequence type of this input.
      • 
-
    
-
    Returns:
    An input type object.
    
-
    Return type:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_value_slot(dim, seq_type=0)
    
-
    Sparse vector. It means the input feature is a sparse vector. Most of the
-elements in this vector are zero, others could be any float value.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • dim (int) – dimension of this vector.
      • 
-
    • seq_type (int) – sequence type of this input.
      • 
-
    
-
    Returns:
    An input type object.
    
-
    Return type:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_vector(dim, seq_type=0)
    
-
    Sparse vector. It means the input feature is a sparse vector. Most of the
-elements in this vector are zero, others could be any float value.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • dim (int) – dimension of this vector.
      • 
-
    • seq_type (int) – sequence type of this input.
      • 
-
    
-
    Returns:
    An input type object.
    
-
    Return type:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_vector_sequence(dim)
    
-
    Data type of a sequence of sparse vector, which most elements are zero,
-others could be any float value.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:dim (int) – dimension of sparse vector.
    Returns:An input type object
    Return type:InputType
    
-
    
-
-
    
-
    
-class paddle.v2.data_type.InputType(dim, seq_type, tp)
    
-
    InputType is the base class for paddle input types.
    
-
    
-
    Note
    
-
    this is a base class, and should never be used by user.
    
-
    
-
---
-
-
-
-
    


    Parameters:
      
-
    • dim (int) – dimension of input. If the input is an integer, it means the
-value range. Otherwise, it means the size of layer.
      • 
-
    • seq_type (int) – sequence type of input. 0 means it is not a sequence. 1
-means it is a variable length sequence. 2 means it is a
-nested sequence.
      • 
-
    • type (int) – data type of input.
      • 
-
    
-
    
-
    
-
-
    
-
    
-
    DataFeeder
    
-
    
-
    
-class paddle.v2.data_feeder.DataFeeder(data_types, feeding=None)
    
-
    DataFeeder converts the data returned by paddle.reader into a data structure
-of Arguments which is defined in the API. The paddle.reader usually returns
-a list of mini-batch data entries. Each data entry in the list is one sample.
-Each sample is a list or a tuple with one feature or multiple features.
-DataFeeder converts this mini-batch data entries into Arguments in order
-to feed it to C++ interface.
    
-
    The simple usage shows below
    
-
    feeding = ['image', 'label']
-data_types = enumerate_data_types_of_data_layers(topology)
-feeder = DataFeeder(data_types=data_types, feeding=feeding)
-
-minibatch_data = [([1.0, 2.0, 3.0, ...], 5)]
-
-arg = feeder(minibatch_data)
-
    
-
    
-
    If mini-batch data and data layers are not one to one mapping, we
-could pass a dictionary to feeding parameter to represent the mapping
-relationship.
    
-
    data_types = [('image', paddle.data_type.dense_vector(784)),
-              ('label', paddle.data_type.integer_value(10))]
-feeding = {'image':0, 'label':1}
-feeder = DataFeeder(data_types=data_types, feeding=feeding)
-minibatch_data = [
-                   ( [1.0,2.0,3.0,4.0], 5, [6,7,8] ),  # first sample
-                   ( [1.0,2.0,3.0,4.0], 5, [6,7,8] )   # second sample
-                 ]
-# or minibatch_data = [
-#                       [ [1.0,2.0,3.0,4.0], 5, [6,7,8] ],  # first sample
-#                       [ [1.0,2.0,3.0,4.0], 5, [6,7,8] ]   # second sample
-#                     ]
-arg = feeder.convert(minibatch_data)
-
    
-
    
-
    
-
    Note
    
-
    This module is for internal use only. Users should use the reader
-interface.
    
-
    
-
---
-
-
-
-
    


    Parameters:
      
-
    • data_types (list) – A list to specify data name and type. Each item is
-a tuple of (data_name, data_type).
      • 
-
    • feeding (dict|collections.Sequence|None) – A dictionary or a sequence to specify the position of each
-data in the input data.
      • 
-
    
-
    
-
    
-
    
-convert(dat, argument=None)
    
-
    
---
-
-
-
-
    


    Parameters:
      
-
    • dat (list) – A list of mini-batch data. Each sample is a list or tuple
-one feature or multiple features.
      • 
-
    • argument (py_paddle.swig_paddle.Arguments) – An Arguments object contains this mini-batch data with
-one or multiple features. The Arguments definition is
-in the API.
      • 
-
    
-
    
-
    
-
-
    
-
-
    
-
    
-
    Reader
    
-
    At training and testing time, PaddlePaddle programs need to read data. To ease
-the users’ work to write data reading code, we define that
    
-
      
-
    • A reader is a function that reads data (from file, network, random number
-generator, etc) and yields data items.
      • 
-
    • A reader creator is a function that returns a reader function.
      • 
-
    • A reader decorator is a function, which accepts one or more readers, and
-returns a reader.
      • 
-
    • A batch reader is a function that reads data (from reader, file, network,
-random number generator, etc) and yields a batch of data items.
      • 
-
    
-
    
-
    Data Reader Interface
    
-
    Indeed, data reader doesn’t have to be a function that reads and yields data
-items. It can be any function with no parameter that creates a iterable
-(anything can be used in for x in iterable):
    
-
    iterable = data_reader()
-
    
-
    
-
    Element produced from the iterable should be a single entry of data,
-not a mini batch. That entry of data could be a single item, or a tuple of
-items.
-Item should be of supported type (e.g., numpy 1d
-array of float32, int, list of int)
    
-
    An example implementation for single item data reader creator:
    
-
    def reader_creator_random_image(width, height):
-    def reader():
-        while True:
-            yield numpy.random.uniform(-1, 1, size=width*height)
-return reader
-
    
-
    
-
    An example implementation for multiple item data reader creator:
    
-
    def reader_creator_random_image_and_label(width, height, label):
-    def reader():
-        while True:
-            yield numpy.random.uniform(-1, 1, size=width*height), label
-return reader
-
    
-
    
-
    TODO(yuyang18): Should we add whole design doc here?
    
-
    
-
    
-paddle.v2.reader.map_readers(func, *readers)
    
-
    Creates a data reader that outputs return value of function using
-output of each data readers as arguments.
    
-
---
-
-
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • func – function to use. The type of func should be (Sample) => Sample
      • 
-
    • readers – readers whose outputs will be used as arguments of func.
      • 
-
    
-
    Type:
    callable
    
-
    Returns:
    the created data reader.
    
-
    Return type:
    callable
    
-
    
-
    
-
-
    
-
    
-paddle.v2.reader.buffered(reader, size)
    
-
    Creates a buffered data reader.
    
-
    The buffered data reader will read and save data entries into a
-buffer. Reading from the buffered data reader will proceed as long
-as the buffer is not empty.
    
-
---
-
-
-
-
-
-
    


    Parameters:
      
-
    • reader (callable) – the data reader to read from.
      • 
-
    • size (int) – max buffer size.
      • 
-
    
-
    Returns:
    the buffered data reader.
    
-
    
-
    
-
-
    
-
    
-paddle.v2.reader.compose(*readers, **kwargs)
    
-
    Creates a data reader whose output is the combination of input readers.
    
-
    If input readers output following data entries:
-(1, 2)    3    (4, 5)
-The composed reader will output:
-(1, 2, 3, 4, 5)
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • readers – readers that will be composed together.
      • 
-
    • check_alignment (bool) – if True, will check if input readers are aligned
-correctly. If False, will not check alignment and trailing outputs
-will be discarded. Defaults to True.
      • 
-
    
-
    Returns:
    the new data reader.
    
-
    Raises:
    ComposeNotAligned – outputs of readers are not aligned.
-Will not raise when check_alignment is set to False.
    
-
    
-
    
-
-
    
-
    
-paddle.v2.reader.chain(*readers)
    
-
    Creates a data reader whose output is the outputs of input data
-readers chained together.
    
-
    If input readers output following data entries:
-[0, 0, 0]
-[1, 1, 1]
-[2, 2, 2]
-The chained reader will output:
-[0, 0, 0, 1, 1, 1, 2, 2, 2]
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:readers – input readers.
    Returns:the new data reader.
    Return type:callable
    
-
    
-
-
    
-
    
-paddle.v2.reader.shuffle(reader, buf_size)
    
-
    Creates a data reader whose data output is shuffled.
    
-
    Output from the iterator that created by original reader will be
-buffered into shuffle buffer, and then shuffled. The size of shuffle buffer
-is determined by argument buf_size.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • reader (callable) – the original reader whose output will be shuffled.
      • 
-
    • buf_size (int) – shuffle buffer size.
      • 
-
    
-
    Returns:
    the new reader whose output is shuffled.
    
-
    Return type:
    callable
    
-
    
-
    
-
-
    
-
    
-paddle.v2.reader.firstn(reader, n)
    
-
    Limit the max number of samples that reader could return.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:
    Returns:
    the decorated reader.
    
-
    Return type:
    callable
    
-
    
-
    
-
-
    
-
    Creator package contains some simple reader creator, which could be used in user
-program.
    
-
    
-
    
-paddle.v2.reader.creator.np_array(x)
    
-
    Creates a reader that yields elements of x, if it is a
-numpy vector. Or rows of x, if it is a numpy matrix.
-Or any sub-hyperplane indexed by the highest dimension.
    
-
---
-
-
-
-
-
-
    


    Parameters:x – the numpy array to create reader from.
    Returns:data reader created from x.
    
-
    
-
-
    
-
    
-paddle.v2.reader.creator.text_file(path)
    
-
    Creates a data reader that outputs text line by line from given text file.
-Trailing new line (‘\n’) of each line will be removed.
    
-
---
-
-
-
-
-
-
    


    Path:path of the text file.
    Returns:data reader of text file
    
-
    
-
-
    
-
    
-
    minibatch
    
-
    
-
    
-paddle.v2.minibatch.batch(reader, batch_size)
    
-
    Create a batched reader.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • reader (callable) – the data reader to read from.
      • 
-
    • batch_size (int) – size of each mini-batch
      • 
-
    
-
    Returns:
    the batched reader.
    
-
    Return type:
    callable
    
-
    
-
    
-
-
    
-
    
-
    Dataset
    
-
    Dataset package.
    
-
    
-
    mnist
    
-
    MNIST dataset.
    
-
    This module will download dataset from http://yann.lecun.com/exdb/mnist/ and
-parse training set and test set into paddle reader creators.
    
-
    
-
    
-paddle.v2.dataset.mnist.train()
    
-
    MNIST training set creator.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 9].
    
-
---
-
-
-
-
-
-
    


    Returns:Training reader creator
    Return type:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.mnist.test()
    
-
    MNIST test set cretor.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 9].
    
-
---
-
-
-
-
-
-
    


    Returns:Test reader creator.
    Return type:callable
    
-
    
-
-
    
-
    
-
    cifar
    
-
    CIFAR dataset.
    
-
    This module will download dataset from
-https://www.cs.toronto.edu/~kriz/cifar.html and parse train/test set into
-paddle reader creators.
    
-
    The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes,
-with 6000 images per class. There are 50000 training images and 10000 test
-images.
    
-
    The CIFAR-100 dataset is just like the CIFAR-10, except it has 100 classes
-containing 600 images each. There are 500 training images and 100 testing
-images per class.
    
-
    
-
    
-paddle.v2.dataset.cifar.train100()
    
-
    CIFAR-100 training set creator.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 99].
    
-
---
-
-
-
-
-
-
    


    Returns:Training reader creator
    Return type:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.cifar.test100()
    
-
    CIFAR-100 test set cretor.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 9].
    
-
---
-
-
-
-
-
-
    


    Returns:Test reader creator.
    Return type:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.cifar.train10()
    
-
    CIFAR-10 training set creator.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 9].
    
-
---
-
-
-
-
-
-
    


    Returns:Training reader creator
    Return type:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.cifar.test10()
    
-
    CIFAR-10 test set cretor.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 9].
    
-
---
-
-
-
-
-
-
    


    Returns:Test reader creator.
    Return type:callable
    
-
    
-
-
    
-
    
-
    conll05
    
-
    Conll05 dataset.
-Paddle semantic role labeling Book and demo use this dataset as an example.
-Because Conll05 is not free in public, the default downloaded URL is test set
-of Conll05 (which is public). Users can change URL and MD5 to their Conll
-dataset. And a pre-trained word vector model based on Wikipedia corpus is used
-to initialize SRL model.
    
-
    
-
    
-paddle.v2.dataset.conll05.get_dict()
    
-
    Get the word, verb and label dictionary of Wikipedia corpus.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.conll05.get_embedding()
    
-
    Get the trained word vector based on Wikipedia corpus.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.conll05.test()
    
-
    Conll05 test set creator.
    
-
    Because the training dataset is not free, the test dataset is used for
-training. It returns a reader creator, each sample in the reader is nine
-features, including sentence sequence, predicate, predicate context,
-predicate context flag and tagged sequence.
    
-
---
-
-
-
-
-
-
    


    Returns:Training reader creator
    Return type:callable
    
-
    
-
-
    
-
    
-
    imdb
    
-
    IMDB dataset.
    
-
    This module downloads IMDB dataset from
-http://ai.stanford.edu/%7Eamaas/data/sentiment/. This dataset contains a set
-of 25,000 highly polar movie reviews for training, and 25,000 for testing.
-Besides, this module also provides API for building dictionary.
    
-
    
-
    
-paddle.v2.dataset.imdb.build_dict(pattern, cutoff)
    
-
    Build a word dictionary from the corpus. Keys of the dictionary are words,
-and values are zero-based IDs of these words.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.imdb.train(word_idx)
    
-
    IMDB training set creator.
    
-
    It returns a reader creator, each sample in the reader is an zero-based ID
-sequence and label in [0, 1].
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:word_idx (dict) – word dictionary
    Returns:Training reader creator
    Return type:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.imdb.test(word_idx)
    
-
    IMDB test set creator.
    
-
    It returns a reader creator, each sample in the reader is an zero-based ID
-sequence and label in [0, 1].
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:word_idx (dict) – word dictionary
    Returns:Test reader creator
    Return type:callable
    
-
    
-
-
    
-
    
-
    imikolov
    
-
    imikolov’s simple dataset.
    
-
    This module will download dataset from
-http://www.fit.vutbr.cz/~imikolov/rnnlm/ and parse training set and test set
-into paddle reader creators.
    
-
    
-
    
-paddle.v2.dataset.imikolov.build_dict()
    
-
    Build a word dictionary from the corpus,  Keys of the dictionary are words,
-and values are zero-based IDs of these words.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.imikolov.train(word_idx, n)
    
-
    imikolov training set creator.
    
-
    It returns a reader creator, each sample in the reader is a word ID
-tuple.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • word_idx (dict) – word dictionary
      • 
-
    • n (int) – sliding window size
      • 
-
    
-
    Returns:
    Training reader creator
    
-
    Return type:
    callable
    
-
    
-
    
-
-
    
-
    
-paddle.v2.dataset.imikolov.test(word_idx, n)
    
-
    imikolov test set creator.
    
-
    It returns a reader creator, each sample in the reader is a word ID
-tuple.
    
-
---
-
-
-
-
-
-
-
-
    


    Parameters:
      
-
    • word_idx (dict) – word dictionary
      • 
-
    • n (int) – sliding window size
      • 
-
    
-
    Returns:
    Test reader creator
    
-
    Return type:
    callable
    
-
    
-
    
-
-
    
-
    
-
    movielens
    
-
    Movielens 1-M dataset.
    
-
    Movielens 1-M dataset contains 1 million ratings from 6000 users on 4000
-movies, which was collected by GroupLens Research. This module will download
-Movielens 1-M dataset from
-http://files.grouplens.org/datasets/movielens/ml-1m.zip and parse training
-set and test set into paddle reader creators.
    
-
    
-
    
-paddle.v2.dataset.movielens.get_movie_title_dict()
    
-
    Get movie title dictionary.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.max_movie_id()
    
-
    Get the maximum value of movie id.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.max_user_id()
    
-
    Get the maximum value of user id.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.max_job_id()
    
-
    Get the maximum value of job id.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.movie_categories()
    
-
    Get movie categoriges dictionary.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.user_info()
    
-
    Get user info dictionary.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.movie_info()
    
-
    Get movie info dictionary.
    
-
    
-
-
    
-
    
-class paddle.v2.dataset.movielens.MovieInfo(index, categories, title)
    
-
    Movie id, title and categories information are stored in MovieInfo.
    
-
    
-
-
    
-
    
-class paddle.v2.dataset.movielens.UserInfo(index, gender, age, job_id)
    
-
    User id, gender, age, and job information are stored in UserInfo.
    
-
    
-
-
    
-
    
-
    sentiment
    
-
    The script fetch and preprocess movie_reviews data set that provided by NLTK
    
-
    TODO(yuyang18): Complete dataset.
    
-
    
-
    
-paddle.v2.dataset.sentiment.get_word_dict()
    
-
    Sorted the words by the frequency of words which occur in sample
-:return:
    
-
    
-
    words_freq_sorted
    
-
    
-
-
    
-
    
-paddle.v2.dataset.sentiment.train()
    
-
    Default training set reader creator
    
-
    
-
-
    
-
    
-paddle.v2.dataset.sentiment.test()
    
-
    Default test set reader creator
    
-
    
-
-
    
-
    
-
    uci_housing
    
-
    UCI Housing dataset.
    
-
    This module will download dataset from
-https://archive.ics.uci.edu/ml/machine-learning-databases/housing/ and
-parse training set and test set into paddle reader creators.
    
-
    
-
    
-paddle.v2.dataset.uci_housing.train()
    
-
    UCI_HOUSING training set creator.
    
-
    It returns a reader creator, each sample in the reader is features after
-normalization and price number.
    
-
---
-
-
-
-
-
-
    


    Returns:Training reader creator
    Return type:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.uci_housing.test()
    
-
    UCI_HOUSING test set creator.
    
-
    It returns a reader creator, each sample in the reader is features after
-normalization and price number.
    
-
---
-
-
-
-
-
-
    


    Returns:Test reader creator
    Return type:callable
    
-
    
-
-
    
-
    
-
    wmt14
    
-
    WMT14 dataset.
-The original WMT14 dataset is too large and a small set of data for set is
-provided. This module will download dataset from
-http://paddlepaddle.cdn.bcebos.com/demo/wmt_shrinked_data/wmt14.tgz and
-parse training set and test set into paddle reader creators.
    
-
    
-
    
-paddle.v2.dataset.wmt14.train(dict_size)
    
-
    WMT14 training set creator.
    
-
    It returns a reader creator, each sample in the reader is source language
-word ID sequence, target language word ID sequence and next word ID
-sequence.
    
-
---
-
-
-
-
-
-
    


    Returns:Training reader creator
    Return type:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.wmt14.test(dict_size)
    
-
    WMT14 test set creator.
    
-
    It returns a reader creator, each sample in the reader is source language
-word ID sequence, target language word ID sequence and next word ID
-sequence.
    
-
---
-
-
-
-
-
-
    


    Returns:Test reader creator
    Return type:callable
    
-
    
-
-
    
 
    
 
    
 
@@ -1262,7 +226,7 @@ sequence.
    
   
     
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,10 +84,10 @@
       
    
         
         
 
         
@@ -109,11 +109,12 @@
         
           
           
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -187,9 +215,31 @@
 
    
 
    
 class paddle.v2.parameters.Parameters
    
-
    Parameters is a dictionary contains Paddle’s parameter. The key of
-Parameters is the name of parameter. The value of Parameters is a plain
-numpy.ndarry .
    
+
    Parameters manages all the learnable parameters in a neural network.
+It stores parameters’ information in an OrderedDict. The key is
+the name of a parameter, and value is a parameter’s configuration(in
+protobuf format), such as initialization mean and std, its size, whether it
+is a static parameter, and so on.
    
+
+++
+
+
+
+
    


    Parameters:
      
+
    • __param_conf__ (OrderedDict) – store the configurations of learnable parameters in
+the network in an OrderedDict. Parameter is added one by one into the
+dict by following their created order in the network: parameters of
+the previous layers in a network are careted first. You can visit the
+parameters from bottom to top by iterating over this dict.
      • 
+
    • __gradient_machines__ (list) – all of the parameters in a neural network are
+appended to a PaddlePaddle gradient machine, which is used internally to
+copy parameter values between C++ and Python end.
      • 
+
    • __tmp_params__ (dict) – a dict to store dummy parameters if no
+__gradient_machines__ is appended to Parameters.
      • 
+
    
+
    
 
    Basically usage is
    
 
    data = paddle.layers.data(...)
 ...
@@ -288,6 +338,26 @@ Parameters is the name of parameter. The value of Parameters is a plain
 
 
    
 
+
    
+
    
+get_grad(key)
    
+
    Get grandient by parameter name.
    
+
+++
+
+
+
+
+
+
+
+
+
+
    


    Note:It will always copy the parameter from C++ side.
    Parameters:key (basestring) – parameter name
    Returns:The grandient matrix.
    Return type:np.ndarray
    
+
    
+
 
    
 
    
 set(parameter_name, value)
    
@@ -318,7 +388,7 @@ Trainer.train.
    
 
 
 
-Parameters:gradient_machine (api.GradientMachine) – Paddle C++ GradientMachine object.
+Parameters:gradient_machine (api.GradientMachine) – PaddlePaddle C++ GradientMachine object.
 
 Returns:
 
@@ -366,6 +436,66 @@ Trainer.train.
    
 
 
    
 
+
    
+
    
+to_tar(f)
    
+
    Save parameters to a tar file.
    
+
    
+
    WARNING: You should use paddle.v2.trainer.SGD.save_parameter_to_tar(f)
    
+
    to save parameters most of the time. Otherwise, some settings such
+as model average will not take effect.
    
+
    
+
+++
+
+
+
+
+
+
    


    Parameters:f (file) – 
    Returns:
    
+
    
+
+
    
+
    
+static from_tar(f)
    
+
    Create a Parameters object from the given file. And
+the Parameters only contains the parameters in this
+file. It is adapted the parameters are same in the
+defined network and the given file. For example, it
+can be used in the inference.
    
+
+++
+
+
+
+
+
+
+
+
    


    Parameters:f (tar file) – the initialized model file.
    Returns:A Parameters object.
    Return type:Parameters.
    
+
    
+
+
    
+
    
+init_from_tar(f)
    
+
    Different from from_tar, this interface can be used to
+init partial network parameters from another saved model.
    
+
+++
+
+
+
+
+
+
    


    Parameters:f (tar file) – the initialized model file.
    Returns:Nothing.
    
+
    
+
 
    
 
 
    
@@ -374,7 +504,7 @@ Trainer.train.
    
 
    Module Trainer
    
 
    
 
    
-class paddle.v2.trainer.SGD(cost, parameters, update_equation, extra_layers=None)
    
+class paddle.v2.trainer.SGD(cost, parameters, update_equation, extra_layers=None, is_local=True, pserver_spec=None, use_etcd=True)
 
    Simple SGD Trainer.
 SGD Trainer combines data reader, network topolopy and update_equation together
 to train/test a neural network.
    
@@ -383,11 +513,18 @@ to train/test a neural network.
    
 
 
 Parameters:
      
-
    • update_equation (paddle.v2.optimizer.Optimizer) – The optimizer object.
      • 
 
    • cost (paddle.v2.config_base.Layer) – Target cost that neural network should be optimized.
      • 
 
    • parameters (paddle.v2.parameters.Parameters) – The parameters dictionary.
      • 
+
    • update_equation (paddle.v2.optimizer.Optimizer) – The optimizer object.
      • 
 
    • extra_layers (paddle.v2.config_base.Layer) – Some layers in the neural network graph are not
 in the path of cost layer.
      • 
+
    • is_local (bool) – Whether trainning locally
      • 
+
    • pserver_spec (string) – comma string for pserver location,
+eg:127.10.0.10:3000,127.10.0.11:3000,
+and this parameter is only used for fault
+tolerant mode cluster training.
      • 
+
    • use_etcd – Whether using etcd pserver.
      • 
+
    • use_etcd – bool
      • 
 
    
 
 
@@ -405,7 +542,7 @@ in the path of cost layer.
 
  • reader (collections.Iterable) – A reader that reads and yeilds data items. Usually we use a
 batched reader to do mini-batch training.
    • 
 
  • num_passes – The total train passes.
    • 
-
  • event_handler ((BaseEvent) => None) – Event handler. A method will be invoked when event
+
  • event_handler ((BaseEvent) => None) – Event handler. A method will be invoked when event
 occurred.
    • 
 
  • feeding (dict|list) – Feeding is a map of neural network input name and array
 index that reader returns.
    • 
@@ -428,7 +565,8 @@ index that reader returns.
 
 
 Parameters:
      
-
    • reader (collections.Iterable) – A reader that reads and yeilds data items.
      • 
+
    • reader (collections.Iterable) – A batch reader that reads and yeilds data items,
+it should be a paddle.v2.batch.
      • 
 
    • feeding (dict) – Feeding is a map of neural network input name and array
 index that reader returns.
      • 
 
    
@@ -469,8 +607,10 @@ index that reader returns.
 
 
    
 
    
-class paddle.v2.event.EndPass(pass_id, evaluator)
    
-
    Event On One Pass Training Complete.
    
+class paddle.v2.event.EndPass(pass_id, evaluator, gm)
+
    Event On One Pass Training Complete.
+To get the output of a specific layer, add “event.gm.getLayerOutputs(‘predict_layer’)”
+in your event_handler call back
    
 
    
 
 
    
@@ -481,8 +621,16 @@ index that reader returns.
 
 
    
 
    
-class paddle.v2.event.EndIteration(pass_id, batch_id, cost, evaluator)
    
-
    Event On One Batch Training Complete.
    
+class paddle.v2.event.EndForwardBackward(pass_id, batch_id, gm)
+
    Event On One Batch ForwardBackward Complete.
    
+
    
+
+
    
+
    
+class paddle.v2.event.EndIteration(pass_id, batch_id, cost, evaluator, gm)
    
+
    Event On One Batch Training Complete.
+To get the output of a specific layer, add “event.gm.getLayerOutputs(‘predict_layer’)”
+in your event_handler call back
    
 
    
 
 
    
@@ -493,9 +641,18 @@ index that reader returns.
 paddle.v2.infer(output_layer, parameters, input, feeding=None, field='value')
 
    Infer a neural network by given neural network output and parameters.  The
 user should pass either a batch of input data or reader method.
    
-
    Example usages:
    
-
    result = paddle.infer(prediction, parameters, input=SomeData,
-                      batch_size=32)
+
    Example usage for sinlge output_layer:
    
+
    result = paddle.infer(output_layer=prediction,
+                      parameters=parameters,
+                      input=SomeData)
+print result
+
    
+
    
+
    Example usage for multiple outout_layers and fields:
    
+
    result = paddle.infer(output_layer=[prediction1, prediction2],
+                      parameters=parameters,
+                      input=SomeData,
+                      field=[id, value]])
 print result
 
    
 
    
@@ -504,7 +661,8 @@ user should pass either a batch of input data or reader method.
    
 
 
 Parameters:
      
-
    • output_layer (paddle.v2.config_base.Layer) – output of the neural network that would be inferred
      • 
+
    • output_layer (paddle.v2.config_base.Layer or a list of
+paddle.v2.config_base.Layer) – output of the neural network that would be inferred
      • 
 
    • parameters (paddle.v2.parameters.Parameters) – parameters of the neural network.
      • 
 
    • input (collections.Iterable) – input data batch. Should be a python iterable object, and each
 element is the data batch.
      • 
@@ -518,7 +676,9 @@ or max_id.
 
    
 
 
-Returns:
    a numpy array
    
+Returns:
    The prediction result. If there are multiple outout_layers and fields,
+the return order is outout_layer1.field1, outout_layer2.field1, ...,
+outout_layer1.field2, outout_layer2.field2 ...
    
 
 
 Return type:
    numpy.ndarray
    
@@ -538,10 +698,10 @@ or max_id.
   
     
   
diff --git a/doc/design/api.html b/doc/design/api.html
index 4c6aba58305d4a6c3ea4fab84374f0728a67283d..e8d652ac2f2895dea62278b5c64e110f692686bc 100644
--- a/doc/design/api.html
+++ b/doc/design/api.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,7 +84,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -180,7 +208,7 @@
 
    
 
    Ingredients
    
 
    As our design principle is starting from the essence: how could we
-allow users to express and solve their problems at neural networks.
+allow users to express and solve their problems as neural networks.
 Some essential concepts that our API have to provide include:
    
 
      
 
    1. A topology is an expression of layers.
      2. 
@@ -383,7 +411,7 @@ access a Kubernetes cluster, s/he should be able to call
      
                   num_parameter_servers=15)
 
    
 
    
-
    The pseudo code if paddle.dist_train is as follows:
    
+
    The pseudo code of paddle.dist_train is as follows:
    
 
    def dist_train(topology, parameters, trainer, reader, ...):
     if os.getenv("KUBERNETES_SERVICE_HOST") == None:
         image_name = k8s_user + '/' + k8s_job
diff --git a/doc/design/auto_gradient_check.html b/doc/design/auto_gradient_check.html
new file mode 100644
index 0000000000000000000000000000000000000000..30987f9c8071a276c5cbffd32f5ecf14302d55d5
--- /dev/null
+++ b/doc/design/auto_gradient_check.html
@@ -0,0 +1,430 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Auto Gradient Checker Design — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Auto Gradient Checker Design
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Auto Gradient Checker Design
    
+
    
+
    
+
    Backgraound：
    
+
      
+
    • Generally, it is easy to check whether the forward computation of an Operator is correct or not. However, backpropagation is a notoriously difficult algorithm to debug and get right:
        
+
      1. you should get the right backpropagation formula according to the forward computation.
        2. 
+
      2. you should implement it right in CPP.
        3. 
+
      3. it’s difficult to prepare test data.
        4. 
+
      
+
      • 
+
    • Auto gradient checking gets a numerical gradient by forward Operator and use it as a reference of the backward Operator’s result. It has several advantages:
        
+
      1. numerical gradient checker only need forward operator.
        2. 
+
      2. user only need to prepare the input data for forward Operator.
        3. 
+
      
+
      • 
+
    
+
    
+
    
+
    Mathematical Theory
    
+
    The following two document from Stanford has a detailed explanation of how to get numerical gradient and why it’s useful.
    
+
    
+
+
    
+
    
+
    
+
    Numeric Gradient Implementation
    
+
    
+
    Python Interface
    
+
    def get_numerical_gradient(op,
+                         input_values,
+                         output_name,
+                         input_to_check,
+                         delta=0.005,
+                         local_scope=None):
+    """
+    Get Numeric Gradient for an operator's input.
+
+    :param op: C++ operator instance, could be an network
+    :param input_values: The input variables. Should be an dictionary, whose key is
+    variable name, and value is numpy array.
+    :param output_name: The final output variable name.
+    :param input_to_check: The input variable with respect to which to compute the gradient.
+    :param delta: The perturbation value for numeric gradient method. The
+    smaller delta is, the more accurate result will get. But if that delta is
+     too small, it will suffer from numerical stability problem.
+    :param local_scope: The local scope used for get_numeric_gradient.
+    :return: The gradient array in numpy format.
+    """
+
    
+
    
+
    
+
    
+
    Explaination:
    
+
      
+
    • Why need output_name
        
+
      • An Operator may have multiple Output, one can get independent gradient from each Output. So caller should specify the name of the output variable.
        • 
+
      
+
      • 
+
    • Why need input_to_check
        
+
      • One operator may have multiple inputs. Gradient Op can calculate the gradient of these inputs at the same time. But Numeric Gradient needs to calculate them one by one. So get_numeric_gradient is designed to calculate the gradient for one input. If you need to compute multiple inputs, you can call get_numeric_gradient multiple times.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    Core Algorithm Implementation
    
+
        # we only compute gradient of one element a time.
+    # we use a for loop to compute the gradient of each element.
+    for i in xrange(tensor_size):
+        # get one input element by its index i.
+        origin = tensor_to_check.get_float_element(i)
+
+        # add delta to it, run op and then get the new value of the result tensor.
+        x_pos = origin + delta
+        tensor_to_check.set_float_element(i, x_pos)
+        y_pos = get_output()
+
+        # plus delta to this element, run op and get the new value of the result tensor.
+        x_neg = origin - delta
+        tensor_to_check.set_float_element(i, x_neg)
+        y_neg = get_output()
+
+        # restore old value
+        tensor_to_check.set_float_element(i, origin)
+
+        # compute the gradient of this element and store it into a numpy array.
+        gradient_flat[i] = (y_pos - y_neg) / delta / 2
+
+    # reshape the gradient result to the shape of the source tensor.
+    return gradient_flat.reshape(tensor_to_check.get_dims())
+
    
+
    
+
    
+
    
+
    
+
    Auto Graident Checker Framework
    
+
    Each Operator Kernel has three kinds of Gradient:
    
+
      
+
    1. Numerical gradient
      2. 
+
    2. CPU kernel gradient
      3. 
+
    3. GPU kernel gradient (if supported)
      4. 
+
    
+
    The numerical gradient only relies on forward Operator. So we use the numerical gradient as the reference value. And the gradient checking is performed in the following three steps:
    
+
      
+
    1. calculate the numerical gradient
      2. 
+
    2. calculate CPU kernel gradient with the backward Operator and compare it with the numerical gradient
      3. 
+
    3. calculate GPU kernel gradient with the backward Operator and compare it with the numeric gradient (if supported)
      4. 
+
    
+
    
+
    Python Interface
    
+
        def check_grad(self,
+                   forward_op,
+                   input_vars,
+                   inputs_to_check,
+                   output_name,
+                   no_grad_set=None,
+                   only_cpu=False,
+                   max_relative_error=0.005):
+        """
+        :param forward_op: used to create backward_op
+        :param input_vars: numpy value of input variable. The following
+            computation will use these variables.
+        :param inputs_to_check: the input variable with respect to which to compute the gradient.
+        :param output_name: The final output variable name.
+        :param max_relative_error: The relative tolerance parameter.
+        :param no_grad_set: used when create backward ops
+        :param only_cpu: only compute and check gradient on cpu kernel.
+        :return:
+        """
+
    
+
    
+
    
+
    
+
    How to check if two numpy array is close enough?
    
+
    if abs_numerical_grad is nearly zero, then use abs error for numerical_grad
    
+
    numerical_grad = ...
+operator_grad = numpy.array(scope.find_var(grad_var_name(name)).get_tensor())
+
+abs_numerical_grad = numpy.abs(numerical_grad)
+# if abs_numerical_grad is nearly zero, then use abs error for numeric_grad, not relative
+# error.
+abs_numerical_grad[abs_numerical_grad < 1e-3] = 1
+
+diff_mat = numpy.abs(abs_numerical_grad - operator_grad) / abs_numerical_grad
+max_diff = numpy.max(diff_mat)
+
    
+
    
+
    
+
    Notes：
    
+
    The Input data for auto gradient checker should be reasonable to avoid numerical  stability problem.
    
+
    
+
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/block.html b/doc/design/block.html
new file mode 100644
index 0000000000000000000000000000000000000000..eb7c39f8a57c6017c1d90a7932de0b97cd325071
--- /dev/null
+++ b/doc/design/block.html
@@ -0,0 +1,570 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Block and Scope — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Block and Scope
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Block and Scope
    
+
    
+
    The Representation of Computation
    
+
    Both deep learning systems and programming languages help users describe computation procedures.  These systems use various representations of computation:
    
+
      
+
    • Caffe, Torch, and Paddle: sequences of layers.
      • 
+
    • TensorFlow, Caffe2, Mxnet: graph of operators.
      • 
+
    • PaddlePaddle: nested blocks, like C++ and Java programs.
      • 
+
    
+
    
+
    
+
    Block in Programming Languages and Deep Learning
    
+
    In programming languages, a block is a pair of curly braces that includes local variables definitions and a sequence of instructions or operators.
    
+
    Blocks work with control flow structures like if, else, and for, which have equivalents in deep learning:
    
+
    | programming languages | PaddlePaddle          |
+|———————–|———————–|
+| for, while loop       | RNN, WhileOp          |
+| if, if-else, switch   | IfElseOp, SwitchOp    |
+| sequential execution  | a sequence of layers  |
    
+
    A key difference is that a C++ program describes a one pass computation, whereas a deep learning program describes both the forward and backward passes.
    
+
    
+
    
+
    Stack Frames and the Scope Hierarchy
    
+
    The existence of the backward pass makes the execution of a block of PaddlePaddle different from traditional programs:
    
+
    | programming languages | PaddlePaddle                    |
+|———————–|———————————|
+| stack                 | scope hierarchy                 |
+| stack frame           | scope                           |
+| push at entering block| push at entering block          |
+| pop at leaving block  | destroy when minibatch completes|
    
+
      
+
    1. In traditional programs:
        
+
      • When the execution enters the left curly brace of a block, the runtime pushes a frame into the stack, where it realizes local variables.
        • 
+
      • After the execution leaves the right curly brace, the runtime pops the frame.
        • 
+
      • The maximum number of frames in the stack is the maximum depth of nested blocks.
        • 
+
      
+
      2. 
+
    2. In PaddlePaddle
        
+
      • When the execution enters a block, PaddlePaddle adds a new scope, where it realizes variables.
        • 
+
      • PaddlePaddle doesn’t pop a scope after the execution of the block because variables therein are used by the backward pass.  So it has a stack forest known as a scope hierarchy.
        • 
+
      • The height of the highest tree is the maximum depth of nested blocks.
        • 
+
      • After the processing of a minibatch, PaddlePaddle destroys the scope hierarchy.
        • 
+
      
+
      3. 
+
    
+
    
+
    
+
    Use Blocks in C++ and PaddlePaddle Programs
    
+
    Let us consolidate the discussion by presenting some examples.
    
+
    
+
    Blocks with if-else and IfElseOp
    
+
    The following C++ programs shows how blocks are used with the if-else structure:
    
+
    namespace pd = paddle;
+
+int x = 10;
+int y = 1;
+int z = 10;
+bool cond = false;
+int o1, o2;
+if (cond) {
+  int z = x + y;
+  o1 = z;
+  o2 = pd::layer::softmax(z);
+} else {
+  int d = pd::layer::fc(z);
+  o1 = d;
+  o2 = d+1;
+}
+
    
+
    
+
    An equivalent PaddlePaddle program from the design doc of the IfElseOp operator is as follows:
    
+
    import paddle as pd
+
+x = minibatch([10, 20, 30]) # shape=[None, 1]
+y = var(1) # shape=[1], value=1
+z = minibatch([10, 20, 30]) # shape=[None, 1]
+cond = larger_than(x, 15) # [false, true, true]
+
+ie = pd.ifelse()
+with ie.true_block():
+    d = pd.layer.add_scalar(x, y)
+    ie.output(d, pd.layer.softmax(d))
+with ie.false_block():
+    d = pd.layer.fc(z)
+    ie.output(d, d+1)
+o1, o2 = ie(cond)
+
    
+
    
+
    In both examples, the left branch computes x+y and softmax(x+y), the right branch computes fc(x) and x+1 .
    
+
    The difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances.
    
+
    
+
    
+
    Blocks with for and RNNOp
    
+
    The following RNN model in PaddlePaddle from the RNN design doc :
    
+
    x = sequence([10, 20, 30]) # shape=[None, 1]
+m = var(0) # shape=[1]
+W = var(0.314, param=true) # shape=[1]
+U = var(0.375, param=true) # shape=[1]
+
+rnn = pd.rnn()
+with rnn.step():
+  h = rnn.memory(init = m)
+  h_prev = rnn.previous_memory(h)
+  a = layer.fc(W, x)
+  b = layer.fc(U, h_prev)  
+  s = pd.add(a, b)
+  act = pd.sigmoid(s)
+  rnn.update_memory(h, act)
+  rnn.output(a, b)
+o1, o2 = rnn()
+
    
+
    
+
    has its equivalent C++ program as follows
    
+
    int* x = {10, 20, 30};
+int* m = {0};
+int* W = {0.314};
+int* U = {0.375};
+
+int mem[sizeof(x) / sizeof(x[0]) + 1];
+int o1[sizeof(x) / sizeof(x[0]) + 1];
+int o2[sizeof(x) / sizeof(x[0]) + 1];
+for (int i = 1; i <= sizeof(x)/sizeof(x[0]); ++i) {
+  int x = x[i-1];
+  if (i == 1) mem[0] = m;
+  int a = W * x;
+  int b = Y * mem[i-1];
+  int s = fc_out + hidden_out;
+  int act = sigmoid(sum);
+  mem[i] = act;
+  o1[i] = act;
+  o2[i] = hidden_out;
+}
+
    
+
    
+
    
+
    
+
    
+
    Compilation and Execution
    
+
    Like TensorFlow, a PaddlePaddle program is written in Python. The first part describes a neural network as a protobuf message, and the rest executes the message for training or inference.
    
+
    The generation of this protobuf message is similar to how a compiler generates a binary executable file. The execution of the message is similar to how the OS executes the binary file.
    
+
    
+
    
+
    The “Binary Executable File Format”
    
+
    The definition of the protobuf message is as follows:
    
+
    message BlockDesc {
+  repeated VarDesc vars = 1;
+  repeated OpDesc ops = 2;
+}
+
    
+
    
+
    The step net in above RNN example would look like
    
+
    BlockDesc {
+  vars = {
+    VarDesc {...} // x
+    VarDesc {...} // h
+    VarDesc {...} // fc_out
+    VarDesc {...} // hidden_out
+    VarDesc {...} // sum
+    VarDesc {...} // act
+  }
+  ops = {
+    OpDesc {...} // matmul
+    OpDesc {...} // add_two
+    OpDesc {...} // sigmoid
+  }
+};
+
    
+
    
+
    Also, the RNN operator in above example is serialized into a protobuf message of type OpDesc and would look like:
    
+
    OpDesc {
+  inputs = {0} // the index of x in vars of BlockDesc above
+  outputs = {5, 3} // indices of act and hidden_out in vars of BlockDesc above
+  attrs {
+    "states" : {1} // the index of h
+    "step_net" : <above step net>
+  }
+};
+
    
+
    
+
    This OpDesc value is in the ops field of the BlockDesc value representing the global block.
    
+
    
+
    
+
    The Compilation of Blocks
    
+
    During the generation of the Protobuf message, the Block should store VarDesc (the Protobuf message which describes Variable) and OpDesc (the Protobuf message which describes Operator).
    
+
    VarDesc in a block should have its name scope to avoid local variables affect parent block’s name scope.
+Child block’s name scopes should inherit the parent’s so that OpDesc in child block can reference a VarDesc that stored in parent block. For example:
    
+
    a = pd.Variable(shape=[20, 20])
+b = pd.fc(a, params=["fc.w", "fc.b"])
+
+rnn = pd.create_rnn()
+with rnn.stepnet():
+    x = a.as_step_input()
+    # reuse fc's parameter
+    fc_without_b = pd.get_variable("fc.w")
+    rnn.output(fc_without_b)
+
+out = rnn()
+
    
+
    
+
    The method pd.get_variable can help retrieve a Variable by the name. The Variable may be stored in a parent block, but might be retrieved in a child block, so block should have a variable scope that supports inheritance.
    
+
    In compiler design, the symbol table is a data structure created and maintained by compilers to store information about the occurrence of various entities such as variable names, function names, classes, etc.
    
+
    To store the definition of variables and operators, we define a C++ class SymbolTable, like the one used in compilers.
    
+
    SymbolTable can do the following:
    
+
      
+
    • store the definitions (some names and attributes) of variables and operators,
      • 
+
    • verify if a variable was declared,
      • 
+
    • make it possible to implement type checking (offer Protobuf message pointers to InferShape handlers).
      • 
+
    
+
    // Information in SymbolTable is enough to trace the dependency graph. So maybe
+// the Eval() interface takes a SymbolTable is enough.
+class SymbolTable {
+ public:
+  SymbolTable(SymbolTable* parent) : parent_(parent) {}
+
+  OpDesc* NewOp(const string& name="");
+
+  // TODO determine whether name is generated by python or C++.
+  // Currently assume that a unique name will be generated by C++ if the
+  // argument name is left default.
+  VarDesc* Var(const string& name="");
+
+  // find a VarDesc by name, if recursive is true, find parent's SymbolTable
+  // recursively.
+  // this interface is introduced to support InferShape, find protobuf messages
+  // of variables and operators, pass pointers into InferShape.
+  //
+  // NOTE maybe some C++ classes such as VarDescBuilder and OpDescBuilder should
+  // be proposed and embedded into pybind to enable python operation on C++ pointers.
+  VarDesc* FindVar(const string& name, bool recursive=true);
+
+  OpDesc* FindOp(const string& name);
+
+  BlockDesc Compile() const;
+
+ private:
+  SymbolTable* parent_;
+
+  map<string, OpDesc> ops_;
+  map<string, VarDesc> vars_;
+};
+
    
+
    
+
    After all the description of variables and operators is added into SymbolTable,
+the block has enough information to run.
    
+
    The Block class takes a BlockDesc as input, and provides Run and InferShape functions.
    
+
    namespace {
+
+class Block : OperatorBase {
+public:
+  Block(const BlockDesc& desc) desc_(desc) {}
+
+  void InferShape(const framework::Scope& scope) const override {
+    if (!symbols_ready_) {
+      CreateVariables(scope);
+      CreateOperators();
+    }
+    // should run InferShape first.
+    for (auto& op : runtime_table_.ops()) {
+      op->InferShape(scope);
+    }
+  }
+
+  void Run(const framework::Scope& scope,
+           const platform::DeviceContext& dev_ctx) const override {
+    PADDLE_ENFORCE(symbols_ready_, "operators and variables should be created first.");
+    for (auto& op : runtime_table_.ops()) {
+      op->Run(scope, dev_ctx);
+    }
+  }
+
+  void CreateVariables(const framework::Scope& scope);
+  void CreateOperators();
+
+  // some other necessary interfaces of NetOp are listed below
+  // ...
+
+private:
+  BlockDesc desc_;
+  bool symbols_ready_{false};
+};
+
    
+
    
+
    
+
    
+
    The Execution of Blocks
    
+
    Block inherits from OperatorBase, which has a Run method.
+Block’s Run method will run its operators sequentially.
    
+
    There is another important interface called Eval, which takes some arguments called targets and generates a minimal graph which treats targets as the end points and creates a new Block. After Run, Eval will get the latest value and return the targets.
    
+
    The definition of Eval is as follows:
    
+
    // clean a block description by targets using the corresponding dependency graph.
+// return a new BlockDesc with minimal number of operators.
+// NOTE: The return type is not a Block but the block's description so that this can be distributed
+// to a cluster.
+BlockDesc Prune(const BlockDesc& desc, vector<string> targets);
+
+void Block::Eval(const vector<string>& targets,
+                 const framework::Scope& scope,
+                 const platform::DeviceContext& dev_ctx) {
+  BlockDesc min_desc = Prune(desc_, targets);
+  Block min_block(min_desc);
+  min_block.Run(scope, dev_ctx);
+}
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/build_system/README.html b/doc/design/build_system/README.html
new file mode 100644
index 0000000000000000000000000000000000000000..18e3bc9f721ca87893eb81a81f110757bf00f60c
--- /dev/null
+++ b/doc/design/build_system/README.html
@@ -0,0 +1,412 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Required CMake Function — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Required CMake Function
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    A few months ago when we were trying to replace CMake with Bazel, @emailweixu suggested that we rewrite those handy Bazel functions using CMake. Now it seems that it’s the right time to get this done, as we are facing problems from the porting of Majel and the development of new the parameter server using Go and C++.
    
+
    Here are some initial thoughts. Your comments are welcome!
    
+
    
+
    Required CMake Function
    
+
    I think we need only the following few CMake functions to make a project description mean and clean:
    
+
    | C++ | CUDA C++ | Go |
+|—|—|—|
+| cc_library | nv_library | go_library |
+| cc_binary | nv_binary | go_binary |
+| cc_test | nv_test | go_test |
    
+
      
+
    • The _library functions generate  .a files from source code.
      • 
+
    • The _binary functions generate executable binary files.
      • 
+
    • The _test functions generate executable unit test files. They work like _binary but links -lgtest and -lgtest_main.
      • 
+
    
+
    The difference between nv_ functions and cc_ functions is that the former use nvcc instead of the system-default C++ compiler.
    
+
    Both nv_ and cc_ functions enables C++11 (-std=c++11).
    
+
    Also,
    
+
      
+
    • to describe external dependencies, we need external_library.
      • 
+
    • to build shared libraries, we need shared_library.
      • 
+
    
+
    
+
    
+
    An Example Project
    
+
    Suppose that we have aforementioned functions defined in our /cmake directory.  The following example CMakeLists.txt describes a project including the following source files:
    
+
      
+
    • tensor.h
      • 
+
    • tensor.cc
      • 
+
    • tensor_test.cc
      • 
+
    • ops.h
      • 
+
    • ops.cu
      • 
+
    • ops_test.cu
      • 
+
    • api.go
      • 
+
    • api_test.go
      • 
+
    
+
    Suppose that ops.cu depends on CUDNN.
    
+
    # cc_binary parses tensor.cc and figures out that target also depend
+# on tensor.h.
+cc_binary(tensor
+  SRCS
+  tensor.cc)
+
+# The dependency to target tensor implies that if any of
+# tensor{.h,.cc,_test.cc} is changed, tensor_test need to be re-built.
+cc_test(tensor_test
+  SRCS
+  tensor_test.cc
+  DEPS
+  tensor)
+
+# I don't have a clear idea what parameters external_library need to
+# have.  @gangliao as a CMake expert would have better ideas.
+external_library(cudnn
+  ....)
+
+# Suppose that ops.cu depends on external target CUDNN.  Also, ops.cu
+# include global functions that take Tensor as their parameters, so
+# ops depend on tensor.  This implies that if any of tensor.{h.cc},
+# ops.{h,cu} is changed, ops need to be re-built.
+nv_library(ops
+  SRCS
+  ops.cu
+  DEPS
+  tensor
+  cudnn)  # cudnn is defined later.
+
+nv_test(ops_test
+  SRCS
+  ops_test.cu
+  DEPS
+  ops)
+
+# Because api.go defines a GO wrapper to ops and tensor, it depends on
+# both.  This implies that if any of tensor.{h,cc}, ops.{h,cu}, or
+# api.go is changed, api need to be re-built.
+go_library(api
+  SRCS
+  api.go
+  DEPS
+  tensor # Because ops depend on tensor, this line is optional.
+  ops)
+
+go_test(api_test
+  SRCS
+  api_test.go
+  DEPS
+  api)
+
+
+# This builds libapi.so.  shared_library might use CMake target
+# api_shared so to distinguish it from above target api.
+shared_library(api
+  DEPS
+  api)
+
    
+
    
+
    
+
    
+
    Implementation
    
+
    As above example CMakeLists.txt executes, each function invocation adds “nodes” to a dependency graph.  It also use this graph to generate CMake commands including add_executable, add_dependencies, target_link_libraries, and add_test.
    
+
    
+
    
+
    Using Package Manager For Go
    
+
    Building Go binaries and libraries need to satisfy their dependencies, generally
+we can do go get ./... to download and compile all external dependencies. The
+problems are:
    
+
      
+
    1. go get will always get the latest code from the default branch of the
+remote repo, so changes of dependents might break the build. This is very
+different with what we already have in cmake/external which download a
+specific version or commit id of the dependency.
      2. 
+
    2. Some locations can not access external dependencies through the internet, as mentioned
+in https://github.com/PaddlePaddle/Paddle/issues/2605. Using package management
+tools can package the dependencies as a “vendor” package, which can be mirrored
+at many cloud file hosting, so users what to compile paddle by themselves can
+download this “vendor” package from a mirror site.
      3. 
+
    
+
    
+
    Choose A Suitable Tool
    
+
    As mentioned by @wangkuiyi, Here
+list dozens of Go package managers. We choose the tool using following principles:
    
+
      
+
    • Most “active” projects with more stars, more pull requests or commits
      • 
+
    • Widely used project
      • 
+
    
+
    After comparing all these projects, we shall choose between the most popular
+tools: Godep and Glide.
    
+
    Here’s a brief comparison between Godep and Glide
+: https://github.com/Masterminds/glide/wiki/Go-Package-Manager-Comparison. There are
+also many complaints about using Godep. There’s also a new “official” pakcage
+management tool has been started at: https://github.com/golang/dep to resolve
+such problems, but it’s currently at Alpha stage. So the best choice now is
+glide obviously.
    
+
    
+
    
+
    Manage Go Packages
    
+
      
+
    • Dependencies: go/glide.yaml will store the dependencies and their versions which
+is directly imported by paddle. go/glide.lock will store all dependencies recursively
+with their commit id. Builds will “lock” to these packages if we don’t glide up
+them
      • 
+
    • Vendor package: go/vendor directory will generated when running cmake command. cmake
+will download the code corresponding to go/glide.lock. If we put a vendor folder
+under go/, cmake will just check the commit id to the packages under the folder,
+if commit id matches, there will be no download at all.
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/dist/README.html b/doc/design/cluster_train/README.html
similarity index 70%
rename from doc/design/dist/README.html
rename to doc/design/cluster_train/README.html
index b384e72b4559b1ae3439fbc9db42e8fe379f3cfe..fb8df57f5639da8bcfc7c2f58f6060b1cd332f27 100644
--- a/doc/design/dist/README.html
+++ b/doc/design/cluster_train/README.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,7 +84,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -191,15 +219,17 @@
 
    Training Job
    
 
    A training job will be created once user asks Paddle cloud to train a model. The training job is made up of different processes that collaboratively consume data and produce a trained model. There are three kinds of processes:
    
 
      
-
    1. the master process, which dispatches tasks to
      2. 
+
    2. the master server process, which dispatches tasks to
      3. 
 
    3. one or more trainer processes, which run distributed training and synchronize gradients/models via
      4. 
-
    4. one or more parameter server processes, where each holds a shard of the global model.
      5. 
+
    5. one or more parameter server processes, where each holds a shard of the global model, and receive the uploaded gradients from every trainer process, so they can run the optimize functions to update their parameters.
      6. 
 
    
 
    Their relation is illustrated in the following graph:
    
 
    
-
    
-
    Master Process
    
-
    The master process will:
    
+
    By coordinating these processes, PaddlePaddle supports use both Synchronize Stochastic Gradient Descent (sync SGD) and Asynchronous Stochastic Gradient Descent (async SGD) to train user-defined neural network topologies.
    
+
    When training with sync SGD, parameter servers wait for all trainers to finish gradients update and then send the updated parameters to trainers, training can not proceed until the trainer received the updated parameters. This creates a synchronization point between trainers. When training with async SGD, each trainer upload gradient and download new parameters individually, without the synchronization with other trainers. Using asyc SGD will be faster in terms of time per pass, but have more noise in gradient since trainers are likely to have a stale model.
    
+
    
+
    Master Server Process
    
+
    The master server process will:
    
 
      
 
    • Partition a dataset into tasks and dispatch tasks to trainers.
      • 
 
    • Keep track of training progress on the dataset with task queue. A training job will iterate on the dataset for a full pass until it goes into next pass.
      • 
@@ -210,10 +240,10 @@
 
    
 
    
 
    Task Queue
    
-
    The master process has three task queues to track training progress. As illustrated in the graph below, Job A and Job B both have one master process. Each master process has three task queues.
    
+
    The master server has three task queues to track training progress. As illustrated in the graph below, Job A and Job B both have one master server. Each master server process has three task queues.
    
 
    
 
      
-
    • The todo queue holds tasks to be dispatched. When a job starts, the master process fills in the todo queue with all tasks.
      • 
+
    • The todo queue holds tasks to be dispatched. When a job starts, the master server fills in the todo queue with all tasks.
      • 
 
    • The pending queue holds tasks that are currently training by trainers.
      • 
 
    • the done queue holds tasks that are already trained.
      • 
 
    
@@ -221,10 +251,10 @@
 
    
 
      
 
    1. When a new pass of training starts, all tasks will be placed in the todo queue.
      2. 
-
    2. The master process will dispatch few tasks to each trainer at a time, puts them in the pending queue and waits for completion.
      3. 
-
    3. The trainer will work on its tasks and tell the master process once a task is completed. The master process will dispatch a new task to that trainer.
      4. 
-
    4. If a task timeout. the master process will move it back to the todo queue. The timeout count will increase by one. If the timeout count is above a threshold, the task is likely to cause a trainer to crash, so it will be discarded.
      5. 
-
    5. The master process will move completed task to the done queue. When the todo queue is empty, the master process will start a new pass by moving all tasks in the done queue to todo queue and reset the timeout counter of all tasks to zero.
      6. 
+
    6. Upon trainer requests for new task, the master server will dispatch a task from todo queue to it, put the task in the pending queue and wait for completion.
      7. 
+
    7. The trainer will work on its task and tell the master server once the task is completed and ask for new task. The master server will dispatch a new task to that trainer.
      8. 
+
    8. If a task fails for any reason in trainer, or takes longer than a specific period of time,  the master server will move the task back to the todo queue. The timeout count for that task will increase by one. If the timeout count is above a threshold, the task is likely to cause a trainer to crash, then it will be discarded.
      9. 
+
    9. The master server will move completed task to the done queue. When the todo queue is empty, the master server will start a new pass by moving all tasks in the done queue to todo queue and reset the timeout counter of all tasks to zero.
      10. 
 
    
 
    
 
    
@@ -232,8 +262,9 @@
 
    Trainer Process
    
 
    The trainer process will:
    
 
      
-
    • Receive tasks from the master.
      • 
-
    • Work on the tasks: calculate and upload gradient to parameter servers, and update local model by downloading new parameters from parameter servers.
      • 
+
    • Request tasks from the master.
      • 
+
    • Work on the tasks
      • 
+
    • Upload gradient to parameter servers, and update local model by downloading new parameters from parameter servers.
      • 
 
    
 
    
 
    
@@ -265,7 +296,7 @@
 
    
 
    
 
    Fault Tolerant
    
-
    The training job will pause if the master processes is dead, or any of the parameter server process is dead. They will be started by Kubernetes and recover in few minutes. Please refer to fault recovery.
    
+
    The training job will pause if the master server processes is dead, or any of the parameter server process is dead. They will be started by Kubernetes and recover in few minutes. Please refer to fault recovery.
    
 
    The training job will continue to make progress if there is at least one training process running. The strategy depends on the type of optimization algorithm:
    
 
      
 
    • sync-SGD
      
@@ -281,27 +312,26 @@
 
      PaddlePaddle uses etcd to keep track of the states of processes. Because etcd is a distributed reliable key-value store, the restarted process can recover its states from etcd. The model parameters are periodically saved into distributed file system, so a restarted parameter server can recover its parameters from the saved file.
      
 
      Now we will introduce how each process recovers from a failure, the graph below shows how etcd is used:
      
 
      
-
      
-
      Master Process
      
+
      
+
      Master Server Process
      
 
      When the master is started by the Kubernetes, it executes the following steps at startup:
      
 
        
 
      1. Grabs a unique master lock in etcd, which prevents concurrent master instantiations.
        2. 
 
      2. Recovers the task queues from etcd if they already exist, otherwise, the master will create them.
        3. 
-
      3. Watches the trainer prefix keys /trainer/ on etcd to find the live trainers.
        4. 
-
      4. Starts dispatching the tasks to the trainers, and updates task queue using an etcd transaction to ensure lock is held during the update.
        5. 
+
      5. Write its ip address to /master/addr so that trainers can discover it.
        6. 
+
      6. Listens to trainers’ request of task, dispatch one upon request, and updates task queue using an etcd transaction to ensure lock is held during the update.
        7. 
 
      
-
      The master process will kill itself if its etcd lease expires.
      
-
      When the master process is dead for any reason, Kubernetes will restart it. It will be online again with all states recovered from etcd in few minutes.
      
+
      When the master server process is dead for any reason, Kubernetes will restart it. It will be online again with all states recovered from etcd in few minutes.
      
 
      
 
      
 
      Trainer Process
      
 
      When the trainer is started by the Kubernetes, it executes the following steps at startup:
      
 
        
-
      1. Watches the available parameter server prefix keys /ps/ on etcd and waits until the count of parameter servers reaches the desired count.
        2. 
-
      2. Generates a unique ID, and sets key /trainer/<unique ID> with its contact address as value. The key will be deleted when the lease expires, so the master will be aware of the trainer being online and offline.
        3. 
-
      3. Waits for tasks from the master to start training.
        4. 
+
      4. Watches the available parameter server prefix keys /ps/ on etcd and waits until the count of parameter servers reaches the desired count /ps_desired.
        5. 
+
      5. Finds and watches /master/addr to get master’s address.
        6. 
+
      6. Requests for tasks from the master to start training.
        7. 
 
      
-
      If trainer’s etcd lease expires, it will try set key /trainer/<unique ID> again so that the master process can discover the trainer again.
      
+
      When a trainer fails, Kuberentes would try to restart it. The recovered trainer would fetch tasks from master and go on training.
      
 
      
 
      
 
      Parameter Server Process
      
@@ -323,6 +353,14 @@
 
      If the parameter server’s etcd lease expires, the parameter server will kill itself.
      
 
      
 
      
+
      
+
      Parameter Server Checkpointing
      
+
      See here
      
+
      
+
      
+
      Store and dispatching trainning data
      
+
      See here
      
+
      
 
      
 
      Dynamic Scaling
      
 
      
diff --git a/doc/design/cluster_train/checkpointing.html b/doc/design/cluster_train/checkpointing.html
new file mode 100644
index 0000000000000000000000000000000000000000..30eb37b8006350cce744c48e2e9f5c3dd33e71ec
--- /dev/null
+++ b/doc/design/cluster_train/checkpointing.html
@@ -0,0 +1,316 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  模型参数检查点（Checkpointing） — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • 模型参数检查点（Checkpointing）
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      模型参数检查点（Checkpointing）
      
+
      模型数据检查点的实现，可以有效的避免parameter server的单点或多点同时故障。模型参数检查点通过定期向磁盘上保存一份存储在parameter server内存中的模型数据的完整镜像，来保证训练过程可以从中间状态重新启动。在一个不可中断并缺少备份的训练任务中，可以通过阶段性的保存每个parameter server的数据快照（snapshot）到 分布式存储服务 达到容灾的目的，比如每隔10分钟最新的快照，并删除更早的快照。在出现单点故障时，只需要恢复这台节点，或者将这台节点迁移到另一个节点并启动即可恢复训练任务。
      
+
      
+
      
+
      快照保存的设计如下：
      
+
      说明：
      
+
        
+
      • parameter server在集群中启动后，自动挂载分布式存储目录，并把快照保存到这个目录下。
        • 
+
      • 注：每个parameter server的检查点各自独立保存，暂时不考虑多个parameter server同步的保存一个特定时间点的全局检查点，因为这样做也没法保证消除随机性。
        • 
+
      
+
      检查点保存程序流程：
      
+
        
+
      1. 如果满足条件”每隔10分钟”时，parameter server会获取parameters内存的read_lock，启动一个新的线程开始保存检查点。如果已经正在执行保存检查点的线程，则忽略。由于对parameters的更新需要获取parameters内存的write_lock，所以在写入快照的过程中，parameter server会暂停参数更新并等待。
        2. 
+
      2. parameter server生成一个UUID，向指定的目录中一个新的文件（文件名为此UUID）写入快照数据。在快照写入完成后，计算这个文件的MD5 sum。然后在etcd的/checkpoints/[pserver_id]中写入json内容：{"uuid": [UUID], "md5", "MD5 sum", "timestamp": xxxx}。
        3. 
+
      3. 删除磁盘目录中不是当前uuid的快照文件。
        4. 
+
      4. 释放对paramters内存的锁定，停止保存检查点的线程。
        5. 
+
      
+
      这里需要用户额外注意，在您的实际环境中，训练任务的运行可能会占满trainer和parameter server之间的网络带宽，如果parameter server此时还需要通过网络访问分布式存储以保存快照，可能会造成网络拥塞，而出现阶段性的运行停滞。
      
+
      
+
      
+
      从快照恢复
      
+
      在parameter server第一次启动或任意时间parameter server故障后被Kubernetes重新启动，则需要回滚到上一个检查点：
      
+
        
+
      1. 从etcd中读取节点：/checkpoints/[pserver_id]获取最新的检查点的文件uuid
        2. 
+
      2. 从磁盘文件中加载uuid文件名的检查点快照文件，并加载其中的参数
        3. 
+
      3. 如果上面两步出现错误，则使用启动参数定义的初始化方法初始化参数
        4. 
+
      4. 开始提供服务
        5. 
+
      
+
      
+
      
+
      
+
      TODO List
      
+
      
+
      推测执行/加速执行（TODO）
      
+
      在异构集群中，如果存在某些trainer执行速度过慢会影响整体集群的速度（如图中Trainer 1），此时master将负责启动一个新的Trainer（Accelerate Trainer 2），使用同样的训练数据block。哪个trainer先完成block的训练，则把另一个慢速的kill掉。
      
+
      
+
      
+
      动态扩容/缩容
      
+
      目前只考虑动态扩容trainer数量，可以减小系统复杂性。
      
+
      
+
      
+
      
+
      术语
      
+
        
+
      • model: 指深度学习训练之后得到的所有参数，使用这个神经网络可以完成对新数据的预测
        • 
+
      • parameters: 神经网络中的参数，包括权重w和偏置b。一个神经网络的模型由大量的参数组成
        • 
+
      • shard: 分片，通常指将一个整体拆分成多份的其中的一份。
        • 
+
      • model shard: 将一个神经网络参数拆分成多份，每个shard分别存储在其中一台parameter server之上
        • 
+
      • parameter block: 多个parameter block构成一个model shard
        • 
+
      • 单点故障: 任意时刻只可能同时有一台服务器故障。由于集群中同时存在两台机器故障的概率极低（（平均故障率*平均故障修复时间）^2）只对特殊在线系统考虑两台以上同时故障的容灾。
        • 
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/cluster_train/data_dispatch.html b/doc/design/cluster_train/data_dispatch.html
new file mode 100644
index 0000000000000000000000000000000000000000..9c210180e281e4ec0291c7d6e1fa789cfe49bba4
--- /dev/null
+++ b/doc/design/cluster_train/data_dispatch.html
@@ -0,0 +1,417 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  训练数据的存储和分发 — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • 训练数据的存储和分发
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      训练数据的存储和分发
      
+
      
+
      概念解释
      
+
      
+
      
+
      流程介绍
      
+
      生产环境中的训练数据集通常体积很大，并被存储在诸如Hadoop HDFS，Ceph，AWS S3之类的分布式存储之上。这些分布式存储服务通常会把数据切割成多个分片分布式的存储在多个节点之上。这样就可以在云端执行多种数据类计算任务，包括：
      
+
        
+
      • 数据预处理任务
        • 
+
      • Paddle训练任务
        • 
+
      • 在线模型预测服务
        • 
+
      
+
      
+
+
      在上图中显示了在一个实际生产环境中的应用（人脸识别）的数据流图。生产环境的日志数据会通过实时流的方式（Kafka）和离线数据的方式（HDFS）存储，并在集群中运行多个分布式数据处理任务，比如流式数据处理（online data process），离线批处理（offline data process）完成数据的预处理，提供给paddle作为训练数据。用户也可以上传labeled data到分布式存储补充训练数据。在paddle之上运行的深度学习训练输出的模型会提供给在线人脸识别的应用使用。
      
+
      
+
      
+
      训练数据存储
      
+
      我们选择CephFS作为存储系统。
      
+
        
+
      • 无论是从PFSClient的角度，还是从Pod中运行任务的角度，统一用/pfs/$DATACENTER/home/$USER来访问用户自己的数据。
        • 
+
      • /pfs/$DATACENTER/common下存放公共数据集合
          
+
        • 做只读挂载
          • 
+
        
+
        • 
+
      
+
      
+
+
      
+
      
+
      文件预处理
      
+
      在开始训练之前, 数据集需要预先被转换成PaddlePaddle分布式训练使用的存储格RecordIO。我们提供两个转换方式：
      
+
        
+
      1. 用户在本地转换好再上传
        2. 
+
      2. 用户上传数据后，在机群上运行转换程序
        3. 
+
      
+
      转换生成的文件名会是以下格式：
      
+
      name_prefix-aaaaa-of-bbbbb
+
      
+
      
+
      “aaaaa”和”bbbbb”都是五位的数字，每一个文件是数据集的一个shard，”aaaaa”代表shard的index，”bbbbb”代表这个shard的最大index。
      
+
      比如ImageNet这个数据集可能被分成1000个shard，它们的文件名是：
      
+
      imagenet-00000-of-00999
+imagenet-00001-of-00999
+...
+imagenet-00999-of-00999
+
      
+
      
+
      
+
      转换库
      
+
      无论是在本地或是云端转换，我们都提供Python的转换库，接口是：
      
+
      def convert(output_path, reader, num_shards, name_prefix)
+
      
+
      
+
        
+
      • output_path: directory in which output files will be saved.
        • 
+
      • reader: a data reader, from which the convert program will read data instances.
        • 
+
      • num_shards: the number of shards that the dataset will be partitioned into.
        • 
+
      • name_prefix: the name prefix of generated files.
        • 
+
      
+
      reader每次输出一个data instance，这个instance可以是单个值，或者用tuple表示的多个值：
      
+
      yield 1 # 单个值
+yield numpy.random.uniform(-1, 1, size=28*28) # 单个值
+yield numpy.random.uniform(-1, 1, size=28*28), 0 # 多个值
+
      
+
      
+
      每个值的类型可以是整形、浮点型数据、字符串，或者由它们组成的list，以及numpy.ndarray。如果是其它类型，会被Pickle序列化成字符串。
      
+
      
+
      
+
      
+
      示例程序
      
+
      
+
      使用转换库
      
+
      以下reader_creator生成的reader每次输出一个data instance，每个data instance包涵两个值：numpy.ndarray类型的值和整型的值：
      
+
      def reader_creator():
+    def reader():
+        for i in range(1000):
+            yield numpy.random.uniform(-1, 1, size=28*28), 0 # 多个值
+    return reader
+
      
+
      
+
      把reader_creator生成的reader传入convert函数即可完成转换：
      
+
      convert("./", reader_creator(), 100, random_images)
+
      
+
      
+
      以上命令会在当前目录下生成100个文件：
      
+
      random_images-00000-of-00099
+random_images-00001-of-00099
+...
+random_images-00099-of-00099
+
      
+
      
+
      
+
      
+
      进行训练
      
+
      PaddlePaddle提供专用的data reader creator，生成给定RecordIO文件对应的data reader。无论在本地还是在云端，reader的使用方式都是一致的：
      
+
      # ...
+reader = paddle.reader.creator.RecordIO("/pfs/datacenter_name/home/user_name/random_images-*-of-*")
+batch_reader = paddle.batch(paddle.dataset.mnist.train(), 128)
+trainer.train(batch_reader, ...)
+
      
+
      
+
      以上代码的reader输出的data instance与生成数据集时，reader输出的data instance是一模一样的。
      
+
      
+
      
+
      
+
      上传训练文件
      
+
      使用下面命令，可以把本地的数据上传到存储集群中。
      
+
      paddle pfs cp filename /pfs/$DATACENTER/home/$USER/folder/
+
      
+
      
+
      比如，把之前示例中转换完毕的random_images数据集上传到云端的/home/可以用以下指令：
      
+
      paddle pfs cp random_images-*-of-* /pfs/$DATACENTER/home/$USER/folder/
+
      
+
      
+
      需要$DATACENTER的配置写到配置文件中，例如
      
+
      # config file
+[datacenter_1]
+username=user
+usercert=user.pem
+userkey=user-key.pem
+endpoint=datacenter1.paddlepaddle.org
+
+[datacenter_2]
+username=user
+usercert=user.pem
+userkey=user-key.pem
+endpoint=datacenter2.paddlepaddle.org
+
      
+
      
+
      
+
      
+
      
+
      TODO
      
+
      
+
      文件访问的权限
      
+
      控制用户权限
      
+
        
+
      • 用户可以把自己的数据分享给别人
        • 
+
      
+
      
+
      
+
      文件访问方式
      
+
      不用mount的方式来访问数据，而是直接用API的接口远程访问
      
+
      例如：
      
+
      f = open('/pfs/datacenter_name/home/user_name/test1.dat')
+
      
+
      
+
      
+
      
+
      支持用户自定义的数据预处理job
      
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/cluster_train/large_model_dist_train.html b/doc/design/cluster_train/large_model_dist_train.html
new file mode 100644
index 0000000000000000000000000000000000000000..5223871c321590b6d8112a2a0b6e6d7f78746762
--- /dev/null
+++ b/doc/design/cluster_train/large_model_dist_train.html
@@ -0,0 +1,354 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Alalysis of large model distributed training in Paddle — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • Alalysis of large model distributed training in Paddle
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      Alalysis of large model distributed training in Paddle
      
+
      NOTE: This is only some note for how we implemeted this scheme in V1, not a new design.
      
+
      
+
      What is it
      
+
      We often encounter cases that the embedding layer parameters(sparse) are so large that we can not store it in the trainer’s memory when training. So we need to put them to several servers, and fetch them row by row instead of fetch all of the parameters.
      
+
      
+
      
+
      How to use
      
+
      Specify command-line argument like  --loadsave_parameters_in_pserver=true --ports_num_for_sparse=1 --use_old_updater=1 when starting the paddle trainer. And also add something like --ports_num_for_sparse=1 --pserver_num_threads=5 when starting pserver processes.
      
+
      Accrodingly, configure your embedding layers like:
      
+
      SPARSE_REMOTE=True
+
+w1 = data_layer(name="w1", size=dict_size)
+emb1 = embedding_layer(input=w1, size=32, param_attr=ParameterAttribute(sparse_update=SPARSE_REMOTE))
+w2 = data_layer(name="w2", size=dict_size)
+emb2 = embedding_layer(input=w2, size=32, param_attr=ParameterAttribute(sparse_update=SPARSE_REMOTE))
+...
+
      
+
      
+
      
+
      
+
      Implementation details
      
+
      enum MatType {
+  MAT_NORMAL,
+  MAT_NORMAL_SHARED,
+  MAT_VALUE_SHARED,
+  MAT_SPARSE_ROW_IDS,
+  MAT_SPARSE_ROW_AUTO_GROW,
+  MAT_CACHE_ROW,
+  MAT_SPARSE_ROW,
+  MAT_SPARSE_ROW_PREFETCH,
+  MAT_SPARSE_ROW_PREFETCH_FULL_SIZE,
+};
+
      
+
      
+
      MAT_SPARSE_ROW_PREFETCH is what we use when configured to fetch only row of matrix when training.
      
+
      In trainer_internal.cpp:L93 trainOneBatch:
      
+
        if (config_->getOptConfig().use_sparse_remote_updater()) {
+    REGISTER_TIMER("prefetch");
+    gradientMachine_->prefetch(inArgs);
+    parameterUpdater_->getParametersRemote();
+  }
+
      
+
      
+
      When doing actual network forward and backward, at the beginning of each batch, the trainer will try to download one row of data from pserver.
      
+
      In trainer/RemoteParameterUpdater.cpp: parameterUpdater_->getParametersRemote();:
      
+
      if (fullSize) {
+    ...
+} else {
+getParams = [&] {
+    parameterClient_->getParameterSparse(
+        /* recvParameterType= */ PARAMETER_VALUE, sendBackParameterType);
+};
+applyL1 = [](Parameter& para, real decayRate) {
+    para.getMat(PARAMETER_VALUE)->applyL1(/*lr=*/1.0f, decayRate);
+};
+}
+
      
+
      
+
      Calling parameterClient_->getParameterSparse will do remote call to pserver’s getParameterSparse:
      
+
      void ParameterServer2::getParameterSparse(const SendParameterRequest& request,
+                                          std::vector<Buffer>& inputBuffers,
+                                          SendParameterResponse* response,
+                                          std::vector<Buffer>* outputBuffers) {
+  (void)inputBuffers;
+  auto& buffer = *readWriteBuffer_;
+  size_t numReals = 0;
+  for (const auto& block : request.blocks()) {
+    numReals += getParameterConfig(block).dims(1);
+  }
+  buffer.resize(numReals);
+
+  VLOG(3) << "pserver: getParameterSparse, numReals=" << numReals;
+
+  ReadLockGuard guard(parameterMutex_);
+  size_t offset = 0;
+  for (const auto& block : request.blocks()) {
+    size_t width = getParameterConfig(block).dims(1);
+    Buffer buf = {buffer.data() + offset, width};
+    int type = request.send_back_parameter_type();
+    sendBackParameterSparse(block, type, response, &buf, width, outputBuffers);
+    offset += width;
+  }
+}
+
      
+
      
+
      getParameterConfig(block).dims(1) returns the width of the current “parameter block”(a shard of parameter object),
+then getParameterSparse remote call returns only one row of data to the client.
      
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/cluster_train/master_server.html b/doc/design/cluster_train/master_server.html
new file mode 100644
index 0000000000000000000000000000000000000000..63c51769316ff4cba97424cb5085902da0a77862
--- /dev/null
+++ b/doc/design/cluster_train/master_server.html
@@ -0,0 +1,352 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Master Server — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • Design Doc: Master Server
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      Design Doc: Master Server
      
+
      For an overview of master server’s role, please refer to distributed training design doc. In this design doc we will discuss the master server in more details. The master will be implemented in Go.
      
+
      
+
      Dataset
      
+
      
+
      A dataset is a list of files in RecordIO format. A RecordIO file consists of chunks, whereas each chunk consists some records.
      
+
      
+
      
+
      Task Queue
      
+
      As mentioned in distributed training design doc, a task is a data shard that the master server assigns to the trainer process to train on. A task consists of one or multiple chunks from one or multiple files. The master server maintains task queues to track the training progress.
      
+
      
+
      Task Queue Creation
      
+
        
+
      1. Each trainer will make an RPC call (using Go’s rpc package) to the master server, telling it the RecordIO files representing the dataset specified by the user. Since every trainer will tell the master server the same dataset, only the first RPC call will be honored.
        
+
        The RPC interface is:
        
+
        func (m *RPCServer) ReportDataset(Paths []string, dummy *int) error {
+}
+
        
+
        
+
        2. 
+
      2. The master server will scan through each RecordIO file to generate the chunk index and know how many chunks does each file have. A chunk can be referenced by the file path and the index of the chunk within the file. The chunk index is in memory data structure that enables fast access to each chunk, and the index of the chunk with the file is an integer start from 0, representing the n-th chunk within the file.
        
+
        The definition of the chunk is:
        
+
        type Chunk struct {
+    Idx   int // index of the chunk within the file
+    Path  string
+    Index recordio.Index // chunk index
+}
+
        
+
        
+
        3. 
+
      3. Chunks are grouped into tasks, and tasks are filled into the todo queue. The pending queue and the done queue are initialized with no element.
        
+
        The definition of the task is:
        
+
        type Task struct {
+    Index  int
+    Chunks []Chunk
+}
+
        
+
        
+
        The elements in the tasks queues is of type TaskEntry, containing a timeout counter (described in task retry logic), and a task:
        
+
        type TaskEntry struct {
+    NumTimeout int
+    Task       Task
+}
+
        
+
        
+
        The definition of task queues is:
        
+
        type TaskQueues struct {
+    Todo    []TaskEntry
+    Pending map[int]TaskEntry // map from task index to task entry
+    Done    []TaskEntry
+}
+
        
+
        
+
        4. 
+
      
+
      
+
      
+
      Task Queue Persistence
      
+
      The task queues need to be persisted on etcd for fault recovery. Since the task queues only change once a task is completed or timed out, which is not very frequent, we can afford to synchronize with etcd every time the task queues change.
      
+
      We will serialize the task queues data structure with gob encoding, compress with gzip, and save into etcd synchronously under key /task_queues.
      
+
      
+
      
+
      Task Dispatch
      
+
      The trainer will make an RPC call to master to get a new task when:
      
+
        
+
      • the trainer first started, or
        • 
+
      • the trainer finishes a task.
        • 
+
      
+
      The RPC interface is:
      
+
      func (m *RPCServer) GetTask(finished *Task, result *Task) error {
+}
+
      
+
      
+
      Argument finished will be nil when the trainer is just started.
      
+
      During the RPC call the master will do the following:
      
+
        
+
      • Make a copy of the task queues, and update the copy reflecting the finished tasks and the new pending tasks.
        • 
+
      • Synchronize the copy of task queues with etcd using a transaction conditioned on holding the master lock.
        • 
+
      • Replace the task queues with the copy and report to the trainer with the new tasks if succeeded, or discard the copy and report the error to the trainer if failed.
        • 
+
      
+
      
+
      
+
      Task Retry Logic
      
+
      When a task is dispatched to the trainer, the master will schedule a function for execution after the timeout duration (based on the moving average of task completion time). If the task entry in still in the pending queue, its timeout counter will increase by one, and the task will be moved to todo queue. If the timeout counter is above the threshold, the master will log the error and discard the task.
      
+
      Please note that since a timed out task could be completed after it has been dispatched for retry, so it is possible for a task to be processed multiple times. We do not try to prevent it from happening since it’s fine to train on the same task multiple times due to the stochastic nature of the stochastic gradient decent algorithm.
      
+
      
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/cluster_train/pserver_client.html b/doc/design/cluster_train/pserver_client.html
new file mode 100644
index 0000000000000000000000000000000000000000..050afdb006b98e94d5c60d58d3d7fd8ec5dc0853
--- /dev/null
+++ b/doc/design/cluster_train/pserver_client.html
@@ -0,0 +1,429 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: The Client Library of Parameter Server — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • Design Doc: The Client Library of Parameter Server
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      Design Doc: The Client Library of Parameter Server
      
+
      For an overview of trainer’s role, please refer to distributed training design doc. In this design doc, we will discuss the parameter server’s client library, which will manage communication with parameter servers. The library will be implemented in Go and made available as a static or dynamic library with a C header file.
      
+
      
+
      Parameter Partition
      
+
      Each parameter will be partitioned into parameter blocks to make the parameters evenly distributed on parameter servers. The partition is done automatically by the client library. The sparse parameter require a little different treatment:
      
+
      
+
      Sparse Parameter
      
+
      The sparse parameter is a parameter that is updated sparsely. The name is somewhat misleading, it does not have a sparse representation, it has the same representation as a dense vector.
      
+
      Because a sparse parameter is updated sparsely, the trainer will have to partition the sparse parameter. Because the parameter server will merge all sparse parameter shard into the same file when saving the parameter. It needs special naming convention:
      
+
      If a sparse parameter is partitioned into n shards, they should be named as:
      
+
      name:sparse-0
+name:sparse-1
+...
+name:sparse-n-1
+
      
+
      
+
      The library is unaware of the partition, and treat each parameter independently. Only when saving parameters, the parameter servers will merge the sparse parameters according to the naming convention.
      
+
      
+
      
+
      
+
      Model Optimization Using Gradients
      
+
      There are two ways to perform model optimization using gradients:
      
+
        
+
      • On Client
        
+
        The client does multiple steps of forward and backward update. In each step, the gradients are calculated and a new model is generated. After some steps, the client will calculate the difference between the newest model and the old model at step 0. The difference will be updated to parameter servers. Parameter servers will just update parameters using the difference without any optimization using gradients (such as Adam and L1 regularization).
        
+
        • 
+
      • On Parameter Server
        
+
        The client will send accumulated gradients to parameter servers, the parameter server will do the optimization using gradients.
        
+
        • 
+
      
+
      
+
      
+
      L1 and L2 Regularization
      
+
      PaddlePaddle allows L1 or L2 regularizations to be specified per parameter, so when the trainer initializes the parameter it needs include a parameter configuration when L1 or L2 regularization is necessary.
      
+
      
+
      
+
      Parameter Initialization
      
+
      The parameters on parameter servers need to be initialized. To provide maximum flexibility, the trainer will initialize the parameters. Only one trainer will do the initialization, the other trainers will wait for the completion of initialization and get the parameters from the parameter servers.
      
+
      
+
      Trainer Selection
      
+
      To select the trainer for initialization, every trainer will try to get a distributed lock, whoever owns the lock will do the initialization. As illustrated below:
      
+
      
+
      
+
      
+
      Trainer Selection Process
      
+
      The trainer select process is encapsulated in the C API function:
      
+
      int paddle_begin_init_params(paddle_pserver_client* client, const char* config_proto);
+
      
+
      
+
      The selected trainer’s call to paddle_begin_init_params will return with 1, and the other trainers’ call to paddle_begin_init_params will return 0. paddle_get_params will be blocked until initialization is completed. As illustrated below:
      
+
      
+
      
+
      
+
      
+
      C Interface
      
+
      typedef enum {
+  PADDLE_ELEMENT_TYPE_INT32   = 0,
+  PADDLE_ELEMENT_TYPE_UINT32  = 1,
+  PADDLE_ELEMENT_TYPE_INT64   = 2,
+  PADDLE_ELEMENT_TYPE_UINT64  = 3,
+  PADDLE_ELEMENT_TYPE_FLOAT32 = 4,
+  PADDLE_ELEMENT_TYPE_FLOAT64 = 5,
+} paddle_element_type;
+
+typedef struct {
+  char*               name;
+  paddle_element_type element_type;
+  unsigned char*      content;
+  int                 content_len;
+} paddle_parameter, paddle_gradient;
+
+typedef int paddle_pserver_client;
+
+/**
+ * @brief creates a pserver client that talks to etcd for coordination.
+ */
+paddle_pserver_client paddle_new_etcd_pserver_client(char* etcd_addr);
+
+/**
+ * @brief creates a pserver client given pserver addresses.
+ *
+ * @param pserver_addrs comma-separated pserver addresses.
+ * @param selected if current pserver client is selected to initialize all parameter servers.
+ */
+paddle_pserver_client paddle_new_pserver_client(char* pserver_addrs, int selected);
+void paddle_pserver_client_release(paddle_pserver_client c);
+
+/**
+ * @brief paddle_begin_init_params begins to initialize parameters on
+ * parameter servers.
+ *
+ * paddle_begin_init_params will be called from multiple trainers,
+ * only one trainer will be selected to initialize the parameters on
+ * parameter servers. Other trainers need to get the initialized
+ * parameters from parameter servers using @paddle_get_params.
+ *
+ * @return 1 if the trainer is selected to initialize parameter
+ * servers, otherwise 0.
+ */
+int paddle_begin_init_params(paddle_pserver_client client);
+
+/**
+ * @brief paddle_init_param initializes the parameter on parameter
+ * servers.
+ *
+ * @param param the parameter to initialize.
+ * @param param_config_proto the configuration for the parameter.
+ * @param config_len the length of param_config_proto
+ * @return 0 if successful, otherwise -1. On failure, the trainer
+ * needs to restart the entire initialization process (starting from
+ * @paddle_begin_init_param). Or simply exit the program and wait for
+ * the cluster management system to restart the trainer.
+ */
+int paddle_init_param(paddle_pserver_client client, paddle_parameter param, const unsigned char* param_config_proto, int config_len);
+
+/**
+ * @brief paddle_finish_init_params tells parameter servers client has
+ * sent all parameters to parameter servers as initialization.
+ *
+ * @return 0 if successful, otherwise -1. On failure, the trainer
+ * needs to restart the entire initialization process (starting from
+ * @paddle_begin_init_param). Or simply exit the program and wait for
+ * the cluster management system to restart the trainer.
+ */
+int paddle_finish_init_params(paddle_pserver_client client);
+
+/**
+ * @brief paddle_send_grads sends gradients to parameter servers for
+ * updating parameters.
+ *
+ * @param grads the array of gradients to send.
+ * @param len the length of the gradient array.
+ * @param learning_rate the learning rate for the gradients.
+ * @return 0 if successful, otherwise -1.
+ */
+int paddle_send_grads(paddle_pserver_client client, const paddle_gradient* grads, int len);
+
+/**
+ * @brief paddle_get_params gets parameters from parameter servers.
+ *
+ * paddle_get_params will block until parameters are initialized on
+ * the parameter servers.
+ *
+ * @param dst the destination array of parameter pointers to save to.
+ * The parameter pointer must be pre-popullated with required parameter name,
+ * and the content of parameter must be pre-allocated of the size of required
+ * parameter on pserver.
+ * @param len the length of the names array and the paddle_parameter
+ * array.
+ * @return 0 if successful, otherwise -1.
+ */
+int paddle_get_params(paddle_pserver_client client, paddle_parameter** dst, int len);
+
+/**
+ * @brief paddle_save_model indicates parameters to save the parameter
+ * to the given path
+ *
+ * @param path the path to save parameters.
+ * @return 0 if successful, otherwise -1.
+ */
+int paddle_save_model(paddle_pserver_client client, const char* path);
+
      
+
      
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/tutorials/rec/ml_dataset_en.html b/doc/design/cluster_train/remote_parameter_updater.html
similarity index 58%
rename from doc/tutorials/rec/ml_dataset_en.html
rename to doc/design/cluster_train/remote_parameter_updater.html
index f0617fed5105b741b307a7151262d0da23bad59a..e78e0f62777f12f85911ba34e5aca61f1d161f26 100644
--- a/doc/tutorials/rec/ml_dataset_en.html
+++ b/doc/design/cluster_train/remote_parameter_updater.html
@@ -8,7 +8,7 @@
   
   
   
-  MovieLens Dataset — PaddlePaddle  documentation
+  Design Doc: Remote Parameter Updater for Cluster Train — PaddlePaddle  documentation
   
 
   
@@ -62,7 +62,7 @@
     
      
     
      
       
      
-        Folk me on Github
+        Fork me on Github
         
      
           
             English
@@ -84,7 +84,7 @@
 
    • GET STARTED
      • 
 
    • HOW TO
      • 
 
    • API
      • 
-
    • ABOUT
      • 
+
    • MOBILE
      • 
 
    
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -166,7 +194,7 @@
 
    
   
      
       
-    
    • MovieLens Dataset
      • 
+    
    • Design Doc: Remote Parameter Updater for Cluster Train
      • 
   
    
 
    
       
@@ -175,110 +203,24 @@
           
    
            
    
             
-  
    
-
    MovieLens Dataset
    
-
    The MovieLens Dataset was collected by GroupLens Research.
-The data set contains some user information, movie information, and many movie ratings from [1-5].
-The data sets have many version depending on the size of set.
-We use MovieLens 1M Dataset as a demo dataset, which contains
-1 million ratings from 6000 users on 4000 movies. Released 2/2003.
    
-
    
-
    Dataset Features
    
-
    In ml-1m Dataset, there are many features in these dataset.
-The data files (which have ”.dat” extension) in ml-1m Dataset
-is basically CSV file that delimiter is ”::”. The description in README we quote here.
    
-
    
-
    RATINGS FILE DESCRIPTION(ratings.dat)
    
-
    All ratings are contained in the file “ratings.dat” and are in the
-following format:
    
-
    UserID::MovieID::Rating::Timestamp
    
-
      
-
    • UserIDs range between 1 and 6040
      • 
-
    • MovieIDs range between 1 and 3952
      • 
-
    • Ratings are made on a 5-star scale (whole-star ratings only)
      • 
-
    • Timestamp is represented in seconds since the epoch as returned by time(2)
      • 
-
    • Each user has at least 20 ratings
      • 
-
    
+  
    
+
    Design Doc: Remote Parameter Updater for Cluster Train
    
+
    For an overview of distribute training, please refer to distributed training design doc. In this design doc, we will discuss the parameter updater that will use parameter server cclient The Client Library of Parameter Server Design Doc to manage and update parameters.
    
+
    
+
    Parameter Updater
    
+
    Parameter Updater is used by trainer to manage and update parameter, there are mainly two kind of parameter updater: local and remote, since this design is for cluster train, we will only discuss remote parameter updater here.
    
+
    
+
    Remote Parameter Updater
    
+
    Remote Parameter Updater manage parameters through remote parameter server with the client that communicate with pserver(The Client Library of Parameter Server Design Doc)
    
+
    In PaddlePaddle Python V2 API, trainer is implemented in python, and the trainer will hold a instance of parameter updater and call it’s functions directly. In this design, we will also expose the api of RemoteParameterUpdater to python with swig.
    
+
    
+
    Sparse Remote Parameter Updater
    
+
    Since we will only implement dense parameter management new, the mechanism for sparse parameter will be discussed in next stage.
    
 
    
-
    
-
    USERS FILE DESCRIPTION(users.dat)
    
-
    User information is in the file “users.dat” and is in the following
-format:
    
-
    UserID::Gender::Age::Occupation::Zip-code
    
-
    All demographic information is provided voluntarily by the users and is
-not checked for accuracy.  Only users who have provided some demographic
-information are included in this data set.
    
-
      
-
    • Gender is denoted by a “M” for male and “F” for female
      • 
-
    • Age is chosen from the following ranges:
        
-
      • 1:  “Under 18”
        • 
-
      • 18:  “18-24”
        • 
-
      • 25:  “25-34”
        • 
-
      • 35:  “35-44”
        • 
-
      • 45:  “45-49”
        • 
-
      • 50:  “50-55”
        • 
-
      • 56:  “56+”
        • 
-
      
-
      • 
-
    • Occupation is chosen from the following choices:
        
-
      • 0:  “other” or not specified
        • 
-
      • 1:  “academic/educator”
        • 
-
      • 2:  “artist”
        • 
-
      • 3:  “clerical/admin”
        • 
-
      • 4:  “college/grad student”
        • 
-
      • 5:  “customer service”
        • 
-
      • 6:  “doctor/health care”
        • 
-
      • 7:  “executive/managerial”
        • 
-
      • 8:  “farmer”
        • 
-
      • 9:  “homemaker”
        • 
-
      • 10:  “K-12 student”
        • 
-
      • 11:  “lawyer”
        • 
-
      • 12:  “programmer”
        • 
-
      • 13:  “retired”
        • 
-
      • 14:  “sales/marketing”
        • 
-
      • 15:  “scientist”
        • 
-
      • 16:  “self-employed”
        • 
-
      • 17:  “technician/engineer”
        • 
-
      • 18:  “tradesman/craftsman”
        • 
-
      • 19:  “unemployed”
        • 
-
      • 20:  “writer”
        • 
-
      
-
      • 
-
    
 
    
-
    
-
    MOVIES FILE DESCRIPTION(movies.dat)
    
-
    Movie information is in the file “movies.dat” and is in the following
-format:
    
-
    MovieID::Title::Genres
    
-
      
-
    • Titles are identical to titles provided by the IMDB (including
-year of release)
      • 
-
    • Genres are pipe-separated and are selected from the following genres:
        
-
      • Action
        • 
-
      • Adventure
        • 
-
      • Animation
        • 
-
      • Children’s
        • 
-
      • Comedy
        • 
-
      • Crime
        • 
-
      • Documentary
        • 
-
      • Drama
        • 
-
      • Fantasy
        • 
-
      • Film-Noir
        • 
-
      • Horror
        • 
-
      • Musical
        • 
-
      • Mystery
        • 
-
      • Romance
        • 
-
      • Sci-Fi
        • 
-
      • Thriller
        • 
-
      • War
        • 
-
      • Western
        • 
-
      
-
      • 
-
    • Some MovieIDs do not correspond to a movie due to accidental duplicate
-entries and/or test entries
      • 
-
    • Movies are mostly entered by hand, so errors and inconsistencies may exist
      • 
-
    
+
    
+
    Interface Design
    
+
    TBD
    
 
    
 
    
 
    
diff --git a/doc/design/cluster_train/save_model.html b/doc/design/cluster_train/save_model.html
new file mode 100644
index 0000000000000000000000000000000000000000..bdf7a97fd8a772cc43c74771c658a91f33677c9f
--- /dev/null
+++ b/doc/design/cluster_train/save_model.html
@@ -0,0 +1,371 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Save Model — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Save Model
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Save Model
    
+
    
+
    Overview
    
+
    The model is the output of the training process. There are two
+ways from which user can obtain a model:
    
+
      
+
    • Save model triggered by user code: user code asks PaddlePaddle to
+save a model.
      • 
+
    • Convert model from the checkpoint: model being converted from
+pservers’ periodic checkpoint. In this way, the user can cancel a
+job at any time, and still have a relatively fresh model (we
+checkpoint around every 5 minutes).
      • 
+
    
+
    
+
    Trainer Saving Model vs. Pservers Saving Model
    
+
    Both trainers and pservers have access to the model. So the model can
+be saved from a trainer or pservers. We need to decide where the model
+is saved from.
    
+
    
+
    Dense Update vs. Sparse Update
    
+
    There are two types of model update methods: dense update and sparse
+update (when the model parameter is configured to be sparse).
    
+
      
+
    • Dense update
      
+
      Every trainer has it’s own full copy of the model. Every model
+update will update the entire model.
      
+
      • 
+
    • Sparse update
      
+
      The training input is sparse, and the trainer does not have the
+entire model. It will only download the sub-model necessary related
+to the input. When updating the model, only the sub-model related to
+the training input is updated.
      
+
      • 
+
    
+
    
+
    
+
    Pservers Saving Model
    
+
    The benefit of letting pservers save model is they have the entire
+model all the time. However, since pservers are on different nodes, it
+requires a merging process to merge model shards into the same
+model. Thus requires the pservers to write models to a distributed
+filesystem, making the checkpoint shards visible to the merge program.
    
+
    
+
    
+
    Trainer Saving Model
    
+
    The benefit of letting one trainer to save the model is it does not
+require a distributed filesystem. And it’s reusing the same save model
+logic when training locally - except when doing sparse update, the
+trainer needs to download the entire model during the saving process.
    
+
    
+
    
+
    Conclusion
    
+
    Given trainer saving model does not require a distributed filesystem,
+and is an intuitive extension to trainer saving model when training
+locally, we decide to let the trainer save the model when doing
+distributed training.
    
+
    
+
    
+
    
+
    Convert Model from Checkpoint
    
+
    TODO
    
+
    
+
    
+
    
+
    Timeline
    
+
    We first implement trainer save the model. Converting the latest
+snapshot to a model will be a TODO for future.
    
+
    
+
    
+
    Trainer Save Model
    
+
    
+
    Trainer Election
    
+
    One trainer will be elected as the one to save the model. When using
+etcd, trainer ID is a randomly generated UUID, the trainer will
+contact the master server requesting to save the model, and find out
+if itself is elected. When the master server is not used, unique
+trainer IDs will be given by the administrator, the trainer whose ID
+is “0” is elected to save the model.
    
+
    
+
    
+
    Model Save Path
    
+
    Each trainer will be given the directory to save the model. The
+elected trainer will save the model to
+given-directory/trainerID. Since the trainer ID is unique, this
+would prevent concurrent save to the same file when multiple trainers
+are elected to save the model when split-brain problem happens.
    
+
    
+
    
+
    What Happens When Model Is Saving
    
+
    It takes some time to save model, we need to define what will happen
+when save model is taking place.
    
+
    When doing dense update, the trainer uses the local model. Pservers
+does not need to pause model update.
    
+
    When doing sparse update. The trainer needs to download the entire
+model while saving. To get the most accurate model, the model update
+needs to be paused before the download starts and resumed after the
+download finishes. Otherwise, the trainer gets a model that is
+“polluted”: some part of the model is old, some part of the model is
+new.
    
+
    It’s unclear that the “polluted” model will be inferior due to the
+stochastic nature of deep learning, and pausing the model update will
+add more complexity to the system. Since supporting sparse update is a
+TODO item. We defer the evaluation of pause the model update or not
+during saving model to the future.
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/cluster_train/submit-job.html b/doc/design/cluster_train/submit-job.html
new file mode 100644
index 0000000000000000000000000000000000000000..ca5a69b34b037337331fbe7daf1171949461fced
--- /dev/null
+++ b/doc/design/cluster_train/submit-job.html
@@ -0,0 +1,400 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Submit a Distributed Training Job — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Submit a Distributed Training Job
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Submit a Distributed Training Job
    
+
    The user can submit a distributed training job with Python code, rather than with a command-line interface.
    
+
    
+
    Runtime Environment On Kubernetes
    
+
    For a distributed training job, there is two Docker image called runtime Docker image and base Docker image. The runtime Docker image is the Docker image that gets scheduled by Kubernetes to run during training. The base Docker image is for building the runtime Docker image.
    
+
    
+
    Base Docker Image
    
+
    Usually, the base Docker image is PaddlePaddle product Docker image including paddle binary files and python package. And of course, users can specify any image name hosted on any docker registry which users have the access right.
    
+
    
+
    
+
    Runtime Docker Image
    
+
    The trainer package which user upload and some Python dependencies are packaged into a runtime Docker image based on base Docker image.
    
+
      
+
    • Handle Python Dependencies
      
+
      You need to provide requirements.txt file in your trainer-package folder. Example:
      
+
      pillow
+protobuf==3.1.0
+
      
+
      
+
      More details about requirements, an example project looks like:
      
+
        paddle_example
+    |-quick_start
+      |-trainer.py
+      |-dataset.py
+      |-requirements.txt
+
      
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    Submit Distributed Training Job With Python Code
    
+
    
+
      
+
    • paddle.job.dist_train() will call the Job Server API /v1/packages to upload the trainer package and save them on CephFS, and then call /v1/trainer/job to submit the PaddlePaddle distributed job.
      • 
+
    • /v1/trainer/job will start a building job for preparing the runtime Docker image. When the building job is finished, Job Server will submit the PaddlePaddle distributed job to Kubernetes.
      • 
+
    • NOTE: For the first version, we will not prepare the runtime Docker image, instead, the package is uploaded to Paddle Cloud, and Paddle Cloud will mount the package in a temporary folder into the base Docker image. We will not support custom Python dependencies in the first version as well.
      • 
+
    
+
    You can call paddle.job.dist_train and provide distributed training configuration as the parameters:
    
+
    paddle.job.dist_train(
+  trainer=dist_trainer(),
+  paddle_job=PaddleJob(
+    job_name = "paddle-cloud",
+    entry_point = "python %s"%__file__,
+    trainer_package = "/example/word2vec",
+    image = "yancey1989/paddle-job",
+    trainers = 10,
+    pservers = 3,
+    trainer_cpu = 1,
+    trainer_gpu = 1,
+    trainer_mem = "10G",
+    pserver_cpu = 1,
+    pserver_mem = "2G"
+  ))
+
    
+
    
+
    The parameter trainer of paddle.job.dist_train is a function and you can implement it as follows:
    
+
    def dist_trainer():
+  def trainer_creator():
+    trainer = paddle.v2.trainer.SGD(...)
+    trainer.train(...)
+  return trainer_creator
+
    
+
    
+
    The pseudo code of paddle.job.dist_train is as follows:
    
+
    def dist_train(trainer, paddle_job):
+  # if the code is running on cloud, set PADDLE_ON_CLOUD=YES
+  if os.getenv("RUNNING_ON_CLOUD", "NO") == "NO":
+    #submit the paddle job
+    paddle_job.submit()
+  else:
+    #start the training
+    trainer()
+
    
+
    
+
    
+
    PaddleJob Parameters
    
+
    parameter | type | explanation
+— | — | —
+job_name | str | the unique name for the training job
+entry_point | str | entry point for startup trainer process
+trainer_package | str | trainer package file path which user have the access right
+image|str|the base image for building the runtime image
+pservers|int| Parameter Server process count
+trainers|int| Trainer process count
+pserver_cpu|int| CPU count for each Parameter Server process
+pserver_mem|str| memory allocated for each Parameter Server process, a plain integer using one of these suffixes: E, P, T, G, M, K
+trainer_cpu|int| CPU count for each Trainer process
+trainer_mem|str| memory allocated for each Trainer process, a plain integer using one of these suffixes: E, P, T, G, M, K
+trainer_gpu|int| GPU count for each Trainer process, if you only want CPU, do not set this parameter
    
+
    
+
    
+
    Deploy Parameter Server, Trainer and Master Process
    
+
      
+
    • Deploy PaddlePaddle Parameter Server processes, it’s a Kubernetes ReplicaSet.
      • 
+
    • Deploy PaddlePaddle Trainer processes, it’s a Kubernetes Job.
      • 
+
    • Deploy PaddlePaddle Master processes, it’s a Kubernetes ReplicaSet.
      • 
+
    
+
    
+
    
+
    
+
    Job Server
    
+
      
+
    • RESTful API
      
+
      Job server provides RESTful HTTP API for receiving the trainer package and displaying
+PaddlePaddle job related informations.
      
+
        
+
      • POST /v1/package receive the trainer package and save them on CephFS
        • 
+
      • POST /v1/trainer/job submit a trainer job
        • 
+
      • GET /v1/jobs/ list all jobs
        • 
+
      • GET /v1/jobs/<job-name> the status of a job
        • 
+
      • DELETE /v1/jobs/<job-name> delete a job
        • 
+
      • GET /v1/version job server version
        • 
+
      
+
      • 
+
    • Build Runtime Docker Image on Kubernetes
      
+
      paddle.job.dist_train will upload the trainer package to Job Server, save them on the distributed filesystem, and then start up a job for building the runtime Docker image that gets scheduled by Kubernetes to run during training.
      
+
      There are some benefits for building runtime Docker image on JobServer:
      
+
        
+
      • On Paddle Cloud, users will run the trainer code in a Jupyter Notebook which is a Kubernetes Pod, if we want to execute docker build in the Pod, we should mount the host’s docker.sock to the Pod, user’s code will connect the host’s Docker Engine directly, it’s not safe.
        • 
+
      • Users only need to upload the training package files, does not need to install docker engine, docker registry as dependencies.
        • 
+
      • If we want to change another image type, such as RKT, users do not need to care about it.
        • 
+
      
+
      • 
+
    • Deploy Parameter Server, Trainer and Master Processes
      
+
      POST /v1/trainer/job receives the distributed training parameters, and deploy the job as follows:
      
+
        
+
      • Deploy PaddlePaddle Parameter Server processes, it’s a Kubernetes ReplicaSet.
        • 
+
      • Deploy PaddlePaddle Trainer processes, it’s a Kubernetes Job.
        • 
+
      • Deploy PaddlePaddle Master processes, it’s a Kubernetes ReplicaSet.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/evaluator.html b/doc/design/evaluator.html
new file mode 100644
index 0000000000000000000000000000000000000000..37c173e257de54aafbeb73312c0b75786102fd7c
--- /dev/null
+++ b/doc/design/evaluator.html
@@ -0,0 +1,323 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Evaluator Design — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Evaluator Design
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Evaluator Design
    
+
    
+
    Problem Statement
    
+
    During training or inference, we provide an evaluation function to measure the model performance, for example, accuracy, precision, etc. In the operator based framework design, the data passes through the network pipeline batch by batch. As a result, inside the operator, we only calculate the metrics for one minibatch. Thus, we need to provide a mechanism to calculate the metrics for each N pass/batch the user wants.
    
+
    
+
    
+
    Evaluator Design
    
+
    Currently, every operation is expressed in the graph. We divide the evaluator process into three steps.
    
+
      
+
    1. Initialize the metric state and add it into the block.
      2. 
+
    2. Calculate the concerned metrics for every mini-batch. The single evaluator operator is only responsible for calculating the necessary statistics for one mini-batch. For example, the accuracy operator only calculates the accuracy for a minibatch data if run once.
      3. 
+
    
+
      
+
    1. Merge the mini-batch statistics to form the evaluation result for multiple mini-batches. When it comes to distributed training/Multi-GPU training, aggregate the value from different devices.
      2. 
+
    
+
    
+
    
+
    Implementation
    
+
    This design is shown in the Python API.
+Each metric operator needs to caculate the metric statistic and return the batch-aware states. Python side is responsible for accumulating the states for each pass.
    
+
    class Evaluator(object):
+    """
+    Evaluator Base class.
+    """
+    def __init__(self, name, **kwargs):
+       """
+       Different evaluator may has different metric states. E.g, Accuracy need two variables, total and right sample counts.
+       Auc need four variables, `true_positives`,
+         `true_negatives`, `false_positives` and `false_negatives`. So every evaluator should create its needed variables and append to main_program
+
+       The initialization of Evaluator should be responsible for:
+       create metric states and append to the main_program
+       """ 
+       pass
+
+    def _update_ops(self, input, label, **kwargs)
+       """
+       Add mini-batch evaluator caculate operators to the main_program.
+       Add increment operator to accumulate the metric states.
+       """
+    
+
+    def reset(self, executor, reset_program=None):
+      """
+      Reset metric states at the begin of each pass/user specified batch number.
+      Execute the reset_program to reset the states.
+      """
+      
+
+    def eval(self, executor, eval_program=None):
+      """
+      Merge the mini-batch statistics to form the evaluation result for multiple mini-batches.
+      Execute the eval_program and return the result.
+      """
+      return eval_result
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/executor.html b/doc/design/executor.html
new file mode 100644
index 0000000000000000000000000000000000000000..c006b3ced058fb4cf209a09f459171fc9b0a7948
--- /dev/null
+++ b/doc/design/executor.html
@@ -0,0 +1,287 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Executor Design Doc — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Executor Design Doc
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Executor Design Doc
    
+
    
+
    Motivation
    
+
    We use executor to do the runtime evaluation of a ProgramDesc.
    
+
    
+
    
+
    Overview
    
+
    An executor takes a ProgramDesc, a block_id and a Scope.  The ProgramDesc is a list of blocks and each block contains the protobuf definition of all the parameters and operators. The block_id specifies the entrance block. And the Scope is the container of all the variable instance, which is persistent throughout different runs.
    
+
    
+
    What does executor do?
    
+
    It evaluates all the operators in the block_idth block of a ProgramDesc.
    
+
    
+
    
+
    What does executor NOT do?
    
+
    It does not do runtime optimization, meaning intelligently parse the dependency of each op a choose which one to be run and in which order they should be run.
    
+
    It does not do graph partitioning, meaning dividing the ProgramDesc into several small pieces and executing them on different devices.
    
+
    
+
    
+
    
+
    Implementation
    
+
    Executor evaluates a ProgramDesc. Essentially, it instantiates Variables and Operators, then run all the operators in sequence. [code]
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/file_manager/README.html b/doc/design/file_manager/README.html
new file mode 100644
index 0000000000000000000000000000000000000000..9b28ae4d8d0ea01b75478b5d4bad3427f4bc63fc
--- /dev/null
+++ b/doc/design/file_manager/README.html
@@ -0,0 +1,376 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  FileManager设计文档 — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • FileManager设计文档
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    FileManager设计文档
    
+
    
+
    目标
    
+
    在本文档中，我们设计说明了名为FileManager系统，方便用户上传自己的训练数据以进行分布式训练
    
+
    主要功能包括：
    
+
      
+
    • 提供常用的命令行管理命令管理文件和目录
      • 
+
    • 支持大文件的断点上传、下载
      • 
+
    
+
    
+
    
+
    名词解释
    
+
      
+
    • PFS：是Paddlepaddle cloud File System的缩写，是对用户文件存储空间的抽象，与之相对的是local filesystem。目前我们用CephFS来搭建。
      • 
+
    • CephFS：一个POSIX兼容的文件系统。
      • 
+
    • Chunk：逻辑划上文件分块的单位。
      • 
+
    
+
    
+
    
+
    模块
    
+
    
+
    架构图
    
+
    
+
    
+
    
+
    PFSClient
    
+
      
+
    • 功能： 详细设计link
        
+
      • 提供用户管理文件的命令
        • 
+
      • 需要可以跨平台执行
        • 
+
      
+
      • 
+
    • 双向验证PFSClient需要和Ingress之间做双向验证tls，所以用户需要首先在cloud.paddlepaddle.org上注册一下，申请用户空间，并且把系统生成的CA(certificate authority)、Key、CRT(CA signed certificate)下载到本地，然后才能使用PFSClient。
      • 
+
    
+
    
+
    
+
    Ingress
    
+
      
+
    • 功能：提供七层协议的反向代理、基于粘性会话的负载均衡功能。
      • 
+
    • 透传用户身份的办法Ingress需要把PFSClient的身份信息传给PFSServer，配置的方法参考link
      • 
+
    
+
    
+
    
+
    PFSServer
    
+
    PFSServer提供RESTful API接口，接收处理PFSClient端的文件管理请求，并且把结果返回PFSClient端。
    
+
    RESTful API
    
+
      
+
    • /api/v1/files
        
+
      • GET /api/v1/files: Get metadata of files or directories.
        • 
+
      • POST /api/v1/files: Create files or directories.
        • 
+
      • PATCH /api/v1/files: Update files or directories.
        • 
+
      • DELETE /api/v1/files: Delete files or directories.
        • 
+
      
+
      • 
+
    • /api/v1/file/chunks
        
+
      • GET /api/v1/storage/file/chunks: Get chunks’s metadata of a file.
        • 
+
      
+
      • 
+
    • /api/v1/storage/files
        
+
      • GET /api/v1/storage/files: Download files or directories.
        • 
+
      • POST /api/v1/storage/files: Upload files or directories.
        • 
+
      
+
      • 
+
    • /api/v1/storage/file/chunks
        
+
      • GET /api/v1/storage/file/chunks: Download chunks’s data.
        • 
+
      • POST /api/v1/storage/file/chunks: Upload chunks’s data.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    文件传输优化
    
+
    
+
    分块文件传输
    
+
    用户文件可能是比较大的，上传到Cloud或者下载到本地的时间可能比较长，而且在传输的过程中也可能出现网络不稳定的情况。为了应对以上的问题，我们提出了Chunk的概念，一个Chunk由所在的文件偏移、数据、数据长度及校验值组成。文件的上传和下载都是通过对Chunk的操作来实现的。由于Chunk比较小（默认256K），完成一个传输动作完成的时间也比较短，不容易出错。PFSClient需要在传输完毕最后一个Chunk的时候检查destination文件的MD5值是否和source文件一致。
    
+
    一个典型的Chunk如下所示：
    
+
    type Chunk struct {
+    fileOffset int64
+    checksum uint32
+    len     uint32
+    data    []byte
+}
+
    
+
    
+
    
+
    
+
    生成sparse文件
    
+
    当destination文件不存在或者大小和source文件不一致时，可以用Fallocate生成sparse文件，然后就可以并发写入多个Chunk。
    
+
    
+
    
+
    覆盖不一致的部分
    
+
    文件传输的的关键在于需要PFSClient端对比source和destination的文件Chunks的checksum是否保持一致，不一致的由PFSClient下载或者传输Chunk完成。这样已经传输成功的部分就不用重新传输了。
    
+
    
+
    
+
    
+
    用户使用流程
    
+
    参考link
    
+
    
+
    
+
    框架生成
    
+
    用swagger生成PFSClient和PFSServer的框架部分，以便我们可以把更多的精力放到逻辑本身上。
    
+
    
+
    
+
    参考文档
    
+
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/file_manager/pfs/pfsclient.html b/doc/design/file_manager/pfs/pfsclient.html
new file mode 100644
index 0000000000000000000000000000000000000000..34830fc9c82044f1d8fa685dc6ac31583314a89e
--- /dev/null
+++ b/doc/design/file_manager/pfs/pfsclient.html
@@ -0,0 +1,402 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  PFSClient — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • PFSClient
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    PFSClient
    
+
    
+
    Description
    
+
    The pfs command is a Command Line Interface to manage your files on PaddlePaddle Cloud
    
+
    
+
    
+
    Synopsis
    
+
    paddle [options] pfs <subcommand> [parameters]
+
    
+
    
+
    
+
    
+
    Options
    
+
    --profile (string)
+    Use a specific profile from your credential file.
+
+--help (string)
+    Display more information about command
+
+--version
+    Output version information and exit
+
+--debug
+    Show detailed debugging log 
+    
+--only-show-errors (boolean) 
+    Only errors and warnings are displayed. All other output is suppressed.
+
    
+
    
+
    
+
    
+
    Path Arguments
    
+
    When using a command, we need to specify path arguments. There are two path argument type: localpath and pfspath.
    
+
    A pfspath begin with /pfs, eg: /pfs/$DATACENTER/home/$USER/folder.
    
+
    Here is how to config datacenters.
    
+
    
+
    
+
    order of Path Arguments
    
+
    Commonly, if there are two path arguments, the first is the source, and the second is the destination.
    
+
    
+
    
+
    Subcommonds
    
+
      
+
    • rm - remove files or directories
      • 
+
    
+
    Synopsis:
+    rm [-r] [-v] <PFSPath> ...
+
+Options:
+    -r 
+        Remove directories and their contents recursively 
+    -v      
+        Cause rm to be verbose, showing files after they are removed.
+    
+Examples:
+    paddle pfs rm /pfs/$DATACENTER/home/$USER/file
+    paddle pfs rm -r /pfs/$DATACENTER/home/$USER/folder
+
    
+
    
+
      
+
    • mv - move (rename) files
      • 
+
    
+
    Synopsis:
+    mv [-f | -n] [-v] <LocalPath> <PFSPath>
+    mv [-f | -n] [-v] <LocalPath> ... <PFSPath>
+    mv [-f | -n] [-v] <PFSPath> <LocalPath> 
+    mv [-f | -n] [-v] <PFSPath> ... <LocalPath> 
+    mv [-f | -n] [-v] <PFSPath> <PFSPath> 
+    mv [-f | -n] [-v] <PFSPath> ... <PFSPath> 
+    
+Options:
+    -f      
+        Do not prompt for confirmation before overwriting the destination path.  (The -f option overrides previous -n options.)
+    -n      
+        Do not overwrite an existing file.  (The -n option overrides previous -f options.)
+    -v      
+        Cause mv to be verbose, showing files after they are moved.
+        
+Examples:
+    paddle pfs mv ./text1.txt /pfs/$DATACENTER/home/$USER/text1.txt
+
    
+
    
+
      
+
    • cp - copy files or directories
      • 
+
    
+
    Synopsis:
+    cp [-r] [-f | -n] [-v] [--preserve--links] <LocalPath> <PFSPath>
+    cp [-r] [-f | -n] [-v] [--preserve--links] <LocalPath> ... <PFSPath>
+    cp [-r] [-f | -n] [-v] [--preserve--links] <PFSPath> <LocalPath> 
+    cp [-r] [-f | -n] [-v] [--preserve--links] <PFSPath> ... <LocalPath>
+    cp [-r] [-f | -n] [-v] [--preserve--links] <PFSPath> <PFSPath> 
+    cp [-r] [-f | -n] [-v] [--preserve--links] <PFSPath> ... <PFSPath>
+
+Options:
+    -r
+        Copy directories recursively
+    -f      
+        Do not prompt for confirmation before overwriting the destination path.  (The -f option overrides previous -n options.)
+    -n      
+        Do not overwrite an existing file.  (The -n option overrides previous -f options.)
+    -v      
+        Cause cp to be verbose, showing files after they are copied.
+    --preserve--links
+       Reserve links when copy links
+       
+Examples:
+    paddle pfs cp ./file /pfs/$DATACENTER/home/$USER/file
+    paddle pfs cp /pfs/$DATACENTER/home/$USER/file ./file
+
    
+
    
+
      
+
    • ls- list files
      • 
+
    
+
    Synopsis:
+    ls [-r] <PFSPath> ...
+    
+Options:
+    -R
+        List directory(ies) recursively
+
+Examples:
+    paddle pfs ls  /pfs/$DATACENTER/home/$USER/file
+    paddle pfs ls  /pfs/$DATACENTER/home/$USER/folder
+
    
+
    
+
      
+
    • mkdir - mkdir directory(ies)
+Create intermediate directory(ies) as required.
      • 
+
    
+
    Synopsis:
+    mkdir <PFSPath> ...
+
+Examples:
+    paddle pfs mkdir  /pfs/$DATACENTER/home/$USER/folder
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/float16.html b/doc/design/float16.html
new file mode 100644
index 0000000000000000000000000000000000000000..1ea944c360cfe31d1b803713b6d8f264136ca079
--- /dev/null
+++ b/doc/design/float16.html
@@ -0,0 +1,381 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: float16 — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: float16
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: float16
    
+
    
+
    Why float16
    
+
    Half precision (float16) is a binary floating-point format that occupies 16 bits in memory. float16 is half the size of traditional 32-bit single precision format (float) and has lower precision and smaller range.
    
+
    When high precision computation is not required, using float16 data type could potentially
    
+
      
+
    • reduce storage space, memory bandwidth, and power usages;
      • 
+
    • increase the chance of data fitting into a smaller cache of lower latency;
      • 
+
    • provide arithmetic speed up if supported by hardware.
      • 
+
    
+
    
+
    
+
    Survey of current float16 support
    
+
    A brief survey of float16 support on different compilers, hardwares, and libraries can be found below. Interested readers can refer to link1 and link2 for more info.
    
+
    The goal of float16 is to serve as a key for the executor to find and run the correct version of compute method specialized for float16 in operator kernel. It should be compatible with various natively supported float16 implementations including __half for cuda, float16_t for ARM, and Eigen::half for Eigen to make writing customized float16 kernels easier.
    
+
    
+
    Compiler
    
+
      
+
    • nvcc supports __half data type after CUDA 7.5.
      • 
+
    • __fp16 or float16_t is supported as storage type for gcc >= 6.1 and clang >= 3.4.
      • 
+
    • __fp16 or float16_t is supported as arithmetic type for gcc >= 7.1 and clang >= 3.9.
      • 
+
    
+
    
+
    
+
    Hardware
    
+
      
+
    • __half is supported on GPU with compute capability >= 5.3.
      • 
+
    • __fp16 is supported as storage type for ARMv7-A, ARMv8-A, and above.
      • 
+
    • __fp16 is supported as arithmetic type after ARMv8.2-A (currently, the only microarchitecture implementing ARMv8.2-A is ARM Cortex-A75, which is announced in May 2017. There seems to be no application processors currently available on market that adopts this architecture. It is reported that Qualcomm Snapdragon 845 uses Cortex-A75 design and will be available in mobile devices in early 2018).
      • 
+
    
+
    
+
    
+
    Libraries
    
+
      
+
    • Eigen >= 3.3 supports float16 calculation on both GPU and CPU using the Eigen::half class. It is mostly useful for Nvidia GPUs because of the overloaded arithmetic operators using cuda intrinsics. It falls back to using software emulation on CPU for calculation and there is no special treatment to ARM processors.
      • 
+
    • ARM compute library >= 17.02.01 supports NEON FP16 kernels (requires ARMv8.2-A CPU).
      • 
+
    
+
    
+
    
+
    CUDA version issue
    
+
    There are currently three versions of CUDA that supports __half data type, namely, CUDA 7.5, 8.0, and 9.0.
+CUDA 7.5 and 8.0 define __half as a simple struct that has a uint16_t data (see cuda_fp16.h) as follows:
    
+
    typedef struct __align__(2) {
+   unsigned short x;
+} __half;
+
+typedef __half half;
+
    
+
    
+
    This struct does not define any overloaded arithmetic operators. So you have to directly use __hadd instead of + to correctly add two half types:
    
+
    __global__ void Add() {
+  half a, b, c;
+  c = __hadd(a, b); // correct
+  c = a + b; // compiler error: no operator "+" matches these operands
+}
+
    
+
    
+
    CUDA 9.0 provides a major update to the half data type. The related code can be found in the updated cuda_fp16.h and the newly added cuda_fp16.hpp.
    
+
    Essentially, CUDA 9.0 renames the original __half type in 7.5 and 8.0 as __half_raw, and defines a new __half class type that has constructors, conversion operators, and also provides overloaded arithmetic operators such as follows:
    
+
    typedef struct __CUDA_ALIGN__(2) {
+    unsigned short x;
+} __half_raw;
+
+
+struct __CUDA_ALIGN__(2) __half {
+protected:
+    unsigned short __x;
+public:
+    // constructors and conversion operators from/to 
+    // __half_raw and other built-in data types
+}
+
+typedef __half half;
+
+__device__ __forceinline__ 
+__half operator+(const __half &lh, const __half &rh) { 
+    return __hadd(lh, rh); 
+}
+
+// Other overloaded operators
+
    
+
    
+
    This new design makes c = a + b work correctly for CUDA half data type.
    
+
    
+
    
+
    
+
    Implementation
    
+
    The float16 class holds a 16-bit uint16_t data internally.
    
+
    struct float16 {
+  uint16_t x;
+};
+
    
+
    
+
    float16 supports the following features:
    
+
      
+
    • constructors / assignment operators that take input from primitive data types including bool, integers of various length, float, and double.
      • 
+
    • constructors / assignment operators that take input from __half on cuda, float16_t on ARM, and Eigen::half on Eigen.
      • 
+
    • conversion operators to primitive data types and half precision data types on cuda, ARM and Eigen.
      • 
+
    • overloaded arithmetic operators for cuda, arm, and non-arm cpu, respectively. These operators will take advantage of the cuda and ARM intrinsics on the corresponding hardware.
      • 
+
    
+
    To support the above features, two fundamental conversion functions are provided:
    
+
    float16 float_to_half_rn(float f);  // convert to half precision in round-to-nearest-even mode
+float half_to_float(float16 h);
+
    
+
    
+
    which provides one-to-one conversion between float32 and float16. These twos functions will do different conversion routines based on the current hardware. CUDA/ARM instrinsics will be used when the corresonding hardware is available. If the hardware or compiler level does not support float32 to float16 conversion, software emulation will be performed to do the conversion.
    
+
    
+
    
+
    To do
    
+
    After float16 class is available, some of the future items are below:
    
+
      
+
    • Update pybind/tensor_py.h to bind c++ float16 with numpy float16.
      • 
+
    • Modify GetKernelType() method in framework/operator.h to make it compatible with float16.
      • 
+
    • Create a type-casting operator that can convert the data type in tensor between float16 and other types.
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/functions_operators_layers.html b/doc/design/functions_operators_layers.html
new file mode 100644
index 0000000000000000000000000000000000000000..c7a19255df9a6da6304f912bbe0734ed23ab0a57
--- /dev/null
+++ b/doc/design/functions_operators_layers.html
@@ -0,0 +1,342 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Functions, Operators, and Layers — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Functions, Operators, and Layers
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Functions, Operators, and Layers
    
+
    In a DL system, we can compose one or more fine grained operators into a coarse grained one.  For example, the FC layer can be composed of a multiplication operator and an add operator.
    
+
    Historically, some fine grained operations are known as operators, and some coarse level ones are known as layers.  But we need a well-defined separation.
    
+
    In general, operators are those very fine grained operations, e.g., mul and add. In the implementation, we can write them as C++ functions:
    
+
    template <typename T> T add(T x, T y) { return x + y; }
+template <typename T> T mul(T x, T y) { return x * y; }
+
    
+
    
+
    Then we can wrap them into operators which are C++ classes and can be created from Python bindings by name.  A C macro can do this. For example, the following macro invocation
    
+
    #define MAKE_FUNCTION_OPERATOR(mul);
+
    
+
    
+
    generates
    
+
    template <typename T> class mulOp : public OperatorBase {...};
+REGISTER_OP(mulOp<float32>, "mul");
+
    
+
    
+
    so that in Python we can create operator mul by:
    
+
    X1 = Var()
+X2 = Var()
+Y = Var()
+paddle.cpp.create_operator("mul", input=[X1, X2], output=Y)
+
    
+
    
+
    Also, at the same time, we can compose a coarse level C++ operator class by composing functions mul and add:
    
+
    template <typename T>
+class FCOp : public OperatorBase {
+ public:
+  void Run(...) {
+    add(mul(Input<T>("X"), Input<T>("W")), Input<T>("b");
+  }
+};
+REGISTER_OP(FCOp, "fc");
+
    
+
    
+
    We need to support such composition in Python as well.  To do so, we need a higher level Python wrapping of operator creation than paddle.cpp.create_operator.  This higher level operator API should be compatible with the layer API.
    
+
    Let’s explain using an example.  Suppose that we are going to compose the FC using mul and add in Python, we’d like to have Python functions mul and add defined in module operator:
    
+
    def operator.mul(X1, X2):
+    O = Var()
+    paddle.cpp.create_operator("mul", input={X1, Y1}, output=O)
+    return O
+
+def operator.add(X1, X2):
+    O = Var()
+    paddle.cpp.create_operator("add", input={X1, X2}, output=O)
+    return O
+
    
+
    
+
    Above code snippets are automatically generated.  Given them, users can define
    
+
    def layer.fc(X):
+    W = Var()
+    b = Var()
+    return operator.add(operator.mul(X, W), b)
+
    
+
    
+
    If we don’t have operator.mul and operator.add, the definiton of layer.fc would be complicated:
    
+
    def layer.fc(X):
+    W = Var()
+    b = Var()
+    O1 = Var()
+    paddle.cpp.create_operator("mul", input=[X, W], output=O1)
+    O2 = Var()
+    paddle.cpp.create_operator("add", input=[O1, b], output=O2)
+    return O2
+
    
+
    
+
    We’d like to have Python bindings to operators in package paddle.operator, and Python compositions of operators in package paddle.layer.  So we have the following concepts in above illustrative example:
    
+
    | C++ functions/functors | mul          | add          |             |          |
+|————————|————–|————–|————-|———-|
+| C++ operator class     | mulOp        | addOp        | FCOp        |          |
+| Python binding         | operator.mul | operator.add | operator.fc |          |
+| Python function        |              |              |             | layer.fc |
    
+
    This is how we differentiate layer and operators in PaddlePaddle:
    
+
      
+
    • those defined in C++ and have a lightweighted Python wrapper in module operators are operators; whereas
      • 
+
    • those who don’t have C++ implementations but a Python implementation that compose C++ operators are known as layers.
      • 
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/gan_api.html b/doc/design/gan_api.html
new file mode 100644
index 0000000000000000000000000000000000000000..18955af7d82256a25b45796c196468cc6401bd49
--- /dev/null
+++ b/doc/design/gan_api.html
@@ -0,0 +1,529 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design for GAN — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design for GAN
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design for GAN
    
+
    GAN (General Adversarial Net [https://arxiv.org/abs/1406.2661]) is an important model for unsupervised learning and widely used in many areas.
    
+
    It applies several important concepts in machine learning system design, including building and running subgraphs, dependency tracing, different optimizers in one executor and so forth.
    
+
    In our GAN design, we wrap it as a user-friendly easily customized python API to design different models. We take the conditional DC-GAN (Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks [https://arxiv.org/abs/1511.06434]) as an example due to its good performance on image generation.
    
+
    
+
    
+Figure 1. The overall running logic of GAN. The black solid arrows indicate the forward pass; the green dashed arrows indicate the backward pass of generator training; the red dashed arrows indicate the backward pass of the discriminator training. The BP pass of the green (red) arrow should only update the parameters in the green (red) boxes. The diamonds indicate the data providers. d\_loss and g\_loss marked in red and green are the two targets we would like to run.
+
    The operators, layers and functions required/optional to build a GAN demo is summarized in https://github.com/PaddlePaddle/Paddle/issues/4563.
    
+
    
+
    
+Figure 2. Photo borrowed from the original DC-GAN paper.
+
    
+
    The Conditional-GAN might be a class.
    
+
    This design we adopt the popular open source design in https://github.com/carpedm20/DCGAN-tensorflow and https://github.com/rajathkmp/DCGAN. It contains following data structure:
    
+
      
+
    • DCGAN(object): which contains everything required to build a GAN model. It provides following member functions methods as API:
      • 
+
    • init(...): Initialize hyper-parameters (like conv dimension and so forth), and declare model parameters of discriminator and generator as well.
      • 
+
    • generator(z, y=None): Generate a fake image from input noise z. If the label y is provided, the conditional GAN model will be chosen.
+Returns a generated image.
      • 
+
    • discriminator(image):
+Given an image, decide if it is from a real source or a fake one.
+Returns a 0/1 binary label.
      • 
+
    • build_model(self):
+build the whole GAN model, define training loss for both generator and discrimator.
      • 
+
    
+
    
+
    
+
    Discussion on Engine Functions required to build GAN
    
+
      
+
    • Trace the tensor and variable dependency in the engine executor. (Very critical, otherwise GAN can’be be trained correctly)
      • 
+
    • Different optimizers responsible for optimizing different loss.
      • 
+
    
+
    To be more detailed, we introduce our design of DCGAN as following:
    
+
    
+
    Class member Function: Initializer
    
+
      
+
    • Set up hyper-parameters, including condtional dimension, noise dimension, batch size and so forth.
      • 
+
    • Declare and define all the model variables. All the discriminator parameters are included in the list self.theta_D and all the generator parameters are included in the list self.theta_G.
      • 
+
    
+
    class DCGAN(object):
+  def __init__(self, y_dim=None):
+  
+    # hyper parameters  
+    self.y_dim = y_dim # conditional gan or not
+    self.batch_size = 100
+    self.z_dim = z_dim # input noise dimension
+
+    # define parameters of discriminators
+    self.D_W0 = pd.Variable(shape=[3,3, 1, 128], data=pd.gaussian_normal_randomizer())
+    self.D_b0 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.D_W1 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer())
+    self.D_b1 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.D_W2 = pd.Varialble(np.random.rand(128, 1))
+    self.D_b2 = pd.Variable(np.zeros(128))
+    self.theta_D = [self.D_W0, self.D_b0, self.D_W1, self.D_b1, self.D_W2, self.D_b2]
+
+    # define parameters of generators
+    self.G_W0 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer())
+    self.G_b0 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.G_W1 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer())
+    self.G_b1 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.G_W2 = pd.Varialble(np.random.rand(128, 1))
+    self.G_b2 = pd.Variable(np.zeros(128))
+    self.theta_G = [self.G_W0, self.G_b0, self.G_W1, self.G_b1, self.G_W2, self.G_b2]
+
    
+
    
+
    
+
    
+
    Class member Function: Generator
    
+
      
+
    • Given a noisy input z, returns a fake image.
      • 
+
    • Concatenation, batch-norm, FC operations required;
      • 
+
    • Deconv layer required, which is missing now...
      • 
+
    
+
    class DCGAN(object):
+  def generator(self, z, y = None):
+    # input z: the random noise
+    # input y: input data label (optional)
+    # output G_im: generated fake images
+    
+    if not self.y_dim:
+      z = pd.layer.concat(1, [z, y])
+      
+    G_h0 = pd.layer.fc(z, self.G_w0, self.G_b0)
+    G_h0_bn = pd.layer.batch_norm(G_h0)
+    G_h0_relu = pd.layer.relu(G_h0_bn)
+    
+    G_h1 = pd.layer.deconv(G_h0_relu, self.G_w1, self.G_b1)
+    G_h1_bn = pd.layer.batch_norm(G_h1)
+    G_h1_relu = pd.layer.relu(G_h1_bn)
+    
+    G_h2 = pd.layer.deconv(G_h1_relu, self.G_W2, self.G_b2))
+    G_im = pd.layer.tanh(G_im)
+    return G_im
+
    
+
    
+
    
+
    
+
    Class member function: Discriminator
    
+
      
+
    • Given a noisy input z, returns a fake image.
      • 
+
    • Concatenation, Convolution, batch-norm, FC, Leaky-ReLU operations required;
      • 
+
    
+
    class DCGAN(object):
+  def discriminator(self, image):
+    # input image: either generated images or real ones
+    # output D_h2: binary logit of the label
+
+    D_h0 = pd.layer.conv2d(image, w=self.D_w0, b=self.D_b0)
+    D_h0_bn = pd.layer.batchnorm(h0)
+    D_h0_relu = pd.layer.lrelu(h0_bn)
+    
+    D_h1 = pd.layer.conv2d(D_h0_relu, w=self.D_w1, b=self.D_b1)
+    D_h1_bn = pd.layer.batchnorm(D_h1)
+    D_h1_relu = pd.layer.lrelu(D_h1_bn)
+    
+    D_h2 = pd.layer.fc(D_h1_relu, w=self.D_w2, b=self.D_b2)
+    return D_h2
+
    
+
    
+
    
+
    
+
    Class member function: Build the model
    
+
      
+
    • Define data readers as placeholders to hold the data;
      • 
+
    • Build generator and discriminators;
      • 
+
    • Define two training losses for discriminator and generator, respectively.
+If we have execution dependency engine to back-trace all tensors, the module building our GAN model will be like this:
      • 
+
    
+
    class DCGAN(object):
+  def build_model(self):
+    if self.y_dim:
+        self.y = pd.data(pd.float32, [self.batch_size, self.y_dim])
+    self.images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    self.faked_images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    self.z = pd.data(tf.float32, [None, self.z_size])
+    
+    # step 1: generate images by generator, classify real/fake images with discriminator
+    if self.y_dim: # if conditional GAN, includes label
+        self.G = self.generator(self.z, self.y)
+        self.D_t = self.discriminator(self.images)
+        # generated fake images
+        self.sampled = self.sampler(self.z, self.y)
+        self.D_f = self.discriminator(self.G)
+    else: # original version of GAN
+        self.G = self.generator(self.z)
+        self.D_t = self.discriminator(self.images)
+        # generate fake images
+        self.sampled = self.sampler(self.z)
+        self.D_f = self.discriminator(self.images)
+    
+    # step 2: define the two losses
+    self.d_loss_real = pd.reduce_mean(pd.cross_entropy(self.D_t, np.ones(self.batch_size))
+    self.d_loss_fake = pd.reduce_mean(pd.cross_entropy(self.D_f, np.zeros(self.batch_size))
+    self.d_loss = self.d_loss_real + self.d_loss_fake
+    
+    self.g_loss = pd.reduce_mean(pd.cross_entropy(self.D_f, np.ones(self.batch_szie))
+
    
+
    
+
    If we do not have dependency engine but blocks, the module building our GAN model will be like this:
    
+
    class DCGAN(object):
+  def build_model(self, default_block):
+    # input data in the default block
+    if self.y_dim:
+        self.y = pd.data(pd.float32, [self.batch_size, self.y_dim])
+    self.images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    # self.faked_images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    self.z = pd.data(tf.float32, [None, self.z_size])
+
+    # step 1: generate images by generator, classify real/fake images with discriminator
+    with pd.default_block().g_block():
+      if self.y_dim: # if conditional GAN, includes label
+        self.G = self.generator(self.z, self.y)
+        self.D_g = self.discriminator(self.G, self.y)
+      else: # original version of GAN
+        self.G = self.generator(self.z)
+        self.D_g = self.discriminator(self.G, self.y)
+      self.g_loss = pd.reduce_mean(pd.cross_entropy(self.D_g, np.ones(self.batch_szie))
+    
+    with pd.default_block().d_block():
+      if self.y_dim: # if conditional GAN, includes label
+        self.D_t = self.discriminator(self.images, self.y)
+        self.D_f = self.discriminator(self.G, self.y)
+      else: # original version of GAN
+        self.D_t = self.discriminator(self.images)
+        self.D_f = self.discriminator(self.G)
+
+      # step 2: define the two losses
+      self.d_loss_real = pd.reduce_mean(pd.cross_entropy(self.D_t, np.ones(self.batch_size))
+      self.d_loss_fake = pd.reduce_mean(pd.cross_entropy(self.D_f, np.zeros(self.batch_size))
+      self.d_loss = self.d_loss_real + self.d_loss_fake
+
    
+
    
+
    Some small confusion and problems with this design:
    
+
      
+
    • D_g and D_f are actually the same thing, but has to be written twice; i.e., if we want to run two sub-graphs conceptually, the same codes have to be written twice if they are shared by the graph.
      • 
+
    • Requires ability to create a block anytime, rather than in if-else or rnn only;
      • 
+
    
+
    
+
    
+
    
+
    Main function for the demo:
    
+
    Generally, the user of GAN just need to the following things:
    
+
      
+
    • Define an object as DCGAN class;
      • 
+
    • Build the DCGAN model;
      • 
+
    • Specify two optimizers for two different losses with respect to different parameters.
      • 
+
    
+
    # pd for short, should be more concise.
+from paddle.v2 as pd
+import numpy as np
+import logging
+
+if __name__ == "__main__":
+    # dcgan class in the default graph/block
+    # if we use dependency engine as tensorflow
+    # the codes, will be slightly different like:
+    # dcgan = DCGAN()
+    # dcgan.build_model()
+    with pd.block() as def_block:
+      dcgan = DCGAN()
+      dcgan.build_model(def_block)
+
+    # load mnist data
+    data_X, data_y = self.load_mnist()
+    
+    # Two subgraphs required!!!
+    with pd.block().d_block():
+      d_optim = pd.train.Adam(lr = .001, beta= .1)
+      d_step = d_optim.minimize(dcgan.d_loss, dcgan.theta_D)
+    with pd.block.g_block():
+      g_optim = pd.train.Adam(lr = .001, beta= .1)
+      g_step = pd.minimize(dcgan.g_loss, dcgan.theta_G)
+
+    # executor
+    sess = pd.executor()
+    
+    # training
+    for epoch in xrange(10000):
+      for batch_id in range(N / batch_size):
+        idx = ...
+        # sample a batch
+        batch_im, batch_label = data_X[idx:idx+batch_size], data_y[idx:idx+batch_size]
+        # sample z
+        batch_z = np.random.uniform(-1., 1., [batch_size, z_dim])
+
+        if batch_id % 2 == 0:
+          sess.run(d_step, 
+                   feed_dict = {dcgan.images: batch_im,
+                                dcgan.y: batch_label,
+                                dcgan.z: batch_z})
+        else:
+          sess.run(g_step,
+                   feed_dict = {dcgan.z: batch_z})
+
    
+
    
+
    
+
    
+
    
+
    More thinking about dependency engine v.s. block design:
    
+
      
+
    • What if we just want to run an intermediate result? Do we need to run the whole block/graph?
      • 
+
    • Should we call eval() to get the fake images in the first stage? And then train the discriminator in the second stage?
      • 
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/graph.html b/doc/design/graph.html
new file mode 100644
index 0000000000000000000000000000000000000000..65e75b308f17a0f1f376a5af757c7da47aa4f389
--- /dev/null
+++ b/doc/design/graph.html
@@ -0,0 +1,323 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Computations as a Graph — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Computations as a Graph
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Computations as a Graph
    
+
    A primary goal of the refactorization of PaddlePaddle is a more flexible representation of deep learning computation, in particular, a graph of operators and variables, instead of sequences of layers as before.
    
+
    This document explains that the construction of a graph as three steps:
    
+
      
+
    • construct the forward part
      • 
+
    • construct the backward part
      • 
+
    • construct the optimization part
      • 
+
    
+
    
+
    The Construction of a Graph
    
+
    Let us take the problem of image classification as a simple example.  The application program that trains the model looks like:
    
+
    x = layer.data("images")
+l = layer.data("label")
+y = layer.fc(x)
+cost = layer.mse(y, l)
+optimize(cost)
+train(cost, reader=mnist.train())
+
    
+
    
+
    
+
    Forward Part
    
+
    The first four lines of above program build the forward part of the graph.
    
+
    
+
    In particular, the first line x = layer.data("images") creates variable x and a Feed operator that copies a column from the minibatch to x.  y = layer.fc(x) creates not only the FC operator and output variable y, but also two parameters, W and b, and the initialization operators.
    
+
    Initialization operators are kind of “run-once” operators – the Run method increments a class data member counter so to run at most once.  By doing so, a parameter wouldn’t be initialized repeatedly, say, in every minibatch.
    
+
    In this example, all operators are created as OpDesc protobuf messages, and all variables are VarDesc.  These protobuf messages are saved in a BlockDesc protobuf message.
    
+
    
+
    
+
    Backward Part
    
+
    The fifth line optimize(cost) calls two functions, ConstructBackwardGraph and ConstructOptimizationGraph.
    
+
    ConstructBackwardGraph traverses the forward graph in the BlockDesc protobuf message and builds the backward part.
    
+
    
+
    According to the chain rule of gradient computation, ConstructBackwardGraph would
    
+
      
+
    1. create a gradient operator G for each operator F,
      2. 
+
    2. make all inputs, outputs, and outputs’ gradient of F as inputs of G,
      3. 
+
    3. create gradients for all inputs of F, except for those who don’t have gradients, like x and l, and
      4. 
+
    4. make all these gradients as outputs of G.
      5. 
+
    
+
    
+
    
+
    Optimization Part
    
+
    For each parameter, like W and b created by layer.fc, marked as double circles in above graphs, ConstructOptimizationGraph creates an optimization operator to apply its gradient.  Here results in the complete graph:
    
+
    
+
    
+
    
+
    
+
    Block and Graph
    
+
    The word block and graph are interchangable in the desgin of PaddlePaddle.  A Block is a metaphore of the code and local variables in a pair of curly braces in programming languages, where operators are like statements or instructions.  A graph of operators and variables is a representation of the block.
    
+
    A Block keeps operators in an array BlockDesc::ops
    
+
    message BlockDesc {
+  repeated OpDesc ops = 1;
+  repeated VarDesc vars = 2;
+}
+
    
+
    
+
    in the order that they appear in user programs, like the Python program at the beginning of this article.  We can imagine that in ops,  we have some forward operators, followed by some gradient operators, and then some optimization operators.
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/graph_survey.html b/doc/design/graph_survey.html
new file mode 100644
index 0000000000000000000000000000000000000000..816ba42302747a5a13ada6071cc0ec3d346bf9a0
--- /dev/null
+++ b/doc/design/graph_survey.html
@@ -0,0 +1,456 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Survey on Graph — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Survey on Graph
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Survey on Graph
    
+
    Neural network framework often provides symbolic API for users to write network topology conveniently. This doc manily focus on symbolic API in most popular neural network frameworks, and try to find out how to parse symbolic configuration to a portable file, such as protobuf or json.
    
+
    
+
    Mxnet
    
+
    The core concept of symbolic API is Symbol. Mxnet implements Symbol class in C++, and export to Python using C-API. Please refer to the comments in Mxnet:
    
+
    Symbol is help class used to represent the operator node in Graph.
+Symbol acts as an interface for building graphs from different components like Variable, Functor and Group. Symbol is also exported to python front-end (while Graph is not) to enable quick test and deployment. Conceptually, symbol is the final operation of a graph and thus including all the information required (the graph) to evaluate its output value.
    
+
    A simple network topology wrote by Symbol is as follows:
    
+
    def get_symbol(num_classes=10, **kwargs):
+    data = mx.symbol.Variable('data')
+    data = mx.symbol.Flatten(data=data)
+    fc1  = mx.symbol.FullyConnected(data = data, name='fc1', num_hidden=128)
+    act1 = mx.symbol.Activation(data = fc1, name='relu1', act_type="relu")
+    fc2  = mx.symbol.FullyConnected(data = act1, name = 'fc2', num_hidden = 64)
+    act2 = mx.symbol.Activation(data = fc2, name='relu2', act_type="relu")
+    fc3  = mx.symbol.FullyConnected(data = act2, name='fc3', num_hidden=num_classes)
+    mlp  = mx.symbol.SoftmaxOutput(data = fc3, name = 'softmax')
+    return mlp
+
    
+
    
+
    Varible here is actually a Symbol. Every basic Symbol will correspond to one Node, and every Node has its own NodeAttr. There is a op field in NodeAttr class, when a Symbol represents Variable(often input data), the op field is null.
    
+
    Symbol contains a data member, std::vector outputs, and NodeEntry cantains a poniter to Node. We can follow the Node pointer to get all the Graph.
    
+
    And Symbol can be saved to a Json file.
    
+
    Here is a detailed example:
    
+
    >>> import mxnet as mx
+>>> data = mx.symbol.Variable('data')
+>>> print data.debug_str()
+Variable:data
+
+>>> data = mx.symbol.Flatten(data=data)
+>>> print data.debug_str()
+Symbol Outputs:
+    output[0]=flatten0(0)
+Variable:data
+--------------------
+Op:Flatten, Name=flatten0
+Inputs:
+    arg[0]=data(0) version=0
+
+>>> fc1  = mx.symbol.FullyConnected(data = data, name='fc1', num_hidden=128)
+>>> print fc1.debug_str()
+Symbol Outputs:
+    output[0]=fc1(0)
+Variable:data
+--------------------
+Op:Flatten, Name=flatten0
+Inputs:
+    arg[0]=data(0) version=0
+Variable:fc1_weight
+Variable:fc1_bias
+--------------------
+Op:FullyConnected, Name=fc1
+Inputs:
+    arg[0]=flatten0(0)
+    arg[1]=fc1_weight(0) version=0
+    arg[2]=fc1_bias(0) version=0
+Attrs:
+    num_hidden=128
+
    
+
    
+
    
+
    
+
    TensorFlow
    
+
    The core concept of symbolic API is Tensor. Tensorflow defines Tensor in Python. Please refer to the comments in TensorFlow:
    
+
    A Tensor is a symbolic handle to one of the outputs of an Operation. It does not hold the values of that operation’s output, but instead provides a means of computing those values in a TensorFlow Session.
    
+
    A simple example is as follows:
    
+
      # Build a dataflow graph.
+  c = tf.constant([[1.0, 2.0], [3.0, 4.0]])
+  d = tf.constant([[1.0, 1.0], [0.0, 1.0]])
+  e = tf.matmul(c, d)
+
+  # Construct a `Session` to execute the graph.
+  sess = tf.Session()
+
+  # Execute the graph and store the value that `e` represents in `result`.
+  result = sess.run(e)
+
    
+
    
+
    The main method of Tensor is as follows:
    
+
    @property
+def op(self):
+  """The `Operation` that produces this tensor as an output."""
+  return self._op
+
+@property
+def dtype(self):
+   """The `DType` of elements in this tensor."""
+  return self._dtype
+
+@property
+def graph(self):
+  """The `Graph` that contains this tensor."""
+  return self._op.graph
+
+@property
+def name(self):
+  """The string name of this tensor."""
+  if not self._op.name:
+    raise ValueError("Operation was not named: %s" % self._op)
+  return "%s:%d" % (self._op.name, self._value_index)
+
+@property
+def device(self):
+  """The name of the device on which this tensor will be produced, or None."""
+  return self._op.device
+
    
+
    
+
    Tensor can be taken as target to run by session. Tensor contains all the information of Graph, and tracks data dependency.
    
+
    Here is a detailed example:
    
+
    >>> import tensorflow as tf
+>>> c = tf.constant([[1.0, 2.0], [3.0, 4.0]])
+>>> print c.graph
+<tensorflow.python.framework.ops.Graph object at 0x10f256d50>
+>>> d = tf.constant([[1.0, 1.0], [0.0, 1.0]])
+>>> print d.graph
+<tensorflow.python.framework.ops.Graph object at 0x10f256d50>
+>>> e = tf.matmul(c, d)
+>>> print e.graph
+<tensorflow.python.framework.ops.Graph object at 0x10f256d50>
+
    
+
    
+
    
+
    
+
    Dynet
    
+
    The core concept of symbolic API is Expression, and Dynet defines Expression class in C++.
    
+
    A simple example is as follows:
    
+
    ComputationGraph cg;
+Expression W = parameter(cg, pW);
+
+Expression in = input(cg, xs[i]);
+Expression label = input(cg, ys[i]);
+Expression pred = W * in;
+Expression loss = square(pred - label);
+
    
+
    
+
    The input data and parameter are also represented by Expression. Every basci Expression corresponds to a Node. And input data is also a Node.
    
+
    Expression has a data member ComputationGraph, and ComputationGraph will be modified in users’ configuring process. Expression can be a running target, beacuse Expression contains all dependency.
    
+
    Here is a detailed example:
    
+
    write topology in C++
    
+
    ComputationGraph cg;
+Expression W = parameter(cg, pW);
+cg.print_graphviz();
+
+Expression pred = W * xs[i];
+cg.print_graphviz();
+
+Expression loss = square(pred - ys[i]);
+cg.print_graphviz();
+
    
+
    
+
    compile and print
    
+
    # first print
+digraph G {
+  rankdir=LR;
+  nodesep=.05;
+  N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"];
+}
+# second print
+digraph G {
+  rankdir=LR;
+  nodesep=.05;
+  N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"];
+  N1 [label="v1 = v0 * -0.98"];
+  N0 -> N1;
+}
+# third print
+digraph G {
+  rankdir=LR;
+  nodesep=.05;
+  N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"];
+  N1 [label="v1 = v0 * -0.98"];
+  N0 -> N1;
+  N2 [label="v2 = -1.88387 - v1"];
+  N1 -> N2;
+  N3 [label="v3 = -v2"];
+  N2 -> N3;
+  N4 [label="v4 = square(v3)"];
+  N3 -> N4;
+}
+
    
+
    
+
    
+
    
+
    Conclusion
    
+
    Actually, Symbol/Tensor/Expression in Mxnet/TensorFlow/Dynet are the same level concepts. We use a unified name Expression here, this level concept has following features:
    
+
      
+
    • Users wirte topoloy with symbolic API, and all return value is Expression, including input data and parameter.
      • 
+
    • Expression corresponds with a global Graph, and Expression can also be composed.
      • 
+
    • Expression tracks all dependency and can be taken as a run target
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/if_else_op.html b/doc/design/if_else_op.html
new file mode 100644
index 0000000000000000000000000000000000000000..49f59a9baece9071e1d4d8a40e39731e81dfbba9
--- /dev/null
+++ b/doc/design/if_else_op.html
@@ -0,0 +1,312 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  The IfElse Operator — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • The IfElse Operator
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    The IfElse Operator
    
+
    PaddlePaddle’s IfElse operator differs from TensorFlow’s:
    
+
      
+
    • the TensorFlow version takes a scalar boolean value as the condition so that the whole mini-batch goes to either the true or the false branch, whereas
      • 
+
    • the PaddlePaddle version takes a vector of boolean value as the condition, and instances corresponding to true values go to the true branch, those corresponding to false values go to the false branch.
      • 
+
    
+
    
+
    Example
    
+
    The following PaddlePaddle program shows the usage of the IfElse operator:
    
+
    import paddle as pd
+
+x = minibatch([10, 20, 30]) # shape=[None, 1]
+y = var(1) # shape=[1], value=1
+z = minibatch([10, 20, 30]) # shape=[None, 1]
+cond = larger_than(x, 15) # [false, true, true]
+
+ie = pd.ifelse()
+with ie.true_block():
+    d = pd.layer.add(x, y)
+    ie.output(d, pd.layer.softmax(d))
+with ie.false_block():
+    d = pd.layer.fc(z)
+    ie.output(d, d+1)
+o1, o2 = ie(cond)
+
    
+
    
+
    A challenge to implement the IfElse operator is to infer those variables to be split, or, say, to identify the variable of the mini-batch or those derived from the mini-batch.
    
+
    An equivalent C++ program is as follows:
    
+
    namespace pd = paddle;
+
+int x = 10;
+int y = 1;
+int z = 10;
+bool cond = false;
+int o1, o2;
+if (cond) {
+  int d = x + y;
+  o1 = z;
+  o2 = pd::layer::softmax(z);
+} else {
+  int d = pd::layer::fc(z);
+  o1 = d;
+  o2 = d+1;
+}
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/infer_var_type.html b/doc/design/infer_var_type.html
new file mode 100644
index 0000000000000000000000000000000000000000..2f443216a43996a9591467650c05a3ec81a612bd
--- /dev/null
+++ b/doc/design/infer_var_type.html
@@ -0,0 +1,329 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: InferVarType — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: InferVarType
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: InferVarType
    
+
    
+
    The Problem Posed
    
+
    The variable in our design can hold variant types. Such as LoDTensor and SelectedRows. An operator should be able to inference the variable types of its output.
    
+
    For example, a lookup table operator takes two LoDTensor; one is a float tensor as the embedding table, the other is an int tensor as word ID. The gradient operator of lookup table will generate a SelectedRows as its output. A sum operator can take both LoDTensor and SelectedRows as its inputs and will generate a LoDTensor if any of its inputs is LoDTensor, otherwise, the sum operator will generate SelectedRows as its output.
    
+
    The variable type will be constant at runtime. Every variable’s type can either be set by the user (input data and parameter) or be inferred by the operator in compile time.
    
+
    
+
    
+
    Proposed Solution
    
+
    The InferVarType is a compile-time function which is registered to each operator. The inferface of that function is:
    
+
    using InferVarTypeFN = std::function<
+    void (const OpDescBind& /*op_desc*/, BlockDescBind* /*block*/)>;
+
    
+
    
+
    It takes an operator description as its input and will write the output variable type and store them in block description.
    
+
    The InferVarTypeFN will be registered in OpInfo, to replace infer_var_type_ field. The OpInfo should be
    
+
    struct OpInfo {
+  InferVarTypeFN infer_var_type_;
+  ...
+};
+
    
+
    
+
    The default InferVarType will set output type as LoDTensor. It can be done by GetInferVarType().
    
+
    void DefaultInferVarType(const OpDescBind& op_desc, BlockDescBind* block) {
+  // set the output type of variable as `LoDTensor`.
+  // ...
+}
+
+struct OpInfo {
+  InferVarTypeFN infer_var_type_;
+  InferVarTypeFN GetInferVarType() const {
+    if (infer_var_type_) {
+      return infer_var_type_;
+    } else {
+      return DefaultInferVarType;
+    }
+  }
+};
+
    
+
    
+
    
+
    
+
    Register InferVarType
    
+
    We provide a thin base class for registering an InferVarTypeFN. To use a base class will ease the implementation of registry since we can detect the registry entry is an InferVarTypeFN or not.
    
+
    class VarTypeInferer {
+public:
+  virtual void operator()(const OpDescBind& op_desc, BlockDescBind* block) const = 0;
+}
+
    
+
    
+
    Operator developers can write the specialize VarTypeInferer as follow.
    
+
    class SpecialVarTypeInferer : public VarTypeInferer {
+public:
+  virtual void operator()(const OpDescBind& op_desc, BlockDescBind* block) const {
+    // .. own logic
+  }
+}
+
    
+
    
+
    Then user can register the InferVarType just like GradOpDescMaker and OpInfoMaker.
    
+
    REGISTER_OPERATOR(some_op, OpType, SpecialVarTypeInferer, ...);
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/model_format.html b/doc/design/model_format.html
new file mode 100644
index 0000000000000000000000000000000000000000..6567c9c280e48449fd7c05f33200d2a9d622e7ff
--- /dev/null
+++ b/doc/design/model_format.html
@@ -0,0 +1,296 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Model Format — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Model Format
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Model Format
    
+
    
+
    Motivation
    
+
    A model is an output of the training process. One complete model consists of two parts, the topology and the parameters. In order to support industrial deployment, the model format must be self-complete and must not expose any training source code.
    
+
    As a result, In PaddlePaddle, the topology is represented as a  ProgramDesc, which describes the model structure. The parameters contain all the trainable weights in the model. We must support large size parameters and efficient serialization/deserialization of parameters.
    
+
    
+
    
+
    Implementation
    
+
    The topology is saved as a plain text in a detailed self-contain protobuf file.
    
+
    The parameters are saved as a binary file. As we all know, the protobuf message has a limit of 64M size. We have done a benchmark experiment, which shows that protobuf is not fit for the task.
    
+
    As a result, we design a particular format for tensor serialization. By default, an arbitrary tensor in Paddle is a LoDTensor, and has a description information proto of LoDTensorDesc. We save the DescProto as the byte string header. It contains all the necessary information, such as the dims, and the LoD information in LoDTensor. A tensor stores values in a continuous memory buffer. For speed we dump the raw memory to disk and save it as the byte string content. So, the binary format of one tensor is,
    
+
    The table below shows a tensor’s byte view in detail. Note that all the signed values are written in the little-endian format.
    
+
    |field name  | type | description |
+| — | — | — |
+| version | uint32_t | Version of saved file. Always 0 now. |
+| tensor desc length | uint32_t | TensorDesc(Protobuf message) length in bytes. |
+| tensor desc | void* | TensorDesc protobuf binary message |
+| tensor data | void* | Tensor’s data in binary format. The length of tensor_data is decided by TensorDesc.dims() and TensorDesc.data_type() |
+| lod_level | uint64_t | Level of LoD |
+| length of lod[0] | uint64_t | [Optional] length of lod[0] in bytes. |
+| data of lod[0] | uint64_t*  | [Optional] lod[0].data() |
+| ... | ... | ... |
    
+
    
+
    
+
    Summary
    
+
      
+
    • We introduce a model format.
      • 
+
    • The model represented by its forward-pass computation procedure is saved in a ProgramDesc protobuf message.
      • 
+
    • A bunch of specified format binary tensors describe the parameters.
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/multi_language_interface/why_plain_c.html b/doc/design/multi_language_interface/00.why_plain_c.html
similarity index 82%
rename from doc/design/multi_language_interface/why_plain_c.html
rename to doc/design/multi_language_interface/00.why_plain_c.html
index eddc60f4d72adeb8546600dd28c3d5e37d2ba30e..bf869bdb286e0966c62e6b74856d0bbf6549ce8a 100644
--- a/doc/design/multi_language_interface/why_plain_c.html
+++ b/doc/design/multi_language_interface/00.why_plain_c.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,7 +84,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -251,29 +279,29 @@
 typedef int paddle_error;
 
 extern "C"
-paddle_error paddle_matrix_shape(paddle_matrix matrix,
-                                 uint64_t* width,
-                                 uint64_t* height);
+paddle_error paddle_matrix_get_shape(paddle_matrix matrix,
+                                     uint64_t* width,
+                                     uint64_t* height);

    而在CPP里面实现这个C的接口，文件 paddle_matrix.cpp

    -
    #include "paddle/math/matrix.hpp"
+
    #include "paddle/math/matrix.h"
 extern "C"
 paddle_error paddle_matrix_shape(paddle_matrix matrix,
                                  uint64_t *width,
                                  uint64_t *height) {
-  auto m = (paddle::math::matrix*)(matrix);
+  auto m = (paddle::capi::CMatrix*)(matrix);
   *width = m->width();
   *height = m->height();
 }
 
    
 
    
-
    其中paddle/math/matrix.hpp文件内容为:
    
+
    其中paddle/capi/CMatrix.hpp文件内容为:
    
 
    namespace paddle {
 namespace math {  
 
-class Matrix {
-  //...
+class CMatrix {
+  std::shared_ptr<paddle::Matrix> mat;
 };
 
 }  // namespace math
@@ -310,8 +338,8 @@
 | 手写多语言绑定 | 不使用SWIG | 使用SWIG需要多语言绑定的开发人员熟练掌握SWIG配置，社区参与困难。SWIG生成的代码不能保证多语言代码风格的一致性 |
    
 
    
 
    
-
    简单实现
    
-
    TBD
    
+
    实现
    
+
    参考Inference implementation
    
 
    
 
    
 
diff --git a/doc/design/multi_language_interface/01.inference_implementation.html b/doc/design/multi_language_interface/01.inference_implementation.html
new file mode 100644
index 0000000000000000000000000000000000000000..5f3283df3c4b99718c192de12af588b33e63a257
--- /dev/null
+++ b/doc/design/multi_language_interface/01.inference_implementation.html
@@ -0,0 +1,401 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  C-API 模型推断实现文档 — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • C-API 模型推断实现文档
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    C-API 模型推断实现文档
    
+
    本文档描述Paddle C-API的实现细节。Paddle C-API是多语言API的基础部分。Paddle需要暴露的API很多。先实现模型推断的API，通过模型推断API的实现作为一个样例，来进行讨论。至于为什么需要C-API，请参考Why Plain C。
    
+
+
    
+
    暴露接口原则
    
+
      
+
    1. 所有的接口均为C接口。即使用extern "C"
      2. 
+
    2. 除构造某种类型的函数(paddle_matrix_create等)，其他函数均返回paddle_error。且调用时不能抛出异常或出现运行时错误。
      3. 
+
    3. 所有类型名为paddle_类型名，所有与类型相关的函数，函数名为paddle_类型名_函数名
      4. 
+
    4. 如果某一个Paddle Core概念(GradientMachine/Matrix)需要被暴露到其他语言，那么
        
+
      • 为了暴露的接口尽量简单。只暴露概念的接口，而不暴露概念的实现。即暴露GradientMachine或者Matrix但不暴露RecurrentGradientMachine和CpuSparseMatrix。
        • 
+
      • 暴露这个概念必要函数。必要是指，即完成某一个任务的最少函数。
        • 
+
      
+
      5. 
+
    5. 不在capi接口层做过多封装。
        
+
      • 如果某一个Paddle概念必须要暴露，但是又过于琐碎。不在capi这一层进行封装，而是直接修改Paddle Core。让Paddle核心中，这一概念不再琐碎。
        • 
+
      
+
      6. 
+
    
+
    
+
    
+
    目录结构
    
+
    Paddle
+  `-- paddle
+        `-- capi
+              `-- examples  # The example project for C-API.
+              `-- tests  # unittests for C-API
+              `-- capi.h  # C-API header file.
+              `-- capi_private.h  # The shared header file between implementation sources.
+              `-- matrix.{h, cpp}
+              `-- gradient_machine.{h, cpp}
+              `-- ...
+
    
+
    
+
    Paddle的C-API目录结构如上图表所示。这个目录中除了capi_private.h之外的所有头文件，均会被安装到include/paddle路径下。C-API生成的二进制文件会被安装到lib目录下。即，安装后的目录结构为
    
+
    `-- include
+      `-- paddle
+             `-- capi.h
+             `-- matrix.h
+             `-- gradient_machine.h
+             `-- ...
+`-- lib
+     `-- libpaddle_capi_shared.{so, dylib}  # In mac, dynamic libary's file name extention is `dylib`
+     `-- libpaddle_capi_whole.a  # static library for all symbols of Paddle.
+
    
+
    
+
    
+
    
+
    实现方式
    
+
    下面分别介绍某一类文件的实现方式。
    
+
    
+
    capi.h
    
+
    capi.h是用户使用C-API时所唯一需要引入的头文件。在capi.h中，引入了类型的头文件，matrix.h, gradient_machine.h。在引入其他类型的头文件时，使用相对路径的引用方式。即#include "matrix.h"
    
+
    
+
    
+
    具体某种类型的头文件
    
+
    具体某种类型的头文件，即例如matrix.h，gradient_machine.h等。在这些头文件中，包含了某种类型的类型定义和暴露的全部函数。
    
+
    这个头文件不假设其他文件的引用顺序，即使用户直接引用某种类型的头文件，也不应该报错(虽然不鼓励这样)。如果某一个类型需要引用另一个类型，例如gradient_machine需要引用matrix，则直接引入另一种类型的头文件，即#include "matrix.h"。
    
+
    
+
    
+
    capi_private.h
    
+
    capi_prviate.h是各个实现中共享的头文件，他主要包含了实际暴露的类型结构。在用户使用C-API时，Paddle的类型全部退化成void *，即typedef paddle_matrix void*。但，对于每种C-API暴露的类型，均是在capi_private.h中实现的结构体。
    
+
    struct CMatrix {
+   int type = MatrixType;
+   std::shared_ptr<paddle::Matrix> mat;
+};
+
    
+
    
+
    通常，这个结构体包含两个项目。
    
+
      
+
    • type是一个类型的标志。对于每种类型，type字段均不尽相同。这样，即使C-API接受的类型全是void *，我们也可以确定每一个参数的类型。
      
+
      void some_c_api_function(void* some_instance) {
+   int* type = (int *) some_instance;
+   switch (*type) {
+     case MatrixType:
+       CMatrix* mat = (CMatrix *) some_instance;
+       ...
+     ...
+   }
+}
+
      
+
      
+
      • 
+
    • 这个结构体中的另一个项目是，Paddle Core中这一类型接口的智能指针(shared_ptr)。
      
+
        
+
      • 使用智能指针的原因是: 用户可以安全的释放某个C-API的实例，而不必在意Paddle Core是否还在使用这个实例。
        • 
+
      • 例如，用户通过C-API获得了神经网络的参数实例。当用户使用完这个参数后，直接删除这个参数即可。即便Paddle Core中的模型还在使用这个参数，这个参数也不会一并删除。
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    具体某种类型的实现文件
    
+
    具体某种类型的实现文件，即matrix.cpp, gradient_machine.cpp等文件。在这些文件中，使用C++ 11实现了C-API的接口，并且使用extern "C"导出这些接口。在实现过程中，对输入参数的安全性进行了必要的判断，并将C-API接口的参数转发给Paddle Core。
    
+
    
+
    
+
    libpaddle_capi_shared.{so, dylib}
    
+
    libpaddle_capi_shared是C-API导出的动态库。这个动态库的连接参数与Paddle的其他二进制(例如paddle_trainer)类似。用户可以直接使用这个动态库来引入Paddle C-API。具体使用方法为-lpaddle_capi_shared。
    
+
    
+
    
+
    libpaddle_capi_whole.a
    
+
    libpaddle_capi_whole是C-API导出的静态库。这个静态库包含了Paddle的全部符号。他是将libpaddle_gserver.a, libpaddle_math.a, libpaddle_capi.a等全部静态库中的目标文件全部打包后产生的文件。具体使用方法为--whole-archive -lpaddle_capi_whole --no-whole-archive。
    
+
    
+
    
+
    examples
    
+
    在样例中，使用C99开发了模型预测的样例代码。具体请参考example/README.md。
    
+
    
+
    
+
    
+
    编译选项
    
+
    C-API的编译选项默认关闭，打开这个编译选项，需要在cmake的时候，设置
    
+
    cmake ${YOUR_SOURCE_ROOT} -DWITH_C_API=ON -DWITH_PYTHON=OFF -DWITH_SWIG_PY=OFF
+
    
+
    
+
    编译C-API的时候推荐Paddle不嵌入Python解释器，也不生成SWIG接口，具体原因参考Why Plain C。
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/ops/rnn.html b/doc/design/ops/rnn.html
new file mode 100644
index 0000000000000000000000000000000000000000..3ec044c0bdbedcf2017cbb2b6b747b1dfe47cd15
--- /dev/null
+++ b/doc/design/ops/rnn.html
@@ -0,0 +1,397 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  RNNOp design — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • RNNOp design
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    RNNOp design
    
+
    This document describes the RNN (Recurrent Neural Network) operator and how it is implemented in PaddlePaddle. The RNN op requires that all instances in a mini-batch have the same length. We will have a more flexible dynamic RNN operator in the future.
    
+
    
+
    RNN Algorithm Implementation
    
+
    
+
+
    The above diagram shows an RNN unrolled into a full network.
    
+
    There are several important concepts here:
    
+
      
+
    • step-net: the sub-graph that runs at each step.
      • 
+
    • memory, $h_t$, the state of the current step.
      • 
+
    • ex-memory, $h_{t-1}$, the state of the previous step.
      • 
+
    • initial memory value, the memory of the first (initial) step.
      • 
+
    
+
    
+
    Step-scope
    
+
    There could be local variables defined in each step-net.  PaddlePaddle runtime realizes these variables in step-scopes which are created for each step.
    
+
    
+
    
+Figure 2 illustrates the RNN's data flow
+
    Please be aware that every step runs the same step-net.  Each step does the following:
    
+
      
+
    1. Creates the step-scope.
      2. 
+
    2. Initializes the local variables including step-outputs, in the step-scope.
      3. 
+
    3. Runs the step-net, which uses the above mentioned variables.
      4. 
+
    
+
    The RNN operator will compose its output from step outputs in each of the step scopes.
    
+
    
+
    
+
    Memory and Ex-memory
    
+
    Let’s give more details about memory and ex-memory using a simple example:
    
+
    $$
+h_t = U h_{t-1} + W x_t
+$$,
    
+
    where $h_t$ and $h_{t-1}$ are the memory and ex-memory (previous memory) of step $t$ respectively.
    
+
    In the implementation, we can make an ex-memory variable either “refer to” the memory variable of the previous step,
+or copy the memory value of the previous step to the current ex-memory variable.
    
+
    
+
    
+
    Usage in Python
    
+
    For more information on Block, please refer to the design doc.
    
+
    We can define an RNN’s step-net using a Block:
    
+
    import paddle as pd
+
+X = some_op() # x is some operator's output and is a LoDTensor
+a = some_op()
+
+# declare parameters
+W = pd.Variable(shape=[20, 30])
+U = pd.Variable(shape=[20, 30])
+
+rnn = pd.create_rnn_op(output_num=1)
+with rnn.stepnet():
+    x = rnn.add_input(X)
+    # declare a memory (rnn's step)
+    h = rnn.add_memory(init=a)
+    # h.pre_state(), the previous memory of rnn
+    new_state = pd.add_two( pd.matmul(W, x) + pd.matmul(U, h.pre_state()))
+    # update current memory
+    h.update(new_state)
+    # indicate that h variables in all step scopes should be merged
+    rnn.add_outputs(h)
+
+out = rnn()
+
    
+
    
+
    Python API functions in above example:
    
+
      
+
    • rnn.add_input: indicates that the parameter is a variable that will be segmented into step-inputs.
      • 
+
    • rnn.add_memory: creates a variable used as the memory.
      • 
+
    • rnn.add_outputs: marks the variables that will be concatenated across steps into the RNN output.
      • 
+
    
+
    
+
    
+
    Nested RNN and LoDTensor
    
+
    An RNN whose step-net includes other RNN operators is known as an nested RNN.
    
+
    For example, we could have a 2-level RNN, where the top level corresponds to paragraphs, and the lower level corresponds to sentences. Each step of the higher level RNN also receives an input from the corresponding step of the lower level, and additionally the output from the previous time step at the same level.
    
+
    The following figure illustrates feeding in text into the lower level, one sentence at a step, and the feeding in step outputs to the top level. The final top level output is about the whole text.
    
+
    
+
+
    import paddle as pd
+
+W = pd.Variable(shape=[20, 30])
+U = pd.Variable(shape=[20, 30])
+
+W0 = pd.Variable(shape=[20, 30])
+U0 = pd.Variable(shape=[20, 30])
+
+# a is output of some op
+a = some_op()
+
+# chapter_data is a set of 128-dim word vectors
+# the first level of LoD is sentence
+# the second level of LoD is a chapter
+chapter_data = pd.Variable(shape=[None, 128], type=pd.lod_tensor, level=2)
+
+def lower_level_rnn(paragraph):
+    '''
+    x: the input
+    '''
+    rnn = pd.create_rnn_op(output_num=1)
+    with rnn.stepnet():
+        sentence = rnn.add_input(paragraph, level=0)
+        h = rnn.add_memory(shape=[20, 30])
+        h.update(
+            pd.matmul(W, sentence) + pd.matmul(U, h.pre_state()))
+        # get the last state as sentence's info
+        rnn.add_outputs(h)
+    return rnn
+
+top_level_rnn = pd.create_rnn_op(output_num=1)
+with top_level_rnn.stepnet():
+    paragraph_data = rnn.add_input(chapter_data, level=1)
+    low_rnn = lower_level_rnn(paragraph_data)
+    paragraph_out = low_rnn()
+
+    h = rnn.add_memory(init=a)
+    h.update(
+        pd.matmul(W0, paragraph_data) + pd.matmul(U0, h.pre_state()))
+    top_level_rnn.add_outputs(h)
+
+# output the last step
+chapter_out = top_level_rnn(output_all_steps=False)
+
    
+
    
+
    In the above example, the construction of the top_level_rnn calls  lower_level_rnn.  The input is an LoD Tensor. The top level RNN segments input text data into paragraphs, and the lower level RNN segments each paragraph into sentences.
    
+
    By default, the RNNOp will concatenate the outputs from all the time steps.
+If the output_all_steps is set to False, it will only output the final time step.
    
+
    
+
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/ops/sequence_decoder.html b/doc/design/ops/sequence_decoder.html
new file mode 100644
index 0000000000000000000000000000000000000000..6c5fd244206e18f60a8399c95ed532b54539e398
--- /dev/null
+++ b/doc/design/ops/sequence_decoder.html
@@ -0,0 +1,472 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design: Sequence Decoder Generating LoDTensors — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design: Sequence Decoder Generating LoDTensors
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design: Sequence Decoder Generating LoDTensors
    
+
    In tasks such as machine translation and visual captioning,
+a sequence decoder is necessary to generate sequences, one word at a time.
    
+
    This documentation describes how to implement the sequence decoder as an operator.
    
+
    
+
    Beam Search based Decoder
    
+
    The beam search algorithm is necessary when generating sequences. It is a heuristic search algorithm that explores the paths by expanding the most promising node in a limited set.
    
+
    In the old version of PaddlePaddle, the C++ class RecurrentGradientMachine implements the general sequence decoder based on beam search, due to the complexity involved, the implementation relies on a lot of special data structures that are quite trivial and hard to be customized by users.
    
+
    There are a lot of heuristic tricks in the sequence generation tasks, so the flexibility of sequence decoder is very important to users.
    
+
    During the refactoring of PaddlePaddle, some new concepts are proposed such as:  LoDTensor and TensorArray that can better support the sequence usage, and they can also help make the implementation of beam search based sequence decoder more transparent and modular .
    
+
    For example, the RNN states, candidates IDs and probabilities of beam search can be represented all as LoDTensors;
+the selected candidate’s IDs in each time step can be stored in a TensorArray, and Packed to the sentences translated.
    
+
    
+
    
+
    Changing LoD’s absolute offset to relative offsets
    
+
    The current LoDTensor is designed to store levels of variable-length sequences. It stores several arrays of integers where each represents a level.
    
+
    The integers in each level represent the begin and end (not inclusive) offset of a sequence in the underlying tensor,
+let’s call this format the absolute-offset LoD for clarity.
    
+
    The relative-offset LoD can retrieve any sequence very quickly but fails to represent empty sequences, for example, a two-level LoD is as follows
    
+
    [[0, 3, 9]
+ [0, 2, 3, 3, 3, 9]]
+
    
+
    
+
    The first level tells that there are two sequences:
    
+
      
+
    • the first’s offset is [0, 3)
      • 
+
    • the second’s offset is [3, 9)
      • 
+
    
+
    while on the second level, there are several empty sequences that both begin and end at 3.
+It is impossible to tell how many empty second-level sequences exist in the first-level sequences.
    
+
    There are many scenarios that rely on empty sequence representation, for example in machine translation or visual captioning, one instance has no translation or the empty candidate set for a prefix.
    
+
    So let’s introduce another format of LoD,
+it stores the offsets of the lower level sequences and is called relative-offset LoD.
    
+
    For example, to represent the same sequences of the above data
    
+
    [[0, 3, 6]
+ [0, 2, 3, 3, 3, 9]]
+
    
+
    
+
    the first level represents that there are two sequences,
+their offsets in the second-level LoD is [0, 3) and [3, 5).
    
+
    The second level is the same with the relative offset example because the lower level is a tensor.
+It is easy to find out the second sequence in the first-level LoD has two empty sequences.
    
+
    The following examples are based on relative-offset LoD.
    
+
    
+
    
+
    Usage in a simple machine translation model
    
+
    Let’s start from a simple machine translation model that is simplified from the machine translation chapter to draw a blueprint of what a sequence decoder can do and how to use it.
    
+
    The model has an encoder that learns the semantic vector from a sequence, and a decoder which uses the sequence encoder to generate new sentences.
    
+
    Encoder
    
+
    import paddle as pd
+
+dict_size = 8000
+source_dict_size = dict_size
+target_dict_size = dict_size
+word_vector_dim = 128
+encoder_dim = 128
+decoder_dim = 128
+beam_size = 5
+max_length = 120
+
+# encoder
+src_word_id = pd.data(
+    name='source_language_word',
+    type=pd.data.integer_value_sequence(source_dict_dim))
+src_embedding = pd.embedding(size=source_dict_size, size=word_vector_dim)
+
+src_word_vec = pd.lookup(src_embedding, src_word_id)
+
+encoder_out_seq = pd.gru(input=src_word_vec, size=encoder_dim)
+
+encoder_ctx = pd.last_seq(encoder_out_seq)
+# encoder_ctx_proj is the learned semantic vector
+encoder_ctx_proj = pd.fc(
+    encoder_ctx, size=decoder_dim, act=pd.activation.Tanh(), bias=None)
+
    
+
    
+
    Decoder
    
+
    def generate():
+    decoder = pd.while_loop()
+    with decoder.step():
+        decoder_mem = decoder.memory(init=encoder_ctx)  # mark the memory
+        generated_ids = decoder.memory() # TODO init to batch_size <s>s
+        generated_scores = decoder.memory() # TODO init to batch_size 1s or 0s
+
+        target_word = pd.lookup(trg_embedding, gendrated_ids)
+        # expand encoder_ctx's batch to fit target_word's lod
+        # for example
+        # decoder_mem.lod is
+        # [[0 1 3],
+        #  [0 1 3 6]]
+        # its tensor content is [a1 a2 a3 a4 a5]
+        # which means there are 2 sentences to translate
+        #   - the first sentence has 1 translation prefixes, the offsets are [0, 1)
+        #   - the second sentence has 2 translation prefixes, the offsets are [1, 3) and [3, 6)
+        # the target_word.lod is
+        # [[0, 1, 6]
+        #  [0, 2, 4, 7, 9 12]]
+        # which means 2 sentences to translate, each has 1 and 5 prefixes
+        # the first prefix has 2 candidates
+        # the following has 2, 3, 2, 3 candidates
+        # the encoder_ctx_expanded's content will be
+        # [a1 a1 a2 a2 a3 a3 a3 a4 a4 a5 a5 a5]
+        encoder_ctx_expanded = pd.lod_expand(encoder_ctx, target_word)
+        decoder_input = pd.fc(
+            act=pd.activation.Linear(),
+            input=[target_word, encoder_ctx],
+            size=3 * decoder_dim)
+        gru_out, cur_mem = pd.gru_step(
+            decoder_input, mem=decoder_mem, size=decoder_dim)
+        scores = pd.fc(
+            gru_out,
+            size=trg_dic_size,
+            bias=None,
+            act=pd.activation.Softmax())
+        # K is an config
+        topk_scores, topk_ids = pd.top_k(scores, K)
+        topk_generated_scores = pd.add_scalar(topk_scores, generated_scores)
+
+        selected_ids, selected_generation_scores = decoder.beam_search(
+            topk_ids, topk_generated_scores)
+
+        # update the states
+        decoder_mem.update(cur_mem)  # tells how to update state
+        generated_ids.update(selected_ids)
+        generated_scores.update(selected_generation_scores)
+
+        decoder.output(selected_ids)
+        decoder.output(selected_generation_scores)
+
+translation_ids, translation_scores = decoder()
+
    
+
    
+
    The decoder.beam_search is an operator that, given the candidates and the scores of translations including the candidates,
+returns the result of the beam search algorithm.
    
+
    In this way, users can customize anything on the input or output of beam search, for example:
    
+
      
+
    1. Make the corresponding elements in topk_generated_scores zero or some small values, beam_search will discard this candidate.
      2. 
+
    2. Remove some specific candidate in selected_ids.
      3. 
+
    3. Get the final translation_ids, remove the translation sequence in it.
      4. 
+
    
+
    The implementation of sequence decoder can reuse the C++ class:  RNNAlgorithm,
+so the python syntax is quite similar to that of an  RNN.
    
+
    Both of them are two-level LoDTensors:
    
+
      
+
    • The first level represents batch_size of (source) sentences.
      • 
+
    • The second level represents the candidate ID sets for translation prefix.
      • 
+
    
+
    For example, 3 source sentences to translate, and has 2, 3, 1 candidates.
    
+
    Unlike an RNN, in sequence decoder, the previous state and the current state have different LoD and shape, and an lod_expand operator is used to expand the LoD of the previous state to fit the current state.
    
+
    For example, the previous state:
    
+
      
+
    • LoD is [0, 1, 3][0, 2, 5, 6]
      • 
+
    • content of tensor is a1 a2 b1 b2 b3 c1
      • 
+
    
+
    the current state is stored in encoder_ctx_expanded:
    
+
      
+
    • LoD is [0, 2, 7][0 3 5 8 9 11 11]
      • 
+
    • the content is
        
+
      • a1 a1 a1 (a1 has 3 candidates, so the state should be copied 3 times for each candidates)
        • 
+
      • a2 a2
        • 
+
      • b1 b1 b1
        • 
+
      • b2
        • 
+
      • b3 b3
        • 
+
      • None (c1 has 0 candidates, so c1 is dropped)
        • 
+
      
+
      • 
+
    
+
    The benefit from the relative offset LoD is that the empty candidate set can be represented naturally.
    
+
    The status in each time step can be stored in TensorArray, and Packed to a final LoDTensor. The corresponding syntax is:
    
+
    decoder.output(selected_ids)
+decoder.output(selected_generation_scores)
+
    
+
    
+
    The selected_ids are the candidate ids for the prefixes, and will be Packed by TensorArray to a two-level LoDTensor, where the first level represents the source sequences and the second level represents generated sequences.
    
+
    Packing the selected_scores will get a LoDTensor that stores scores of each translation candidate.
    
+
    Packing the selected_generation_scores will get a LoDTensor, and each tail is the probability of the translation.
    
+
    
+
    
+
    LoD and shape changes during decoding
    
+
    
+  
+
    According to the image above, the only phase that changes the LoD is beam search.
    
+
    
+
    
+
    Beam search design
    
+
    The beam search algorithm will be implemented as one method of the sequence decoder and has 3 inputs:
    
+
      
+
    1. topk_ids, the top K candidate ids for each prefix.
      2. 
+
    2. topk_scores, the corresponding scores for topk_ids
      3. 
+
    3. generated_scores, the score of the prefixes.
      4. 
+
    
+
    All of these are LoDTensors, so that the sequence affiliation is clear. Beam search will keep a beam for each prefix and select a smaller candidate set for each prefix.
    
+
    It will return three variables:
    
+
      
+
    1. selected_ids, the final candidate beam search function selected for the next step.
      2. 
+
    2. selected_scores, the scores for the candidates.
      3. 
+
    3. generated_scores, the updated scores for each prefix (with the new candidates appended).
      4. 
+
    
+
    
+
    
+
    Introducing the LoD-based Pack and Unpack methods in TensorArray
    
+
    The selected_ids, selected_scores and generated_scores are LoDTensors that exist at each time step,
+so it is natural to store them in arrays.
    
+
    Currently, PaddlePaddle has a module called TensorArray which can store an array of tensors. It is better to store the results of beam search in a TensorArray.
    
+
    The Pack and UnPack in TensorArray are used to pack tensors in the array to an LoDTensor or split the LoDTensor to an array of tensors.
+It needs some extensions to support the packing or unpacking an array of LoDTensors.
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/optimizer.html b/doc/design/optimizer.html
new file mode 100644
index 0000000000000000000000000000000000000000..19c1dfdf899326b89a8ff195188ce4a08a7df503
--- /dev/null
+++ b/doc/design/optimizer.html
@@ -0,0 +1,352 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Optimizer Design — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Optimizer Design
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Optimizer Design
    
+
    
+
    The Problem
    
+
    A PaddlePaddle program, or a block, is a sequence of operators operating variables.  A training program needs to do three kinds of works:
    
+
      
+
    1. the forward pass, which computes intermediate results and the cost(s),
      2. 
+
    2. the backward pass, which derives gradients from intermediate results and costs, and
      3. 
+
    3. the optimization pass, which update model parameters to optimize the cost(s).
      4. 
+
    
+
    These works rely on three kinds of operators:
    
+
      
+
    1. forward operators,
      2. 
+
    2. gradient operators, and
      3. 
+
    3. optimization operators.
      4. 
+
    
+
    It’s true that users should be able to create all these operators manually by calling some low-level API, but it would be much more convenient if they could only describe the forward pass and let PaddlePaddle create the backward and optimization operators automatically.
    
+
    In this design, we propose a high-level API that automatically derives the optimisation pass and operators from the forward pass.
    
+
    
+
    
+
    High-level Python API to describe the training process
    
+
      
+
    1. User write code to describe the network:
      
+
      images = layer.data("images")
+labels = layer.data("labels")
+w1 = pd.var("w1")
+b1 = pd.var("b1")
+hidden = layer.fc(images, w=w1, b=b1)
+cost = layer.mse(hidden, labels)
+
      
+
      
+
      The above code snippet will create forward operators in Block.
      
+
      2. 
+
    
+
      
+
    1. Users create a certain kind of Optimizer with some argument.
      
+
      optimizer = AdagradOptimizer(learing_rate=0.001)
+
      
+
      
+
      2. 
+
    2. Users use the optimizer to minimize a certain cost through updating parameters in parameter_list.
      
+
      opt_op_list = optimizer.minimize(cost, parameter_list=[w1, b1])
+
      
+
      
+
      The above code snippet will create gradient and optimization operators in Block. The return value of minimize() is list of optimization operators that will be run by session.
      
+
      3. 
+
    3. Users use Session/Executor to run this opt_op_list as target to do training.
      
+
      sess.run(target= opt_op_list, ...)
+
      
+
      
+
      4. 
+
    
+
    
+
    Optimizer Python interface:
    
+
    class Optimizer(object):
+    """Optimizer Base class.
+
+    """
+
+    def __init__(self):
+        pass
+
+    def create_optimization_pass(self, parameters_and_grads):
+        """Add optimization operators to update gradients to variables.
+
+        Args:
+          parameters_and_grads: a list of (variable, gradient) pair to update.
+
+        Returns:
+          optmization_op_list: a list of optimization operator that will update parameter using gradient.
+        """
+        return None
+
+    def minimize(self, loss, parameter_list):
+        """Add operations to minimize `loss` by updating `parameter_list`.
+
+        This method combines interface `append_backward_ops()` and
+        `create_optimization_pass()` into one.
+        """
+        params_grads = self.create_backward_pass(loss, parameter_list)
+        update_ops = self.create_optimization_pass(params_grads)
+        return update_ops
+
+
    
+
    
+
    Users can inherit the Optimizer above to create their own Optimizer with some special logic, such as AdagradOptimizer.
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/parameter_average.html b/doc/design/parameter_average.html
new file mode 100644
index 0000000000000000000000000000000000000000..d9458274677da6bad67c5fa6604997ee2288d450
--- /dev/null
+++ b/doc/design/parameter_average.html
@@ -0,0 +1,342 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Averaging Parameter in PaddlePaddle — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Averaging Parameter in PaddlePaddle
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Averaging Parameter in PaddlePaddle
    
+
    
+
    Why Averaging
    
+
    In a large scale machine learning setup where the size of the training data is huge, it could take us a large number of iterations over the training data before we can achieve the optimal values of parameters of our model. Looking at the problem setup, it is desirable if we can obtain the optimal values of parameters by going through the data in as few passes as we can.
    
+
    Polyak and Juditsky (1992) showed that the test performance of simple average of parameters obtained by Stochastic Gradient Descent (SGD) is as good as that of parameter values that are obtained by training the model over and over again, over the training dataset.
    
+
    Hence, to accelerate the speed of Stochastic Gradient Descent, Averaged Stochastic Gradient Descent (ASGD) was proposed in Polyak and Juditsky (1992). For ASGD, the running average of parameters obtained by SGD, is used as the estimator for 
     . The averaging is done as follows:
    
+

    
+
    We propose averaging for any optimizer similar to how ASGD performs it, as mentioned above.
    
+
    
+
    How to perform Parameter Averaging in PaddlePaddle
    
+
    Parameter Averaging in PaddlePaddle works in the following way during training :
    
+
      
+
    1. It will take in an instance of a normal optimizer as an input, e.g. RMSPropOptimizer
      2. 
+
    2. The optimizer itself is responsible for updating the parameters.
      3. 
+
    3. The ParameterAverageOptimizer maintains a separate copy of the parameters for itself:
        
+
      1. In concept, the values of this copy are the average of the values of the parameters in the most recent N batches.
        2. 
+
      2. However, saving all the N instances of the parameters in memory is not feasible.
        3. 
+
      3. Therefore, an approximation algorithm is used.
        4. 
+
      
+
      4. 
+
    
+
    Hence, overall we have have two copies of the parameters: one for the optimizer itself, and one for the ParameterAverageOptimizer. The former should be used in back propagation, while the latter should be used during testing and should be saved.
    
+
    During the testing/ saving the model phase, we perform the following steps:
    
+
      
+
    1. Perform the delayed operations.
      2. 
+
    2. Save current values of the parameters to a temporary variable.
      3. 
+
    3. Replace the values of the parameters with the averaged values.
      4. 
+
    4. Perform testing and/or save the parameters.
      5. 
+
    5. Restore the values of the parameters once done.
      6. 
+
    
+
    
+
    
+
    How to implement Averaging of Parameter in PaddlePaddle
    
+
    We can add the ParameterAverageOptimizer op to the graph through Python API. Using this approach, we manually add this op to the graph and direct the output of the optimizer op to this op during training.
    
+
    **Advantages**:
+- Allows for greater flexibility to the users of PaddlePaddle. Using this approach, the users can plug different optimizers into ParameterAverageOptimizer by passing in the optimizer to the op.
+- Makes it easy for the users to customize and extend the framework.
+
+**Disadvantages**:
+- Implementation requires re-writing the averaging methodology in Python.  
+
    
+
    
+
    
+
    
+
    Low-Level implementation
    
+
    In the new design, we propose to create a new operation for averaging parameter updates (ParameterAverageOptimizer). For now, we can add an op that takes in the following as input:
    
+
      
+
    • the optimizer
      • 
+
    • the window_size to keep the updates
      • 
+
    
+
    The ParameterAverageOptimizer op can be like any other operator with its own CPU/GPU implementation either using Eigen or separate CPU and GPU kernels. As the initial implementation, we can implement the kernel using Eigen following the abstraction pattern implemented for Operators. We also want to support the case when the Trainer/Optimizer runs on the GPU while ParameterAverageOptimizer runs on a CPU.
    
+
    The idea of building an op for averaging is in sync with the refactored PaddlePaddle philosophy of using operators to represent any computation unit. The way the op will be added to the computation graph will be decided by the layer functions in Python API.
    
+
    
+
    
+
    Python API implementation for ParameterAverageOptimizer
    
+
    Based on Polyak and Juditsky (1992), we can generalize the averaging of updates to any optimizer. The input to the op would be the following:
    
+
      
+
    • Any optimizer (RMSProp , AdaGrad etc.)
      • 
+
    • A window size. The op keeps accumulating updated parameter values over a window of N batches and takes an average. Move the averaged value to a buffer when window is full to avoid loss of precision.
      • 
+
    
+
    Using the ParameterAverageOptimizer op, any user can add the operation to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support averaging. As per the PaddlePaddle Python API design, the layer functions are responsible for creating operators, operator parameters and variables. Since ParameterAverageOptimizer will be an operator, it makes sense to create it in the layer functions.
+We will have a wrapper written in Python that will support the functionality and implement the actual core computation in C++ core as we have done for other Optimizers
    
+
    
+
    Creation of the ParameterAverageOptimizer operator
    
+
    There are two ways for creating the ParameterAverageOptimizer op:
    
+
      
+
    1. We create the op immediately while building the computation graph.
      2. 
+
    2. We add the op in a lazy manner, just before the backward pass, similar to the way the optimization ops are added.
      3. 
+
    
+
    The proposal is to add the op immediately while building the computation graph.
    
+
    
+
    
+
    High-level API
    
+
    In PaddlePaddle Python API, users will primarily rely on layer functions to create neural network layers. Hence, we also need to provide parameter average functionality in layer functions.
    
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/parameters_in_cpp.html b/doc/design/parameters_in_cpp.html
new file mode 100644
index 0000000000000000000000000000000000000000..1734eb5b1f19a5560788074a6063a40c7c131109
--- /dev/null
+++ b/doc/design/parameters_in_cpp.html
@@ -0,0 +1,301 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: The C++ Class Parameters — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: The C++ Class Parameters
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: The C++ Class Parameters
    
+
    Parameters is a concept we designed in PaddlePaddle V2 API. Parameters is a container of parameters, which makes PaddlePaddle capable of  sharing parameter between topologies. We described usages of Parameter in api.md.
    
+
    We used Python to implement Parameters when designing V2 API before. There are several defects for the current implementation:
    
+
      
+
    • We just use memcpy to share Parameters between topologies, but this is very inefficient.
      • 
+
    • We did not support sharing Parameters while training. We just trigger memcpy when start training.
      • 
+
    
+
    It is necessary that we implement Parameters in CPP side. However, it could result a code refactoring for PaddlePaddle, because PaddlePaddle was designed for training only one topology before, i.e., each GradientMachine contains its Parameter as a data member. In current PaddlePaddle implementation, there are three concepts associated with Parameters:
    
+
      
+
    1. paddle::Parameter. A Parameters is a container for paddle::Parameter.
+It is evident that we should use paddle::Parameter when developing Parameters.
+However, the Parameter class contains many functions and does not have a clear interface.
+It contains create/store Parameter, serialize/deserialize, optimize(i.e SGD), randomize/zero.
+When we developing Parameters, we only use create/store Parameter functionality.
+We should extract functionalities of Parameter into many classes to clean PaddlePaddle CPP implementation.
      2. 
+
    2. paddle::GradientMachine and its sub-classes, e.g., paddle::MultiGradientMachine, paddle::NeuralNetwork.
+We should pass Parameters to paddle::GradientMachine when forward/backward to avoid memcpy between topologies.
+Also, we should handle multi-GPU/CPU training, because forward and backward would perform on multi-GPUs and multi-CPUs.
+Parameters should dispatch the parameter value to each device, and gather the parameter gradient from each device.
      3. 
+
    3. paddle::ParameterUpdater. The ParameterUpdater is used to update parameters in Paddle.
+So Parameters should be used by paddle::ParameterUpdater, and paddle::ParameterUpdater should optimize Parameters (by SGD).
      4. 
+
    
+
    The step by step approach for implementation Parameters in PaddlePaddle C++ core is listed below. Each step should be a PR and could be merged into PaddlePaddle one by one.
    
+
      
+
    1. Clean paddle::Parameter interface. Extract the functionalities of paddle::Parameter to prepare for the implementation of Parameters.
      2. 
+
    2. Implementation a Parameters class. It just stores the paddle::Parameter inside. Make GradientMachine uses Parameters as a class member.
      3. 
+
    3. Make Parameters support Multi-CPU and Multi-GPU training to prepare for sharing Parameter between topologies.
+Because we need share Parameters between topologies, it is Parameters‘s response to exchange Parameters between GPUs.
+GradientMachine should not handle how to exchange Parameters because GradientMachine only used to train one topology and we need to support train many topologies in Paddle, i.e., there could be many GradientMachines use one Parameters.
        
+
      • We should use a global function to exchange Parameters between GPUs, not a member function in Parameters. The MultiGradientMachine invoke this function, which uses Parameters as this function inputs.
        • 
+
      • The MultiGradientMachine contains many functionalities. Extracting the Parameters exchanging logic could make MultiGradientMachine clearer and simpler.
        • 
+
      
+
      4. 
+
    4. Make Parameters as an argument for forward/backward function, not a data member for GradientMachine. For example, forward could be forward(const Parameters& params, ...) and backward could be backward(Parameters* params, ...). After this step, Paddle could share Parameters between topologies.
      5. 
+
    5. ParameterUpdater is invoked by GradientMachine and Trainer, but it updates Parameters. In the end of this code refactoring, we could change ParameterUpdater directly uses Parameters to make ParameterUpdater‘s implementation clear.
      6. 
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/program.html b/doc/design/program.html
new file mode 100644
index 0000000000000000000000000000000000000000..5fb7e753a7aa78bb24f74a14536b5276fbd1a63a
--- /dev/null
+++ b/doc/design/program.html
@@ -0,0 +1,389 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: PaddlePaddle Programs — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: PaddlePaddle Programs
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: PaddlePaddle Programs
    
+
    
+
    Compile and Execution
    
+
    A PaddlePaddle program consists of two parts – the first generates a ProgramDesc protobuf message that describes the program, and the second runs this message using a C++ class Executor.
    
+
    A simple example PaddlePaddle program can be found in graph.md:
    
+
    x = layer.data("images")
+l = layer.data("label")
+y = layer.fc(x)
+cost = layer.mse(y, l)
+optimize(cost)
+train(cost, reader=mnist.train())
+
    
+
    
+
    The first five lines of the following PaddlePaddle program generates, or, compiles, the ProgramDesc message.  The last line runs it.
    
+
    
+
    
+
    Programs and Blocks
    
+
    The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.
    
+
      
+
    • program: some nested blocks
      • 
+
    • block:
        
+
      • some local variable definitions, and
        • 
+
      • a sequence of operators
        • 
+
      
+
      • 
+
    
+
    The concept of block comes from usual programs.  For example, the following C++ program has three blocks:
    
+
    int main() { // block 0
+  int i = 0;
+  if (i < 10) { // block 1
+    for (int j = 0; j < 10; j++) { // block 2
+    }
+  }
+  return 0;
+}
+
    
+
    
+
    The following PaddlePaddle program has three blocks:
    
+
    import paddle as pd  // block 0
+
+x = minibatch([10, 20, 30]) # shape=[None, 1]
+y = var(1) # shape=[1], value=1
+z = minibatch([10, 20, 30]) # shape=[None, 1]
+cond = larger_than(x, 15) # [false, true, true]
+
+ie = pd.ifelse()
+with ie.true_block():  // block 1
+    d = pd.layer.add_scalar(x, y)
+    ie.output(d, pd.layer.softmax(d))
+with ie.false_block():  // block 2
+    d = pd.layer.fc(z)
+    ie.output(d, d+1)
+o1, o2 = ie(cond)
+
    
+
    
+
    
+
    
+
    BlockDesc and ProgramDesc
    
+
    All protobuf messages are defined in framework.proto.
    
+
    BlockDesc is straight-forward – it includes local variable definitions, vars, and a sequence of operators, ops.
    
+
    message BlockDesc {
+  required int32 parent = 1;
+  repeated VarDesc vars = 2;
+  repeated OpDesc ops = 3;
+}
+
    
+
    
+
    The parent ID indicates the parent block so that operators in a block can refer to variables defined locally and also those defined in their ancestor blocks.
    
+
    All hierarchical blocks in a program are flattened and stored in an array. The block ID is the index of the block in this array.
    
+
    message ProgramDesc {
+  repeated BlockDesc blocks = 1;
+}
+
    
+
    
+
    
+
    Global Block
    
+
    The global block is the first one in the above array.
    
+
    
+
    
+
    
+
    Operators that Use Blocks
    
+
    In the above example, the operator IfElseOp has two blocks – the true branch and the false branch.
    
+
    The definition of OpDesc shows that an operator could have some attributes:
    
+
    message OpDesc {
+  AttrDesc attrs = 1;
+  ...
+}
+
    
+
    
+
    and an attribute could be of type block, which is, in fact, a block ID as described above:
    
+
    message AttrDesc {
+  required string name = 1;
+
+  enum AttrType {
+    INT = 1,
+    STRING = 2,
+    ...
+    BLOCK = ...
+  }
+  required AttrType type = 2;
+
+  optional int32 block = 10; // when type == BLOCK
+  ...
+}
+
    
+
    
+
    
+
    
+
    InferShape
    
+
    With this design, the InferShape function should take the following parameters:
    
+
    void InferShape(int current_block,
+                int current_operator,
+                ProgramDesc* program // might change VarDesc values.
+                ) {
+  ...
+}
+
    
+
    
+
    where
    
+
      
+
    • current_block indices into ProgramDesc::blocks,
      • 
+
    • current_operator indices into BlockDesc::ops.
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/prune.html b/doc/design/prune.html
new file mode 100644
index 0000000000000000000000000000000000000000..2e5de800398db6a4dd0d2abbf2fdc20ec2e2467b
--- /dev/null
+++ b/doc/design/prune.html
@@ -0,0 +1,323 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Prune — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Prune
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Prune
    
+
    
+
    Motivation
    
+
    We want to support running inference, training and checkpointing in one ProgramDesc. We implement
+void Prune(const ProgramDesc* input, ProgramDesc* output) function, which takes a ProgramDesc
+and generate a pruned ProgramDesc.
    
+
    
+
    
+
    Challenge
    
+
    Pruning need to support both variables and operators being evaluation targets. Consider the following
+different situations.
    
+
    # Case 1: run foward pass.
+cost_np = session.run(target=cost)
+# Case 2: run backward passing.
+opts_np, _ = session.run(target=[cost, opt])
+# Case 3: run checkpointing
+_ = session.run(target=checkpoint)
+
    
+
    
+
    
+
    
+
    Solution
    
+
    To support evaluation of operators, we add is_target field in the OpDesc.
    
+
    message OpDesc {
+  required string type = 3;
+  repeated Var inputs = 1;
+  repeated Var outputs = 2;
+  repeated Attr attrs = 4;
+  optional bool is_target = 5 [ default = false ];
+};
+
    
+
    
+
    To support evaluation of variables, we add fetch_op.
+For each variable in the target, we insert a fetch_op into the ProgramDesc with variable being
+fetch_op‘s input. Then we also set fetch_op is a target.
    
+
    
+
    Algorithm
    
+
    If an operator needs to be run, it must fall into one of the following cases:
    
+
      
+
    1. It is the target.
      2. 
+
    2. It is depended by some other ops, meaning its output is some other op’s input.
      3. 
+
    
+
    The first case can be checked by op_desc.is_traget() . The second case can be implement as
    
+
    bool HasDependentVar(const OpDesc& op_desc, const std::set<string>& dependent_vars) {
+  for (auto& var : op_desc.outputs()) {
+    for (auto& argu : var.arguments()) {
+      if (dependent_vars.count(argu) != 0) {
+        return true;
+      }
+    }
+  }
+  return false;
+}
+
    
+
    
+
    Then the whole algorithm can be implemented as the following code.
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/python_api.html b/doc/design/python_api.html
new file mode 100644
index 0000000000000000000000000000000000000000..f1a496634422857fe9a714820e477f3aaf9af653
--- /dev/null
+++ b/doc/design/python_api.html
@@ -0,0 +1,520 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Python API — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Python API
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Python API
    
+
    Due to the refactorization of the PaddlePaddle core, we need Python classes to construct corresponding protobuf messages that describe a DL program.
    
+
    | Python classes | Protobuf messages |
+| — | — |
+| Program | ProgramDesc |
+| Block | BlockDesc |
+| Operator | OpDesc |
+| Variable | VarDesc |
    
+
    Please be aware that these Python classes need to maintain some construction-time information, which are not part of the protobuf messages.
    
+
    
+
    Core Concepts
    
+
    
+
    Program
    
+
    A ProgramDesc describes a DL program, which is composed of an array of BlockDescs.  The BlockDescs in a ProgramDesc can have a tree-like hierarchical structure. However, the ProgramDesc onlys stores a flattened array of BlockDescs. A BlockDesc refers to its parent block by its index in the array.  For example, operators in the step block of an RNN operator need to be able to access variables in its ancestor blocks.
    
+
    Whenever we create a block, we need to set its parent block to the current block, hence the Python class Program needs to maintain a data member current_block.
    
+
    class Program(objects):
+    def __init__(self):
+        self.desc = core.NewProgram() # a C++ ProgramDesc pointer.
+        self.blocks = vector<Block>()
+        self.blocks.append(Block(self, -1)) # the global block
+        self.current_block = 0          # initialized to the global block
+
+    def global_block():
+        return self.blocks[0]
+
+    def current_block():
+        return self.get_block(self.current_block)
+
+    def rollback():
+        self.current_block = self.current_block().parent_idx
+
+    def create_block():
+        new_block_idx = len(self.block)
+        self.blocks.append(Block(self, self.current_block))
+        self.current_block = new_block_idx
+        return current_block()
+
    
+
    
+
    Program is an accessor to the protobuf message ProgramDesc, which is created in C++ space, because the InferShape function is in C++, which manipulates VarDesc messages, which are in turn members of BlockDesc, which is a member of ProgramDesc.
    
+
    Program creates the first block as the global block in its constructor.  All parameters and their initializer operators are in the global block.
    
+
    
+
    
+
    Block
    
+
    A Block includes
    
+
      
+
    1. a map from variable names to an instance of the Python Variable class, and
      2. 
+
    2. a list of Operator instances.
      3. 
+
    
+
    class Block(objects):
+    def __init__(self, program, parent_idx):
+        self.desc = core.NewBlock(program.desc)
+        self.program = program
+        self.vars = map<string, Variable>()
+        self.ops = vector<Operator>()
+        self.parent_idx = parent_idx
+
+    def create_var(self, ...):
+        return Variable(self, ...)
+
+    def _create_global_var(self, ...):
+        program.global_block().create_var(...)
+
+    def create_parameter(self, name, ...):
+        # Parameter is a subclass of variable. See Parameter section for details.
+        self.vars[name] = Parameter(self._create_global_var(...), ...)
+        return self.vars[name]
+
+    def append_operator(self, ...):
+        self.ops.append(Operator(self, ...))
+
+    def prepend_operator(self, ...): # Parameter's ctor prepands initialize operators.
+       self.ops.prepend(Operator(self, ...))
+
    
+
    
+
    create_parameter is necessary because parameters are global variables, defined in the global block, but can be created in some sub-blocks. For example, an FC layer in the step block of an RNN operator.
    
+
    prepend_operator is necessary because the constructor of Parameter needs to create the initialize (or load) operator of the parameter, and would like to put it in the preamble of the global block.
    
+
    
+
    
+
    Operator
    
+
    The Operator class fills in the OpDesc message and calls the C++ function InferShape to infer the output shapes from the input shapes.
    
+
    class Operator(object):
+    def __init__(self,
+                 block,  # Block
+                 type,   # string
+                 inputs, # dict<string, Variable>
+                 outputs,# dict<stirng, Variable>
+                 attrs   # dict<string, Any>
+                 ):
+        self.desc = core.NewOpDesc(block.desc, type, inputs, outputs, attrs)
+        core.infer_shape(self.desc, inputs, outputs)
+
+    def type(self):
+        return self.desc.type()
+
    
+
    
+
    Operator creates the OpDesc message in C++ space, so that it can call the InferShape function, which is in C++.
    
+
    
+
    
+
    Variable
    
+
    Operators take Variables as its inputs and outputs.
    
+
    class Variable(object):
+    def __init__(self,
+                 block=None,      # Block
+                 name=None,       # string
+                 shape,           # tuple
+                 dtype="float32", # string
+                 lod_level=None   # int
+                 ):
+        if name is None:
+            name = unique_name_generator()
+        self.name = name
+        self.block = block
+        self.desc = core.NewVarDesc(block.desc, name, shape, lod_level)
+        self.writer = None
+
    
+
    
+
    Please be aware of self.writer, that tracks operator who creates the variable.  It possible that there are more than one operators who write a variable, but in Python space, each write to a variable is represented by a Variable class.  This is guaranteed by the fact that core.NewVarDesc must NOT create a new VarDesc message if its name already exists in the specified block.
    
+
    
+
    
+
    Parameter
    
+
    A parameter is a global variable with an initializer (or load) operator.
    
+
    class Parameter(Variable):
+    def __init__(self,
+                 block=None,      # Block
+                 name=None,       # string
+                 shape,           # tuple
+                 dtype="float32", # string
+                 lod_level=None   # int
+                 trainable,       # bool
+                 initialize_op_attrs,
+                 optimize_op_attrs):
+        super(Parameter, self).__init__(block, name, shape, dtype, lod_level)
+        self.trainable = trainable
+        self.optimize_op_attrs = optimize_op_attrs
+        block.prepend(Operator(block,  # Block
+                               initialize_op_attrs['type'],   # string
+                               None,   # no inputs
+                               self,   # output is the parameter
+                               initialize_op_attrs)
+
    
+
    
+
    When users create a parameter, they can call
    
+
    program.create_parameter(
+  ...,
+  init_attr={
+    type: "uniform_random",
+    min: -1.0,
+    max: 1.0,
+  })
+)
+
    
+
    
+
    In above example, init_attr.type names an initialize operator.  It can also name the load operator
    
+
    init_attr={
+ type: "load",
+ filename: "something.numpy",
+}
+
    
+
    
+
    optimize_op_attrs is not in the VarDesc message, but kept in the Python instance, as it will be used in the Python space when creating the optimize operator’s OpDesc, and will be in the OpDesc message.
    
+
    
+
    
+
    
+
    Layer Function
    
+
    A layer is a Python function that creates some operators and variables. Layers simplify the work of application programmers.
    
+
    Layer functions take Variable and configuration parameters as its input and return the output variable(s).
    
+
    For example, FullyConnected take one or more variable as its input. The input could be input data or another layer’s output. There are many configuration options for a FullyConnected layer, such as layer size, activation, parameter names, initialization strategies of parameters, and so on. The FullyConnected layer will return an output variable.
    
+
    
+
    Necessity for reusing code between layer functions
    
+
    There are a lot of code that can be reused. Such as
    
+
      
+
    • Give the default value of configuration. e.g., default initialize strategy for parameters is uniform random with min = -1.0, max = 1.0. and default initialize strategy for bias is to fill zero.
      • 
+
    • Append the activation operator.
      • 
+
    • Create a temporary variable.
      • 
+
    • Create parameter.
      • 
+
    • Generate a unique name.
      • 
+
    • Add a bias.
      • 
+
    • ...
      • 
+
    
+
    A mechanism to reuse code between layer functions is necessary. It will be around 150 lines of code if we write a FullyConnected layer without any helper functions.
    
+
    
+
    
+
    Comparision between global functions and helper class
    
+
    The FullyConnected layer will be as follow when we provide global functions:
    
+
    def fc_layer(input, size, param_attr=None, bias_attr=None, act=None, name=None):
+  if name is None:
+    name = unique_name("fc")
+  input = multiple_input(input)
+  param_attr = default_param_attr(param_attr)
+  param_attr = multiple_param_attr(param_attr, len(input))
+
+  # mul
+  mul_results = []
+  for ipt, attr in zip(input, param_attr):
+    shape = ipt.shape[1:] + [size]
+    w = g_program.global_block().create_parameter(shape, ipt.dtype, name, attr)
+    tmp = create_tmp_var(name)
+    g_program.current_block().append_op("mul", {ipt, w}, {tmp})
+  mul_results.append(tmp)
+
+  # add sum
+  ...
+  # add bias
+  ...
+  # add activation
+  ...
+  return out
+
    
+
    
+
    We can provide many helpers functions for layer developers. However, there are several disadvantages for global helper functions:
    
+
      
+
    1. We need a namespace for these methods, then layer developers can quickly figure out what method they can use.
      2. 
+
    2. Global functions will force layer developers to pass its parameter time by time.
      3. 
+
    
+
    So we provide a helper class, LayerHelper, to share code between layer functions. The FullyConnected Layer will be as follow.
    
+
    def fc_layer(input, size, param_attr=None, bias_attr=None, act=None, name=None):
+  helper = LayerHelper(locals())  # pass all parameter to LayerHelper
+
+  mul_results = []
+  for ipt, param in helper.iter_multiple_input_and_param():
+    w = helper.create_parameter(shape=ipt.shape[1:] + [size], dtype = ipt.dtype)
+    tmp = helper.create_tmp_variable()
+    helper.append_op('mul', {ipt, w}, {tmp})
+    mul_results.append(tmp)
+
+  pre_bias = helper.add_sum(mul_results)
+  pre_activation = helper.add_bias(pre_bias)
+  return helper.add_activation(pre_activation)
+
    
+
    
+
    We not only use the fewer lines of code to write fc_layer but also make the code clearer to understand. At the same time, layer developers can figure out what function they can invoke by typing helper. in a python editor.
    
+
    
+
    
+
    Implementation of layer helper
    
+
    We just keep all parameters of a layer function as a dictionary in layer helper as a private data member. Every method of layer helper will look up the dictionary after it is invoked. In that way, we can implement a layer helper for all layer functions even some layer does not contain some operator. For example, The activation is used by the FullyConnected layer or convolution layers, but a cross-entropy layer does not use it. The example code of add_activation are:
    
+
    class LayerHelper(object):
+  def __init__(self, **kwargs):  # kwargs is short for `keyword arguments`
+    self.kwargs = kwargs
+
+  def add_activation(self, input_var):
+    act = self.kwargs.get("act", None)  # default value is None
+    if act is None:  # do nothing if no act
+      return input_var
+
+    tmp = self.create_tmp_var(self)
+    self.append_op(type=act, input=input_var, output=tmp)
+    return tmp
+
    
+
    
+
    
+
    
+
    
+
    Optimizer
    
+
    Optimizer Design Doc
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/reader/README.html b/doc/design/reader/README.html
index 4de366bd53ddbb2d646443eb4747b219070d324e..161e05061eaf5486da3114fd79942818015ffe08 100644
--- a/doc/design/reader/README.html
+++ b/doc/design/reader/README.html
@@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -84,7 +84,7 @@
  • GET STARTED
  • HOW TO
  • API
    • -
  • ABOUT
    • +
  • MOBILE
    • @@ -108,9 +108,10 @@ -
  • Run Distributed Training
    • +
  • PaddlePaddle Distributed Training
  • Paddle On Kubernetes
  • Distributed PaddlePaddle Training on AWS with Kubernetes
  • Write New Layers
  • Contribute Code
    • -
  • RNN Models
    • +
  • Contribute Documentation
    • +
  • RNN Models +
  • Tune GPU Performance
    • @@ -135,17 +140,40 @@
  • Model Configuration
    • -
  • Data Reader Interface and DataSets
    • +
  • Data Reader Interface and DataSets +
  • Training and Inference
    • +
  • Fluid +
    • + + +
  • MOBILE
    • -
  • ABOUT
    • @@ -177,22 +205,22 @@

    Python Data Reader Design Doc

    -

    At training and testing time, PaddlePaddle programs need to read data. To ease the users’ work to write data reading code, we define that

    +

    During the training and testing phases, PaddlePaddle programs need to read data. To help the users write code that performs reading input data, we define the following:

      -
    • A reader is a function that reads data (from file, network, random number generator, etc) and yields data items.
      • -
    • A reader creator is a function that returns a reader function.
      • -
    • A reader decorator is a function, which accepts one or more readers, and returns a reader.
      • -
    • A batch reader is a function that reads data (from reader, file, network, random number generator, etc) and yields a batch of data items.
      • +
    • A reader: A function that reads data (from file, network, random number generator, etc) and yields the data items.
      • +
    • A reader creator: A function that returns a reader function.
      • +
    • A reader decorator: A function, which takes in one or more readers, and returns a reader.
      • +
    • A batch reader: A function that reads data (from reader, file, network, random number generator, etc) and yields a batch of data items.
    -

    and provide function which converts reader to batch reader, frequently used reader creators and reader decorators.

    +

    and also provide a function which can convert a reader to a batch reader, frequently used reader creators and reader decorators.

    Data Reader Interface

    -

    Indeed, data reader doesn’t have to be a function that reads and yields data items. It can be any function with no parameter that creates a iterable (anything can be used in for x in iterable):

    +

    Data reader doesn’t have to be a function that reads and yields data items. It can just be any function without any parameters that creates an iterable (anything can be used in for x in iterable) as follows:

    iterable = data_reader()
    -

    Element produced from the iterable should be a single entry of data, not a mini batch. That entry of data could be a single item, or a tuple of items. Item should be of supported type (e.g., numpy 1d array of float32, int, list of int)

    -

    An example implementation for single item data reader creator:

    +

    The item produced from the iterable should be a single entry of data and not a mini batch. The entry of data could be a single item or a tuple of items. Item should be of one of the supported types (e.g., numpy 1d array of float32, int, list of int etc.)

    +

    An example implementation for single item data reader creator is as follows:

    def reader_creator_random_image(width, height):
     def reader():
         while True:
@@ -200,7 +228,7 @@
     return reader
    -

    An example implementation for multiple item data reader creator:

    +

    An example implementation for multiple item data reader creator is as follows:

    def reader_creator_random_image_and_label(width, height, label):
     def reader():
         while True:
@@ -211,8 +239,8 @@

    Batch Reader Interface

    -

    batch reader can be any function with no parameter that creates a iterable (anything can be used in for x in iterable). The output of the iterable should be a batch (list) of data items. Each item inside the list must be a tuple.

    -

    Here are valid outputs:

    +

    Batch reader can be any function without any parameters that creates an iterable (anything can be used in for x in iterable). The output of the iterable should be a batch (list) of data items. Each item inside the list should be a tuple.

    +

    Here are some valid outputs:

    # a mini batch of three data items. Each data item consist three columns of data, each of which is 1.
 [(1, 1, 1),
 (2, 2, 2),
@@ -221,24 +249,24 @@
 # a mini batch of three data items, each data item is a list (single column).
 [([1,1,1],),
 ([2,2,2],),
-([3,3,3],),
+([3,3,3],)]

    Please note that each item inside the list must be a tuple, below is an invalid output:

     # wrong, [1,1,1] needs to be inside a tuple: ([1,1,1],).
- # Otherwise it's ambiguous whether [1,1,1] means a single column of data [1, 1, 1],
- # or three column of datas, each of which is 1.
+ # Otherwise it is ambiguous whether [1,1,1] means a single column of data [1, 1, 1],
+ # or three columns of data, each of which is 1.
 [[1,1,1],
 [2,2,2],
 [3,3,3]]
    -

    It’s easy to convert from reader to batch reader:

    +

    It is easy to convert from a reader to a batch reader:

    mnist_train = paddle.dataset.mnist.train()
 mnist_train_batch_reader = paddle.batch(mnist_train, 128)
    -

    Also easy to create custom batch reader:

    +

    It is also straight forward to create a custom batch reader:

    def custom_batch_reader():
     while True:
         batch = []
@@ -252,7 +280,8 @@

    Usage

    -

    batch reader, mapping from item(s) read to data layer, batch size and number of total pass will be passed into paddle.train:

    +

    Following is how we can use the reader with PaddlePaddle: +The batch reader, a mapping from item(s) to data layer, the batch size and the number of total passes will be passed into paddle.train as follows:

    # two data layer is created:
 image_layer = paddle.layer.data("image", ...)
 label_layer = paddle.layer.data("label", ...)
@@ -265,11 +294,11 @@

    Data Reader Decorator

    -

    Data reader decorator takes a single or multiple data reader, returns a new data reader. It is similar to a python decorator, but it does not use @ syntax.

    -

    Since we have a strict interface for data readers (no parameter, return a single data item). Data reader can be used flexiable via data reader decorators. Following are a few examples:

    +

    The Data reader decorator takes in a single reader or multiple data readers and returns a new data reader. It is similar to a python decorator, but it does not use @ in the syntax.

    +

    Since we have a strict interface for data readers (no parameters and return a single data item), a data reader can be used in a flexible way using data reader decorators. Following are a few examples:

    Prefetch Data

    -

    Since reading data may take time and training can not proceed without data. It is generally a good idea to prefetch data.

    +

    Since reading data may take some time and training can not proceed without data, it is generally a good idea to prefetch the data.

    Use paddle.reader.buffered to prefetch data:

    buffered_reader = paddle.reader.buffered(paddle.dataset.mnist.train(), 100)
    @@ -278,8 +307,8 @@

    Compose Multiple Data Readers

    -

    For example, we want to use a source of real images (reusing mnist dataset), and a source of random images as input for Generative Adversarial Networks.

    -

    We can do:

    +

    For example, if we want to use a source of real images (say reusing mnist dataset), and a source of random images as input for Generative Adversarial Networks.

    +

    We can do the following :

    def reader_creator_random_image(width, height):
     def reader():
         while True:
@@ -297,14 +326,14 @@
 
 reader = paddle.reader.compose(paddle.dataset.mnist.train(), data_reader_creator_random_image(20, 20), true_reader, false_reader)
 # Skipped 1 because paddle.dataset.mnist.train() produces two items per data entry.
-# And we don't care second item at this time.
+# And we don't care about the second item at this time.
 paddle.train(paddle.batch(reader, 128), {"true_image":0, "fake_image": 2, "true_label": 3, "false_label": 4}, ...)

    Shuffle

    -

    Given shuffle buffer size n, paddle.reader.shuffle will return a data reader that buffers n data entries and shuffle them before a data entry is read.

    +

    Given the shuffle buffer size n, paddle.reader.shuffle returns a data reader that buffers n data entries and shuffles them before a data entry is read.

    Example:

    reader = paddle.reader.shuffle(paddle.dataset.mnist.train(), 512)
    @@ -313,21 +342,21 @@

    Q & A

    -
    -

    Why reader return only a single entry, but not a mini batch?

    -

    Always returning a single entry make reusing existing data readers much easier (e.g., if existing reader return not a single entry but 3 entries, training code will be more complex because it need to handle cases like batch size 2).

    -

    We provide function paddle.batch to turn (single entry) reader into batch reader.

    +
    +

    Why does a reader return only a single entry, and not a mini batch?

    +

    Returning a single entry makes reusing existing data readers much easier (for example, if an existing reader returns 3 entries instead if a single entry, the training code will be more complicated because it need to handle cases like a batch size 2).

    +

    We provide a function: paddle.batch to turn (a single entry) reader into a batch reader.

    -
    -

    Why do we need batch reader, isn’t train take reader and batch_size as arguments sufficient?

    -

    In most of the case, train taking reader and batch_size as arguments would be sufficent. However sometimes user want to customize order of data entries inside a mini batch. Or even change batch size dynamically.

    +
    +

    Why do we need a batch reader, isn’t is sufficient to give the reader and batch_size as arguments during training ?

    +

    In most of the cases, it would be sufficient to give the reader and batch_size as arguments to the train method. However sometimes the user wants to customize the order of data entries inside a mini batch, or even change the batch size dynamically. For these cases using a batch reader is very efficient and helpful.

    -
    -

    Why use a dictionary but not a list to provide mapping?

    -

    We decided to use dictionary ({"image":0, "label":1}) instead of list (["image", "label"]) is because that user can easily resue item (e.g., using {"image_a":0, "image_b":0, "label":1}) or skip item (e.g., using {"image_a":0, "label":2}).

    +
    +

    Why use a dictionary instead of a list to provide mapping?

    +

    Using a dictionary ({"image":0, "label":1}) instead of a list (["image", "label"]) gives the advantage that the user can easily reuse the items (e.g., using {"image_a":0, "image_b":0, "label":1}) or even skip an item (e.g., using {"image_a":0, "label":2}).

    -
    -

    How to create custom data reader creator

    +
    +

    How to create a custom data reader creator ?

    def image_reader_creator(image_path, label_path, n):
     def reader():
         f = open(image_path)
@@ -350,7 +379,7 @@

    How is paddle.train implemented

    -

    An example implementation of paddle.train could be:

    +

    An example implementation of paddle.train is:

    def train(batch_reader, mapping, batch_size, total_pass):
     for pass_idx in range(total_pass):
         for mini_batch in batch_reader(): # this loop will never end in online learning.
diff --git a/doc/design/refactor/distributed_architecture.html b/doc/design/refactor/distributed_architecture.html
new file mode 100644
index 0000000000000000000000000000000000000000..1d66a091cce89a2b7eefcdacbf9400068a8c0c12
--- /dev/null
+++ b/doc/design/refactor/distributed_architecture.html
@@ -0,0 +1,379 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Distributed Training Architecture — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Distributed Training Architecture
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Distributed Training Architecture
    
+
    
+
    Abstract
    
+
    PaddlePaddle version 0.10.0 uses the “trainer-parameter server” architecture. We run multiple instances of trainers (where each trainer runs the same model) and parameter servers for distributed training. This architecture serves well, but has few limitations:
    
+
      
+
    1. There is a need to write special code that handles tasks which should only be run on a single trainer. E.g., initializing the model, saving the model etc.
      2. 
+
    2. Model parallelism is hard: It would need all the if-else branches conditioned on the trainer ID to partition the model onto the trainers, and eventually manually writing out the inter-model-shard communication code to communicate between different trainers.
      3. 
+
    3. The user can not directly specify the parameter update rule: This would need to modify the parameter server code and compile a new binary. This makes things more complicated for researchers: A lot of extra effort is required to make this work. Besides, the training job submission program may not allow running arbitrary binaries.
      4. 
+
    
+
    This design doc discusses PaddlePaddle’s new distributed training architecture that addresses the above mentioned limitations.
    
+
    
+
    
+
    Analysis
    
+
    The assumption is that the user writes the trainer program in either Python or C++.
    
+
    
+
    Limitation 1
    
+
    There are two basic functionalities in the trainer program:
    
+
      
+
    1. The training logic such as loading / saving the model and printing out the logs.
      2. 
+
    2. The neural network definition such as the definition of the data layer, the fully connected layer, the cost function and the
+optimizer.
      3. 
+
    
+
    When we train using PaddlePaddle v0.10.0 in a distributed fashion, multiple instances of the same Python code are run on different nodes, hence both: the
+training logic as well as the neural network computation logic, is replicated.
    
+
    The tasks that only need to be run once belong to the training logic. Hence if we only replicate the neural network computation part, and do not
+replicate the training logic, the limitation mentioned above can be avoided.
    
+
    
+
    
+
    Limitation 2
    
+
    Model parallelism means that a single model is partitioned into different components and each node runs one of the component separately. This comes at the extra cost of managing the
+inter-model-shard communication between nodes.
    
+
    PaddlePaddle should ideally be able to modify the neural network computation and figure out the support for model parallelism automatically. However, the
+computation is only specified in Python code which sits outside of PaddlePaddle, hence PaddlePaddle can not support the feature in this setup.
    
+
    Similar to how a compiler uses an intermediate representation (IR) so that the programmer does not need to manually optimize their code for most of the cases, we can have an intermediate representation in PaddlePaddle as well. The compiler optimizes the IR as follows:
    
+
    
+
    PaddlePaddle can support model parallelism by converting the IR so that the user no longer needs to manually perform the computation and operations in the Python component:
    
+
    
+
    The IR for PaddlePaddle after refactoring is called a Block, it specifies the computation dependency graph and the variables used in the computation.
    
+
    
+
    
+
    Limitation 3
    
+
    The user can not directly specify the parameter update rule for the parameter server in the Python module, since the parameter server does not use the same computation definition as the trainer. Instead, the update rule is baked inside the parameter server. The user can not specify the update rule explicitly.
    
+
    This could be fixed by making the parameter server run the same computation definition as the trainer (the user’s Python module). For a detailed explanation, refer to this document -
+Design Doc: Operation Graph Based Parameter Server
    
+
    
+
    
+
    
+
    Distributed Training Architecture
    
+
    The revamped distributed training architecture can address the above discussed limitations. Below is the illustration of how it does so:
    
+
    
+
    The major components in the architecture are: PaddlePaddle Python, PaddlePaddle converter and PaddlePaddle runtime.
    
+
    
+
    PaddlePaddle Python
    
+
    PaddlePaddle Python is the Python library that user’s Python code invokes, to read the data. build the neural network topology, start training, etc.
    
+
    paddle.init()
+input = paddle.op.recordIO("/home/data/mnist.recordio") # file stored on the cluster
+img, label = input[0], input[1]
+hidden = paddle.layer.fc(input=img, size=200, act=paddle.activation.Tanh())
+prediction = paddle.layer.fc(input=img, size=10, act=paddle.activation.Softmax())
+cost = paddle.layer.classification_cost(input=prediction, label=label)
+optimizer = paddle.optimizer.SGD(cost, learning_rate=0.01)
+session = paddle.session.NewRemote(num_trainer=3, num_ps=2, GPU_per_trainer=1)
+for i in range(1000):
+    _, cost_val = session.eval(targets=[cost, optimizer])
+    print cost_val
+
    
+
    
+
    The above code is what a typical Python trainer code is, the neural network topology is built using the helper functions such as paddle.layer.fc. Training is done by calling session.eval iteratively.
    
+
    
+
    session.eval
    
+
    As shown in the graph, session.eval sends the IR and the evaluation inputs or targets to the PaddlePaddle cluster for evaluation.
+The targets can be any variable in the computation graph. When the target is say, the optimizer variable, the neural network will be optimized once. When the target is the cost variable, session.eval returns the cost value. Based on what the target is, an appropriate action is taken.
    
+
    The Python session is a wrapper of the C++ Session class. For more information about Session, refer to this document - Design Doc: Session.
    
+
    
+
    
+
    
+
    PaddlePaddle Converter
    
+
    The PaddlePaddle converter automatically converts the IR in the request (IR and evaluation inputs/targets) from PaddlePaddle Python to partitioned IRs and dispatches the new IRs and evaluation inputs/targets to different PaddlePaddle runtimes. Below are the steps that are followed :
    
+
      
+
    1. Add a feed OP that feeds the eval inputs, and a fetch OP that fetches the eval targets to the IR.
      2. 
+
    2. Extract a new computation (sub)graph with the feed and fetch OPs as the boundary. The runtime does not need to run the OP that is not dependent on the fetch OP.
      3. 
+
    3. Optimize the computation graph.
      4. 
+
    4. Place the OPs in the graph onto different devices on different PaddlePaddle runtime according to a placement algorithm and the device constraints specified by the user.
      5. 
+
    5. Partition the graph according to runtime boundaries and add send / recv OP pair on the runtime boundaries.
      6. 
+
    6. Dispatch the partitioned graph to different PaddlePaddle runtimes.
      7. 
+
    7. PaddlePaddle runtimes with the fetch OP reports evaluation results back to the converter, the converter reports the evaluation results back to the PaddlePaddle Python.
      8. 
+
    
+
    The output IRs will be cached to optimize the conversion latency.
    
+
    
+
    Placement Algorithm
    
+
    Our first implementation will only support “trainer-parameter server” placement: the parameters, initializers, and optimizers are all placed on the PaddlePaddle runtimes with the parameter server role. Everything else will be placed on the PaddlePaddle runtimes with the trainer role. This has the same functionality as the “trainer-parameter server” architecture of PaddlePaddle v0.10.0, but is more generic and flexible.
    
+
    In the future, a more general placement algorithm should be implemented, which makes placements according to the input IR, and a model of device computation time and device communication time. Model parallelism requires the generic placement algorithm.
    
+
    
+
    
+
    
+
    PaddlePaddle Runtime
    
+
    The PaddlePaddle runtime owns multiple devices (e.g., CPUs, GPUs) and runs the IR. The runtime does not need to do OP placement since it is already done by the converter.
    
+
    
+
    
+
    Local Training Architecture
    
+
    The local training architecture will be the same as the distributed training architecture, the difference is that everything runs locally, and there is just one PaddlePaddle runtime:
    
+
    
+
    
+
    
+
    Training Data
    
+
    In PaddlePaddle v0.10.0, training data is typically read with a data reader from Python. This approach is no longer efficient when training in a distributed fashion since the Python process no longer runs on the same node with the trainer processes. The Python reader will need to read from the distributed filesystem (assuming it has the required access) and send to the trainers, doubling the network traffic.
    
+
    When doing distributed training, the user can still use Python data reader: the training data are sent with session.eval. However this should be used for debugging purpose only. The users are encouraged to use the read data OPs.
    
+
    
+
    
+
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/refactor/parameter_server.html b/doc/design/refactor/parameter_server.html
new file mode 100644
index 0000000000000000000000000000000000000000..4b2f85149eb91787d5a8fdf205a7a2291456bbc2
--- /dev/null
+++ b/doc/design/refactor/parameter_server.html
@@ -0,0 +1,369 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Operation Graph Based Parameter Server — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Operation Graph Based Parameter Server
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Operation Graph Based Parameter Server
    
+
    
+
    Abstract
    
+
    We propose an approach to implement the parameter server. In this
+approach, there is no fundamental difference between the trainer and
+the parameter server: they both run subgraphs, but subgraphs of
+different purposes.
    
+
    
+
    
+
    Background
    
+
    The previous implementations of the parameter server does not run a
+subgraph. parameter initialization, optimizer computation, network
+communication and checkpointing are implemented twice on both the
+trainer and the parameter server.
    
+
    It would be great if we can write code once and use them on both the
+trainer and the parameter server: reduces code duplication and
+improves extensibility. Given that after the current refactor, we are
+representing everything as a computing graph on the
+trainer. Representing everything as a computing graph on the parameter
+server becomes a natural extension.
    
+
    
+
    
+
    Design
    
+
    
+
    Graph Converter
    
+
    The graph converter converts the user-defined operation (OP) graph
+into subgraphs to be scheduled on different nodes with the following
+steps:
    
+
      
+
    1. OP placement: the OPs will be placed on different nodes according
+to heuristic that minimizes estimated total computation
+time. Currently we will use a simple heuristic that puts parameter
+varable on parameter server workers and everything else on trainer
+workers.
      2. 
+
    2. Add communication OPs to enable the communication between nodes.
      3. 
+
    
+
    We will need these OPs: Send, Recv, Enqueue, Dequeue.
    
+
    Below is an example of converting the user defined graph to the
+subgraphs for the trainer and the parameter server:
    
+
    
+
    After converting:
    
+
    
+
      
+
    1. The parameter variable W and it’s optimizer subgraph are placed on the parameter server.
      2. 
+
    2. Operators are added to the subgraphs.
        
+
      • Send sends data to the connected Recv operator.  The
+scheduler on the receive node will only schedule Recv operator
+to run when the Send operator has ran (the Send OP will mark
+the Recv OP runnable automatically).
        • 
+
      • Enueue enqueues the input variable, it can block until space
+become available in the queue.
        • 
+
      • Dequeue outputs configurable numbers of tensors from the
+queue. It will block until the queue have the required number of
+tensors.
        • 
+
      
+
      3. 
+
    
+
    
+
    
+
    Benefits
    
+
      
+
    • Model parallelism become easier to implement: it’s an extension to
+the trainer - parameter server approach. we already have the
+communication OPs, but need to extend the graph converter’s
+placement functionality.
      • 
+
    • User-defined optimizer is easier to add - user can now express it as
+a subgraph.
      • 
+
    • No more duplication logic inside the trainer and the parameter
+server mentioned in the background section.
      • 
+
    
+
    
+
    
+
    Challenges
    
+
      
+
    • It might be hard for the graph converter to cut a general graph
+(without any hint for which subgraph is the optimizer). We may need
+to label which subgraph inside the OP graph is the optimizer.
      • 
+
    • It’s important to balance the parameter shards of on multiple
+parameter server. If a single parameter is very big (some
+word-embedding, fully connected, softmax layer), we need to
+automatically partition the single parameter onto different
+parameter servers when possible (only element-wise optimizer depends
+on the parameter variable).
      • 
+
    
+
    
+
    
+
    Discussion
    
+
      
+
    • In the “Aync SGD” figure, the “W” variable on the parameter server
+could be read and wrote concurrently, what is our locking strategy?
+E.g., each variable have a lock cpp method to be invoked by every
+OP, or, have a lock OP.
      • 
+
    • Can the Enqueue OP be implemented under our current tensor design
+(puts the input tensor into the queue tensor)?
      • 
+
    • Dequeue OP will have variable numbers of output (depends on the
+min_count attribute), does our current design support it? (similar
+question for the Add OP)
      • 
+
    
+
    
+
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/refactor/session.html b/doc/design/refactor/session.html
new file mode 100644
index 0000000000000000000000000000000000000000..73bb238dc5cde4900e31d497d4e19f0150967858
--- /dev/null
+++ b/doc/design/refactor/session.html
@@ -0,0 +1,427 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Session — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Session
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Session
    
+
    
+
    Abstract
    
+
    The session object encapsulates the environment in which the
+computation graph is executed.
    
+
    We will have the local session and remote session, they offer the
+same interface. The local session encapsulates the local
+runtime environment and the remote session encapsulates the cluster
+runtime environment.
    
+
    The local runtime environment contains:
    
+
      
+
    1. computation devices (i.e., CPU, GPU) handles, and
      2. 
+
    2. the scope which holds all variables.
      3. 
+
    
+
    The remote runtime environment contains:
    
+
      
+
    1. computation devices (i.e., CPU and GPU on node 0, 1) in a cluster,
+and
      2. 
+
    2. the distributed scope in a cluster which holds all
+variables.
      3. 
+
    
+
    The user can create a remote session on Paddle Cloud and evaluate the
+computation graph with it. In this way, the user can control the
+remote computation resource in a cluster from his local computer.
    
+
    
+
    
+
    Background
    
+
    The current design has an implicit global session in which
+paddle.eval() is executed. The pain point is:
    
+
    Since the user is not able to explicitly switch between runtime
+environments, the user cannot run a topology in two independent
+environments.
    
+
    For example, in reinforcement learning, the user may want to have a
+stale model for inference and a fresh model for training, and only
+replace the stale model with the fresh model periodically.
    
+
    Furthermore, we have no concept that encapsulates a remote environment
+that executes a computation graph.
    
+
    We need the session object to address above issues.
    
+
    
+
    
+
    Session
    
+
    A session is an object that owns the runtime environment. All
+computations are executed through session.eval().
    
+
    
+
    Interface
    
+
    eval(
+    targets,
+    feed_dict=None,
+)
+
    
+
    
+
    Evaluates the target Operations or Variables in targets.
    
+
      
+
    • targets: the evaluation targets. Can be a single Operation or
+Variable, or a list with the Operations or Variables as
+elements. The value returned by eval() has the same shape as the
+target argument.
      
+
      The PaddlePaddle program is represented by
+the ProgramDesc, eval() will infer the
+ProgramDesc from the given targets and run the PaddlePaddle
+program. Please
+see
+this graph for
+the detailed illustration for the local session
+and
+this graph for
+the detailed illustration for the remote session.
      
+
      • 
+
    • feed_dict: a dictionary that contains the tensors which override
+the edges of the computation graph.
      
+
      feed_dict not only can provide the input data, it can override any
+OP’s input as well:
      
+
      a = pd.constant(2.0, name="a")
+b = pd.variable(name="b")
+c = pd.mul(a,b)
+sess.eval(targets=c, feed_dict={"b":3.0}) # returns 6.0
+
      
+
      
+
      • 
+
    
+
    close()
+
    
+
    
+
    Closes the session and releases the scope that the session owns.
    
+
    
+
    
+
    Create a Local Session
    
+
    session(
+    devices=None
+)
+
    
+
    
+
    Creates a new session. One session owns one global scope, so creating
+multiple sessions will create different scopes.
    
+
      
+
    • devices: a single string or a list of string of device names,
+the corresponding devices will be the computation devices for
+eval(). If not specified, all available devices (e.g., all GPUs)
+will be used. The user doesn’t need to specify the CPU device since
+it will be always used. Multiple sessions can use the same device.
      • 
+
    
+
    
+
    Example
    
+
    a = paddle.constant(1.0)
+b = paddle.constant(2.0)
+c = a + b
+sess = paddle.session(devices=["gpu:0", "gpu:1", "fpga:0"])
+sess.eval(c)
+sess.close()
+
    
+
    
+
    
+
    
+
    
+
    Create a Remote Session
    
+
    create_cloud_job(
+    name,
+    num_trainer,
+    mem_per_trainer,
+    gpu_per_trainer,
+    cpu_per_trainer,
+    num_ps,
+    mem_per_ps,
+    cpu_per_ps,
+)
+
    
+
    
+
    Creates a Paddle Cloud job. Fails if the job name exists.
    
+
    get_cloud_job(
+    name
+)
+
    
+
    
+
    Gets a Paddle Cloud job.
    
+
    remote_session(
+    job
+)
+
    
+
    
+
      
+
    • job: the Paddle Cloud job.
      • 
+
    
+
    
+
    Example
    
+
    reader = paddle.reader.recordio("/pfs/home/peter/mnist-train-*") # data stored on Paddle Cloud
+image = reader.column(0)
+label = reader.column(1)
+fc1 = paddle.op.fc(image, size=256, act="sigmoid")
+fc2 = paddle.op.fc(fc1, size=10, act="softmax")
+cost = paddle.op.cross_entropy(fc2, label)
+opt = paddle.optimizer.sgd(cost)
+
+job = paddle.create_cloud_job("test", 3, "1G", 1, 1, 2, "1G", 1)
+sess = paddle.remote_ession(job)
+for i in range(1000):
+    sess.eval(opt)
+sess.close()
+
    
+
    
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/refactorization.html b/doc/design/refactorization.html
new file mode 100644
index 0000000000000000000000000000000000000000..a29f56eccefa92b7c0a61fd0e95ab50debaacac8
--- /dev/null
+++ b/doc/design/refactorization.html
@@ -0,0 +1,594 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Refactorization Overview — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Refactorization Overview
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Refactorization Overview
    
+
    The goals of refactoring include:
    
+
      
+
    1. Making it easy for external contributors to write new elementary computation operations.
      2. 
+
    2. Making the codebase clean and readable.
      3. 
+
    3. Designing a new computation representation – a computation graph of operators and variables.
      4. 
+
    4. Implementing auto-scalability and auto fault recoverable distributed computing with the help of computation graphs.
      5. 
+
    
+
    
+
    Computation Graphs
    
+
      
+
    1. PaddlePaddle represents the computation, training and inference of Deep Learning models, by computation graphs.
      2. 
+
    2. Please refer to computation graphs for a concrete example.
      3. 
+
    3. Users write Python programs to describe the graphs and run them (locally or remotely).
      4. 
+
    4. A graph is composed of variables and operators.
      5. 
+
    5. The description of graphs must be serializable/deserializable, so that:
        
+
      1. It can be sent to the cloud for distributed execution, and
        2. 
+
      2. It can be sent to clients for mobile or enterprise deployment.
        3. 
+
      
+
      6. 
+
    6. The Python program does two things
        
+
      1. Compilation runs a Python program to generate a protobuf message representation of the graph and send it to
          
+
        1. the C++ library libpaddle.so for local execution,
          2. 
+
        2. the master process of a distributed training job for training, or
          3. 
+
        3. the server process of a Kubernetes serving job for distributed serving.
          4. 
+
        
+
        2. 
+
      2. Execution executes the graph by constructing instances of class Variable and OperatorBase, according to the protobuf message.
        3. 
+
      
+
      7. 
+
    
+
    
+
    
+
    Description and Realization of Computation Graph
    
+
    At compile time, the Python program generates a protobuf message representation of the graph, or a description of the graph.
    
+
    At runtime, the C++ program realizes the graph and runs it.
    
+
    | | Representation (protobuf messages) | Realization (C++ class objects) |
+|—|—|—|
+|Data|VarDesc|Variable|
+|Operation|OpDesc|Operator|
+|Block|BlockDesc|Block|
    
+
    The word graph is interchangeable with block in this document.  A graph consists of computation steps and local variables similar to a C++/Java program block, or a pair of parentheses({ and }).
    
+
    
+
    
+
    Compilation and Execution
    
+
      
+
    1. Run a Python program to describe the graph.  In particular, the Python application program does the following:
        
+
      1. Create VarDesc to represent local/intermediate variables,
        2. 
+
      2. Create operators and set attributes,
        3. 
+
      3. Validate attribute values,
        4. 
+
      4. Infer the type and the shape of variables,
        5. 
+
      5. Plan memory-reuse for variables,
        6. 
+
      6. Generate the backward graph
        7. 
+
      7. Add optimization operators to the computation graph.
        8. 
+
      8. Optionally, split the graph for distributed training.
        9. 
+
      
+
      2. 
+
    2. The invocation of train or infer methods in the Python program does the following:
        
+
      1. Create a new Scope instance in the scope hierarchy for each run of a block,
          
+
        1. realize local variables defined in the BlockDesc message in the new scope,
          2. 
+
        2. a scope is similar to the stack frame in programming languages,
          3. 
+
        
+
        2. 
+
      2. Create an instance of class Block, in which,
          
+
        1. realize operators in the BlockDesc message,
          2. 
+
        
+
        3. 
+
      3. Run the Block by calling
          
+
        1. Block::Eval(vector<Variable>* targets) for forward and backward computations, or
          2. 
+
        2. Block::Eval(vector<Operator>* targets) for optimization.
          3. 
+
        
+
        4. 
+
      
+
      3. 
+
    
+
    
+
    
+
    Intermediate Representation (IR)
    
+
    Compile Time -> IR -> Runtime
+
    
+
    
+
    
+
    Benefits of IR
    
+
      
+
    • Optimization
      
+
      Compile Time -> IR -> Optimized IR -> Runtime
+
      
+
      
+
      • 
+
    • Automatically send partitioned IR to different nodes.
      
+
        
+
      • Automatic Data Parallelism
        
+
        Compile Time
+|-> Single GPU IR
+    |-> [trainer-IR-0, trainer-IR-1, pserver-IR]
+        |-> Node-0 (runs trainer-IR-0)
+        |-> Node-1 (runs trainer-IR-1)
+        |-> Node-2 (runs pserver-IR)
+
        
+
        
+
        • 
+
      • Automatic Model Parallelism (planned for future)
        
+
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    
+
    
+
    Operator/OpWithKernel/OpKernel
    
+
    class_diagram
    
+
    
+
    
+
    
+
    Operator
    
+
    class_diagram
    
+
      
+
    • Operator is the fundamental building block of the user interface.
        
+
      • Operator stores input/output variable names and attributes.
        • 
+
      • The InferShape interface is used to infer the shape of the output variables based on the shapes of the input variables.
        • 
+
      • Use Run to compute the output variables from the input variables.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    OpWithKernel/Kernel
    
+
    class_diagram
    
+
      
+
    • OpWithKernel inherits Operator.
      • 
+
    • OpWithKernel contains a Kernel map.
        
+
      • OpWithKernel::Run get device’s kernel, and invoke OpKernel::Compute.
        • 
+
      • OpKernelKey is the map key. Only device place now, but may be data type later.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    Why separate Kernel and Operator
    
+
      
+
    • Separate GPU and CPU code.
        
+
      • Make Paddle capable of running without GPU.
        • 
+
      
+
      • 
+
    • Make one operator (which is a user interface) and create many implementations.
        
+
      • For example, same multiplication op can have different implementations kernels such as FP16 kernel, FP32 kernel, MKL, eigen kernel.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    Libraries for Kernel development
    
+
      
+
    • Eigen::Tensor contains basic math and element-wise functions.
        
+
      • Note that Eigen::Tensor has broadcast implementation.
        • 
+
      • Limit the number of tensor.device(dev) = in your code.
        • 
+
      
+
      • 
+
    • thrust::transform and std::transform.
        
+
      • thrust has the same API as C++ standard library. Using transform, one can quickly implement customized element-wise kernels.
        • 
+
      • thrust, in addition, supports more complex APIs, like scan, reduce, reduce_by_key.
        • 
+
      
+
      • 
+
    • Hand-writing GPUKernel and CPU code
        
+
      • Do not write in header (.h) files. CPU Kernel should be in cpp source (.cc) and GPU kernels should be in cuda (.cu) files. (GCC cannot compile GPU code.)
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    Operator Registration
    
+
    
+
    Why is registration necessary?
    
+
    We need a method to build mappings between Op type names and Op classes.
    
+
    
+
    
+
    How is registration implemented?
    
+
    Maintaining a map, whose key is the type name and the value is the corresponding Op constructor.
    
+
    
+
    
+
    
+
    
+
    The Registry Map
    
+
    
+
    OpInfoMap
    
+
    op_type(string) -> OpInfo
    
+
    OpInfo:
    
+
      
+
    • creator: The Op constructor.
      • 
+
    • grad_op_type: The type of the gradient Op.
      • 
+
    • proto: The Op’s Protobuf, including inputs, outputs and required attributes.
      • 
+
    • checker: Used to check attributes.
      • 
+
    
+
    
+
    
+
    
+
+
    
+
    
+
    Registration Process
    
+
      
+
    1. Write an Op class and its gradient Op class, if required.
      2. 
+
    2. Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator.
      3. 
+
    3. Invoke the macro REGISTER_OP. This macro will
        
+
      1. Call maker class to complete proto and checker
        2. 
+
      2. Using the completed proto and checker, it will add a new key-value pair to the OpInfoMap
        3. 
+
      
+
      4. 
+
    
+
    
+
    
+
    
+
    Backward Module (1/2)
    
+
    
+
    Create Backward Operator
    
+
      
+
    • Mapping from forward Op to backward Op
+backward
      • 
+
    
+
    
+
    
+
    
+
    
+
    Backward Module (2/2)
    
+
    
+
    Build Backward Network
    
+
      
+
    • Input: a graph of forward operators
      • 
+
    • Output: a graph of backward operators
      • 
+
    • Corner cases in construction
        
+
      • Shared Variables => insert an Add operator to combine gradients
        • 
+
      • No Gradient => insert a fill_zero_grad operator
        • 
+
      • Recursive NetOp => call Backward recursively
        • 
+
      • RNN Op => recursively call Backward on stepnet
        • 
+
      • RNN Op => recursively call Backward on stepnet
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    
+
    Scope, Variable, Tensor
    
+
      
+
    • Tensor is an n-dimension array with type.
        
+
      • Only dims and data pointers are stored in Tensor.
        • 
+
      • All operations on Tensor are written in Operator or global functions.
        • 
+
      • Variable length Tensor design LoDTensor
        • 
+
      
+
      • 
+
    • Variable instances are the inputs and the outputs of an operator, not just Tensor.
        
+
      • step_scopes in RNN is a variable and not a tensor.
        • 
+
      
+
      • 
+
    • Scope is where variables are stored.
        
+
      • map<string var name, Variable>
        • 
+
      • Scope has a hierarchical structure. The local scope can get variables from its parent scope.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    Block (in design)
    
+
    
+
    the difference between original RNNOp and Block
    
+
      
+
    • As an operator is more intuitive than RNNOp,
      • 
+
    • Offers a new interface Eval(targets) to deduce the minimal block to Run,
      • 
+
    • Fits the compile-time/ runtime separation design paradigm.
        
+
      • During the compilation, SymbolTable stores VarDescs and OpDescs and serialize to a BlockDesc
        • 
+
      • When graph executes, a Block with BlockDesc is passed. It then creates Op and Var instances and then invokes Run.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    
+
    Milestone
    
+
      
+
    • Take Paddle/books as the main line, the requirement of the models motivates framework refactoring,
      • 
+
    • Model migration
        
+
      • Framework development gives priority support to model migration, for example,
          
+
        • the MNIST demo needs a Python interface,
          • 
+
        • the RNN models require the framework to support LoDTensor.
          • 
+
        
+
        • 
+
      • Determine some timelines,
        • 
+
      • Frequently used Ops need to be migrated first,
        • 
+
      • Different models can be migrated in parallel.
        • 
+
      
+
      • 
+
    • Improve the framework at the same time
      • 
+
    • Accept imperfection, concentrate on solving the specific problem at the right price.
      • 
+
    
+
    
+
    
+
    
+
    Control the migration quality
    
+
      
+
    • Compare the performance of migrated models with old ones.
      • 
+
    • Follow the google C++ style guide.
      • 
+
    • Build the automatic workflow of generating Python/C++ documentations.
        
+
      • The documentation of layers and ops should be written inside the code.
        • 
+
      • Take the documentation quality into account when submitting pull requests.
        • 
+
      • Preview the documentations, read and improve them from a user’s perspective.
        • 
+
      
+
      • 
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/register_grad_op.html b/doc/design/register_grad_op.html
new file mode 100644
index 0000000000000000000000000000000000000000..25e9697e8161b8f5b6329549b953dd3c13338d6f
--- /dev/null
+++ b/doc/design/register_grad_op.html
@@ -0,0 +1,339 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Gradient Operators Registration — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Gradient Operators Registration
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Gradient Operators Registration
    
+
    
+
    The Problem Posed
    
+
    Currently, for each C++ operator class definition, a gradient operator creator function is registered, which takes as input a C++ operator instance and returns the corresponding gradient operator instance.
    
+
    However, we noticed two problems with the current design:
    
+
      
+
    1. As we decided to separate the compilation and the execution phases, we need to change the creator to take an OpDesc protobuf message in a ProgramDesc and inserts corresponding OpDesc messages into the ProgramDesc message.
      2. 
+
    2. For some operators, the gradient computation can be written in terms of existing operators.  For example, the gradient of minus operator consists of two operators – an identity operator followed by a scale operator.  Hence the registration mechanism needs to support mapping from an operator to a set of operators for the gradient computation.
      3. 
+
    
+
    
+
    
+
    The Current Implementation
    
+
    Instances of the C++ class OpInfo are stored an associative map whose key is the operator type. The grad_op_type indicates the associated gradient operator type. An operator can create the gradient operator by invoking OpInfo::creator_ of the gradient operator. The pseudo code is as follows
    
+
    struct OpInfo {
+  std::function<OperatorBase*(...)> creator_;
+  std::string grad_op_type_;
+  ...
+};
+
+map<string, OpInfo> OpInfoMap;
+
+OperatorBase* CreateGradientOperator(const OperatorBase& op) {
+  return OpInfoMap.at(op.Type()).creator_(...);
+}
+
    
+
    
+
    
+
    
+
    Proposed Solution
    
+
    The mapping relationship between an operator and its gradient operators is a function. The interface of this function is:
    
+
    // (OpDesc) --> vector<OpDesc>
+std::function<std::vector<OpDescBind>(const OpDescBind&)>;
+
    
+
    
+
    The function takes an OpDescBind of the forward operator and returns one or many gradient operator descriptions. OpDescBind is a C++ wrapper for  the protobuf message OpDesc for rapid manipulation of OpDesc.
    
+
    The GradOpDescMaker will be registered in OpInfo and will replace the grad_op_type_ field. The OpInfo should look like
    
+
    struct OpInfo {
+  std::function<std::vector<std::unique_ptr<OpDescBind>>(const OpDescBind&)>  grad_op_maker_;
+  ...
+};
+
    
+
    
+
    The grad_op_maker_ is a nullptr if the operator does not have any associated gradient operators.
    
+
    We propose a base class called GradOpDescMakerBase to let operator developers generate Gradient Operators easily. The public interface of that class is
    
+
    class GradOpDescMakerBase {
+public:
+  GradOpDescMakerBase(const OpDescBind& );
+  virtual std::vector<std::unique_ptr<OpDescBind>> operator()()const = 0;
+};
+
    
+
    
+
    We can convert GradOpDescMakerBase to std::function<std::vector<std::unique_ptr<OpDescBind>>(const OpDescBind&)> by
    
+
    using GradOpMaker = ...;
+std::function<std::vector<OpDescBind>(const OpDescBind&)> func;
+func = [] (const OpDescBind& fwd_op) {
+  GradOpMaker maker(fwd_op);
+  return maker();
+};
+
    
+
    
+
    We can write many helper functions since the GradOpDescMakerBase is a class now. The basic helper functions get the variables of Input, Output, InputGradient and OutputGradient in the forwarding operator.
    
+
    We should change register macros at the same time. In the current solution, there is no difference between forwarding operators and backward operators. So REGISTER_OP just register one operator. If the REGISTER_OPERATOR contains OpProtoAndCheckerMaker and GradOpDescMaker, we just list them in the same macro. It can be done by a macro contains __VA_ARGS__.
    
+
    The user interface should be
    
+
    vector<OpDesc> MinusOpGradMaker(OpDesc) {...}
+REGISTER_OPERATOR(minus, MinusOp, MinusOpProtoAndCheckerMaker, SumOpGradMaker);
+// Developers can still manually implement gradient operator.
+REGISTER_OPERATOR(minus_grad, MinusGradOp);
+
    
+
    
+
    The interface of current REGISTER_OP macro could not be changed. In REGISTER_OP, it will invoke REGISTER_OPERATOR two times and generate GradOpDescMaker inside.
    
+
    REGISTER_OP(minus, MinusOp, MinusOpProtoAndCheckerMaker, minus_grad, MinusGradOp);
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/regularization.html b/doc/design/regularization.html
new file mode 100644
index 0000000000000000000000000000000000000000..eb382e6884ebf536b0ca643d2ef153b34949cd91
--- /dev/null
+++ b/doc/design/regularization.html
@@ -0,0 +1,331 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Regularization in PaddlePaddle — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Regularization in PaddlePaddle
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Regularization in PaddlePaddle
    
+
    
+
    Introduction to Regularization
    
+
    A central problem in machine learning is how to design an algorithm that will perform well not just on the training data, but also on new data. A frequently faced problem is the problem of overfitting, where the model does not make reliable predictions on new unseen data. Regularization is the process of introducing additional information in order to prevent overfitting. This is usually done by adding extra penalties to the loss function that restricts the parameter spaces that an optimization algorithm can explore.
    
+
    
+
    Parameter Norm Penalties
    
+
    Most common regularization approaches in deep learning are based on limiting the capacity of the models by adding a parameter norm penalty to the objective function J. This is given as follows:
    
+

    
+
    The parameter alpha is a hyperparameter that weights the relative contribution of the norm penalty term, omega, relative to the standard objective function J.
    
+
    The most commonly used norm penalties are the L2 norm penalty and the L1 norm penalty. These are given as follows:
    
+
    
+
    L2 Regularization:
    
+

    
+
    
+
    
+
    L1 Regularization
    
+

    
+
    A much more detailed mathematical background of regularization can be found here.
    
+
    
+
    
+
    
+
    
+
    Regularization Survey
    
+
    A detailed survey of regularization in various deep learning frameworks can be found here.
    
+
    
+
    
+
    Proposal for Regularization in PaddlePaddle
    
+
    
+
    Low-Level implementation
    
+
    In the new design, we propose to create new operations for regularization. For now, we can add 2 ops that correspond to the most frequently used regularizations:
    
+
      
+
    • L2_regularization_op
      • 
+
    • L1_regularization_op
      • 
+
    
+
    These ops can be like any other ops with their own CPU/GPU implementations either using Eigen or separate CPU and GPU kernels. As the initial implementation, we can implement their kernels using Eigen following the abstraction pattern implemented for Activation Ops. This abstraction pattern can make it very easy to implement new regularization schemes other than L1 and L2 norm penalties.
    
+
    The idea of building ops for regularization is in sync with the refactored Paddle philosophy of using operators to represent any computation unit. The way these ops will be added to the computation graph, will be decided by the layer functions in Python API.
    
+
    
+
    
+
    Computation Graph
    
+
    Below is an example of a really simple feed forward neural network.
    
+

    
+
    The Python API will modify this computation graph to add regularization operators. The modified computation graph will look as follows:
    
+

    
+
    
+
    
+
    Python API implementation for Regularization
    
+
    Using the low level ops, L2_regularization_op and L1_regularization_op, any user can add regularization to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support regularization. An example of such an API can be seen in Keras. As per the PaddlePaddle Python API design, the layer functions are responsible for creating operators, operator parameters and variables. Since regularization is a property of parameters, it makes sense to create these in the layer functions.
    
+
    
+
    Creation of Regularization ops
    
+
    There are two possibilities for creating the regularization ops:
    
+
      
+
    1. We create these ops immediately while building the computation graph.
      2. 
+
    2. We add these ops in a lazy manner, just before the backward, similar to the way the optimization ops are added.
      3. 
+
    
+
    The proposal is to add these ops in a lazy manner just before the backward pass.
    
+
    
+
    
+
    Storage of Regularization attributes
    
+
    Since we want to create the regularization ops in a lazy manner, the regularization attributes (type of regularization and weight of regularization penalty) can be stored as attributes of the Parameter class. This is because regularization is a property of the parameters and storing regularization properties with Parameters also allows for shared parameters.
    
+
    
+
    
+
    High-level API
    
+
    In PaddlePaddle Python API, users will primarily rely on layer functions to create neural network layers. Hence, we also need to provide regularization functionality in layer functions. The design of these APIs can be postponed for later right now. A good reference for these APIs can be found in Keras and also by looking at Tensorflow in tf.contrib.layers.
    
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/releasing_process.html b/doc/design/releasing_process.html
new file mode 100644
index 0000000000000000000000000000000000000000..eb80f045d6be58c6ba527f3048bb832a7e1ed864
--- /dev/null
+++ b/doc/design/releasing_process.html
@@ -0,0 +1,355 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  PaddlePaddle发行规范 — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • PaddlePaddle发行规范
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    PaddlePaddle发行规范
    
+
    PaddlePaddle使用git-flow branching model做分支管理，使用Semantic Versioning标准表示PaddlePaddle版本号。
    
+
    PaddlePaddle每次发新的版本，遵循以下流程:
    
+
      
+
    1. 从develop分支派生出新的分支，分支名为release/版本号。例如，release/0.10.0
      
+
      2. 
+
    2. 将新分支的版本打上tag，tag为版本号rc.Patch号。第一个tag为0.10.0rc1，第二个为0.10.0rc2，依次类推。
      
+
      3. 
+
    3. 对这个版本的提交，做如下几个操作:
      
+
        
+
      • 修改python/setup.py.in中的版本信息,并将istaged字段设为True。
        
+
        • 
+
      • 编译这个版本的Docker发行镜像，发布到dockerhub。如果失败，修复Docker编译镜像问题，Patch号加一，返回第二步
        
+
        • 
+
      • 编译这个版本的Ubuntu Deb包。如果失败，修复Ubuntu Deb包编译问题，Patch号加一，返回第二步。
        
+
        • 
+
      • 使用Regression Test List作为检查列表，测试Docker镜像/ubuntu安装包的功能正确性
        
+
          
+
        • 如果失败，记录下所有失败的例子，在这个release/版本号分支中，修复所有bug后，Patch号加一，返回第二步
          • 
+
        
+
        • 
+
      • 编译这个版本的python wheel包，并发布到pypi。
        
+
          
+
        • 由于pypi.python.org目前遵循严格的命名规范PEP 513，在使用twine上传之前，需要重命名wheel包中platform相关的后缀，比如将linux_x86_64修改成manylinux1_x86_64。
          
+
          • 
+
        • pypi上的package名称为paddlepaddle和paddlepaddle_gpu，如果要上传GPU版本的包，需要修改build/python/setup.py中，name: “paddlepaddle_gpu”并重新打包wheel包：python setup.py bdist_wheel。
          
+
          • 
+
        • 上传方法：
          
+
          cd build/python
+pip install twine
+twine upload dist/[package to upload]
+
          
+
          
+
          • 
+
        
+
        • 
+
      
+
      4. 
+
    4. 第三步完成后，将release/版本号分支合入master分支，并删除release/版本号分支。将master分支的合入commit打上tag，tag为版本号。同时再将master分支合入develop分支。最后删除release/版本号分支。
      
+
      5. 
+
    5. 编译master分支的Docker发行镜像，发布到dockerhub。编译ubuntu的deb包，发布到github release页面
      
+
      6. 
+
    6. 协同完成Release Note的书写
      
+
      7. 
+
    
+
    需要注意的是:
    
+
      
+
    • release/版本号分支一旦建立，一般不允许再从develop分支合入release/版本号。这样保证release/版本号分支功能的封闭，方便测试人员测试PaddlePaddle的行为。
      • 
+
    • 在release/版本号分支存在的时候，如果有bugfix的行为，需要将bugfix的分支同时merge到master, develop和release/版本号这三个分支。
      • 
+
    
+
    
+
    PaddlePaddle 分支规范
    
+
    PaddlePaddle开发过程使用git-flow分支规范，并适应github的特性做了一些区别。
    
+
      
+
    • PaddlePaddle的主版本库遵循git-flow分支规范。其中:
        
+
      • master分支为稳定(stable branch)版本分支。每一个master分支的版本都是经过单元测试和回归测试的版本。
        • 
+
      • develop分支为开发(develop branch)版本分支。每一个develop分支的版本都经过单元测试，但并没有经过回归测试。
        • 
+
      • release/版本号分支为每一次Release时建立的临时分支。在这个阶段的代码正在经历回归测试。
        • 
+
      
+
      • 
+
    • 其他用户的fork版本库并不需要严格遵守git-flow分支规范，但所有fork的版本库的所有分支都相当于特性分支。
        
+
      • 建议，开发者fork的版本库使用develop分支同步主版本库的develop分支
        • 
+
      • 建议，开发者fork的版本库中，再基于develop版本fork出自己的功能分支。
        • 
+
      • 当功能分支开发完毕后，向PaddlePaddle的主版本库提交Pull Reuqest，进而进行代码评审。
          
+
        • 在评审过程中，开发者修改自己的代码，可以继续在自己的功能分支提交代码。
          • 
+
        
+
        • 
+
      
+
      • 
+
    • BugFix分支也是在开发者自己的fork版本库维护，与功能分支不同的是，BugFix分支需要分别给主版本库的master、develop与可能有的release/版本号分支，同时提起Pull Request。
      • 
+
    
+
    
+
    
+
    PaddlePaddle回归测试列表
    
+
    本列表说明PaddlePaddle发版之前需要测试的功能点。
    
+
    
+
    PaddlePaddle Book中所有章节
    
+
    PaddlePaddle每次发版本首先要保证PaddlePaddle Book中所有章节功能的正确性。功能的正确性包括验证PaddlePaddle目前的paddle_trainer训练和纯使用Python训练模型正确性。
    
+
    | | 新手入门章节 | 识别数字 | 图像分类 | 词向量 | 情感分析 | 语意角色标注 | 机器翻译 | 个性化推荐 |
+| — | — | — | — | — | — | — | — | — |
+| API.V2 + Docker + GPU  |  |  |  |  |  |  |  |  |
+| API.V2 + Docker + CPU  |  |  |  |  |  |  |  |  |
+| paddle_trainer + Docker + GPU |  |  |  |  |  |  |  |  |
+| paddle_trainer + Docker + CPU |  |  |  |  |  |  |  |  |
+| API.V2 + Ubuntu + GPU |  |  |  |  |  |  |  |  |
+| API.V2 + Ubuntu + CPU |  |  |  |  |  |  |  |  |
+| paddle_trainer + Ubuntu + GPU |  |  |  |  |  |  |  |  |
+| paddle_trainer + Ubuntu + CPU |  |  |  |  |  |  |  |  |
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/scope.html b/doc/design/scope.html
new file mode 100644
index 0000000000000000000000000000000000000000..639803f8e01f36c11eadcf7011be0fabc9b7a1b6
--- /dev/null
+++ b/doc/design/scope.html
@@ -0,0 +1,384 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design of Scope in Paddle — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design of Scope in Paddle
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design of Scope in Paddle
    
+
    
+
    Overview
    
+
    Scope is an important concept in programming languages, which defines a program region that a set of bindings between names and entities applies. In a specific scope, a valid name is uniquely associated with an entity, such as a variable. And in another scope, this name may refer to other entity or nothing at all. It clearly restricts the visibility and validity of names in a program. Hence Scope is introduced to PaddlePaddle to manage variables in context. But different from the original abstract concept, Scope now becomes an object with two important attributes:
    
+
      
+
    • Scope is an association of a name to variable.
      • 
+
    • Variables in a parent scope can be retrieved from local scope.
      • 
+
    
+
    A detailed explanation of these two attributes goes as following.
    
+
    
+
    
+
    Scope is an association of a name to variable.
    
+
    Scope is an association of a name to variable. All variables belong to Scope. You need to specify a scope to run a Net, i.e., net.Run(&scope). One net can run in different scopes and update different variable in the scope.
    
+
      
+
    1. Scope only contains a map of a name to variable.
      
+
      All parameters, data, states in a Net should be variables and stored inside a scope. Each op should get inputs and outputs to do computation from a scope, such as data buffer, state (momentum) etc.
      
+
      2. 
+
    2. Variable can only be created by Scope and a variable can only be got from Scope. User cannot create or get a variable outside a scope. This is a constraints of our framework, and will keep our framework simple and clear.
      
+
      3. 
+
    3. Scope only contains methods that are used to Create and Get Variables. Scope do not contain Operators and have no information to run them.
+Net is designed to drive the computation and Scope only contains a map of variables. There is no computation logic inside a Scope. Scope just handles the lifetime management of variables.
      
+
        
+
      • Create is used to create a Variable by its name and add the mapping relation.
        • 
+
      • Get is used to find a Variable by name.
        • 
+
      
+
      4. 
+
    4. Every variable only belongs to one certain Scope.
      
+
      Variable can not belong to many scopes. If you want to use variables from parent scope, you can use parent scope.
      
+
      5. 
+
    5. Scope should destruct all Variables inside it when itself is destructed. User can never store Variable pointer somewhere else.
      
+
      Because Variable can only be got from Scope. When destroying Scope, we also need to destroy all the Variables in it. If user store Variable pointer to private data member or some global variable, the pointer will be an invalid pointer when associated Scope is destroyed.
      
+
      6. 
+
    
+
    class Scope {
+ public:
+  Variable* Var(const std::string& name);
+  const Variable* FindVar(const std::string& name) const;
+
+ private:
+    std::unordered_map<std::string, std::unique_ptr<Variable>> vars_;
+};
+
    
+
    
+
    
+
    
+
    Parent scope and local scope
    
+
    Just like scope in programming languages, Scope in the neural network can also be a local scope. There are two attributes about local scope.
    
+
      
+
    1. We can create local variables in a local scope. When that local scope is destroyed, all local variables should also be destroyed.
      2. 
+
    2. Variables in a parent scope can be retrieved from local scopes of that parent scope, i.e., when user get a variable from a scope, it will try to search this variable in current scope. If there is no such variable in the local scope, scope will keep searching from its parent, until the variable is found or there is no parent.
      3. 
+
    
+
    class Scope {
+ public:
+  Scope(const std::shared_ptr<Scope>& scope): parent_(scope) {}
+
+  Variable* FindVar(const std::string& name) const {
+    auto it = vars_.find(name);
+    if (it != vars_.end()) {
+      return it->second.get();
+    } else if (parent_ != nullptr) {
+      return parent_->FindVar(name);
+    } else {
+      return nullptr;
+    }
+  }
+
+ private:
+  std::shared_ptr<Scope> parent_ {nullptr};
+};
+
    
+
    
+
    In Scope class, there is a private data member called parent_. parent_ is a smart pointer to its parent scope. When user Get a variable by its name, the name will be searched inside the current scope. If the variable cannot be found locally and parent scope is not a nullptr, the variable will be searched inside that parent scope. parent_ pointer’s default value is nullptr. It means that the scope is a global scope when parent_ is nullptr.
    
+
    A local scope is very useful when we implement Recurrent Neural Network. Each timestep of an RNN should be a Net. Each Net of timestep (StepNet for short) should use an independent local scope. Just like variables in a while loop is inside a local scope in programming languages. By using a single StepNet and changing local scope, we can implement an RNN easily.
    
+
    
+
    
+
    
+
    Interface Design
    
+
    class Variable {
+ private:
+  Variable() = default;
+  friend class Scope;
+};
+
+class Scope {
+ private:
+  Scope(const std::shared_ptr<Scope>& parent = nullptr);
+
+ public:
+  static std::shared_ptr<Scope> Create(const std::shared_ptr<Scope>& parent = nullptr);
+
+  // return nullptr if not found.
+  Variable* FindVar(const std::string& name) const;
+
+  // return if already contains same name variable.
+  Variable* Var(const std::string& name);
+
+ private:
+  std::shared_ptr<Scope> parent_;
+  std::unordered_map<std::string, std::unique_ptr<Variable>> vars_;
+};
+
    
+
    
+
    
+
    Only scope can create a variable
    
+
    To ensure only scope can create a variable, we should mark Variable‘s constructor as a private member function, and Scope is a friend class of Variable. And then only Var can construct Variable.
    
+
    
+
    
+
    When scope destroyed, all variables inside this scope should be destroyed together
    
+
    The scope hold unique pointers for all variables. User can FindVar from scope, but he should not hold this pointer as a member variable. Because when scope is destroyed, all variables inside this scope will be destroyed together.
    
+
    
+
    
+
    Sharing a parent scope
    
+
    Local scope contains a parent_ pointer. It is a linked-list for scopes. Using a shared_ptr because when a local scope is using, its parents cannot be destroyed.
    
+
    Also, as the parent scope is a shared_ptr, we can only Create() a scope shared pointer. We cannot construct a scope variable, because it cannot be passed to other scope as parent pointer.
    
+
    
+
    
+
    Orthogonal interface
    
+
    FindVar will return nullptr when name is not found. It can be used as Contains method. Var will return an Error when there is a name conflict locally. Combine FindVar and Var, we can implement Var easily.
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/selected_rows.html b/doc/design/selected_rows.html
new file mode 100644
index 0000000000000000000000000000000000000000..aa12d4136ea85cead1c873981275688f31e20404
--- /dev/null
+++ b/doc/design/selected_rows.html
@@ -0,0 +1,330 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Selected Rows — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Selected Rows
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Selected Rows
    
+
    SelectedRows is a type of sparse tensor data type, which is designed to support embedding operators. The gradient of embedding table is a sparse tensor. Only a few rows are non-zero values in this tensor. It is straight-forward to represent a sparse tensor by the following sparse tensor data structure:
    
+
    class SelectedRows {
+ private:
+  vector<int> rows_;
+  Tensor value_;
+  int height_;
+};
+
    
+
    
+
    The field height_ is the first dimension of SelectedRows. The rows are the indices of the non-zero rows of SelectedRows. The value_ field is an N-dim tensor of shape [rows.size() /* NUM_ROWS */, ...], which supplies values for each row. The dimension of SelectedRows satisfies [height_] + value_.shape[1:].
    
+
    Suppose that a SelectedRows-typed variable x has many rows, but only two of them have values – row 73 is [1, 2] and row 84 is [3, 4], the SelectedRows representation would be:
    
+
    x = SelectedRow {
+  rows = [73, 84],
+  value = [[1, 2], [3,4]]
+}
+
    
+
    
+
    
+
    SelectedRows in Protobuf
    
+
    SelectedRows is a type of Variable. VarDesc in protobuf should describe the SelectedRows information. Only the tensor dimension of a SelectedRows will be described in compile-time because the rows_ and value_ are dependent on the training data.
+So we use TensorDesc to unify data_type and dims. A LodTensorDesc contains a TensorDesc and lod_level. The description of SelectedRows is a Tensor description.
    
+
    message TensorDesc {
+  required DataType data_type = 1;
+  repeated int64 dims = 2; // [UNK, 640, 480] is saved as [-1, 640, 480]
+}
+
+message LodTensorDesc {
+  required TensorDesc tensor = 1;
+  optional int lod_level = 2;
+}
+
+message VarDesc {
+  required string name = 1;
+  enum VarType { 
+    LOD_TENSOR = 0;
+    SELECTED_ROWS = 1;
+  }
+  required VarType type = 2;
+  optional LodTensorDesc lod_desc = 3;
+  optional TensorDesc selected_rows_desc = 4;
+  optional bool persistable = 5 [ default = false ];
+}
+
    
+
    
+
    
+
    
+
    InferShape for Selected Rows
    
+
    Just like LoD information, InferShape method will infer the output tensor type as well. The operator should decide whether its output is a SelectedRows or Dense tensor.
    
+
    For example, the gradient operator of TableLookup will always generate SelectedRows. Its InferShape method should be like following
    
+
    void TableLookupGrad::InferShape(context) {
+  ...
+  context.SetDataType("Embedding.Grad", kSelectedRows);
+}
+
    
+
    
+
    
+
    
+
    Sparse Operators
    
+
    There are several operators that need to be written to support SelectedRows. These are:
    
+
      
+
    1. Operators which generate SelectedRows gradient. e.g. Gradient of TableLookupOp.
      2. 
+
    2. Optimize operators which support SelectedRows gradient. e.g. SGD or AdaGrad for SelectedRows. However, there should be only one SGD operator. OpWithKernel::Run should select a suitable kernel for both dense tensor or SelectedRows.
      3. 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/simple_op_design.html b/doc/design/simple_op_design.html
new file mode 100644
index 0000000000000000000000000000000000000000..e2c8cd89f98fc15bb2c4434d939e16c691249c12
--- /dev/null
+++ b/doc/design/simple_op_design.html
@@ -0,0 +1,452 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Interaction between C++ and Python — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Interaction between C++ and Python
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Interaction between C++ and Python
    
+
    Users employ API in Python to describe their own network, however, the network construction actually happens in C++. so Protobuf is introduced to send the message between Python and C++.
    
+
    The Interaction between Python and C++ can be simplified as two steps:
    
+
      
+
    1. C++ tells Python how many Ops there are, and what parameter do users need to offer to initialize a new Op. Python then builds API for each Op at compile time.
      2. 
+
    2. Users invoke APIs built by Python and provide necessary parameters. These parameters will be sent to C++ for finishing the Op construction task.
      3. 
+
    
+
    
+
    Message from C++ to Python
    
+
    We define a Protobuf message class OpProto to hold message needed in the first step. What should an OpProto contain? This question is equivalent to “What message do we need to offer, to build a Python API which is legal and user oriented and can use to describe a whole Op.”
    
+
    Following message are necessary:
    
+
      
+
    1. Op’s name, and its simple comment.
      2. 
+
    2. Input and output variable number; each variable’s name, type, and comment.
      3. 
+
    3. Op’s attributes; each attribute includes name, type, comment, default value and value range.
      4. 
+
    
+
    So OpProto can be defined as follows:
    
+
    enum AttrType {
+    INT = 1;
+    FLOAT = 2;
+    STRING = 3;
+    INTS = 4;
+    FLOATS = 5;
+    STRINGS = 6;
+};
+
+message AttrValue {
+    AttrType type = 1;
+    optional int iv = 2;
+    optional float fv = 3;
+    optional string sv = 4;
+    repeated int ivs = 5;
+    repeated float fvs = 6;
+    repeated string svs = 7;
+};
+
+message AttrProto {
+    required string name = 1;
+    required string comment = 2;
+    required AttrType type = 3;
+};
+
+message VarProto {
+    required string name = 1;
+    required string comment = 2;
+    required bool is_tensor = 3;
+};
+
+message OpProto {
+    repeated VarProto inputs = 1;
+    repeated VarProto outputs = 2;
+    repeated AttrProto attrs = 3;
+    required string type = 4;
+    required string comment = 5;
+};
+
    
+
    
+
    To generate Python code automatically:
    
+
    def create_python_ops_creatation_functions():
+    op_protos = paddle.framework.OpRegistry.get_all_op_proto()
+    for type_name in op_protos:
+        op_proto = op_protos[type_name]
+        def __impl__(**kwargs):  # User must use key word args in Paddle API
+            inputs = [kwargs.get(ipt.name, "") for ipt in op_proto.inputs]
+            outputs = [kwargs.get(opt.name, "") for opt in op_proto.outputs]
+            attrs = [cast_to_op_attr(attr, kwargs.get(attr.name, None)) for attr in op_proto.attrs]
+            opdesc = （input, outputs, type_name, attrs）
+            return paddle.framework.OpRegistry.CreateOp(opdesc)
+        __impl__.__doc__ = create_doc_string(op_proto)
+        globals()[type_name] = __impl__
+
+create_python_ops_creatation_functions()
+
    
+
    
+
    
+
    
+
    Message from Python to C++
    
+
    To hold message needed in the above second step, we define Protobuf message class OpDesc. It is used to hold user-specified parameters in Op describing.
    
+
    message OpDesc {
+    required string type = 1;   
+    repeated string inputs = 2;
+    repeated string outputs = 3;
+    map<string, AttrValue> attrs = 4;
+};
+
    
+
    
+
    
+
    
+
    
+
    OpProto Register
    
+
    Every Op has its own OpProto. For using convenience, we need to register them and record all their messages. For each Op class, we define a corresponding OpMaker class, in whose constructor we implement the OpProto‘s building process. OpMaker‘s constructor will be invoked by another function OpRegistry::RegisterOp().
    
+
    class OpProtoMaker {
+public:
+    OpProtoMaker(OpProto* proto): proto_(proto) {}
+protected:
+    OpProto* proto_;
+    void AddInput(const std::string& name, const std::string& desc) {...}
+    void AddAttr(const std::string& name, const std::string& desc, TypeId type) {...}
+    void AddComment(const std::string& comment) { ... }
+};
+
+class OpRegistry {
+public:
+    using OpCreator = std::function<OperatorBase* (OpDesc& desc)>;
+    
+    template <typename OpType, typename OpMaker>
+    static void RegisterOp(const std::string& name) {
+        gCreators_[name] = [](const OpDesc& desc) {
+            return new OpType(desc);
+        };
+        OpProto& opProto = gProtos_[name];
+        OpMaker()(&opProto);
+    }
+
+    static map<string, OpCreator> gCreators_;
+    static map<string, OpProto> gProtos_;
+};
+
+template <typename OpType, typename OpMaker>
+class OpRegister {
+  public:
+    OpRegister(std::string type) {
+        OpRegistry::RegisterOp<OpType, OpMaker>(type);
+    }
+};
+
+#define REGISTER_OP(op_class, op_maker_class, type_name)         \
+    class op_class##Register {                                   \
+      private:                                                   \
+        const static OpRegister<#op_class, #op_maker_class> reg; \
+    };                                                           \
+    const Register op_class##Register::reg(#type_name);
+    
+class CosineOp {
+// ...
+}
+
+struct CosineOpProtoMaker : public OpProtoMaker {
+    CosineOpProtoMaker(OpProto* proto) : OpProtoMaker(proto) {
+        AddInput("input", "input of cosine op");
+        AddAttr("scale", "scale of cosine op", float).Default(1.0).GreaterThan(0.0);
+        AddType("cos");
+        AddComment("This is cos op");
+    }
+}
+
+REGISTER_OP(CosineOp, CosineOpProtoMaker, cos);
+
    
+
    
+
    In REGISTER_OP(CosineOp, CosineOpProtoMaker, cos), we register not only CosineOp but also CosineOpProto. As fields of CosineOpProto, the default value and value range of scale are also registered here.
    
+
    
+
    
+
    Python API
    
+
    Python  APIs are divided into two types, high-level API and low-level API.
    
+
    
+
    High-Level API
    
+
    High-level API is called by users directly, so it should keep its style consistent with existing V2 APIs.
    
+
    Here is a sample about how a define a fc layer:
    
+
    hd = fc_layer(input=data, size=56, with_bias=True, activation="sigmoid");
+
    
+
    
+
    hd is the output of fc_layer and it’s a variable. It can be further sent into other layers as input.
    
+
    The definition of fc_layer():
    
+
    def fc_layer(input, size, with_bias, activation):
+    attr_map = {"size":size}
+    check_attrs(attr_map)
+    w = make_variable('w')
+    if with_bias:
+        b = make_variable('b')
+    else:
+        b = None
+    fc_output = make_variable('fc_output');
+    fc_op(input, w, b, fc_output, attr_map)
+    act_output = make_variable('sigmod_output');
+    if activation == "sigmod":
+        sigmod_op(fc_output, act_output);
+    elif:
+        # ...
+    return act_output;
+
    
+
    
+
    
+
    
+
    Low Leval API
    
+
    In above sample, fc_op and sigmod_op are low-level API. They build OpDesc and invoke corresponding C++ code.
    
+
    TODO
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/tensor_array.html b/doc/design/tensor_array.html
new file mode 100644
index 0000000000000000000000000000000000000000..60a6de242e3a9d4d4a22b52c8c53753c0c5021c6
--- /dev/null
+++ b/doc/design/tensor_array.html
@@ -0,0 +1,514 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design for TensorArray — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design for TensorArray
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design for TensorArray
    
+
    This design doc presents the necessity of a new C++ class TensorArray.
+In addition to the very simple C++ implementation
    
+
    class TensorArray {
+ public:
+  explicit TensorArray(const LoDTensor&);
+  explicit TensorArray(size_t size);
+
+ private:
+  vector<LoDTensor> values_;
+};
+
    
+
    
+
    We also need to expose it to PaddlePaddle’s Python API,
+because users would want to use it with our very flexible operators WhileLoop.
+An example for a RNN based on dynamic operators is
    
+
    input = pd.data(...)
+num_steps = Var(12)
+
+TensorArray states(size=num_steps)
+TensorArray step_inputs(unstack_from=input)
+TensorArray step_outputs(size=num_steps)
+
+W = Tensor(...)
+U = Tensor(...)
+default_state = some_op()
+
+step = Var(1)
+
+wloop = paddle.create_whileloop(loop_vars=[step])
+with wloop.frame():
+    wloop.break_if(pd.equal(step, num_steps)
+    pre_state = states.read(step-1, default_state)
+    step_input = step_inputs.read(step)
+    state = pd.sigmoid(pd.matmul(U, pre_state) + pd.matmul(W, step_input))
+    states.write(step, state)
+    step_outputs.write(step, state) # output state
+    step.update(state+1)
+
+output = step_outputs.stack()
+
    
+
    
+
    
+
    Background
    
+
    Steps are one of the core concepts of RNN. In each time step of RNN, there should be several input segments, states, and output segments; all these components act like arrays, for example, call states[step_id] will get the state in step_idth time step.
    
+
    An RNN can be implemented with the following pseudocode
    
+
    Array states;
+Array input_segments;
+Array output_segments;
+Parameter W, U;
+
+step = 1
+seq_len = 12
+while_loop {
+   if (step == seq_len) break;
+    states[step] = sigmoid(W * states[step-1] + U * input_segments[step]);
+    output_segments[step] = states[step] // take state as output
+   step++;
+}
+
    
+
    
+
    According to the RNN roadmap, there are several different RNNs that PaddlePaddle will eventually support.
    
+
    Currently, the basic RNN implementation supported by PaddlePaddle is the recurrent_op which takes tensors as input and splits them into input_segments.
    
+
    Since a tensor cannot store variable-length sequences directly, PaddlePaddle implements the tensor with level of details (LoDTensor for short).
+Segmenting the LoDTensor is much more complicated than splitting a tensor, that makes it necessary to refactor the recurrent_op with LoDTensor segmenting support.
    
+
    As the next step in RNN support, dynamic_recurrent_op should be introduced to handle inputs with variable-length sequences.
    
+
    The implementation is similar to recurrent_op.
+The key difference is the way the original input LoDTensors and outupts are split to get the input_segments and the output_segments.
    
+
    Though it can’t be built over recurrent_op or dynamic_recurrent_op directly,
+the logic behind splitting a tensor or a LoD tensor into input_segments remains the same.
    
+
    
+
    
+
    Why TensorArray
    
+
    The logic behind splitting the inputs to segments, states and outputs is similar and can be shared in a seperate module.
    
+
    The array of states, input_segments and output_segments would be exposed to users when writing a dynamic RNN model similar to the above pseudo codes.
    
+
    So there should be an array-like container, which can store the segments of a tensor or LoD tensor.
    
+
    This container can store an array of tensors and provides several methods to split a tensor or a LoD tensor .
+This is where the notion of TensorArray comes from.
    
+
    
+
    
+
    Introduce TensorArray to uniform all the three RNNs
    
+
    TensorArray as a new concept is borrowed from TensorFlow,
+it is meant to be used with dynamic iteration primitives such as while_loop and map_fn.
    
+
    This concept can be used to support our new design of dynamic operations, and help to refactor some existing variant-sentence-related layers,
+such as recurrent_op, RecurrentGradientMachine.
    
+
    In our design for dynamic RNN,
+TensorArray is used to segment inputs and store states in all time steps.
+By providing some methods similar to a C++ array,
+the definition of some state-based dynamic models such as RNN can be more natural and highly flexible.
    
+
    
+
    
+
    Dynamic-operations on TensorArray
    
+
    TensorArray will be used directly when defining dynamic models, so some operators listed below should be implemented
    
+
    # several helper operators for TensorArray
+def tensor_array_stack(ta, tensor):
+    '''
+    get a tensor array `ta`, return a packed `tensor`.
+    '''
+    pass
+
+def tensor_array_unstack(tensor, ta):
+    '''
+    get a `tensor`, unstack it and get a tensor array `ta`.
+    '''
+    pass
+
+def tensor_array_write(ta, index, tensor, data_shared):
+    '''
+    get a `tensor` and a scalar tensor `index`, write `tensor` into index-th
+    value of the tensor array `ta`.
+    `data_shared` is an attribute that specifies whether to copy or reference the tensors.
+    '''
+    pass
+
+def tensor_array_read(ta, index, tensor):
+    '''
+    get a tensor array `ta`, a scalar tensor `index`, read the index-th value of
+    `ta` and return as the `tensor`.
+    '''
+    pass
+
+def tensor_array_size(ta, tensor):
+    '''
+    get a tensor array `ta`, return the size of `ta` and return as the scalar `tensor`.
+    '''
+    pass
+
    
+
    
+
    It is trivial for users to use so many low-level operators, so some helper methods should be proposed in python wrapper to make TensorArray easier to use,
+for example
    
+
    class TensorArray:
+    def __init__(self, name):
+        self.name = name
+        self.desc = TensorArrayDesc()
+
+    def stack(self, name=None):
+        '''
+        Pack the values in a `TensorArray` into a tensor with rank one higher
+        than each tensor in `values`.
+        `stack` can be used to split tensor into time steps for RNN or whileloop.
+
+        @name: str
+            the name of the variable to output.
+        '''
+        tensor = Var(name)
+        tensor_array_stack(self.name, tensor)
+        return tensor
+
+    def unstack(self, input):
+        '''
+        Unpacks the given dimension of a rank-`R` tensor into rank-`(R-1)` tensors.
+        `unstack` can be used to concatenate all the time steps for RNN or whileloop.
+
+        @input: str
+            the name of input tensor
+        '''
+        tensor_array_unstack(tensor, self.name)
+
+    def write(self, index, value, data_shared=True):
+        '''
+        Write value into index of the TensorArray.
+        If `data_shared` is set to True, than the index-th value in TensorArray will
+        be shared with the tensor passed in.
+
+        @index: str
+            name of a scalar tensor
+        @value: str
+            name of a tensor
+        @data_shared: bool
+        '''
+        tensor_array_write(self.name, index, value, data_shared)
+
+    def read(self, index, output):
+        '''
+        Read the value at location `index` in the `TensorArray`.
+
+        @index: str
+            name of a scalar tensor
+        @output:
+            name of a output variable
+        '''
+        tensor_array_read(self.name, index, output)
+
+
+    def size(self, output):
+        '''
+        Return the number of values.
+
+        @output: str
+            name of a scalar tensor
+        '''
+        tensor_array_size(self.name, output)
+
    
+
    
+
    
+
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/design/var_desc.html b/doc/design/var_desc.html
new file mode 100644
index 0000000000000000000000000000000000000000..506383506ecb565db983a2b05cdf2cda980ba07e
--- /dev/null
+++ b/doc/design/var_desc.html
@@ -0,0 +1,328 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Background — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Background
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Background
    
+
    PaddlePaddle divides the description of neural network computation graph into two stages: compile time and runtime.
    
+
    PaddlePaddle use proto message to describe compile time graph because
    
+
      
+
    1. Computation graph should be able to be saved to a file.
      2. 
+
    2. In distributed training, the graph will be serialized and send to multiple workers.
      3. 
+
    
+
    The computation graph is constructed by Data Node and Operation Node. The concept to represent them is in the table below.
    
+
    | |compile time|runtime|
+|—|—|—|
+|Data|VarDesc(proto)|Variable(cpp)|
+|Operation|OpDesc(proto)|Operator(cpp)|
    
+
    
+
    
+
    Definition of VarDesc
    
+
    A VarDesc should have a name, and value. The are two kinds of variable type in compile time, they are LoDTensor and SelectedRows.
    
+
    message VarDesc {
+  required string name = 1;
+  enum VarType {
+    LOD_TENSOR = 0;
+    SELECTED_ROWS = 1;
+  }
+  required VarType type = 2;
+  optional LoDTensorDesc lod_desc = 3;
+  optional TensorDesc selected_rows_desc = 4;
+  optional bool persistable = 5 [ default = false ];
+}
+
    
+
    
+
    
+
    
+
    Definition of TensorDesc
    
+
    enum DataType {
+  BOOL = 0;
+  INT16 = 1;
+  INT32 = 2;
+  INT64 = 3;
+  FP16 = 4;
+  FP32 = 5;
+  FP64 = 6;
+}
+
+message TensorDesc {
+  required DataType data_type = 1;
+  repeated int64 dims = 2; // [UNK, 640, 480] is saved as [-1, 640, 480]
+}
+
    
+
    
+
    A TensorDesc describes SelectedRows and LoDTensor. For details of SelectedRows, please reference SelectedRows.
    
+
    
+
    
+
    Definition of LodTensorDesc
    
+
    message LoDTensorDesc {
+  required TensorDesc tensor = 1;
+  optional int lod_level = 2;
+}
+
    
+
    
+
    A LoDTensorDesc contains a tensor and a lod_level.
    
+
    
+
    
+
    Definition of Variable in Python
    
+
    For Variable in Python, please reference Python API.
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/genindex.html b/doc/genindex.html
index ee5993c9cbdcd98fa7d49a811d8745561c0b3b13..1116f4befaeb4456ea5eb110a7e41d1911dfecc9 100644
--- a/doc/genindex.html
+++ b/doc/genindex.html
@@ -63,7 +63,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -85,7 +85,7 @@
  • GET STARTED
  • HOW TO
  • API
    • -
  • ABOUT
    • +
  • MOBILE
    • @@ -109,9 +109,10 @@ -
  • Run Distributed Training
    • +
  • PaddlePaddle Distributed Training
  • Paddle On Kubernetes
  • Distributed PaddlePaddle Training on AWS with Kubernetes
  • Write New Layers
  • Contribute Code
    • -
  • RNN Models
    • +
  • Contribute Documentation
    • +
  • RNN Models +
  • Tune GPU Performance
    • @@ -136,17 +141,40 @@
  • Model Configuration
    • -
  • Data Reader Interface and DataSets
    • +
  • Data Reader Interface and DataSets +
  • Training and Inference
    • +
  • Fluid +
    • + + +
  • MOBILE
    • -
  • ABOUT
    • @@ -180,54 +208,100 @@

    Index

    - D - | E + B + | C + | L + | M | P + | R | S + | T
    -

    D

    +

    B

    + + +
    + +

    C

    -

    E

    +

    L

    -

    P

    +

    M

    +
    + +

    P

    + +
    +

    R

    + + + +
    +

    S

    + +
    + +

    T

    + +
    diff --git a/doc/getstarted/basic_usage/index_en.html b/doc/getstarted/basic_usage/index_en.html deleted file mode 100644 index 974ef3ce8a27aeac57aa745201e2b0750ad79cab..0000000000000000000000000000000000000000 --- a/doc/getstarted/basic_usage/index_en.html +++ /dev/null @@ -1,331 +0,0 @@ - - - - - - - - - - - Simple Linear Regression — PaddlePaddle documentation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    -
      - -
    • Simple Linear Regression
      • -
    -
    - -
    -
    -
    -
    - -
    -

    Simple Linear Regression

    -

    PaddlePaddle is a deep learning platform open-sourced by Baidu. With PaddlePaddle, you can easily train a classic neural network within a couple lines of configuration, or you can build sophisticated models that provide state-of-the-art performance on difficult learning tasks like sentiment analysis, machine translation, image caption and so on.

    -
    -

    Problem Background

    -

    Now, to give you a hint of what using PaddlePaddle looks like, let’s start with a fundamental learning problem - simple linear regression: you have observed a set of two-dimensional data points of X and Y, where X is an explanatory variable and Y is corresponding dependent variable, and you want to recover the underlying correlation between X and Y. Linear regression can be used in many practical scenarios. For example, X can be a variable about house size, and Y a variable about house price. You can build a model that captures relationship between them by observing real estate markets.

    -
    -
    -

    Prepare the Data

    -

    Suppose the true relationship can be characterized as Y = 2X + 0.3, let’s see how to recover this pattern only from observed data. Here is a piece of python code that feeds synthetic data to PaddlePaddle. The code is pretty self-explanatory, the only extra thing you need to add for PaddlePaddle is a definition of input data types.

    -
    -
    # dataprovider.py
-from paddle.trainer.PyDataProvider2 import *
-import random
-
-# define data types of input: 2 real numbers
-@provider(input_types=[dense_vector(1), dense_vector(1)],use_seq=False)
-def process(settings, input_file):
-    for i in xrange(2000):
-        x = random.random()
-        yield [x], [2*x+0.3]
-
    -
    -
    -
    -
    -

    Train a NeuralNetwork

    -

    To recover this relationship between X and Y, we use a neural network with one layer of linear activation units and a square error cost layer. Don’t worry if you are not familiar with these terminologies, it’s just saying that we are starting from a random line Y' = wX + b , then we gradually adapt w and b to minimize the difference between Y' and Y. Here is what it looks like in PaddlePaddle:

    -
    -
    # trainer_config.py
-from paddle.trainer_config_helpers import *
-
-# 1. read data. Suppose you saved above python code as dataprovider.py
-data_file = 'empty.list'
-with open(data_file, 'w') as f: f.writelines(' ')
-define_py_data_sources2(train_list=data_file, test_list=None,
-        module='dataprovider', obj='process',args={})
-
-# 2. learning algorithm
-settings(batch_size=12, learning_rate=1e-3, learning_method=MomentumOptimizer())
-
-# 3. Network configuration
-x = data_layer(name='x', size=1)
-y = data_layer(name='y', size=1)
-y_predict = fc_layer(input=x, param_attr=ParamAttr(name='w'), size=1, act=LinearActivation(), bias_attr=ParamAttr(name='b'))
-cost = mse_cost(input=y_predict, label=y)
-outputs(cost)
-
    -
    -
    -

    Some of the most fundamental usages of PaddlePaddle are demonstrated:

    -
      -
    • The first part shows how to feed data into PaddlePaddle. In general cases, PaddlePaddle reads raw data from a list of files, and then do some user-defined process to get real input. In this case, we only need to create a placeholder file since we are generating synthetic data on the fly.
      • -
    • The second part describes learning algorithm. It defines in what ways adjustments are made to model parameters. PaddlePaddle provides a rich set of optimizers, but a simple momentum based optimizer will suffice here, and it processes 12 data points each time.
      • -
    • -
      Finally, the network configuration. It usually is as simple as “stacking” layers. Three kinds of layers are used in this configuration:
      -
        -
      • Data Layer: a network always starts with one or more data layers. They provide input data to the rest of the network. In this problem, two data layers are used respectively for X and Y.
        • -
      • FC Layer: FC layer is short for Fully Connected Layer, which connects all the input units to current layer and does the actual computation specified as activation function. Computation layers like this are the fundamental building blocks of a deeper model.
        • -
      • Cost Layer: in training phase, cost layers are usually the last layers of the network. They measure the performance of current model, and provide guidence to adjust parameters.
        • -
      -
      -
      -
      • -
    -

    Now that everything is ready, you can train the network with a simple command line call:

    -
    -
    paddle train --config=trainer_config.py --save_dir=./output --num_passes=30
-
    -
    -
    -

    This means that PaddlePaddle will train this network on the synthectic dataset for 30 passes, and save all the models under path ./output. You will see from the messages printed out during training phase that the model cost is decreasing as time goes by, which indicates we are getting a closer guess.

    -
    -
    -

    Evaluate the Model

    -

    Usually, a different dataset that left out during training phase should be used to evalute the models. However, we are lucky enough to know the real answer: w=2, b=0.3, thus a better option is to check out model parameters directly.

    -

    In PaddlePaddle, training is just to get a collection of model parameters, which are w and b in this case. Each parameter is saved in an individual file in the popular numpy array format. Here is the code that reads parameters from last pass.

    -
    -
    import numpy as np
-import os
-
-def load(file_name):
-    with open(file_name, 'rb') as f:
-        f.read(16) # skip header for float type.
-        return np.fromfile(f, dtype=np.float32)
-
-print 'w=%.6f, b=%.6f' % (load('output/pass-00029/w'), load('output/pass-00029/b'))
-# w=1.999743, b=0.300137
-
    -
    -../../_images/parameters.png -
    -

    Although starts from a random guess, you can see that value of w changes quickly towards 2 and b changes quickly towards 0.3. In the end, the predicted line is almost identical with real answer.

    -

    There, you have recovered the underlying pattern between X and Y only from observed data.

    -
    -
    - - -
    -
    -
    - - -
    - -
    -

    - © Copyright 2016, PaddlePaddle developers. - -

    -
    - Built with Sphinx using a theme provided by Read the Docs. - -
    - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc/getstarted/build_and_install/build_from_source_en.html b/doc/getstarted/build_and_install/build_from_source_en.html index 7b8c78f7377a7cd2fe111ac352460349b250637b..7116e58aed86be65d0f301dcd90bb4ada3041287 100644 --- a/doc/getstarted/build_and_install/build_from_source_en.html +++ b/doc/getstarted/build_and_install/build_from_source_en.html @@ -8,7 +8,7 @@ - Installing from Sources — PaddlePaddle documentation + Build from Sources — PaddlePaddle documentation @@ -35,7 +35,7 @@ - + @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -87,7 +87,7 @@
  • GET STARTED
  • HOW TO
  • API
    • -
  • ABOUT
    • +
  • MOBILE
    • @@ -111,9 +111,10 @@ -
  • Run Distributed Training
    • +
  • PaddlePaddle Distributed Training
  • Paddle On Kubernetes
  • Distributed PaddlePaddle Training on AWS with Kubernetes
  • Write New Layers
  • Contribute Code
    • -
  • RNN Models
    • +
  • Contribute Documentation
    • +
  • RNN Models +
  • Tune GPU Performance
    • @@ -138,17 +143,40 @@
  • Model Configuration
    • -
  • Data Reader Interface and DataSets
    • +
  • Data Reader Interface and DataSets +
  • Training and Inference
    • +
  • Fluid +
    • + + +
  • MOBILE
    • -
  • ABOUT
    • @@ -173,7 +201,7 @@
  • Install and Build >
    • -
  • Installing from Sources
    • +
  • Build from Sources
    • @@ -182,216 +210,206 @@
    -
    -

    Installing from Sources

    - -
    -

    Download and Setup

    -

    You can download PaddlePaddle from the github source.

    -
    git clone https://github.com/PaddlePaddle/Paddle paddle
-cd paddle
+  
    
+
    Build from Sources
    
+
    
+
    How To Build
    
+
    PaddlePaddle mainly uses CMake and GCC, G++ as compile
+tools. We recommend you to use our pre-built Docker image to run the build
+to avoid installing dependencies by yourself. We have several build environment
+Docker images here .
    
+
    If you choose not to use Docker image for your build, you need to install the
+below Compile Dependencies before run the build.
    
+
    Then run:
    
+
    git clone https://github.com/PaddlePaddle/Paddle.git
+cd Paddle
+# run the following command to build a CPU-Only binaries if you are using docker
+docker run -it -v $PWD:/paddle -e "WITH_GPU=OFF" -e "WITH_TESTING=OFF" paddlepaddle/paddle_manylinux_devel:cuda8.0_cudnn5 bash -x /paddle/paddle/scripts/docker/build.sh
+# else run these commands
+mkdir build
+cd build
+cmake -DWITH_GPU=OFF -DWITH_TESTING=OFF ..
+make
 
    
 
    
-
    
-
    
-
    Requirements
    
-
    To compile the source code, your computer must be equipped with the following dependencies.
    
-
      
-
    • Compiler: GCC >= 4.8 or Clang >= 3.3 (AppleClang >= 5.1) and gfortran compiler
      • 
-
    • CMake: CMake >= 3.0 (at least CMake 3.4 on Mac OS X)
      • 
-
    • BLAS: MKL, OpenBlas or ATLAS
      • 
-
    • Python: only support Python 2.7
      • 
-
    
-
    Note: For CUDA 7.0 and CUDA 7.5, GCC 5.0 and up are not supported!
-For CUDA 8.0, GCC versions later than 5.3 are not supported!
    
-
    
-
    Options
    
-
    PaddlePaddle supports some build options.
    
-
    
- 
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
    OptionalDescription
    WITH_GPUCompile PaddlePaddle with NVIDIA GPU
    WITH_AVXCompile PaddlePaddle with AVX intrinsics
    WITH_DSOCompile PaddlePaddle with dynamic linked CUDA
    WITH_TESTINGCompile PaddlePaddle with unit testing
    WITH_SWIG_PYCompile PaddlePaddle with inference api
    WITH_STYLE_CHECKCompile PaddlePaddle with style check
    WITH_PYTHONCompile PaddlePaddle with python interpreter
    WITH_DOUBLECompile PaddlePaddle with double precision
    WITH_RDMACompile PaddlePaddle with RDMA support
    WITH_TIMERCompile PaddlePaddle with stats timer
    WITH_PROFILERCompile PaddlePaddle with GPU profiler
    WITH_DOCCompile PaddlePaddle with documentation
    WITH_COVERAGECompile PaddlePaddle with code coverage
    COVERALLS_UPLOADPackage code coverage data to coveralls
    ON_TRAVISExclude special unit test on Travis CI
    
-
    Note:
    
-
      
-
    • The GPU version works best with Cuda Toolkit 8.0 and cuDNN v5.
      • 
-
    • Other versions like Cuda Toolkit 7.0, 7.5 and cuDNN v3, v4 are also supported.
      • 
-
    • To utilize cuDNN v5, Cuda Toolkit 7.5 is prerequisite and vice versa.
      • 
-
    
-
    As a simple example, consider the following:
    
-
      
-
    1. BLAS Dependencies(optional)
      
-
      CMake will search BLAS libraries from system. If not found, OpenBLAS will be downloaded, built and installed automatically.
-To utilize preinstalled BLAS， you can simply specify MKL, OpenBLAS or ATLAS via MKL_ROOT, OPENBLAS_ROOT or ATLAS_ROOT.
      
-
      # specify MKL
-cmake .. -DMKL_ROOT=<mkl_path>
-# or specify OpenBLAS
-cmake .. -DOPENBLAS_ROOT=<openblas_path>
+
      When the compile finishes, you can get the output whl package under
+build/python/dist, then you can choose to install the whl on local
+machine or copy it to the target machine.
      
+
      pip install build/python/dist/*.whl
 
      
 
      
-
      2. 
-
    2. Doc Dependencies(optional)
      
-
      To generate PaddlePaddle’s documentation, install dependencies and set -DWITH_DOC=ON as follows:
      
-
      pip install 'sphinx>=1.4.0'
-pip install sphinx_rtd_theme recommonmark
-
-# install doxygen on Ubuntu
-sudo apt-get install doxygen 
-# install doxygen on Mac OS X
-brew install doxygen
-
-# active docs in cmake
-cmake .. -DWITH_DOC=ON`
-
      
 
      
-
      3. 
-
    
-
    
-
    
-
    
-
    Build on Ubuntu 14.04
    
-
    
-
    Install Dependencies
    
-
      
-
    • Paddle Dependencies
      
-
      # necessary
-sudo apt-get update
-sudo apt-get install -y git curl gcc g++ gfortran make build-essential automake
-sudo apt-get install -y python python-pip python-numpy libpython-dev bison
-sudo pip install 'protobuf==3.1.0.post1'
-
-# install cmake 3.4
-curl -sSL https://cmake.org/files/v3.4/cmake-3.4.1.tar.gz | tar -xz && \
-    cd cmake-3.4.1 && ./bootstrap && make -j4 && sudo make install && \
-    cd .. && rm -rf cmake-3.4.1
+
      
+
      Run Tests
      
+
      If you wish to run the tests, you may follow the below steps:
      
+
      When using Docker, set RUN_TEST=ON and WITH_TESTING=ON will run test immediately after the build.
+Set WITH_GPU=ON Can also run tests on GPU.
      
+
      docker run -it -v $PWD:/paddle -e "WITH_GPU=OFF" -e "WITH_TESTING=ON" -e "RUN_TEST=ON" paddlepaddle/paddle_manylinux_devel:cuda8.0_cudnn5 bash -x paddle/paddle/scripts/docker/build.sh
 
      
 
      
-
      • 
-
    • GPU Dependencies (optional)
      
-
      To build GPU version, you will need the following installed:
      
-
        1. a CUDA-capable GPU
-  2. A supported version of Linux with a gcc compiler and toolchain
-  3. NVIDIA CUDA Toolkit (available at http://developer.nvidia.com/cuda-downloads)
-  4. NVIDIA cuDNN Library (availabel at https://developer.nvidia.com/cudnn)
+
      If you don’t use Docker, just run ctest will start the tests:
      
+
      mkdir build
+cd build
+cmake -DWITH_GPU=OFF -DWITH_TESTING=ON ..
+make
+ctest
+# run a single test like test_mul_op
+ctest -R test_mul_op
 
      
 
      
-
      The CUDA development environment relies on tight integration with the host development environment,
-including the host compiler and C runtime libraries, and is therefore only supported on
-distribution versions that have been qualified for this CUDA Toolkit release.
      
-
      After downloading cuDNN library, issue the following commands:
      
-
      sudo tar -xzf cudnn-7.5-linux-x64-v5.1.tgz -C /usr/local
-sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*
-
      
 
      
-
      Then you need to set LD_LIBRARY_PATH, PATH environment variables in ~/.bashrc.
      
-
      export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
-export PATH=/usr/local/cuda/bin:$PATH
-
      
-
      
-
      • 
-
    
-
    
-
    
-
    Build and Install
    
-
    As usual, the best option is to create build folder under paddle project directory.
    
-
    mkdir build && cd build
-
    
-
    
-
    Finally, you can build and install PaddlePaddle:
    
-
    # you can add build option here, such as:    
-cmake .. -DCMAKE_INSTALL_PREFIX=<path to install>
-# please use sudo make install, if you want to install PaddlePaddle into the system
-make -j `nproc` && make install
-# set PaddlePaddle installation path in ~/.bashrc
-export PATH=<path to install>/bin:$PATH
-# install PaddlePaddle Python modules.
-sudo pip install <path to install>/opt/paddle/share/wheels/*.whl
-
    
-
    
-
    
-
    
-
    
-
    Build on Centos 7
    
-
    
-
    Install Dependencies
    
-
      
-
    • CPU Dependencies
      
-
      # necessary
-sudo yum update
-sudo yum install -y epel-release
-sudo yum install -y make cmake3 python-devel python-pip gcc-gfortran swig git
-sudo pip install wheel numpy
-sudo pip install 'protobuf>=3.0.0'
-
      
-
      
-
      • 
-
    • GPU Dependencies (optional)
      
-
      To build GPU version, you will need the following installed:
      
-
        1. a CUDA-capable GPU
-  2. A supported version of Linux with a gcc compiler and toolchain
-  3. NVIDIA CUDA Toolkit (available at http://developer.nvidia.com/cuda-downloads)
-  4. NVIDIA cuDNN Library (availabel at https://developer.nvidia.com/cudnn)
-
      
+
      
+
      Compile Dependencies
      
+
      PaddlePaddle need the following dependencies when compiling, other dependencies
+will be downloaded automatically.
      
+
+
+++++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
      PaddlePaddle Compile Dependencies




      DependencyVersionDescription
      CMake>=3.5 
      GCC4.8.2Recommend devtools2 for CentOS
      Python2.7.xNeed libpython2.7.so
      pip>=9.0
      numpy 
      SWIG>=2.0 
      Go>=1.8Optional
      
 
      
-
      The CUDA development environment relies on tight integration with the host development environment,
-including the host compiler and C runtime libraries, and is therefore only supported on
-distribution versions that have been qualified for this CUDA Toolkit release.
      
-
      After downloading cuDNN library, issue the following commands:
      
-
      sudo tar -xzf cudnn-7.5-linux-x64-v5.1.tgz -C /usr/local
-sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*
-
      
+
      
+
      Build Options
      
+
      Build options include whether build binaries for CPU or GPU, which BLAS
+library to use etc. You may pass these settings when running cmake.
+For detailed cmake tutorial please refer to here 。
      
 
      
-
      Then you need to set LD_LIBRARY_PATH, PATH environment variables in ~/.bashrc.
      
-
      export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
-export PATH=/usr/local/cuda/bin:$PATH
+
      
+
      Bool Type Options
      
+
      You can add -D argument to pass such options, like:
      
+
      cmake .. -DWITH_GPU=OFF
 
      
 
      
-
      • 
-
    
+
+
+++++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
    Bool Type Options




    OptionDescriptionDefault
    WITH_GPUBuild with GPU supportON
    WITH_C_APIBuild only CAPIOFF
    WITH_DOUBLEBuild with double precisionOFF
    WITH_DSODynamically load CUDA librariesON
    WITH_AVXBuild with AVX supportON
    WITH_PYTHONBuild with integrated Python interpreterON
    WITH_STYLE_CHECKCheck code style when buildingON
    WITH_TESTINGBuild unit testsON
    WITH_DOCBuild documentationsOFF
    WITH_SWIG_PYBuild Python SWIG interface for V2 APIAuto
    WITH_GOLANGBuild fault-tolerant parameter server written in goON
    WITH_MKLUse MKL as BLAS library, else use OpenBLASON
    
+
    
+
    BLAS
    
+
    PaddlePaddle supports MKL and
+OpenBlAS as BLAS library。By default it uses MKL.
+If you are using MKL and your machine supports AVX2, MKL-DNN will also be downloaded
+and used, for more details .
    
+
    If you choose not to use MKL, then OpenBlAS will be used.
    
 
    
-
    
-
    Build and Install
    
-
    As usual, the best option is to create build folder under paddle project directory.
    
-
    mkdir build && cd build
-
    
+
    
+
    CUDA/cuDNN
    
+
    PaddlePaddle will automatically find CUDA and cuDNN when compiling and running.
+parameter -DCUDA_ARCH_NAME=Auto can be used to detect SM architecture
+automatically in order to speed up the build.
    
+
    PaddlePaddle can build with any version later than cuDNN v5.1, and we intend to
+keep on with latest cuDNN versions. Be sure to run with the same version of cuDNN
+you built.
    
 
    
-
    Finally, you can build and install PaddlePaddle:
    
-
    # you can add build option here, such as:    
-cmake3 .. -DCMAKE_INSTALL_PREFIX=<path to install>
-# please use sudo make install, if you want to install PaddlePaddle into the system
-make -j `nproc` && make install
-# set PaddlePaddle installation path in ~/.bashrc
-export PATH=<path to install>/bin:$PATH
-# install PaddlePaddle Python modules.
-sudo pip install <path to install>/opt/paddle/share/wheels/*.whl
+
    
+
    Pass Compile Options
    
+
    You can pass compile options to use intended BLAS/CUDA/Cudnn libraries.
+When running cmake command, it will search system paths like
+/usr/lib:/usr/local/lib and then search paths that you
+passed to cmake, i.e.
    
+
    cmake .. -DWITH_GPU=ON -DWITH_TESTING=OFF -DCUDNN_ROOT=/opt/cudnnv5
 
    
 
    
+
    NOTE: These options only take effect when running cmake for the first time, you need to clean the cmake cache or clean the build directory ( rm -rf ) if you want to change it.
    
 
    
 
    
 
    
@@ -406,7 +424,7 @@ sudo pip install <path to install>/opt/paddle/share/wheels/*.whl
         Next 
       
       
-         Previous
+         Previous
       
     
    
   
diff --git a/doc/getstarted/build_and_install/docker_install_en.html b/doc/getstarted/build_and_install/docker_install_en.html
index 31fef38e3f5e7ad94607732ea6065caed3d507bf..f90b10a756fd9692e49eb07bc3547cdf46bbf555 100644
--- a/doc/getstarted/build_and_install/docker_install_en.html
+++ b/doc/getstarted/build_and_install/docker_install_en.html
@@ -8,7 +8,7 @@
   
   
   
-  PaddlePaddle in Docker Containers — PaddlePaddle  documentation
+  Run in Docker Containers — PaddlePaddle  documentation
   
 
   
@@ -34,8 +34,8 @@
         
     
         
-        
-         
+        
+         
 
   
   
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -173,7 +201,7 @@
       
         
  • Install and Build > 
    • 
       
-    
  • PaddlePaddle in Docker Containers
    • 
+    
  • Run in Docker Containers
    • 
   
 
    
       
@@ -182,217 +210,129 @@
           
    
            
    
             
-  
    
-
    PaddlePaddle in Docker Containers
    
-
    Docker container is currently the only officially-supported way to
-running PaddlePaddle.  This is reasonable as Docker now runs on all
-major operating systems including Linux, Mac OS X, and Windows.
-Please be aware that you will need to change Dockers settings to make full use
-of your hardware resource on Mac OS X and Windows.
    
-
    
-
    Working With Docker
    
-
    Docker is simple as long as we understand a few basic concepts:
    
-
      
-
    • image: A Docker image is a pack of software. It could contain one or more programs and all their dependencies. For example, the PaddlePaddle’s Docker image includes pre-built PaddlePaddle and Python and many Python packages. We can run a Docker image directly, other than installing all these software. We can type
      
-
      docker images
-
      
-
      
-
      to list all images in the system. We can also run
      
-
      docker pull paddlepaddle/paddle:0.10.0rc2
-
      
-
      
-
      to download a Docker image, paddlepaddle/paddle in this example,
-from Dockerhub.com.
      
-
      • 
-
    • container: considering a Docker image a program, a container is a
-“process” that runs the image. Indeed, a container is exactly an
-operating system process, but with a virtualized filesystem, network
-port space, and other virtualized environment. We can type
      
-
      docker run paddlepaddle/paddle:0.10.0rc2
+  
      
+
      Run in Docker Containers
      
+
      Run PaddlePaddle in Docker container so that you don’t need to care about
+runtime dependencies, also you can run under Windows system. You can get
+tutorials at here .
      
+
      If you are using Windows, please refer to
+this
+tutorial to start running docker under windows.
      
+
      After you’ve read above tutorials you may proceed the following steps.
      
+
      
+
      Pull PaddlePaddle Docker Image
      
+
      Run the following command to download the latest Docker images:
      
+
      
+
      docker pull paddlepaddle/paddle
 
      
 
      
-
      to start a container to run a Docker image, paddlepaddle/paddle in this example.
      
-
      • 
-
    • By default docker container have an isolated file system namespace,
-we can not see the files in the host file system. By using volume,
-mounted files in host will be visible inside docker container.
-Following command will mount current dirctory into /data inside
-docker container, run docker container from debian image with
-command ls /data.
      
-
      docker run --rm -v $(pwd):/data debian ls /data
+
      
+
      For users in China, we provide a faster mirror:
      
+
      
+
      docker pull docker.paddlepaddle.org/paddle
 
      
 
      
-
      • 
-
    
-
    
-
    
-
    Usage of CPU-only and GPU Images
    
-
    We package PaddlePaddle’s compile environment into a Docker image,
-called the develop image, it contains all compiling tools that
-PaddlePaddle needs. We package compiled PaddlePaddle program into a
-Docker image as well, called the production image, it contains all
-runtime environment that running PaddlePaddle needs. For each version
-of PaddlePaddle, we release both of them. Production image includes
-CPU-only version and a CUDA GPU version and their no-AVX versions.
    
-
    We put the docker images on dockerhub.com. You can find the
-latest versions under “tags” tab at dockerhub.com. If you are in
-China, you can use our Docker image registry mirror to speed up the
-download process. To use it, please replace all paddlepaddle/paddle in
-the commands to docker.paddlepaddle.org/paddle.
    
-
      
-
    1. Production images, this image might have multiple variants:
      
-
        
-
      • GPU/AVX：paddlepaddle/paddle:<version>-gpu
        • 
-
      • GPU/no-AVX：paddlepaddle/paddle:<version>-gpu-noavx
        • 
-
      • CPU/AVX：paddlepaddle/paddle:<version>
        • 
-
      • CPU/no-AVX：paddlepaddle/paddle:<version>-noavx
        • 
-
      
-
      Please be aware that the CPU-only and the GPU images both use the
-AVX instruction set, but old computers produced before 2008 do not
-support AVX.  The following command checks if your Linux computer
-supports AVX:
      
-
      if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
+
      
+
      Download GPU version images:
      
+
      
+
      docker pull paddlepaddle/paddle:latest-gpu
+docker pull docker.paddlepaddle.org/paddle:latest-gpu
 
      
 
      
-
      To run the CPU-only image as an interactive container:
      
-
      docker run -it --rm paddlepaddle/paddle:0.10.0rc2 /bin/bash
+
      
+
      Choose between different BLAS version:
      
+
      
+
      # image using MKL by default
+docker pull paddlepaddle/paddle
+# image using OpenBLAS
+docker pull paddlepaddle/paddle:latest-openblas
 
      
 
      
-
      Above method work with the GPU image too – the recommended way is
-using nvidia-docker.
      
-
      Please install nvidia-docker first following this tutorial.
      
-
      Now you can run a GPU image:
      
-
      nvidia-docker run -it --rm paddlepaddle/paddle:0.10.0rc2-gpu /bin/bash
+
      
+
      If you want to use legacy versions, choose a tag from
+DockerHub
+and run:
      
+
      
+
      docker pull paddlepaddle/paddle:[tag]
+# i.e.
+docker pull docker.paddlepaddle.org/paddle:0.10.0-gpu
 
      
 
      
-
      2. 
-
    2. development image paddlepaddle/paddle:<version>-dev
      
-
      This image has packed related develop tools and runtime
-environment. Users and developers can use this image instead of
-their own local computer to accomplish development, build,
-releasing, document writing etc. While different version of paddle
-may depends on different version of libraries and tools, if you
-want to setup a local environment, you must pay attention to the
-versions.  The development image contains:
      
-
        
-
      • gcc/clang
        • 
-
      • nvcc
        • 
-
      • Python
        • 
-
      • sphinx
        • 
-
      • woboq
        • 
-
      • sshd
        • 
-
      
-
      Many developers use servers with GPUs, they can use ssh to login to
-the server and run docker exec to enter the docker
-container and start their work.  Also they can start a development
-docker image with SSHD service, so they can login to the container
-and start work.
      
-
      3. 
-
    
+
    
 
    
-
    
-
    Train Model Using Python API
    
-
    Our official docker image provides a runtime for PaddlePaddle
-programs. The typical workflow will be as follows:
    
-
    Create a directory as workspace:
    
-
    mkdir ~/workspace
+
    
+
    Launch your training program in Docker
    
+
    Assume that you have already written a PaddlePaddle program
+named train.py under directory /home/work (refer to
+PaddlePaddleBook
+for more samples), then run the following command:
    
+
    
+
    cd /home/work
+docker run -it -v $PWD:/work paddlepaddle/paddle /work/train.py
 
    
 
    
-
    Edit a PaddlePaddle python program using your favourite editor
    
-
    emacs ~/workspace/example.py
-
    
-
    
-
    Run the program using docker:
    
-
    docker run --rm -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2 python /workspace/example.py
-
    
-
    
-
    Or if you are using GPU for training:
    
-
    nvidia-docker run --rm -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2-gpu python /workspace/example.py
-
    
-
    
-
    Above commands will start a docker container by running python
-/workspace/example.py. It will stop once python
-/workspace/example.py finishes.
    
-
    Another way is to tell docker to start a /bin/bash session and
-run PaddlePaddle program interactively:
    
-
    docker run -it -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2 /bin/bash
-# now we are inside docker container
-cd /workspace
-python example.py
-
    
-
    
-
    Running with GPU is identical:
    
-
    nvidia-docker run -it -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2-gpu /bin/bash
-# now we are inside docker container
-cd /workspace
-python example.py
-
    
-
    
-
    
-
    
-
    Develop PaddlePaddle or Train Model Using C++ API
    
-
    We will be using PaddlePaddle development image since it contains all
-compiling tools and dependencies.
    
-
      
-
    1. Build PaddlePaddle develop image
      
-
      Use following command to build PaddlePaddle develop image:
      
-
      git clone https://github.com/PaddlePaddle/Paddle.git && cd Paddle
-docker build -t paddle:dev .
-
      
-
      
-
      2. 
-
    2. Build PaddlePaddle production image
      
-
      There are two steps for building production image, the first step is to run:
      
-
      docker run -v $(pwd):/paddle -e "WITH_GPU=OFF" -e "WITH_AVX=OFF" -e "WITH_TEST=ON" paddle:dev
-
      
-
      
-
      The above command will compile PaddlePaddle and create a Dockerfile for building production image. All the generated files are in the build directory. “WITH_GPU” controls if the generated production image supports GPU. “WITH_AVX” controls if the generated production image supports AVX. “WITH_TEST” controls if the unit test will be generated.
      
-
      The second step is to run:
      
-
      docker build -t paddle:prod -f build/Dockerfile ./build
-
      
-
      
-
      The above command will generate the production image by copying the compiled PaddlePaddle program into the image.
      
-
      3. 
-
    3. Run unit test
      
-
      Following command will run unit test:
      
-
      docker run -it -v $(pwd):/paddle paddle:dev bash -c "cd /paddle/build && ctest"
-
      
-
      
-
      4. 
-
    
+
    
+
    In the above command, -it means run the container interactively;
+-v $PWD:/work means mount the current directory ($PWD will expand
+to current absolute path in Linux) under /work in the container.
+paddlepaddle/paddle to specify image to use; finnally
+/work/train.py is the command to run inside docker.
    
+
    Also, you can go into the container shell, run or debug your code
+interactively:
    
+
    
+
    
+
    NOTE: We did not install vim in the default docker image to reduce the image size, you can run apt-get install -y vim to install it if you need to edit python files.
    
 
    
 
    
-
    PaddlePaddle Book
    
-
    The Jupyter Notebook is an open-source web application that allows
-you to create and share documents that contain live code, equations,
-visualizations and explanatory text in a single browser.
    
-
    PaddlePaddle Book is an interactive Jupyter Notebook for users and developers.
-We already exposed port 8888 for this book. If you want to
+
    PaddlePaddle Book
    
+
    You can create a container serving PaddlePaddle Book using Jupyter Notebook in
+one minute using Docker. PaddlePaddle Book is an interactive Jupyter Notebook
+for users and developers.If you want to
 dig deeper into deep learning, PaddlePaddle Book definitely is your best choice.
    
 
    We provide a packaged book image, simply issue the command:
    
-
    docker run -p 8888:8888 paddlepaddle/book
+
    
+
    docker run -p 8888:8888 paddlepaddle/book
 
    
 
    
+
    
 
    Then, you would back and paste the address into the local browser:
    
-
    http://localhost:8888/
+
    
+
    http://localhost:8888/
 
    
 
    
+
    
 
    That’s all. Enjoy your journey!
    
 
    
-
    
-
    Documentation
    
-
    Paddle Docker images include an HTML version of C++ source code
-generated using woboq code browser.  This makes it easy
-for users to browse and understand the C++ source code.
    
-
    As long as we give the Paddle Docker container a name, we can run an
-additional Nginx Docker container to serve the volume from the Paddle
-container:
    
-
    docker run -d --name paddle-cpu-doc paddle:<version>
-docker run -d --volumes-from paddle-cpu-doc -p 8088:80 nginx
+
    
+
    Train with Docker with GPU
    
+
    We recommend using
+nvidia-docker
+to run GPU training jobs. Please ensure you have latest
+GPU driver installed before move on.
    
+
    
+
    nvidia-docker run -it -v $PWD:/work paddledev/paddle:latest-gpu /bin/bash
+
    
+
    
+
    
+
    NOTE: If you don’t have nvidia-docker installed, try the following method to mount CUDA libs and devices into the container.
    
+
    
+
    export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
+export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
+docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:latest-gpu
+
    
+
    
+
    
+
    About AVX:
    
+
    AVX is a kind of CPU instruction can accelerate PaddlePaddle’s calculations.
+The latest PaddlePaddle Docker image turns AVX on by default, so, if your
+computer doesn’t support AVX, you’ll probably need to
+build with WITH_AVX=OFF.
    
+
    The following command will tell you whether your computer supports AVX.
    
+
    
+
    if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
 
    
 
    
-
    Then we can direct our Web browser to the HTML version of source code
-at http://localhost:8088/paddle/
    
+
    
 
    
 
    
 
@@ -403,10 +343,10 @@ at http://loc
   
     
   
diff --git a/doc/getstarted/build_and_install/index_en.html b/doc/getstarted/build_and_install/index_en.html
index 97f9451697b8edf5f48138f5138bb1896e611e93..13e25a31c48ea63de3cdf7662c7da85783a47b8f 100644
--- a/doc/getstarted/build_and_install/index_en.html
+++ b/doc/getstarted/build_and_install/index_en.html
@@ -34,7 +34,7 @@
         
     
         
-        
+        
          
 
   
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -182,27 +210,33 @@
             
   
    
 
    Install and Build
    
-
    
-
    Install PaddlePaddle
    
+
    
+
    Install Steps
    
+
    You can choose either pip or Docker to complete your install:
    
 
    
 
 
    
-
    
 
    
-
    Build from Source
    
+
    Build from Source
    
 
    
 
    Warning
    
-
    Please use deb package or docker image to install paddle. The building guide is used for hacking or contributing PaddlePaddle source code.
    
+
    We recommend to directly install via above installation steps, you’ll only need to build PaddlePaddle from source when you need a modifed binary.
    
 
    
 
    
 
 
    
 
    
+
    
+
    
+
    FAQ
    
+
    FAQ
    
+
    
 
    
 
 
@@ -212,7 +246,7 @@
   
     
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -86,7 +86,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -110,9 +110,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -137,17 +142,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -179,14 +207,57 @@
             
   
    
 
    GET STARTED
    
+
    
+
    Quick Install
    
+
    You can use pip to install PaddlePaddle with a single command, supports
+CentOS 6 above, Ubuntu 14.04 above or MacOS 10.12, with Python 2.7 installed.
+Simply run the following command to install:
    
+
    
+
    pip install paddlepaddle
+
    
+
    
+
    
+
    If you need to install GPU version, run:
    
+
    
+
    pip install paddlepaddle-gpu
+
    
+
    
+
    
+
    For more details about installation and build:
    
 
    
 
 
    
-
+
    
+
    Quick Start
    
+
    Create a new file called housing.py, and paste this Python
+code:
    
+
    
+
    import paddle.v2 as paddle
+
+# Initialize PaddlePaddle.
+paddle.init(use_gpu=False, trainer_count=1)
+
+# Configure the neural network.
+x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13))
+y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear())
+
+# Infer using provided test data.
+probs = paddle.infer(
+    output_layer=y_predict,
+    parameters=paddle.dataset.uci_housing.model(),
+    input=[item for item in paddle.dataset.uci_housing.test()()])
+
+for i in xrange(len(probs)):
+    print 'Predicted price: ${:,.2f}'.format(probs[i][0] * 1000)
+
    
+
    
+
    
+
    Run python housing.py and voila! It should print out a list of predictions
+for the test housing data.
    
+
    
+
    
 
    
 
 
diff --git a/doc/howto/deep_model/rnn/index_en.html b/doc/howto/deep_model/rnn/index_en.html
index 6c3b311ca5c6f0b37a907658ed2d30322c08fb06..e72452cf798ba163abd76dffcac5208d5aa1257e 100644
--- a/doc/howto/deep_model/rnn/index_en.html
+++ b/doc/howto/deep_model/rnn/index_en.html
@@ -34,8 +34,8 @@
         
     
         
-        
-         
+        
+         
 
   
   
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -182,6 +210,11 @@
             
   
    
 
    RNN Models
    
+
    
+
+
    
 
    
 
 
@@ -191,10 +224,10 @@
   
     
   
diff --git a/doc/howto/deep_model/rnn/rnn_config_en.html b/doc/howto/deep_model/rnn/rnn_config_en.html
index fa9f70dff2622ad7278979e4178cb09ca2dd87d1..34ed1c8601abc00689b5774b3b9ede9c24e57f0d 100644
--- a/doc/howto/deep_model/rnn/rnn_config_en.html
+++ b/doc/howto/deep_model/rnn/rnn_config_en.html
@@ -32,7 +32,10 @@
         
         
-     
+    
+        
+        
+         
 
   
   
@@ -62,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -80,11 +83,11 @@
       
    
       
    
         
-        
    
 
    
 
    PaddlePaddle uses memory to construct step function. Memory is the most important concept when constructing recurrent neural networks in PaddlePaddle. A memory is a state that is used recurrently in step functions, such as \(x_{t+1} = f_x(x_t)\). One memory contains an output and a input. The output of memory at the current time step is utilized as the input of the memory at the next time step. A memory can also has a boot layer, whose output is utilized as the initial value of the memory. In our case, the output of the gated recurrent unit is employed as the output memory. Notice that the name of the layer rnn_out is the same as the name of out_mem. This means the output of the layer rnn_out (\(x_{t+1}\)) is utilized as the output of out_mem memory.
    
@@ -247,43 +267,52 @@ Its output function simply takes \(x_t\)We will use the sequence to sequence model with attention as an example to demonstrate how you can configure complex recurrent neural network models. An illustration of the sequence to sequence model with attention is shown in the following figure.
    
 ../../../_images/encoder-decoder-attention-model.png
 
    In this model, the source sequence \(S = \{s_1, \dots, s_T\}\) is encoded with a bidirectional gated recurrent neural networks. The hidden states of the bidirectional gated recurrent neural network \(H_S = \{H_1, \dots, H_T\}\) is called encoder vector The decoder is a gated recurrent neural network. When decoding each token \(y_t\), the gated recurrent neural network generates a set of weights \(W_S^t = \{W_1^t, \dots, W_T^t\}\), which are used to compute a weighted sum of the encoder vector. The weighted sum of the encoder vector is utilized to condition the generation of the token \(y_t\).
    
-
    The encoder part of the model is listed below. It calls grumemory to represent gated recurrent neural network. It is the recommended way of using recurrent neural network if the network architecture is simple, because it is faster than recurrent_group. We have implemented most of the commonly used recurrent neural network architectures, you can refer to Layers for more details.
    
+
    The encoder part of the model is listed below. It calls grumemory to represent gated recurrent neural network. It is the recommended way of using recurrent neural network if the network architecture is simple, because it is faster than recurrent_group. We have implemented most of the commonly used recurrent neural network architectures, you can refer to api_trainer_config_helpers_layers for more details.
    
 
    We also project the encoder vector to decoder_size dimensional space, get the first instance of the backward recurrent network, and project it to decoder_size dimensional space:
    
 
    # Define the data layer of the source sentence.
-src_word_id = data_layer(name='source_language_word', size=source_dict_dim)
+src_word_id = paddle.layer.data(
+    name='source_language_word',
+    type=paddle.data_type.integer_value_sequence(source_dict_dim))
 # Calculate the word embedding of each word.
-src_embedding = embedding_layer(
+src_embedding = paddle.layer.embedding(
     input=src_word_id,
     size=word_vector_dim,
-    param_attr=ParamAttr(name='_source_language_embedding'))
+    param_attr=paddle.attr.ParamAttr(name='_source_language_embedding'))
 # Apply forward recurrent neural network.
-src_forward = grumemory(input=src_embedding, size=encoder_size)
+src_forward = paddle.networks.simple_gru(
+    input=src_embedding, size=encoder_size)
 # Apply backward recurrent neural network. reverse=True means backward recurrent neural network.
-src_backward = grumemory(input=src_embedding,
-                          size=encoder_size,
-                          reverse=True)
+src_backward = paddle.networks.simple_gru(
+    input=src_embedding, size=encoder_size, reverse=True)
 # Mix the forward and backward parts of the recurrent neural network together.
-encoded_vector = concat_layer(input=[src_forward, src_backward])
+encoded_vector = paddle.layer.concat(input=[src_forward, src_backward])
 
 # Project encoding vector to decoder_size.
-encoder_proj = mixed_layer(input = [full_matrix_projection(encoded_vector)],
-                           size = decoder_size)
+encoded_proj = paddle.layer.mixed(
+    size=decoder_size,
+    input=paddle.layer.full_matrix_projection(encoded_vector))
 
 # Compute the first instance of the backward RNN.
-backward_first = first_seq(input=src_backward)
+backward_first = paddle.layer.first_seq(input=src_backward)
 
 # Project the first instance of backward RNN to decoder size.
-decoder_boot = mixed_layer(input=[full_matrix_projection(backward_first)], size=decoder_size, act=TanhActivation())
+decoder_boot = paddle.layer.mixed(
+   size=decoder_size,
+   act=paddle.activation.Tanh(),
+   input=paddle.layer.full_matrix_projection(backward_first))
 
    
 
    
 
    The decoder uses recurrent_group to define the recurrent neural network. The step and output functions are defined in gru_decoder_with_attention:
    
-
    group_inputs=[StaticInput(input=encoded_vector,is_seq=True),
-              StaticInput(input=encoded_proj,is_seq=True)]
-trg_embedding = embedding_layer(
-    input=data_layer(name='target_language_word',
-                     size=target_dict_dim),
-    size=word_vector_dim,
-    param_attr=ParamAttr(name='_target_language_embedding'))
+
    group_input1 = paddle.layer.StaticInput(input=encoded_vector, is_seq=True)
+group_input2 = paddle.layer.StaticInput(input=encoded_proj, is_seq=True)
+group_inputs = [group_input1, group_input2]
+trg_embedding = paddle.layer.embedding(
+        input=paddle.layer.data(
+            name='target_language_word',
+            type=paddle.data_type.integer_value_sequence(target_dict_dim)),
+        size=word_vector_dim,
+        param_attr=paddle.attr.ParamAttr(name='_target_language_embedding'))
+    group_inputs.append(trg_embedding)
 group_inputs.append(trg_embedding)
 
 # For decoder equipped with attention mechanism, in training,
@@ -292,9 +321,10 @@ Its output function simply takes \(x_t\)# StaticInput means the same value is utilized at different time steps.
 # Otherwise, it is a sequence input. Inputs at different time steps are different.
 # All sequence inputs should have the same length.
-decoder = recurrent_group(name=decoder_group_name,
-                          step=gru_decoder_with_attention,
-                          input=group_inputs)
+decoder = paddle.layer.recurrent_group(
+        name=decoder_group_name,
+        step=gru_decoder_with_attention,
+        input=group_inputs)
 
    
 
    
 
    The implementation of the step function is listed as below. First, it defines the memory of the decoder network. Then it defines attention, gated recurrent unit step function, and the output function:
    
@@ -302,27 +332,32 @@ Its output function simply takes \(x_t\)# Defines the memory of the decoder.
     # The output of this memory is defined in gru_step.
     # Notice that the name of gru_step should be the same as the name of this memory.
-    decoder_mem = memory(name='gru_decoder',
-                         size=decoder_size,
-                         boot_layer=decoder_boot)
+    decoder_mem = paddle.layer.memory(
+        name='gru_decoder', size=decoder_size, boot_layer=decoder_boot)
     # Compute attention weighted encoder vector.
-    context = simple_attention(encoded_sequence=enc_vec,
-                               encoded_proj=enc_proj,
-                               decoder_state=decoder_mem)
+    context = paddle.networks.simple_attention(
+        encoded_sequence=enc_vec,
+        encoded_proj=enc_proj,
+        decoder_state=decoder_mem)
     # Mix the current word embedding and the attention weighted encoder vector.
-    decoder_inputs = mixed_layer(inputs = [full_matrix_projection(context),
-                                           full_matrix_projection(current_word)],
-                                 size = decoder_size * 3)
+    decoder_inputs = paddle.layer.mixed(
+        size=decoder_size * 3,
+        input=[
+            paddle.layer.full_matrix_projection(input=context),
+            paddle.layer.full_matrix_projection(input=current_word)
+        ])
     # Define Gated recurrent unit recurrent neural network step function.
-    gru_step = gru_step_layer(name='gru_decoder',
-                              input=decoder_inputs,
-                              output_mem=decoder_mem,
-                              size=decoder_size)
+    gru_step = paddle.layer.gru_step(
+        name='gru_decoder',
+        input=decoder_inputs,
+        output_mem=decoder_mem,
+        size=decoder_size)
     # Defines the output function.
-    out = mixed_layer(input=[full_matrix_projection(input=gru_step)],
-                      size=target_dict_dim,
-                      bias_attr=True,
-                      act=SoftmaxActivation())
+    out = paddle.layer.mixed(
+        size=target_dict_dim,
+        bias_attr=True,
+        act=paddle.activation.Softmax(),
+        input=paddle.layer.full_matrix_projection(input=gru_step))
     return out
 
    
 
    
@@ -340,44 +375,36 @@ Its output function simply takes \(x_t\)max_length: the maximum length of the generated sentences.
 
 
-
  • use seqtext_printer_evaluator to print text according to index matrix and dictionary. This function needs to set:
      
-
    • id_input: the integer ID of the data, used to identify the corresponding output in the generated files.
      • 
-
    • dict_file: the dictionary file for converting word id to word.
      • 
-
    • result_file: the path of the generation result file.
      • 
-
    
-
    • 
 
 
    The code is listed below:
    
-
    group_inputs=[StaticInput(input=encoded_vector,is_seq=True),
-              StaticInput(input=encoded_proj,is_seq=True)]
+
    group_input1 = paddle.layer.StaticInput(input=encoded_vector, is_seq=True)
+group_input2 = paddle.layer.StaticInput(input=encoded_proj, is_seq=True)
+group_inputs = [group_input1, group_input2]
 # In generation, decoder predicts a next target word based on
 # the encoded source sequence and the last generated target word.
 # The encoded source sequence (encoder's output) must be specified by
 # StaticInput which is a read-only memory.
 # Here, GeneratedInputs automatically fetchs the last generated word,
 # which is initialized by a start mark, such as <s>.
-trg_embedding = GeneratedInput(
-    size=target_dict_dim,
-    embedding_name='_target_language_embedding',
-    embedding_size=word_vector_dim)
+trg_embedding = paddle.layer.GeneratedInput(
+        size=target_dict_dim,
+        embedding_name='_target_language_embedding',
+        embedding_size=word_vector_dim)
 group_inputs.append(trg_embedding)
-beam_gen = beam_search(name=decoder_group_name,
-                       step=gru_decoder_with_attention,
-                       input=group_inputs,
-                       bos_id=0, # Beginnning token.
-                       eos_id=1, # End of sentence token.
-                       beam_size=beam_size,
-                       max_length=max_length)
-
-seqtext_printer_evaluator(input=beam_gen,
-                          id_input=data_layer(name="sent_id", size=1),
-                          dict_file=trg_dict_path,
-                          result_file=gen_trans_file)
-outputs(beam_gen)
+beam_gen = paddle.layer.beam_search(
+        name=decoder_group_name,
+        step=gru_decoder_with_attention,
+        input=group_inputs,
+        bos_id=0, # Beginnning token.
+        eos_id=1, # End of sentence token.
+        beam_size=beam_size,
+        max_length=max_length)
+
+return beam_gen
 
    
 
    
-
    Notice that this generation technique is only useful for decoder like generation process. If you are working on sequence tagging tasks, please refer to Semantic Role labeling Tutorial for more details.
    
-
    The full configuration file is located at demo/seqToseq/seqToseq_net.py.
    
+
    Notice that this generation technique is only useful for decoder like generation process. If you are working on sequence tagging tasks, please refer to book/06.understand_sentiment for more details.
    
+
    The full configuration file is located at book/08.machine_translation/train.py .
    
 
    
 
    
 
@@ -386,6 +413,15 @@ Its output function simply takes \(x_t\)
           
    
   
+    
+  
 
   
    
 
diff --git a/doc/howto/dev/build_en.html b/doc/howto/dev/build_en.html
new file mode 100644
index 0000000000000000000000000000000000000000..52d76382fd9d764526f96767b313f88cbbc08f30
--- /dev/null
+++ b/doc/howto/dev/build_en.html
@@ -0,0 +1,385 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Build using Docker — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+    
+        
+        
+         
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Build using Docker
    
+
    
+
    What Developers Need
    
+
    To contribute to PaddlePaddle, you need
    
+
      
+
    1. A computer – Linux, BSD, Windows, MacOS, and
      2. 
+
    2. Docker.
      3. 
+
    
+
    Nothing else.  Not even Python and GCC, because you can install all build tools into a Docker image.  We run all the tools by running this image.
    
+
    
+
    
+
    General Process
    
+
      
+
    1. Retrieve source code.
      
+
      git clone https://github.com/paddlepaddle/paddle
+
      
+
      
+
      2. 
+
    2. Install build tools into a Docker image.
      
+
      cd paddle; docker build -t paddle:dev .
+
      
+
      
+
      Please be aware of the . at the end of the command, which refers to the ./Dockerfile file.  docker build follows instructions in this file to create a Docker image named paddle:dev, and installs building tools into it.
      
+
      3. 
+
    3. Build from source.
      
+
      This following command starts a Docker container that executes the Docker image paddle:dev, mapping the current directory to /paddle/ in the container, and runs the default entry-point build.sh as specified in the Dockefile.  build.sh invokes cmake and make to build PaddlePaddle source code, which had been mapped to /paddle, and writes outputs to /paddle/build, which maps to build in the current source directory on the computer.
      
+
      docker run -v $PWD:/paddle paddle:dev
+
      
+
      
+
      Above command builds a CUDA-enabled version.  If we want to build a CPU-only version, we can type
      
+
      docker run -e WITH_GPU=OFF -v $PWD:/paddle paddle:dev
+
      
+
      
+
      4. 
+
    4. Run unit tests.
      
+
      To run all unit tests using the first GPU of a node:
      
+
      NV_GPU=0 nvidia-docker run -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest"
+
      
+
      
+
      If we used WITH_GPU=OFF at build time, it generates only CPU-based unit tests, and we don’t need nvidia-docker to run them.  We can just run
      
+
      docker run -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest"
+
      
+
      
+
      Sometimes we want to run a specific unit test, say memory_test, we can run
      
+
      nvidia-docker run -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest -V -R memory_test"
+
      
+
      
+
      5. 
+
    5. Clean Build.
      
+
      Sometimes, we might want to clean all thirt-party dependents and built binaries.  To do so, just
      
+
      rm -rf build
+
      
+
      
+
      6. 
+
    
+
    
+
    
+
    Docker, Or Not?
    
+
      
+
    • What is Docker?
      
+
      If you haven’t heard of it, consider it something like Python’s virtualenv.
      
+
      • 
+
    • Docker or virtual machine?
      
+
      Some people compare Docker with VMs, but Docker doesn’t virtualize any hardware nor running a guest OS, which means there is no compromise on the performance.
      
+
      • 
+
    • Why Docker?
      
+
      Using a Docker image of build tools standardizes the building environment, which makes it easier for others to reproduce your problems and to help.
      
+
      Also, some build tools don’t run on Windows or Mac or BSD, but Docker runs almost everywhere, so developers can use whatever computer they want.
      
+
      • 
+
    • Can I choose not to use Docker?
      
+
      Sure, you don’t have to install build tools into a Docker image; instead, you can install them in your local computer.  This document exists because Docker would make the development way easier.
      
+
      • 
+
    • How difficult is it to learn Docker?
      
+
      It takes you ten minutes to read an introductory article and saves you more than one hour to install all required build tools, configure them, especially when new versions of PaddlePaddle require some new tools.  Not even to mention the time saved when other people trying to reproduce the issue you have.
      
+
      • 
+
    • Can I use my favorite IDE?
      
+
      Yes, of course.  The source code resides on your local computer, and you can edit it using whatever editor you like.
      
+
      Many PaddlePaddle developers are using Emacs.  They add the following few lines into their ~/.emacs configure file:
      
+
      (global-set-key "\C-cc" 'compile)
+(setq compile-command
+ "docker run --rm -it -v $(git rev-parse --show-toplevel):/paddle paddle:dev")
+
      
+
      
+
      so they could type Ctrl-C and c to build PaddlePaddle from source.
      
+
      • 
+
    • Does Docker do parallel building?
      
+
      Our building Docker image runs a Bash script, which calls make -j$(nproc) to starts as many processes as the number of your CPU cores.
      
+
      • 
+
    
+
    
+
    
+
    Some Gotchas
    
+
      
+
    • Docker requires sudo
      
+
      An owner of a computer has the administrative privilege, a.k.a., sudo, and Docker requires this privilege to work properly.  If you use a shared computer for development, please ask the administrator to install and configure Docker.  We will do our best to support rkt, another container technology that doesn’t require sudo.
      
+
      • 
+
    • Docker on Windows/MacOS builds slowly
      
+
      On Windows and MacOS, Docker containers run in a Linux VM.  You might want to give this VM some more memory and CPUs so to make the building efficient.  Please refer to this issue for details.
      
+
      • 
+
    • Not enough disk space
      
+
      Examples in this article uses option --rm with the docker run command.  This option ensures that stopped containers do not exist on hard disks.  We can use docker ps -a to list all containers, including stopped.  Sometimes docker build generates some intermediate dangling images, which also take disk space.  To clean them, please refer to this article.
      
+
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/howto/dev/contribute_to_paddle_en.html b/doc/howto/dev/contribute_to_paddle_en.html
index 4c2dcae5a29fb65c5c69f6f39cf722621a7ba23d..da3aa543e2a8853e02ab136b13ac104e0e809e25 100644
--- a/doc/howto/dev/contribute_to_paddle_en.html
+++ b/doc/howto/dev/contribute_to_paddle_en.html
@@ -34,7 +34,7 @@
         
     
         
-        
+        
          
 
   
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -182,136 +210,132 @@
             
   
    
 
    Contribute Code
    
-
    We sincerely appreciate your contributions. You can use fork and pull request
-workflow to merge your code.
    
-
    
-
    Code Requirements
    
-
      
-
    • Your code must be fully documented by
-doxygen style.
      • 
-
    • Make sure the compiler option WITH_STYLE_CHECK is on and the compiler
-passes the code style check.
      • 
-
    • All code must have unit test.
      • 
-
    • Pass all unit tests.
      • 
-
    
-
    The following tutorial guides you into submitting your contibution.
    
-
    
-
    
-
    Creating a Fork
    
-
    Just head over to the GitHub page and click the “Fork” button.
-It’s just that simple.
    
-
    
-
    
-
    Clone
    
-
    Paddle is currently using git-flow branching model.
-The develop is the main branch, and other user’s branches are feature branches.
    
-
    Once you’ve created a fork, you can use your favorite git client to clone your
-repo or just head straight to the command line:
    
-
    # Clone your fork to your local machine
-git clone --branch develop https://github.com/USERNAME/Paddle.git
+
    We sincerely appreciate your contribution.  This document explains our workflow and work style.
    
+
    
+
    Workflow
    
+
    PaddlePaddle uses this Git branching model.  The following steps guide usual contributions.
    
+
      
+
    1. Fork
      
+
      Our development community has been growing fastly; it doesn’t make sense for everyone to write into the official repo.  So, please file Pull Requests from your fork.  To make a fork,  just head over to the GitHub page and click the “Fork” button.
      
+
      2. 
+
    2. Clone
      
+
      To make a copy of your fork to your local computers, please run
      
+
      git clone https://github.com/your-github-account/paddle
+cd paddle
 
      
 
      
-
      If your repository doesn’t contain develop branch, just create it by your own.
      
-
      git clone https://github.com/USERNAME/Paddle.git Paddle
-cd Paddle
-git checkout -b develop  # create develop branch.
-git remote add upstream https://github.com/PaddlePaddle/Paddle.git  # add upstream to baidu/Paddle
-git pull upstream develop  # update to upstream
+
      3. 
+
    3. Create the local feature branch
      
+
      For daily works like adding a new feature or fixing a bug, please open your feature branch before coding:
      
+
      git checkout -b my-cool-stuff
 
      
 
      
-
      Then you can start to develop by making a local developement branch
      
-
      git checkout -b MY_COOL_STUFF_BRANCH
+
      4. 
+
    4. Commit
      
+
      Before issuing your first git commit command, please install pre-commit by running the following commands:
      
+
      pip install pre-commit
+pre-commit install
 
      
 
      
-
    
-
    
-
    Using pre-commit hook
    
-
    Paddle developers use pre-commit tool to manage git
-pre-commit hooks. It can help us format source codes (cpp, python), check some
-basic thing before commit (only one EOL for each file, do not add a huge file
-in git). pre-commit tests is a part of unit tests in Travis-CI now, every
-PR doesn’t fit hook can not be merged into Paddle.
    
-
    To use pre-commit, you should install it by
-pip install pre-commit, and currently, Paddle uses clang-format to format
-c/cpp sources. Please make sure clang-format 3.8+ installed.
    
-
    Then just run pre-commit install in your Paddle clone directory. When you
-commit your code, the pre-commit hook will check the local code if there is
-anything not suitable to commit, and so on.
    
-
    
-
    
-
    Commit
    
-
    Commit your changes by following command lines:
    
-
    # show the working tree status
-git status
-# add modified files
-git add xx
-env EDITOR=vim git commit  # You can write your comments by vim/nano/emacs.
+
    Our pre-commit configuration requires clang-format 3.8 for auto-formating C/C++ code and yapf for Python.
    
+
    Once installed, pre-commit checks the style of code and documentation in every commit.  We will see something like the following when you run git commit:
    
+
    ➜  git commit
+CRLF end-lines remover...............................(no files to check)Skipped
+yapf.................................................(no files to check)Skipped
+Check for added large files..............................................Passed
+Check for merge conflicts................................................Passed
+Check for broken symlinks................................................Passed
+Detect Private Key...................................(no files to check)Skipped
+Fix End of Files.....................................(no files to check)Skipped
+clang-formater.......................................(no files to check)Skipped
+[my-cool-stuff c703c041] add test file
+ 1 file changed, 0 insertions(+), 0 deletions(-)
+ create mode 100644 233
 
    
 
    
-
    The first line of commit infomation is the title. The second and later lines
-are the details if any.
    
+
+
  • Build and test
    
+
    Users can build PaddlePaddle natively on Linux and Mac OS X.  But to unify the building environment and to make it easy for debugging, the recommended way is using Docker.
    
+
    • 
+
  • Keep pulling
    
+
    An experienced Git user pulls from the official repo often – daily or even hourly, so they notice conflicts with others work early, and it’s easier to resolve smaller conflicts.
    
+
    git remote add upstream https://github.com/PaddlePaddle/Paddle
+git pull upstream develop
+
    
 
    
-
    
-
    Keeping Fork Up to Date
    
-
    Before pull your request, you should sync your code from the latest PaddlePaddle.
-To do this, you’ll need to add a remote at first:
    
-
    # see the current configured remote repository
-git remote -v
-# add upstream repository
-git remote add upstream https://github.com/PaddlePaddle/Paddle.git
-# verify the new upstream
-git remote -v
+
    • 
+
  • Push and file a pull request
    
+
    You can “push” your local work into your forked repo:
    
+
    git push origin my-cool-stuff
 
    
 
    
-
    Update your fork with the latest upstream changes:
    
-
    git pull --rebase upstream develop
+
    The push allows you to create a pull request, requesting owners of this official repo to pull your change into the official one.
    
+
    To create a pull request, please follow these steps.
    
+
    If your change is for fixing an issue, please write “Fixes <issue-URL>” in the description section of your pull request.  Github would close the issue when the owners merge your pull request.
    
+
    Please remember to specify some reviewers for your pull request.  If you don’t know who are the right ones, please follow Github’s recommendation.
    
+
    • 
+
+
      
+
    1. Delete local and remote branches
      
+
      To keep your local workspace and your fork clean, you might want to remove merged branches:
      
+
      git push origin :my-cool-stuff
+git checkout develop
+git pull upstream develop
+git branch -d my-cool-stuff
 
      
 
      
-
      If there are no unique commits locally, git will simply perform a fast-forward.
-However, if you have been making changes (in the vast majority of cases you
-probably shouldn’t be), you may have to deal with conflicts.
      
-
      Now, your local master branch is up-to-date with everything modified upstream.
      
+
      2. 
+
    
+
    
+
    Code Review
    
+
      
+
    • Please feel free to ping your reviewers by sending them the URL of your pull request via IM or email.  Please do this after your pull request passes the CI.
      • 
+
    • Please answer reviewers’ every comment.  If you are to follow the comment, please write “Done”; please give a reason otherwise.
      • 
+
    • If you don’t want your reviewers to get overwhelmed by email notifications, you might reply their comments by in a batch.
      • 
+
    • Reduce the unnecessary commits.  Some developers commit often.  It is recommended to append a sequence of small changes into one commit by running git commit --amend instead of git commit.
      • 
+
    
 
    
-
    
-
    Push to GitHub
    
-
    # push to your repository in Github
-git push -u origin MY_COOL_STUFF_BRANCH  # create remote branch MY_COOL_STUFF_BRANCH to origin.
-
    
 
    
+
    
+
    Coding Standard
    
+
    
+
    Code Style
    
+
    Our C/C++ code follows the Google style guide.
    
+
    Our Python code follows the PEP8 style guide.
    
+
    Our build process helps to check the code style.  In build.sh, the entry point of our builder Docker image, the CMake argument WITH_STYLE_CHECK is set to ON by default.  This flag is on
    
+
    Please install pre-commit, which automatically reformat the changes to C/C++ and Python code whenever we run git commit.  To check the whole codebase, we can run the command pre-commit run -a, as in the check_style.sh file, which is invoked by our Travis CI configuration.
    
 
    
-
    
-
    Pull Request
    
-
    Go to the page for your fork on GitHub, select your development branch,
-and click the pull request button.
    
+
    
+
    Unit Tests
    
+
    Please remember to add related unit tests.
    
+
 
    
-
    
-
    Update your pull request with the lastest version
    
-
    During the code review, your pull request may become stale because new commits in
-baidu/Paddle. GitHub allows autmotic update if there is no conflict. You can do this
-by clicking the “Update Branch” button in your pull request page. However, in the case
-of conflict, you need to do the update manually. You need to do the following on
-your local repository:
    
-
    git checkout MY_COOL_STUFF_BRANCH
-git pull upstream develop
-# You may need to resolve the conflict according to the git prompt.
-# Make and test your code.
-git push origin MY_COOL_STUFF_BRANCH
+
    
+
    Writing Logs
    
+
    We use glog for logging in our C/C++ code.
    
+
    For general information, please use LOG.  For debug information, please use VLOG.  The reason is at here.
    
+
    VLOG requires a verbose level parameter.  For example:
    
+
    VLOG(3) << "Operator FC is taking " << num_inputs << "inputs."
 
    
 
    
-
    Now your Pull Request is updated with the latest version.
    
-
    
-
    
-
    Revise your pull request
    
-
    When you revise your pull request according to reviewer’s comments, please use ‘git commit’ instead of ‘git commit –amend’ to commit your changes so that the reviewers can see the difference between the new pull requrest and the old pull request.
    
-
    The possible commands are
    
-
    git checkout MY_COOL_STUFF_BRANCH
-git pull upstream develop   # update local to newest code base.
-# May be some conflicts will occured.
-# And develop your cool stuff
-env EDITOR=vim git commit  # add your revise log
-git push origin MY_COOL_STUFF_BRANCH
+
    When we run a PaddlePaddle application or test, we can specify a verbose threshold.  For example:
    
+
    GLOG_vmodule=buddy_allocator=2 \
+GLOG_v=10 \
+python \
+../python/paddle/v2/framework/tests/test_recurrent_op.py
 
    
 
    
+
    This will enable VLOG messages generated by buddy_allocator.{h,cc} and in the verbose range of 0 to 3, so you will see above example VLOG message, which is in level 3.  This suggests that we output overall messages in lower verbose levels, so they display with higher probability.  When coding C++, please follow the verbose level convention as follows:
    
+
+
    
 
    
 
    
 
@@ -322,7 +346,7 @@ git push origin MY_COOL_STUFF_BRANCH
   
     
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -203,7 +231,7 @@ Fully connected layer takes a dense input vector with dimension \(c(y)\), then
    
 
    
 \[\frac{\partial c(y)}{\partial x} = \frac{\partial c(y)}{\partial y} \frac{\partial y}{\partial x}\]
    
-
    Suppose \(z = f(W^T x + b)\), then
    
+
    Suppose \(z = W^T x + b\), then
    
 
    
 \[\frac{\partial y}{\partial z} = \frac{\partial f(z)}{\partial z}\]
    
 
    This derivative can be automatically computed by our base layer class.
    
diff --git a/doc/howto/dev/new_op_en.html b/doc/howto/dev/new_op_en.html
new file mode 100644
index 0000000000000000000000000000000000000000..703d6ede47d3a01e1cf44bc370b81b13596acb9a
--- /dev/null
+++ b/doc/howto/dev/new_op_en.html
@@ -0,0 +1,580 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  How to write a new operator — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • How to write a new operator
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    How to write a new operator
    
+
+
    
+
    Background
    
+
    Here are the base types needed. For details, please refer to the design docs.
    
+
      
+
    • framework::OperatorBase: Operator (Op)base class.
      • 
+
    • framework::OpKernel: Base class for Op computation.
      • 
+
    • framework::OperatorWithKernel: Inherited from OperatorBase, describing an operator with computation.
      • 
+
    • class OpProtoAndCheckerMaker: Describes an Operator’s input, output, attributes and description, mainly used to interface with Python API.
      • 
+
    
+
    An operator can be differentiated by whether in has kernel methods. An operator with kernel inherits from OperatorWithKernel while the ones without inherit from OperatorBase. This tutorial focuses on implementing operators with kernels. In short, an operator includes the following information:
    
+
    Information           | Where is it defined
+————–  | :———————-
+OpProtoMake definition  | .ccfiles, Backward Op does not need an OpProtoMake interface.
+Op definition           | .cc files
+Kernel implementation       | The kernel methods shared between CPU and GPU are defined in .h files. CPU-specific kernels live in .cc files, while GPU-specific kernels are implemented in .cufiles.
+Registering the Op           | Ops are registered in .cc files; For Kernel registration, .cc files contain the CPU implementation, while .cu files contain the GPU implementation.
    
+
    New Operator implementations are added to the list paddle/operators, with file names in the format *_op.h (if applicable), *_op.cc, *_op.cu (if applicable).** The system will use the naming scheme to automatically build operators and their corresponding Python extensions. **
    
+
    Let’s take matrix multiplication operator, MulOp, as an example to introduce the writing of an Operator with Kernel.
    
+
    
+
    
+
    Implementing C++ Types
    
+
    
+
    1. Defining Class ProtoMaker
    
+
    Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output.
    
+
    First, define ProtoMaker to describe the Operator’s input, output, and additional comments:
    
+
    class MulOpMaker : public framework::OpProtoAndCheckerMaker {
+ public:
+  MulOpMaker(framework::OpProto *proto, framework::OpAttrChecker *op_checker)
+      : OpProtoAndCheckerMaker(proto, op_checker) {
+    AddInput("X", "(Tensor), 2D tensor of size (M x K)");
+    AddInput("Y", "(Tensor), 2D tensor of size (K x N)");
+    AddOutput("Out", "(Tensor), 2D tensor of size (M x N)");
+    AddComment(R"DOC(
+Two Element Mul Operator.
+The equation is: Out = X * Y
+)DOC");
+  }
+};
+
    
+
    
+
    MulOpMakeris inherited fromframework::OpProtoAndCheckerMaker, consisting of 2 variables in the constructor：
    
+
      
+
    • framework::OpProto stores Operator input and variable attribute, used for generating Python API interfaces.
      • 
+
    • framework::OpAttrChecker is used to validate variable attributes.
      • 
+
    
+
    The constructor utilizes AddInput, AddOutput, and AddComment, so that the corresponding information will be added to OpProto.
    
+
    The code above adds two inputs X and Y to MulOp, an output Out, and their corresponding descriptions, in accordance to Paddle’s naming convention.
    
+
    An additional example ScaleOp is implemented as follows:
    
+
    template <typename AttrType>
+class ScaleOpMaker : public framework::OpProtoAndCheckerMaker {
+ public:
+  ScaleOpMaker(framework::OpProto *proto, framework::OpAttrChecker *op_checker)
+      : OpProtoAndCheckerMaker(proto, op_checker) {
+    AddInput("X", "The input tensor of scale operator.").NotInGradient();
+    AddOutput("Out", "The output tensor of scale operator.").NotInGradient();
+    AddComment(R"DOC(Scale operator
+The equation is: Out = scale*X
+)DOC");
+    AddAttr<AttrType>("scale", "scale of scale operator.").SetDefault(1.0);
+  }
+};
+
    
+
    
+
    There are two changes in this example:
    
+
      
+
    • AddInput("X","...").NotInGradient() expresses that input X is not involved in ScaleOp‘s corresponding computation. If an input to an operator is not participating in back-propagation, please explicitly set .NotInGradient().
      • 
+
    • AddAttr<AttrType>("scale", "...").SetDefault(1.0);  adds scaleconstant as an attribute, and sets the default value to 1.0.
      • 
+
    
+
    
+
    
+
    2. Defining Operator
    
+
    The following code defines the interface for MulOp:
    
+
    class MulOp : public framework::OperatorWithKernel {
+ public:
+  using framework::OperatorWithKernel::OperatorWithKernel;
+
+ protected:
+  void InferShape(const framework::InferShapeContext &ctx) const override {
+    auto dim0 = ctx.Input<Tensor>("X")->dims();
+    auto dim1 = ctx.Input<Tensor>("Y")->dims();
+    PADDLE_ENFORCE_EQ(dim0.size(), 2,
+                      "input X(%s) should be a tensor with 2 dims, a matrix",
+                      ctx.op_.Input("X"));
+    PADDLE_ENFORCE_EQ(dim1.size(), 2,
+                      "input Y(%s) should be a tensor with 2 dims, a matrix",
+                      ctx.op_.Input("Y"));
+    PADDLE_ENFORCE_EQ(
+        dim0[1], dim1[0],
+        "First matrix's width must be equal with second matrix's height.");
+    ctx.Output<Tensor>("Out")->Resize({dim0[0], dim1[1]});
+  }
+};
+
    
+
    
+
    MulOp is inherited from OperatorWithKernel. Its public member
    
+
    using framework::OperatorWithKernel::OperatorWithKernel;
+
    
+
    
+
    expresses an operator constructor using base class OperatorWithKernel, alternatively written as
    
+
    MulOp(const std::string &type, const framework::VariableNameMap &inputs,
+      const framework::VariableNameMap &outputs,
+      const framework::AttributeMap &attrs)
+  : OperatorWithKernel(type, inputs, outputs, attrs) {}
+
    
+
    
+
    InferShape interface needs to be re-written.InferShape is a constant method and cannot modify Op’s member variables, its constant member const framework::InferShapeContext &ctx can be used to extract input, output, and attributes. It functions to
    
+
      
+
    • 1). validate and error out early: it checks input data dimensions and types.
      • 
+
    • 2). configures the tensor shape in the output.
      • 
+
    
+
    Usually OpProtoMaker and Op‘s type definitions are written in .cc files, which also include the registration methods introduced later.
    
+
    
+
    
+
    3. Defining OpKernel
    
+
    MulKernel inherits framework::OpKernel, which includes the following templates:
    
+
      
+
    • typename Place denotes device type. When different devices, namely the CPU and the GPU, share the same kernel, this template needs to be added. If they don’t share kernels, this must not be added. An example of a non-sharing kernel is OnehotCrossEntropyOpKernel.
      • 
+
    • typename T denotes data type, such as float or double.
      • 
+
    
+
    MulKernel types need to rewrite the interface for Compute.
    
+
      
+
    • Compute takes one input variable const framework::ExecutionContext& context.
      • 
+
    • Compared with InferShapeContext, ExecutionContext includes device types, and can similarly extract input, output, and attribute variables.
      • 
+
    • Compute implements the computation logics of an OpKernel.
      • 
+
    
+
    MulKernel‘s implementation of Compute is as follows:
    
+
    template <typename Place, typename T>
+class MulKernel : public framework::OpKernel {
+public:
+void Compute(const framework::ExecutionContext& context) const override {
+  auto* X = context.Input<Tensor>("X");
+  auto* Y = context.Input<Tensor>("Y");
+  auto* Z = context.Output<Tensor>("Out");
+  Z->mutable_data<T>(context.GetPlace());
+  auto* device_context =
+      const_cast<platform::DeviceContext*>(context.device_context_);
+  math::matmul<Place, T>(*X, false, *Y, false, 1, Z, 0, device_context);
+}
+};
+
    
+
    
+
    Note that different devices (CPU, GPU)share an Op definition; whether or not they share the same OpKernel depends on whether Compute calls functions that support both devices.
    
+
    MulOp‘s CPU and GPU share the same Kernel. A non-sharing  OpKernel example can be seen in OnehotCrossEntropyOpKernel.
    
+
    To ease the writing of OpKernel compute, and for reusing code cross-device, Eigen-unsupported Tensor module is used to implement Compute interface. To learn about how the Eigen library is used in PaddlePaddle, please see usage document.
    
+
    This concludes the forward implementation of an operator. Next its operation and kernel need to be registered in a .cc file.
    
+
    The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. Note that a backward operator does not include a ProtoMaker.
    
+
    
+
    
+
    4. Registering Operator
    
+
      
+
    • In .cc files, register forward and backward operator classes and the CPU kernel.
      
+
      namespace ops = paddle::operators;
+REGISTER_OP(mul, ops::MulOp, ops::MulOpMaker, mul_grad, ops::MulOpGrad);
+REGISTER_OP_CPU_KERNEL(mul, ops::MulKernel<paddle::platform::CPUPlace, float>);
+REGISTER_OP_CPU_KERNEL(mul_grad,
+              ops::MulGradKernel<paddle::platform::CPUPlace, float>);
+
      
+
      
+
      In that code block,
      
+
        
+
      • REGISTER_OP registers the ops::MulOp class, type named mul, its type ProtoMaker is ops::MulOpMaker, registering ops::MulOpGrad as mul_grad.
        • 
+
      • REGISTER_OP_WITHOUT_GRADIENT registers an operator without gradient.
        • 
+
      • REGISTER_OP_CPU_KERNEL registers ops::MulKernel class and specialized template types paddle::platform::CPUPlace and float, which also registers ops::MulGradKernel.
        • 
+
      
+
      • 
+
    
+
      
+
    • Registering GPU Kernel in .cu files
      
+
        
+
      • Note that if GPU Kernel is implemented using the Eigen unsupported module, then on top of .cu, a macro definition #define EIGEN_USE_GPU is needed, such as
        • 
+
      
+
      // if use Eigen unsupported module before include head files
+#define EIGEN_USE_GPU
+
+namespace ops = paddle::operators;
+REGISTER_OP_GPU_KERNEL(mul, ops::MulKernel<paddle::platform::GPUPlace, float>);
+REGISTER_OP_GPU_KERNEL(mul_grad,
+                       ops::MulGradKernel<paddle::platform::GPUPlace, float>);
+
      
+
      
+
      • 
+
    
+
    
+
    
+
    5. Compilation
    
+
    Run the following commands to compile.
    
+
    make mul_op
+
    
+
    
+
    
+
    
+
    
+
    Python Binding
    
+
    The system will automatically bind to Python and link it to a generated library.
    
+
    
+
    
+
    Unit Tests
    
+
    Unit tests for an operator include
    
+
      
+
    1. comparing a forward operator’s implementations on different devices,
      2. 
+
    2. comparing a backward operator’s implementation on different devices, and
      3. 
+
    3. a scaling test for the backward operator.
      4. 
+
    
+
    Here, we introduce the unit tests for MulOp.
    
+
    
+
    Testing Forward Operators
    
+
    A forward operator unit test inherits unittest.TestCase and defines metaclass __metaclass__ = OpTestMeta. More concrete tests are performed in OpTestMeta. Testing a forward operator requires the following:
    
+
      
+
    1. Defining input, output and relevant attributes in setUp method.
      2. 
+
    2. Generating random input data.
      3. 
+
    3. Implementing the same computation logic in a Python script:
      4. 
+
    
+
    import unittest
+import numpy as np
+from gradient_checker import GradientChecker, create_op
+from op_test_util import OpTestMeta
+
+class TestMulOp(unittest.TestCase):
+    __metaclass__ = OpTestMeta
+
+    def setUp(self):
+        self.type = "mul"
+        self.inputs = {
+            'X': np.random.random((32, 84)).astype("float32"),
+            'Y': np.random.random((84, 100)).astype("float32")
+        }
+        self.outputs = {'Out': np.dot(self.inputs['X'], self.inputs['Y'])}
+
    
+
    
+
    Get its output, and compare it with the forward operator’s own output.
    
+
    The code above first loads required packages. In addition, we have
    
+
      
+
    • self.type = "mul" defines the type that is identical to what the operator’s registered type.
      • 
+
    • self.inputs defines input, with type numpy.array and initializes it.
      • 
+
    • self.outputs defines output and completes the same operator computation in the Python script, and returns its result from the Python script.
      • 
+
    
+
    
+
    
+
    Testing Backward Operators
    
+
    A backward operator unit test inherits GradientChecker, which inherits unittest.TestCase. As a result, a backward operator unit test needs to be have the prefix test_.
    
+
    class TestMulGradOp(GradientChecker):
+    def setUp(self):
+        self.op = create_op("mul")
+        self.inputs = {
+            'X': np.random.random((32, 84)).astype("float32"),
+            'Y': np.random.random((84, 100)).astype("float32")
+        }
+
+    def test_check_grad_normal(self):
+        # mul op will enlarge the relative error
+        self.check_grad(['X', 'Y'], 'Out', max_relative_error=0.5)
+
+    def test_check_grad_ingore_x(self):
+        self.check_grad(
+            ['Y'], 'Out', max_relative_error=0.5, no_grad_set=set("X"))
+
+    def test_check_grad_ingore_y(self):
+        self.check_grad(
+            ['X'], 'Out', max_relative_error=0.5, no_grad_set=set('Y'))
+
    
+
    
+
    Some key points in the code above include:
    
+
      
+
    • create_op("mul") creates the backward operator’s corresponding forward operator.
      • 
+
    • test_normal calls check_grad to validate scaling tests’ correctness and stability through numeric methods.
        
+
      • The first variable ["X", "Y"] appoints X and Y to be scale tested.
        • 
+
      • The second variable "Out" points to the network’s final output target Out.
        • 
+
      • The third variable max_relative_error points to the maximum relative tolerance error during scaling tests.
        • 
+
      
+
      • 
+
    • test_check_grad_ingore_x and test_check_grad_ingore_ybranches test the cases where there is only one scaling input.
      • 
+
    
+
    
+
    
+
    Compiling and Running
    
+
    Any new unit testing file of the format test_*.py  added to the director python/paddle/v2/framework/tests is automatically added to the project to compile.
    
+
    Note that unlike the compile test for Ops, running unit tests requires compiling the entire project and requires compiling with flag WITH_TESTING on i.e. cmake paddle_dir -DWITH_TESTING=ON.
    
+
    After successfully compiling the project, run the following command to run unit tests:
    
+
    make test ARGS="-R test_mul_op -V"
+
    
+
    
+
    Or,
    
+
    ctest -R test_mul_op
+
    
+
    
+
    
+
    
+
    
+
    Remarks
    
+
      
+
    • Every *_op.h (if applicable), *_op.cc, and *_op.cu (if applicable) must be created for a unique Op. Compiling will fail if multiple operators are included per file.
      • 
+
    • The type with which an operator is registered needs to be identical to the Op’s name. Registering REGISTER_OP(B, ...) in A_op.cc will cause unit testing failures.
      • 
+
    • If the operator does not implement a GPU kernel, please refrain from creating an empty *_op.cu file, or else unit tests will fail.
      • 
+
    • If multiple operators rely on some shared methods, a file NOT named *_op.* can be created to store them, such as gather.h.
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/howto/dev/use_eigen_en.html b/doc/howto/dev/use_eigen_en.html
new file mode 100644
index 0000000000000000000000000000000000000000..f4d376195ba2ad9be1d577b695d9e8b3e1871352
--- /dev/null
+++ b/doc/howto/dev/use_eigen_en.html
@@ -0,0 +1,391 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  How to use Eigen in Paddle — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • How to use Eigen in Paddle
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    How to use Eigen in Paddle
    
+
    Essentially, a neural network is a compute graph. T data needed for the computation is stored in Tensors and its computation procedure is described by Operators. An Operator calls the Compute interface in its corresponding OpKernel and operates on the Tensor.
    
+
    
+
    Eigen Tensor Module
    
+
    The Eigen Tensor module supports powerful element-wise computation. In addition, a piece of code written using it can be run on both the CPU and the GPU.
    
+
    Note that Eigen Tensor is still being actively developed, so its tests are not completely covered and its documentation may be sparse.
    
+
    For details on Eigen Tensor module, please see doc 1 and doc 2.
    
+
    
+
    
+
    paddle::framework::Tensor
    
+
    Paddle Tensor’s is defined in the framework directory with the following interface:
    
+
    class Tensor {
+ public:
+  /*! Return a pointer to mutable memory block. */
+  template <typename T>
+  inline T* data();
+
+  /**
+   * @brief   Return a pointer to mutable memory block.
+   * @note    If not exist, then allocation.
+   */
+  template <typename T>
+  inline T* mutable_data(platform::Place place);
+
+  /**
+   * @brief     Return a pointer to mutable memory block.
+   *
+   * @param[in] dims    The dimensions of the memory block.
+   * @param[in] place   The place of the memory block.
+   *
+   * @note      If not exist, then allocation.
+   */
+  template <typename T>
+  inline T* mutable_data(DDim dims, platform::Place place);
+
+  /*! Resize the dimensions of the memory block. */
+  inline Tensor& Resize(const DDim& dims);
+
+  /*! Return the dimensions of the memory block. */
+  inline const DDim& dims() const;
+
+ private:
+  /*! holds the memory block if allocated. */
+  std::shared_ptr<Placeholder> holder_;
+
+  /*! points to dimensions of memory block. */
+  DDim dim_;
+};
+
    
+
    
+
    Placeholder is used to delay memory allocation; that is, we can first define a tensor, using Resize to configure its shape, and then call mutuable_data to allocate the actual memory.
    
+
    paddle::framework::Tensor t;
+paddle::platform::CPUPlace place;
+// set size first
+t.Resize({2, 3});
+// allocate memory on CPU later
+t.mutable_data(place);
+
    
+
    
+
    
+
    
+
    paddle::framework::Tensor Usage
    
+
    AddOp demonstrates Tensor’s usage.
    
+
      
+
    • InferShape
      • 
+
    
+
    When computing a neural network’s compute graph, first call every Operator‘s InferShape method, and use Resize to configure the size of the output tensor.
    
+
    void InferShape(const framework::InferShapeContext &ctx) const override {
+  PADDLE_ENFORCE_EQ(ctx.Input<Tensor>("X")->dims(),
+                    ctx.Input<Tensor>("Y")->dims(),
+                    "Two input of Add Op's dimension must be same.");
+  ctx.Output<Tensor>("Out")->Resize(ctx.Input<Tensor>("X")->dims());
+}
+
    
+
    
+
      
+
    • Run
      • 
+
    
+
    void Compute(const framework::ExecutionContext& context) const override {
+  auto* input0 = context.Input<Tensor>("X");
+  auto* input1 = context.Input<Tensor>("Y");
+  auto* output = context.Output<Tensor>("Out");
+
+  output->mutable_data<T>(context.GetPlace());
+
+  auto x = EigenVector<T>::Flatten(*input0);
+  auto y = EigenVector<T>::Flatten(*input1);
+  auto z = EigenVector<T>::Flatten(*output);
+
+  auto place = context.GetEigenDevice<Place>();
+
+  z.device(place) = x + y;
+}
+
    
+
    
+
    
+
    
+
    paddle::framework::Tensor到EigenTensor的转换
    
+
    As shown above, in actual computation, we need to transform the input and output Tensors into formats Eigen supports. We show some functions in eigen.h to implement the transformation from paddle::framework::Tensorto EigenTensor/EigenMatrix/EigenVector/EigenScalar.
    
+
    Using EigenTensor as an example:
    
+
    Tensor t;
+float* p = t.mutable_data<float>(make_ddim({1, 2, 3}), platform::CPUPlace());
+for (int i = 0; i < 1 * 2 * 3; i++) {
+  p[i] = static_cast<float>(i);
+}
+
+EigenTensor<float, 3>::Type et = EigenTensor<float, 3>::From(t);
+
    
+
    
+
    From is an interfacing method provided by the EigenTensor template, which implements the transformation from a paddle::framework::Tensor object to an EigenTensor. Since rank is a template parameter, it needs to be explicitly specified at the time of the transformation.
    
+
    In Eigen, tensors with different ranks are different types, with Vector bring a rank-1 instance. Note that EigenVector<T>::From uses a transformation from an 1-dimensional Paddle tensor to a 1-dimensional Eigen tensor while EigenVector<T>::Flatten reshapes a paddle tensor and flattens it into a 1-dimensional Eigen tensor. Both resulting tensors are still typed EigenVector.
    
+
    For more transformations, see the unit tests in the eigen_test.cc file.
    
+
    
+
    
+
    Implementing Computation
    
+
    While computing, the device interface is needed from the EigenTensors on the left hand side of the assignments. Note that the computation between EigenTensors only changes the data originally inthe Tensor and does not change all the shape information associated with the Tensor.
    
+
    auto x = EigenVector<T>::Flatten(*input0);
+auto y = EigenVector<T>::Flatten(*input1);
+auto z = EigenVector<T>::Flatten(*output);
+auto place = context.GetEigenDevice<Place>();
+z.device(place) = x + y;
+
    
+
    
+
    In this code segment, input0/input1/output can be Tensors of arbitrary dimension. We are calling Flatten from EigenVector, transforming a tensor of any dimension into a 1-dimensional EigenVector. After completing computation, input0/input1/output will retain the same shape information, and they can be resized using the Resize interface.
    
+
    Because the Eigen Tensor module is under-documented, please refer to OpKernel‘s computation code in TensorFlow’s kernel module documentation.
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/howto/dev/write_docs_en.html b/doc/howto/dev/write_docs_en.html
new file mode 100644
index 0000000000000000000000000000000000000000..3b3673068c8ac344d8f9cc6e90775298b97a1745
--- /dev/null
+++ b/doc/howto/dev/write_docs_en.html
@@ -0,0 +1,340 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Contribute Documentation — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+    
+        
+        
+         
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+        
    • HOW TO > 
      • 
+      
+    
    • Contribute Documentation
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Contribute Documentation
    
+
    PaddlePaddle supports English documentation doc and Chinese documentation doc_cn.
+Both are compiled by cmake and sphinx , the compiled documentations will be stored under doc and doc_cn directories.
+When using the PaddlePaddle.org to compile documentations, the compiled documentations will be stored under a consolidated directory: .ppo_workspace/content
    
+
    
+
    How to Build Documentations
    
+
    We recommend using PaddlePaddle.org tool to build documentation
    
+
    
+
    Use PaddlePaddle.org tool
    
+
    This is the recommended method to build documentation. It can compile documentation and preview the documentation in a web browser.
    
+
    The tool uses Docker, please install it on your system. Please check Docker official website on how to install Docker. You may use the following commands to activate the tool
    
+
    mkdir paddlepaddle # Create paddlepaddle working directory
+cd paddlepaddle
+
+# Clone the content repositories. You may only clone the contents you need
+git clone https://github.com/PaddlePaddle/Paddle.git
+git clone https://github.com/PaddlePaddle/book.git
+git clone https://github.com/PaddlePaddle/models.git
+git clone https://github.com/PaddlePaddle/Mobile.git
+
+# Please specify the working directory through -v
+docker run -it -p 8000:8000 -v `pwd`:/var/content paddlepaddle/paddlepaddle.org:latest
+
    
+
    
+
    Note: PaddlePaddle.org will read the content repos specified in the -v (volume) flag of the docker run command
+Use a web browser and navigate to http://localhost:8000, click the buttons to compile the documentation
+The compiled documentations will be stored in <paddlepaddle working directory>/.ppo_workspace/content
    
+
    If you don’t wish to use Docker, you can also activate the tool through Django. Use the following the commands to set up
    
+
    mkdir paddlepaddle # Create paddlepaddle working directory
+cd paddlepaddle
+
+# Clone the content repositories and PaddlePaddle.org
+git clone https://github.com/PaddlePaddle/Paddle.git
+git clone https://github.com/PaddlePaddle/book.git
+git clone https://github.com/PaddlePaddle/models.git
+git clone https://github.com/PaddlePaddle/Mobile.git
+git clone https://github.com/PaddlePaddle/PaddlePaddle.org.git
+
+# Please specify the PaddlePaddle working directory. In the current setting, it should be pwd
+export CONTENT_DIR=<path_to_paddlepaddle_working_directory>
+export ENV=''
+cd PaddlePaddle.org/portal/
+pip install -r requirements.txt
+python manage.py runserver
+
    
+
    
+
    Use a web browser and navigate to http://localhost:8000, click the buttons to compile the documentation
+The compiled documentations will be stored in <paddlepaddle working directory>/.ppo_workspace/content
    
+
    If you want to learn more on the PaddlePaddle.org, please click here 。
    
+
    
+
    
+
    
+
    How to write Documentations
    
+
    PaddlePaddle uses sphinx to compile documentations，Please check sphinx official website for more detail.
    
+
    
+
    
+
    How to update www.paddlepaddle.org
    
+
    Please create PRs and submit them to github, please check Contribute Code 。
+PaddlePaddle develop branch will update the documentation once the PR is merged. User may check latest Chinese Docs and
+English Docs 。
    
+
    
+
    
+
+
+           
    
+          
    
+          
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/howto/index_en.html b/doc/howto/index_en.html
index 11f58305f734fa9cc30e1ce8585542af4298aae2..a8f392893b3198d4512eaae2cc594086509a3ace 100644
--- a/doc/howto/index_en.html
+++ b/doc/howto/index_en.html
@@ -34,7 +34,7 @@
         
     
         
-         
+         
 
   
   
@@ -64,7 +64,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -86,7 +86,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -110,9 +110,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -137,17 +142,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -184,7 +212,7 @@
 
    
 
@@ -196,6 +224,7 @@
 
 
    
 
    
@@ -227,7 +256,7 @@
         Next 
       
       
-         Previous
+         Previous
       
     
    
   
diff --git a/doc/howto/optimization/cpu_profiling.html b/doc/howto/optimization/cpu_profiling.html
new file mode 100644
index 0000000000000000000000000000000000000000..1a0af562d6f62290f83bcdff077eed8cec2350b0
--- /dev/null
+++ b/doc/howto/optimization/cpu_profiling.html
@@ -0,0 +1,432 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Profiling the Python Code — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Profiling the Python Code
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    This tutorial introduces techniques we use to profile and tune the
+CPU performance of PaddlePaddle.  We will use Python packages
+cProfile and yep, and Google’s perftools.
    
+
    Profiling is the process that reveals performance bottlenecks,
+which could be very different from what’s in the developers’ mind.
+Performance tuning is done to fix these bottlenecks. Performance optimization
+repeats the steps of profiling and tuning alternatively.
    
+
    PaddlePaddle users program AI applications by calling the Python API, which calls
+into libpaddle.so. written in C++.  In this tutorial, we focus on
+the profiling and tuning of
    
+
      
+
    1. the Python code and
      2. 
+
    2. the mixture of Python and C++ code.
      3. 
+
    
+
    
+
    Profiling the Python Code
    
+
    
+
    Generate the Performance Profiling File
    
+
    We can use Python standard
+package, cProfile,
+to generate Python profiling file.  For example:
    
+
    python -m cProfile -o profile.out main.py
+
    
+
    
+
    where main.py is the program we are going to profile, -o specifies
+the output file.  Without -o, cProfile would outputs to standard
+output.
    
+
    
+
    
+
    Look into the Profiling File
    
+
    cProfile generates profile.out after main.py completes. We can
+use cprofilev to look into
+the details:
    
+
    cprofilev -a 0.0.0.0 -p 3214 -f profile.out main.py
+
    
+
    
+
    where -a specifies the HTTP IP, -p specifies the port, -f
+specifies the profiling file, and main.py is the source file.
    
+
    Open the Web browser and points to the local IP and the specifies
+port, we will see the output like the following:
    
+
       ncalls  tottime  percall  cumtime  percall filename:lineno(function)
+        1    0.284    0.284   29.514   29.514 main.py:1(<module>)
+     4696    0.128    0.000   15.748    0.003 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/executor.py:20(run)
+     4696   12.040    0.003   12.040    0.003 {built-in method run}
+        1    0.144    0.144    6.534    6.534 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/__init__.py:14(<module>)
+
    
+
    
+
    where each line corresponds to Python function, and the meaning of
+each column is as follows:
    
+
    | column | meaning |
+| — | — |
+| ncalls | the number of calls into a function |
+| tottime | the total execution time of the function, not including the
+execution time of other functions called by the function |
+| percall | tottime divided by ncalls |
+| cumtime | the total execution time of the function, including the execution time of other functions being called |
+| percall | cumtime divided by ncalls |
+| filename:lineno(function) | where the function is defined |
    
+
    
+
    
+
    Identify Performance Bottlenecks
    
+
    Usually, tottime and the related percall time is what we want to
+focus on. We can sort above profiling file by tottime:
    
+
         4696   12.040    0.003   12.040    0.003 {built-in method run}
+   300005    0.874    0.000    1.681    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/dataset/mnist.py:38(reader)
+   107991    0.676    0.000    1.519    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:219(__init__)
+     4697    0.626    0.000    2.291    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp)
+        1    0.618    0.618    0.618    0.618 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/__init__.py:1(<module>)
+
    
+
    
+
    We can see that the most time-consuming function is the built-in method run, which is a C++ function in libpaddle.so.  We will
+explain how to profile C++ code in the next section.  At this
+moment, let’s look into the third function sync_with_cpp, which is a
+Python function.  We can click it to understand more about it:
    
+
    Called By:
+
+   Ordered by: internal time
+   List reduced from 4497 to 2 due to restriction <'sync_with_cpp'>
+
+Function                                                                                                 was called by...
+                                                                                                             ncalls  tottime  cumtime
+/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp)  <-    4697    0.626    2.291  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp)
+/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp)  <-    4696    0.019    2.316  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:487(clone)
+                                                                                                                  1    0.000    0.001  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:534(append_backward)
+
+
+Called:
+
+   Ordered by: internal time
+   List reduced from 4497 to 2 due to restriction <'sync_with_cpp'>
+
    
+
    
+
    The lists of the callers of sync_with_cpp might help us understand
+how to improve the function definition.
    
+
    
+
    
+
    
+
    Profiling Python and C++ Code
    
+
    
+
    Generate the Profiling File
    
+
    To profile a mixture of Python and C++ code, we can use a Python
+package, yep, that can work with Google’s perftools, which is a
+commonly-used profiler for C/C++ code.
    
+
    In Ubuntu systems, we can install yep and perftools by running the
+following commands:
    
+
    apt update
+apt install libgoogle-perftools-dev
+pip install yep
+
    
+
    
+
    Then we can run the following command
    
+
    python -m yep -v main.py
+
    
+
    
+
    to generate the profiling file.  The default filename is
+main.py.prof.
    
+
    Please be aware of the -v command line option, which prints the
+analysis results after generating the profiling file.  By examining the
+the print result, we’d know that if we stripped debug
+information from libpaddle.so at build time.  The following hints
+help make sure that the analysis results are readable:
    
+
      
+
    1. Use GCC command line option -g when building libpaddle.so so to
+include the debug information.  The standard building system of
+PaddlePaddle is CMake, so you might want to set
+CMAKE_BUILD_TYPE=RelWithDebInfo.
      2. 
+
    2. Use GCC command line option -O2 or -O3 to generate optimized
+binary code. It doesn’t make sense to profile libpaddle.so
+without optimization, because it would anyway run slowly.
      3. 
+
    3. Profiling the single-threaded binary file before the
+multi-threading version, because the latter often generates tangled
+profiling analysis result.  You might want to set environment
+variable OMP_NUM_THREADS=1 to prevents OpenMP from automatically
+starting multiple threads.
      4. 
+
    
+
    
+
    
+
    Examining the Profiling File
    
+
    The tool we used to examine the profiling file generated by
+perftools is pprof, which
+provides a Web-based GUI like cprofilev.
    
+
    We can rely on the standard Go toolchain to retrieve the source code
+of pprof and build it:
    
+
    go get github.com/google/pprof
+
    
+
    
+
    Then we can use it to profile main.py.prof generated in the previous
+section:
    
+
    pprof -http=0.0.0.0:3213 `which python`  ./main.py.prof
+
    
+
    
+
    Where -http specifies the IP and port of the HTTP service.
+Directing our Web browser to the service, we would see something like
+the following:
    
+
    result
    
+
    
+
    
+
    Identifying the Performance Bottlenecks
    
+
    Similar to how we work with cprofilev, we’d focus on tottime and
+cumtime.
    
+
    kernel_perf
    
+
    We can see that the execution time of multiplication and the computing
+of the gradient of multiplication takes 2% to 4% of the total running
+time, and MomentumOp takes about 17%. Obviously, we’d want to
+optimize MomentumOp.
    
+
    pprof would mark performance critical parts of the program in
+red. It’s a good idea to follow the hints.
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/howto/optimization/gpu_profiling_en.html b/doc/howto/optimization/gpu_profiling_en.html
index 913cfe90b030ed9ce49762650fc3ecf0c4349306..af935bca19442c3b7b74aba20b1cc290cfeec216 100644
--- a/doc/howto/optimization/gpu_profiling_en.html
+++ b/doc/howto/optimization/gpu_profiling_en.html
@@ -35,7 +35,7 @@
     
         
         
-         
+         
 
   
   
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -514,7 +542,7 @@ Your mileage may vary!
    
         Next 
       
       
-         Previous
+         Previous
       
     
    
   
diff --git a/doc/howto/usage/cluster/cluster_train_en.html b/doc/howto/usage/cluster/cluster_train_en.html
index e4153c2e3c8cfc4639369703d11ced3da31717c2..5749ea95f62a62e1006a752868ac4df51a3ebde3 100644
--- a/doc/howto/usage/cluster/cluster_train_en.html
+++ b/doc/howto/usage/cluster/cluster_train_en.html
@@ -8,7 +8,7 @@
   
   
   
-  Run Distributed Training — PaddlePaddle  documentation
+  PaddlePaddle Distributed Training — PaddlePaddle  documentation
   
 
   
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -171,7 +199,7 @@
       
         
  • HOW TO > 
    • 
       
-    
  • Run Distributed Training
    • 
+    
  • PaddlePaddle Distributed Training
    • 
   
 
    
       
@@ -180,112 +208,219 @@
           
    
            
    
             
-  
    
-
    Run Distributed Training
    
-
    In this article, we explain how to run distributed Paddle training jobs on clusters.  We will create the distributed version of the single-process training example, recommendation.
    
-
    Scripts used in this article launch distributed jobs via SSH.  They also work as a reference for users running more sophisticated cluster management systems like MPI and Kubernetes.
    
-
    
-
    Prerequisite
    
-
      
-
    1. Aforementioned scripts use a Python library fabric to run SSH commands.  We can use pip to install fabric:
      
-
      pip install fabric
-
      
-
      
+  
      
+
      PaddlePaddle Distributed Training
      
+
+
      
+
      Introduction
      
+
      In this article, we’ll explain how to run distributed training jobs with PaddlePaddle on different types of clusters. The diagram below shows the main architecture of a distributed trainning job:
      
+
      
+
        
+
      • Data shard: training data will be split into multiple partitions, trainers use the partitions of the whole dataset to do the training job.
        • 
+
      • Trainer: each trainer reads the data shard, and train the neural network. Then the trainer will upload calculated “gradients” to parameter servers, and wait for parameters to be optimized on the parameter server side. When that finishes, the trainer download optimized parameters and continues its training.
        • 
+
      • Parameter server: every parameter server stores part of the whole neural network model data. They will do optimization calculations when gradients are uploaded from trainers, and then send updated parameters to trainers.
        • 
+
      
+
      PaddlePaddle can support both synchronize stochastic gradient descent (SGD) and asynchronous SGD.
      
+
      When training with synchronize SGD, PaddlePaddle uses an internal “synchronize barrier” which makes gradients update and parameter download in strict order. On the other hand, asynchronous SGD won’t wait for all trainers to finish upload at a single step, this will increase the parallelism of distributed training: parameter servers do not depend on each other, they’ll do parameter optimization concurrently. Parameter servers will not wait for trainers, so trainers will also do their work concurrently. But asynchronous SGD will introduce more randomness and noises in the gradient.
      
+
      
+
      
+
      Preparations
      
+
        
+
      1. Prepare your computer cluster. It’s normally a bunch of Linux servers connected by LAN. Each server will be assigned a unique IP address. The computers in the cluster can be called “nodes”.
        2. 
+
      2. Install PaddlePaddle on every node. If you are going to take advantage of GPU cards, you’ll also need to install proper driver and CUDA libraries. To install PaddlePaddle please read this build and install document. We strongly recommend using Docker installation.
        3. 
 
      
+
      After installation, you can check the version by typing the below command (run a docker container  if using docker: docker run -it paddlepaddle/paddle:[tag] /bin/bash):
      
+
      $ paddle version
+PaddlePaddle 0.10.0rc, compiled with
+    with_avx: ON
+    with_gpu: OFF
+    with_double: OFF
+    with_python: ON
+    with_rdma: OFF
+    with_timer: OFF
+
      
+
      
+
      We’ll take doc/howto/usage/cluster/src/word2vec as an example to introduce distributed training using PaddlePaddle v2 API.
      
+
      
+
      
+
      Command-line arguments
      
+
      
+
      Starting parameter server
      
+
      Type the below command to start a parameter server which will wait for trainers to connect:
      
+
      $ paddle pserver --port=7164 --ports_num=1 --ports_num_for_sparse=1 --num_gradient_servers=1
+
      
+
      
+
      If you wish to run parameter servers in background, and save a log file, you can type:
      
+
      $ stdbuf -oL /usr/bin/nohup paddle pserver --port=7164 --ports_num=1 --ports_num_for_sparse=1 --num_gradient_servers=1 &> pserver.log
+
      
+
      
+
      | param  | required | default | description |
+| ————- | ————- | ————- | ————- |
+| port  | required | 7164 | port which parameter server will listen on. If ports_num greater than 1, parameter server will listen on multiple ports for more network throughput |
+| ports_num  | required | 1 | total number of ports will listen on  |
+| ports_num_for_sparse  | required | 1 | number of ports which serves sparse parameter update  |
+| num_gradient_servers  | required | 1 | total number of gradient servers |
      
+
      
+
      
+
      Starting trainer
      
+
      Type the command below to start the trainer(name the file whatever you want, like “train.py”)
      
+
      $ python train.py
+
      
+
      
+
      Trainers’ network need to be connected with parameter servers’ network to finish the job. Trainers need to know port and IPs to locate parameter servers. You can pass arguments to trainers through environment variables or pass to paddle.init() function. Arguments passed to the paddle.init() function will overwrite environment variables.
      
+
      Use environment viriables:
      
+
      export PADDLE_INIT_USE_GPU=False
+export PADDLE_INIT_TRAINER_COUNT=1
+export PADDLE_INIT_PORT=7164
+export PADDLE_INIT_PORTS_NUM=1
+export PADDLE_INIT_PORTS_NUM_FOR_SPARSE=1
+export PADDLE_INIT_NUM_GRADIENT_SERVERS=1
+export PADDLE_INIT_TRAINER_ID=0
+export PADDLE_INIT_PSERVERS=127.0.0.1
+python train.py
+
      
 
      
-
      
-
      Prepare Job Workspace
      
-
      We refer to the directory where we put dependent libraries, config files, etc., as workspace.
      
-
      These train/test data should be prepared before launching cluster job. To  satisfy the requirement that train/test data are placed in different directory from workspace, PADDLE refers train/test data according to index file named as train.list/test.list which are used in model config file. So the train/test data also contains train.list/test.list two list file. All local training demo already provides scripts to help you create these two files,  and all nodes in cluster job will handle files with same logical code in normal condition.
      
-
      Generally, you can use same model file from local training for cluster training. What you should have in mind that, the batch_size set in setting function in model file means batch size in each node of cluster job instead of total batch size if synchronization SGD was used.
      
-
      Following steps are based on demo/recommendation demo in demo directory.
      
-
      You just go through demo/recommendation tutorial doc until Train section, and at last you will get train/test data and model configuration file. Finaly, just use demo/recommendation as workspace for cluster training.
      
-
      At last your workspace should look like as follow:
      
+
      Pass arguments:
      
+
      paddle.init(
+        use_gpu=False,
+        trainer_count=1,
+        port=7164,
+        ports_num=1,
+        ports_num_for_sparse=1,
+        num_gradient_servers=1,
+        trainer_id=0,
+        pservers="127.0.0.1")
+
      
+
      
+
      | param  | required | default | description |
+| ————- | ————- | ————- | ————- |
+| use_gpu  | optional | False | set to “True” to enable GPU training |
+| trainer_count  | required | 1 | total count of trainers in the training job |
+| port  | required | 7164 | port to connect to parameter server  |
+| ports_num  | required | 1 | number of ports for communication |
+| ports_num_for_sparse  | required | 1 | number of ports for sparse type caculation |
+| num_gradient_servers  | required | 1 | total number of gradient server |
+| trainer_id  | required | 0 | ID for every trainer, start from 0 |
+| pservers  | required | 127.0.0.1 | list of IPs of parameter servers, separated by ”,” |
      
+
      
+
      
+
      Prepare Training Dataset
      
+
      Here’s some example code prepare.py, it will download public imikolov dataset and split it into multiple files according to job parallelism(trainers count). Modify SPLIT_COUNT at the begining of prepare.py to change the count of output files.
      
+
      In the real world, we often use MapReduce job’s output as training data, so there will be lots of files. You can use mod to assign training file to trainers:
      
+
      import os
+train_list = []
+flist = os.listdir("/train_data/")
+for f in flist:
+  suffix = int(f.split("-")[1])
+  if suffix % TRAINER_COUNT == TRAINER_ID:
+    train_list.append(f)
+
      
+
      
+
      Example code prepare.py will split training data and testing data into 3 files with digital suffix like -00000, -00001 and-00002:
      
+
      train.txt
+train.txt-00000
+train.txt-00001
+train.txt-00002
+test.txt
+test.txt-00000
+test.txt-00001
+test.txt-00002
+
      
+
      
+
      When job started, every trainer needs to get it’s own part of data. In some distributed systems a storage service will be provided, so the date under that path can be accessed by all the trainer nodes. Without the storage service, you must copy the training data to each trainer node.
      
+
      Different training jobs may have different data format and reader() function, developers may need to write different data prepare scripts and reader() functions for their job.
      
+
      
+
      
+
      Prepare Training program
      
+
      We’ll create a workspace directory on each node, storing your training program, dependencies, mounted or downloaded dataset directory.
      
+
      Your workspace may looks like:
      
 
      .
-|-- common_utils.py
-|-- data
-|   |-- config.json
-|   |-- config_generator.py
-|   |-- meta.bin
-|   |-- meta_config.json
-|   |-- meta_generator.py
-|   |-- ml-1m
-|   |-- ml_data.sh
-|   |-- ratings.dat.test
-|   |-- ratings.dat.train
-|   |-- split.py
-|   |-- test.list
-|   `-- train.list
-|-- dataprovider.py
-|-- evaluate.sh
-|-- prediction.py
-|-- preprocess.sh
-|-- requirements.txt
-|-- run.sh
-`-- trainer_config.py
+|-- my_lib.py
+|-- word_dict.pickle
+|-- train.py
+|-- train_data_dir/
+|   |-- train.txt-00000
+|   |-- train.txt-00001
+|   |-- train.txt-00002
+`-- test_data_dir/
+    |-- test.txt-00000
+    |-- test.txt-00001
+    `-- test.txt-00002
 
      
 
      
-
      Not all of these files are needed for cluster training, but it’s not necessary to remove useless files.
      
-
      trainer_config.py
-Indicates the model config file.
      
-
      train.list and test.list
-File index. It stores all relative or absolute file paths of all train/test data at current node.
      
-
      dataprovider.py
-used to read train/test samples. It’s same as local training.
      
-
      data
-all files in data directory are refered by train.list/test.list which are refered by data provider.
      
-
      
-
      
-
      Prepare Cluster Job Configuration
      
-
      The options below must be carefully set in cluster_train/conf.py
      
-
      HOSTS  all nodes hostname or ip that will run cluster job. You can also append user and ssh port with hostname, such as root@192.168.100.17:9090.
      
-
      ROOT_DIR workspace ROOT directory for placing JOB workspace directory
      
-
      PADDLE_NIC the NIC(Network Interface Card) interface name for cluster communication channel, such as eth0 for ethternet, ib0 for infiniband.
      
-
      PADDLE_PORT port number for cluster commnunication channel
      
-
      PADDLE_PORTS_NUM the number of port used for cluster communication channle. if the number of cluster nodes is small(less than 5~6nodes), recommend you set it to larger, such as 2 ~ 8, for better network performance.
      
-
      PADDLE_PORTS_NUM_FOR_SPARSE the number of port used for sparse updater cluster commnunication channel. if sparse remote update is used, set it like PADDLE_PORTS_NUM
      
-
      LD_LIBRARY_PATH set addtional LD_LIBRARY_PATH for cluster job. You can use it to set CUDA libraries path.
      
-
      Default Configuration as follow:
      
-
      HOSTS = [
-        "root@192.168.100.17",
-        "root@192.168.100.18",
-        ]
-
-'''
-workspace configuration
-'''
-
-#root dir for workspace
-ROOT_DIR = "/home/paddle"
-
-'''
-network configuration
-'''
-#pserver nics
-PADDLE_NIC = "eth0"
-#pserver port
-PADDLE_PORT = 7164
-#pserver ports num
-PADDLE_PORTS_NUM = 2
-#pserver sparse ports num
-PADDLE_PORTS_NUM_FOR_SPARSE = 2
-
-#environments setting for all processes in cluster job
-LD_LIBRARY_PATH="/usr/local/cuda/lib64:/usr/lib64"
+
        
+
      • my_lib.py: user defined libraries, like PIL libs. This is optional.
        
+
        • 
+
      • word_dict.pickle: dict file for training word embeding.
        
+
        • 
+
      • train.py: training program. Sample code: api_train_v2_cluster.py. NOTE: You may need to modify the head part of train.py when using different cluster platform to retrive configuration environment variables:
        
+
        cluster_train_file = "./train_data_dir/train/train.txt"
+cluster_test_file = "./test_data_dir/test/test.txt"
+node_id = os.getenv("OMPI_COMM_WORLD_RANK")
+if not node_id:
+    raise EnvironmentError("must provied OMPI_COMM_WORLD_RANK")
 
        
 
        
+
        • 
+
      • train_data_dir: containing training data. Mount from storage service or copy trainning data to here.
        
+
        • 
+
      • test_data_dir: containing testing data.
        
+
        • 
+
      
+
      
+
      
+
      
+
      Use cluster platforms or cluster management tools
      
+
      PaddlePaddle supports running jobs on several platforms including:
      
+
        
+
      • Kubernetes open-source system for automating deployment, scaling, and management of containerized applications from Google.
        • 
+
      • OpenMPI Mature high performance parallel computing framework.
        • 
+
      • Fabric A cluster management tool. Write scripts to submit jobs or manage the cluster.
        • 
+
      
+
      We’ll introduce cluster job management on these platforms. The examples can be found under cluster_train_v2.
      
+
      These cluster platforms provide API or environment variables for training processes, when the job is dispatched to different nodes. Like node ID, IP or total number of nodes etc.
      
+
      
+
      Cluster Training Using Fabric
      
+
      
+
      Prepare a Linux cluster
      
+
      Run kubectl -f ssh_servers.yaml under the directory:  paddle/scripts/cluster_train_v2/fabric/docker_cluster will launch a demo cluster. Run kubectl get po -o wide to get IP addresses of these nodes.
      
+
      
 
      
-
      Launching Cluster Job
      
-
      paddle.py provides automatical scripts to start all PaddlePaddle cluster processes in different nodes. By default, all command line options can set as paddle.py command options and paddle.py will transparently and automatically set these options to PaddlePaddle lower level processes.
      
+
      Launching Cluster Job
      
+
      paddle.py provides automatical scripts to start all PaddlePaddle cluster processes in different nodes. By default, all command line options can be set as paddle.py command options and paddle.py will transparently and automatically set these options to PaddlePaddle lower level processes.
      
 
      paddle.pyprovides two distinguished command option for easy job launching.
      
-
      job_dispatch_package  set it with local workspacedirectory, it will be dispatched to all nodes set in conf.py. It could be helpful for frequent hacking workspace files, otherwise frequent mulit-nodes workspace deployment could make your crazy.
-job_workspace  set it with already deployed workspace directory, paddle.py will skip dispatch stage to directly launch cluster job with all nodes. It could help to reduce heavy
-dispatch latency.
      
+
        
+
      • job_dispatch_package set it with local workspace directory, it will be dispatched to all nodes which is set in conf.py. It could be helpful for frequently manipulating workspace files. otherwise, frequent multi-nodes workspace deployment is very annoying.
        • 
+
      • job_workspace  set it with already deployed workspace directory, paddle.py will skip dispatch stage to directly launch cluster job with all nodes. It could help to reduce heavy
+dispatch latency.
        • 
+
      
 
      cluster_train/run.sh provides command line sample to run demo/recommendation cluster job, just modify job_dispatch_package and job_workspace with your defined directory, then:
      
 
      sh run.sh
 
      
@@ -293,27 +428,72 @@ dispatch latency.
      
 
      The cluster Job will start in several seconds.
      
 
      
 
      
-
      Kill Cluster Job
      
-
      paddle.py can capture Ctrl + C SIGINT signal to automatically kill all processes launched by it. So just stop paddle.py to kill cluster job. You should mannally kill job if program crashed.
      
+
      Kill Cluster Job
      
+
      paddle.py can capture Ctrl + C SIGINT signal to automatically kill all processes launched by it. So just stop paddle.py to kill cluster job. You should manually kill the job if the program crashed.
      
 
      
 
      
-
      Check Cluster Training Result
      
+
      Check Cluster Training Result
      
 
      Check log in $workspace/log for details, each node owns same log structure.
      
 
      paddle_trainer.INFO
-It provides almost all interal output log for training,  same as local training. Check runtime model convergence here.
      
+It provides almost all internal output log for training,  same as local training. Check runtime model convergence here.
      
 
      paddle_pserver2.INFO
-It provides pserver running log, which could help to diagnose distributed error.
      
+It provides parameter server running log, which could help to diagnose distributed error.
      
 
      server.log
-It provides stderr and stdout of pserver process. Check error log if training crashs.
      
+It provides stderr and stdout of parameter server process. Check error log if training crashes.
      
 
      train.log
-It provides stderr and stdout of trainer process. Check error log if training crashs.
      
+It provides stderr and stdout of trainer process. Check error log if training crashes.
      
 
      
 
      
-
      Check Model Output
      
-
      After one pass finished, model files will be writed in output directory in node 0.
+
      Check Model Output
      
+
      After one pass finished, model files will be written in output directory in node 0.
 nodefile in workspace indicates the node id of current cluster job.
      
 
      
 
      
+
      
+
      Cluster Training Using OpenMPI
      
+
      
+
      Prepare an OpenMPI cluster
      
+
      Run the following command to start a 3-node MPI cluster and one “head” node.
      
+
      cd paddle/scripts/cluster_train_v2/openmpi/docker_cluster
+kubectl create -f head.yaml
+kubectl create -f mpi-nodes.yaml
+
      
+
      
+
      Then you can log in to every OpenMPI node using ssh without input any passwords.
      
+
      
+
      
+
      Launching Cluster Job
      
+
      Follow the steps to launch a PaddlePaddle training job in OpenMPI cluster:\
      
+
      # find out node IP addresses
+kubectl get po -o wide
+# generate a "machines" file containing node IP addresses
+kubectl get po -o wide | grep nodes | awk '{print $6}' > machines
+# copy necessary files onto "head" node
+scp -i ssh/id_rsa.mpi.pub machines prepare.py train.py start_mpi_train.sh tutorial@[headIP]:~
+# login to head node using ssh
+ssh -i ssh/id_rsa.mpi.pub tutorial@[headIP]
+# --------------- in head node ---------------
+# prepare training data
+python prepare.py
+# copy training data and dict file to MPI nodes
+cat machines | xargs -i scp word_dict.pickle train.py start_mpi_train.sh machines {}:/home/tutorial
+# creat a directory for storing log files
+mpirun -hostfile machines -n 3 mkdir /home/tutorial/logs
+# copy training data to every node
+scp train.txt-00000 test.txt-00000 [node1IP]:/home/tutorial
+scp train.txt-00001 test.txt-00001 [node2IP]:/home/tutorial
+scp train.txt-00002 test.txt-00002 [node3IP]:/home/tutorial
+# start the job
+mpirun -hostfile machines -n 3  /home/tutorial/start_mpi_train.sh
+
      
+
      
+
      
+
      
+
      
+
      Cluster Training Using Kubernetes
      
+
      The details can be found here
      
+
      
+
      
 
      
 
 
diff --git a/doc/howto/usage/cmd_parameter/arguments_en.html b/doc/howto/usage/cmd_parameter/arguments_en.html
index 5235f793b003493f668f59a8044e017f909f85e3..f609dbbbf35d9091e6f396ae7984c3f699f10452 100644
--- a/doc/howto/usage/cmd_parameter/arguments_en.html
+++ b/doc/howto/usage/cmd_parameter/arguments_en.html
@@ -65,7 +65,7 @@
     
      
     
      
       
      
-        Folk me on Github
+        Fork me on Github
         
      
           
             English
@@ -87,7 +87,7 @@
 
    2. GET STARTED
      3. 
 
    3. HOW TO
      4. 
 
    4. API
      5. 
-
    5. ABOUT
      6. 
+
    6. MOBILE
      7. 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
    7. Run Distributed Training
      8. 
+
    8. PaddlePaddle Distributed Training
      9. 
 
    9. Paddle On Kubernetes
      10. 
 
    10. Distributed PaddlePaddle Training on AWS with Kubernetes
      11. 
 
    11. Write New Layers
      12. 
 
    12. Contribute Code
      13. 
-
    13. RNN Models
      14. 
+
    14. Contribute Documentation
      15. 
+
    15. RNN Models
+
      16. 
 
    16. Tune GPU Performance
      17. 
 
 
@@ -138,17 +143,40 @@
 
    17. Model Configuration
 
      18. 
-
    18. Data Reader Interface and DataSets
      19. 
+
    19. Data Reader Interface and DataSets
+
      20. 
 
    20. Training and Inference
      21. 
+
    21. Fluid
+
      22. 
+
+
+
    22. MOBILE
 
      23. 
-
    23. ABOUT
      24. 
 
 
         
diff --git a/doc/howto/usage/cmd_parameter/detail_introduction_en.html b/doc/howto/usage/cmd_parameter/detail_introduction_en.html
index 037ae4ce951bd9be494ec54b63c059a5ef4ac636..51ccccc3a87a786061b6237d21ac17f3f24fab51 100644
--- a/doc/howto/usage/cmd_parameter/detail_introduction_en.html
+++ b/doc/howto/usage/cmd_parameter/detail_introduction_en.html
@@ -34,7 +34,7 @@
         
     
         
-        
+        
          
 
   
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -628,7 +656,7 @@
   
     
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
diff --git a/doc/howto/usage/cmd_parameter/use_case_en.html b/doc/howto/usage/cmd_parameter/use_case_en.html
index 490771bb9c2fc2b50e475e3f4a6d44b535e97c01..6bc19847f4885f7ca3448f067f514e77dedce45c 100644
--- a/doc/howto/usage/cmd_parameter/use_case_en.html
+++ b/doc/howto/usage/cmd_parameter/use_case_en.html
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
diff --git a/doc/howto/usage/k8s/k8s_aws_en.html b/doc/howto/usage/k8s/k8s_aws_en.html
index 4ec021b26875891cb020059cdc54c5bee6621e37..cce00ad6fb84f9f1969260b940ab1f9baa7dd7e7 100644
--- a/doc/howto/usage/k8s/k8s_aws_en.html
+++ b/doc/howto/usage/k8s/k8s_aws_en.html
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
diff --git a/doc/howto/usage/k8s/k8s_en.html b/doc/howto/usage/k8s/k8s_en.html
index d23d440e9aca1646e0d7d5ac5603cef8e148419a..1d1265f4da58ca1f11508cad9f65107ee58ac5a2 100644
--- a/doc/howto/usage/k8s/k8s_en.html
+++ b/doc/howto/usage/k8s/k8s_en.html
@@ -35,7 +35,7 @@
     
         
         
-         
+         
 
   
   
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -385,7 +413,7 @@ Volumes:
         Next 
       
       
-         Previous
+         Previous
       
     
    
   
diff --git a/doc/howto/usage/k8s/src/k8s_data/README.html b/doc/howto/usage/k8s/src/k8s_data/README.html
index 48f563142949bb8e773f50cdd3a546fb5f9a3181..c58e23648086f896c754e6d018e23972b4be017b 100644
--- a/doc/howto/usage/k8s/src/k8s_data/README.html
+++ b/doc/howto/usage/k8s/src/k8s_data/README.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,7 +84,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
diff --git a/doc/howto/usage/k8s/src/k8s_train/README.html b/doc/howto/usage/k8s/src/k8s_train/README.html
index 9725be05449bbd86927db0d08ecd2440a9e82366..4914dd45aac9aba1085827f79861fe9e14e82e47 100644
--- a/doc/howto/usage/k8s/src/k8s_train/README.html
+++ b/doc/howto/usage/k8s/src/k8s_train/README.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,7 +84,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
diff --git a/doc/index_en.html b/doc/index_en.html
index 4c6d8694402296e9dad2fb3c72000c4621df48ca..3a8cec821cb7d0d7f54f92d97070fff38c75d1c5 100644
--- a/doc/index_en.html
+++ b/doc/index_en.html
@@ -63,7 +63,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -85,7 +85,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -109,9 +109,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -136,17 +141,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -183,7 +211,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
    
 
    
diff --git a/doc/mobile/cross_compiling_for_android_en.html b/doc/mobile/cross_compiling_for_android_en.html
new file mode 100644
index 0000000000000000000000000000000000000000..a64a783d3f39c4758fe1263ba60f2e703a25235a
--- /dev/null
+++ b/doc/mobile/cross_compiling_for_android_en.html
@@ -0,0 +1,441 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Build PaddlePaddle for Android — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+    
+        
+        
+         
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+        
    • MOBILE > 
      • 
+      
+    
    • Build PaddlePaddle for Android
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Build PaddlePaddle for Android
    
+
    There are two approaches to build PaddlePaddle for Android: using Docker and on Linux without Docker.
    
+
    
+
    Cross-Compiling Using Docker
    
+
    Docker-based cross-compiling is the recommended approach because Docker runs on all major operating systems, including Linux, Mac OS X, and Windows.
    
+
    
+
    Build the Docker Image
    
+
    The following steps pack all the tools that we need to build PaddlePaddle into a Docker image.
    
+
    $ git clone https://github.com/PaddlePaddle/Paddle.git
+$ cd Paddle
+$ docker build -t paddle:dev-android . -f Dockerfile.android
+
    
+
    
+
    
+
    
+
    Build the Inference Library
    
+
    We can run the Docker image we just created to build the inference library of PaddlePaddle for Android using the command below:
    
+
    $ docker run -it --rm -v $PWD:/paddle -e "ANDROID_ABI=armeabi-v7a" -e "ANDROID_API=21" paddle:dev-android
+
    
+
    
+
    The Docker image accepts two arguments ANDROID_ABI and ANDROID_API:
    
+
++  +  +  +
+
+  
+  
+  
+  
+
+
+
+  
+  
+  
+  
+
+
+  
+  
+  
+
+
+
    




    ArgumentOptional ValuesDefault
    ANDROID_ABIarmeabi-v7a, arm64-v8aarmeabi-v7a
    ANDROID_API>= 2121
    The ARM-64 architecture (arm64-v8a) requires at least level 21 of Android API.
    
+
    The default entry-point of the Docker image, paddle/scripts/docker/build_android.sh generates the Android cross-compiling standalone toolchain based on the argument: ANDROID_ABI or ANDROID_API.  For information about other configuration arguments, please continue reading.
    
+
    The above command generates and outputs the inference library in $PWD/install_android and puts third-party libraries in $PWD/install_android/third_party.
    
+
    
+
    
+
    
+
    Cross-Compiling on Linux
    
+
    The Linux-base approach to cross-compile is to run steps in Dockerfile.android manually on a Linux x64 computer.
    
+
    
+
    Setup the Environment
    
+
    To build for Android’s, we need Android NDK:
    
+
    wget -q https://dl.google.com/android/repository/android-ndk-r14b-linux-x86_64.zip
+unzip -q android-ndk-r14b-linux-x86_64.zip
+
    
+
    
+
    Android NDK includes everything we need to build the standalone toolchain, which in then used to build PaddlePaddle for Android.  (We plan to remove the intermediate stage of building the standalone toolchain in the near future.)
    
+
      
+
    • To build the standalone toolchain for armeabi-v7a and Android API level 21:
      
+
      your/path/to/android-ndk-r14b-linux-x86_64/build/tools/make-standalone-toolchain.sh \
+        --arch=arm --platform=android-21 --install-dir=your/path/to/arm_standalone_toolchain
+
      
+
      
+
      The generated standalone toolchain will be in your/path/to/arm_standalone_toolchain.
      
+
      • 
+
    • To build the standalone toolchain for arm64-v8a and Android API level 21:
      
+
      your/path/to/android-ndk-r14b-linux-x86_64/build/tools/make-standalone-toolchain.sh \
+        --arch=arm64 --platform=android-21 --install-dir=your/path/to/arm64_standalone_toolchain
+
      
+
      
+
      The generated standalone toolchain will be in your/path/to/arm64_standalone_toolchain.
      
+
      • 
+
    
+
    Please be aware that the minimum level of Android API required by PaddlePaddle is 21.
    
+
    
+
    
+
    Cross-Compiling Arguments
    
+
    CMake supports choosing the toolchain.  PaddlePaddle provides android.cmake, which configures the Android cross-compiling toolchain for CMake.  android.cmake is not required for CMake >= 3.7, which support Android cross-compiling. PaddlePaddle detects the CMake version, for those newer than 3.7, it uses the official version.
    
+
    Some other CMake arguments you need to know:
    
+
      
+
    • CMAKE_SYSTEM_NAME must be Android.  This tells PaddlePaddle’s CMake system to cross-compile third-party dependencies.  This also changes some other CMake arguments like WITH_GPU=OFF, WITH_AVX=OFF, WITH_PYTHON=OFF, and WITH_RDMA=OFF.
      • 
+
    • WITH_C_API must be ON, to build the C-based inference library for Android.
      • 
+
    • WITH_SWIG_PY must be OFF because the Android platform doesn’t support SWIG-based API.
      • 
+
    
+
    Some Android-specific arguments:
    
+
      
+
    • ANDROID_STANDALONE_TOOLCHAIN: the absolute path of the Android standalone toolchain, or the path relative to the CMake build directory.  PaddlePaddle’s CMake extensions would derive the cross-compiler, sysroot and Android API level from this argument.
      • 
+
    • ANDROID_TOOLCHAIN: could be gcc or clang.  The default value is clang.
        
+
      • For CMake >= 3.7, it should anyway be clang.  For older versions, it could be gcc.
        • 
+
      • Android’s official clang requires glibc >= 2.15.
        • 
+
      
+
      • 
+
    • ANDROID_ABI: could be armeabi-v7a or arm64-v8a.  The default value is armeabi-v7a.
      • 
+
    • ANDROID_NATIVE_API_LEVEL: could be derived from the value of ANDROID_STANDALONE_TOOLCHAIN.
      • 
+
    • ANROID_ARM_MODE:
        
+
      • could be ON or OFF, and defaults to ON, when ANDROID_ABI=armeabi-v7a;
        • 
+
      • no need to specify when ANDROID_ABI=arm64-v8a.
        • 
+
      
+
      • 
+
    • ANDROID_ARM_NEON: indicates if to use NEON instructions.
        
+
      • could be ON or OFF, and defaults to ON, when ANDROID_ABI=armeabi-v7a;
        • 
+
      • no need to specify when ANDROID_ABI=arm64-v8a.
        • 
+
      
+
      • 
+
    
+
    Other useful arguments:
    
+
      
+
    • USE_EIGEN_FOR_BLAS: indicates if using Eigen.  Could be ON or OFF, defaults to OFF.
      • 
+
    • HOST_C/CXX_COMPILER: specifies the host compiler, which is used to build the host-specific protoc and target-specific OpenBLAS.  It defaults to the value of the environment variable CC, or cc.
      • 
+
    
+
    Some frequent configurations for your reference:
    
+
    cmake -DCMAKE_SYSTEM_NAME=Android \
+      -DANDROID_STANDALONE_TOOLCHAIN=your/path/to/arm_standalone_toolchain \
+      -DANDROID_ABI=armeabi-v7a \
+      -DANDROID_ARM_NEON=ON \
+      -DANDROID_ARM_MODE=ON \
+      -DUSE_EIGEN_FOR_BLAS=ON \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_C_API=ON \
+      -DWITH_SWIG_PY=OFF \
+      ..
+
    
+
    
+
    cmake -DCMAKE_SYSTEM_NAME=Android \
+      -DANDROID_STANDALONE_TOOLCHAIN=your/path/to/arm64_standalone_toolchain \
+      -DANDROID_ABI=arm64-v8a \
+      -DUSE_EIGEN_FOR_BLAS=OFF \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_C_API=ON \
+      -DWITH_SWIG_PY=OFF \
+      ..
+
    
+
    
+
    There are some other arguments you might want to configure.
    
+
      
+
    • CMAKE_BUILD_TYPE=MinSizeRel minimizes the size of library.
      • 
+
    • CMAKE_BUILD_TYPE-Release optimizes the runtime performance.
      • 
+
    
+
    Our own tip for performance optimization to use clang and Eigen or OpenBLAS:
    
+
      
+
    • CMAKE_BUILD_TYPE=Release
      • 
+
    • ANDROID_TOOLCHAIN=clang
      • 
+
    • USE_EIGEN_BLAS=ON for armeabi-v7a, or USE_EIGEN_FOR_BLAS=OFF for arm64-v8a.
      • 
+
    
+
    
+
    
+
    Build and Install
    
+
    After running cmake, we can run make; make install to build and install.
    
+
    Before building, you might want to remove the third_party and build directories including pre-built libraries for other architectures.
    
+
    After building，in the directory CMAKE_INSTALL_PREFIX, you will find three sub-directories:
    
+
      
+
    • include: the header file of the inference library,
      • 
+
    • lib: the inference library built for various Android ABIs,
      • 
+
    • third_party: dependent third-party libraries built for Android.
      • 
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/mobile/cross_compiling_for_raspberry_en.html b/doc/mobile/cross_compiling_for_raspberry_en.html
new file mode 100644
index 0000000000000000000000000000000000000000..549d4d842503b28cee82660cad0c3377390dc4e5
--- /dev/null
+++ b/doc/mobile/cross_compiling_for_raspberry_en.html
@@ -0,0 +1,325 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Build PaddlePaddle for Raspberry Pi — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+    
+        
+         
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+        
    • MOBILE > 
      • 
+      
+    
    • Build PaddlePaddle for Raspberry Pi
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Build PaddlePaddle for Raspberry Pi
    
+
    You may use any of the following two approaches to build the inference library of PaddlePaddle for Raspberry Pi:
    
+
      
+
    1. Build using SSH: Log in to a Raspberry Pi using SSH and build the library. The required development tools and third-party dependencies are listed in here: /Dockerfile.
      2. 
+
    2. Cross-compile: We talk about how to cross-compile PaddlePaddle for Raspberry Pi on a Linux/x64 machine, in more detail in this article.
      3. 
+
    
+
    
+
    The Cross-Compiling Toolchain
    
+
    Step 1. Clone the Github repo by running the following command.
    
+
    git clone https://github.com/raspberrypi/tools.git
+
    
+
    
+
    Step 2. Use the pre-built cross-compiler found in ./tools/tree/master/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64.  To run it on a Linux computer, glibc version >= 2.14 is needed.
    
+
    
+
    
+
    CMake Arguments
    
+
    CMake supports cross-compiling.  All CMake configuration arguments required for the cross-compilation for Raspberry Pi can be found in cmake/cross_compiling/raspberry_pi.cmake.
    
+
    Some important arguments that need to be set:
    
+
      
+
    • CMAKE_SYSTEM_NAME: The target platform.  Must be RPi.
      • 
+
    • RPI_TOOLCHAIN: The absolute path of the cross-compiling toolchain.
      • 
+
    • RPI_ARM_NEON: Use ARM NEON Intrinsics. This is a required argument and set default to ON.
      • 
+
    • HOST_C/CXX_COMPILER: The C/C++ compiler for the host.  It is used to build building tools running on the host, for example, protoc.
      • 
+
    
+
    A commonly-used CMake configuration is as follows:
    
+
    cmake -DCMAKE_SYSTEM_NAME=RPi \
+      -DRPI_TOOLCHAIN=your/path/to/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64 \
+      -DRPI_ARM_NEON=ON \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_GPU=OFF \
+      -DWITH_C_API=ON \
+      -DWITH_PYTHON=OFF \
+      -DWITH_SWIG_PY=OFF \
+      ..
+
    
+
    
+
    To build the inference library, please set the argument WITH_C_API to ON: WITH_C_API=ON.
    
+
    You can add more arguments. For example, to minimize the size of the generated inference library, you may use CMAKE_BUILD_TYPE=MinSizeRel. For performance optimization, you may use CMAKE_BUILD_TYPE=Release.
    
+
    
+
    
+
    Build and Install
    
+
    The following commands build the inference library of PaddlePaddle for Raspberry Pi and third-party dependencies.
    
+
    make
+make install
+
    
+
    
+
    The intermediate files will be stored in build. Third-party libraries will be located in build/third_party. If you have already built it for other platforms like Android or iOS, you may want to clear these directories by running the command: rm -rf build.
    
+
    The infernece library will be in your/path/to/install/lib, with related header files in your/path/to/install/include.
    
+
    
+
    
+
+
+           
    
+          
    
+          
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/about/index_en.html b/doc/mobile/index_en.html
similarity index 67%
rename from doc/about/index_en.html
rename to doc/mobile/index_en.html
index 70a56744565756a5e92c229abf8a1bbf6aa7e474..de74244d3f6a1585437d84924a43f38c775fe79b 100644
--- a/doc/about/index_en.html
+++ b/doc/mobile/index_en.html
@@ -8,7 +8,7 @@
   
   
   
-  ABOUT — PaddlePaddle  documentation
+  MOBILE — PaddlePaddle  documentation
   
 
   
@@ -33,7 +33,8 @@
               href="../genindex.html"/>
         
     
-         
+        
+         
 
   
   
@@ -63,7 +64,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -85,7 +86,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -109,9 +110,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -136,17 +142,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -167,7 +196,7 @@
 
    
   
      
       
-    
    • ABOUT
      • 
+    
    • MOBILE
      • 
   
    
 
    
       
@@ -176,15 +205,13 @@
           
    
            
    
             
-  
    
-
    ABOUT
    
-
    PaddlPaddle is an easy-to-use, efficient, flexible and scalable deep learning platform,
-which is originally developed by Baidu scientists and engineers for the purpose of applying deep learning to many products at Baidu.
    
-
    PaddlePaddle is now open source but far from complete, which is intended to be built upon, improved, scaled, and extended.
-We hope to build an active open source community both by providing feedback and by actively contributing to the source code.
    
-
    
-
    Credits
    
-
    We owe many thanks to all contributors and developers of PaddlePaddle!
    
+  
 
@@ -195,8 +222,10 @@ We hope to build an active open source community both by providing feedback and
   
     
   
diff --git a/doc/objects.inv b/doc/objects.inv
index 186579d2607a16f4c0bf496eecaa725727d56b85..cf7d48907a2e58aeee28473704b33817ba5eed82 100644
Binary files a/doc/objects.inv and b/doc/objects.inv differ
diff --git a/doc/operators.json b/doc/operators.json
new file mode 100644
index 0000000000000000000000000000000000000000..873dbfee1b897104985329cf0851f1c2d86e35bc
--- /dev/null
+++ b/doc/operators.json
@@ -0,0 +1,5254 @@
+[
+{
+ "type" : "sgd",
+ "comment" : "\n\nSGD operator\n\nThis operator implements one step of the stochastic gradient descent algorithm.\n\n$$param\\_out = param - learning\\_rate * grad$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Param",
+   "comment" : "(Tensor) Input parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LearningRate",
+   "comment" : "(Tensor) Learning rate of SGD",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Grad",
+   "comment" : "(Tensor) Input gradient",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ParamOut",
+   "comment" : "(Tensor) Output parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "adagrad",
+ "comment" : "\n\nAdaptive Gradient Algorithm (Adagrad).\n\nThe update is done as follows:\n\n$$momentOut = moment + grad * grad \\break\nparamOut = param - learningRate * grad / ($\\sqrt{momentOut}$ + \\epsilon) \\break\n$$\n\nThe original paper(http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf)\ndoes not have the epsilon attribute. It is added here in our implementation\nas also proposed here: http://cs231n.github.io/neural-networks-3/#ada\nfor numerical stability to avoid the division by zero error.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Param",
+   "comment" : "(Tensor) Input parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Grad",
+   "comment" : "(Tensor) Input gradient",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Moment",
+   "comment" : "(Tensor) Second moment",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LearningRate",
+   "comment" : "(Tensor) Learning rate",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ParamOut",
+   "comment" : "(Tensor) Output parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "MomentOut",
+   "comment" : "(Tensor) Output second moment",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "epsilon",
+   "type" : "float",
+   "comment" : "(float, default 1.0e-6) Constant for numerical stability",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "conv3d",
+ "comment" : "\nConvolution3D Operator.\n\nThe convolution operation calculates the output based on the input, filter\nand strides, paddings, dilations, groups parameters. The size of each dimension of the\nparameters is checked in the infer-shape.\nInput(Input) and output(Output) are in NCDHW format, where N is batch\nsize, C is the number of channels,D is the depth of the feature, H is the height of\nthe feature, and W is the width of the feature.\nFilters(Input) is MCDHW format, where M is the number of output image channels,\nC is the number of input image channels, D is the depth of the filter,\nH is the height of the filter, and W is the width of the filter.\nParameters(strides, paddings, dilations) are three elements. These three elements\nrepresent depth, height and width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:\n  Input:\n       Input shape: $(N, C_{in}, D_{in}, H_{in}, W_{in})$\n       Filter shape: $(C_{out}, C_{in}, D_f, H_f, W_f)$\n  Output:\n       Output shape: $(N, C_{out}, D_{out}, H_{out}, W_{out})$\n  Where\n  $$\n       D_{out}= \\frac{(D_{in} + 2 * paddings[0] - (dilations[0] * (D_f - 1) + 1))}{ strides[0]}+ 1 \\\\\n       H_{out}= \\frac{(H_{in} + 2 * paddings[1] - (dilations[1] * (H_f - 1) + 1))}{ strides[1]}+ 1 \\\\\n       W_{out}= \\frac{(W_{in} + 2 * paddings[2] - (dilations[2] * (W_f - 1) + 1))}{ strides[2]}+ 1\n  $$\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(Tensor) The input tensor of convolution operator. The format of input tensor is NCDHW. Where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Filter",
+   "comment" : "(Tensor) The filter tensor of convolution operator. The format of the filter tensor is MCDHW, where M is the number of output image channels, C is the number of input image channels, D is the depth of the filter, H is the height of the filter, and W is the width of the filter.If the groups attribute is greater than 1, C equals the number of input image channels divided by the groups.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Output",
+   "comment" : "(Tensor) The output tensor of convolution operator.The format of output tensor is also NCDHW.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector, default:{1, 1, 1}), the strides(d_stride, h_stride, w_stride) of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector, default:{0, 0, 0}), the paddings(d_pad, h_pad, w_pad) of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "groups",
+   "type" : "int",
+   "comment" : "(int default:1), the groups number of the convolution operator. According to grouped convolution in Alex Krizhevsky's Deep CNN paper: when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels.",
+   "generated" : 0
+ }, { 
+   "name" : "dilations",
+   "type" : "int array",
+   "comment" : "(vector default:{1, 1, 1}), the dilations(d_dilation, h_dilation, w_dilation) of convolution operator.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "conv2d",
+ "comment" : "\nConvolution Operator.\n\nThe convolution operation calculates the output based on the input, filter\nand strides, paddings, dilations, groups parameters. The size of each dimension of the\nparameters is checked in the infer-shape.\nInput(Input) and Output(Output) are in NCHW format. Where N is batch\nsize, C is the number of channels, H is the height of the feature, and W is\nthe width of the feature.\nFilters(Input) is MCHW format. Where M is the number of output image channels, C is\nthe number of input image channels, H is the height of the filter, and W\nis the width of the filter.\nParameters(strides, paddings, dilations) are two elements. These two elements represent\nheight and width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:\n  Input:\n       Input shape: $(N, C_{in}, H_{in}, W_{in})$\n       Filter shape: $(C_{out}, C_{in}, H_f, W_f)$\n  Output:\n       Output shape: $(N, C_{out}, H_{out}, W_{out})$\n  Where\n$$\n       H_{out}= \\frac{(H_{in} + 2 * paddings[0] - (dilations[0] * (H_f - 1) + 1))}{strides[0]}+ 1 \\\\\n       W_{out}= \\frac{(W_{in} + 2 * paddings[1] - (dilations[1] * (W_f - 1) + 1))}{strides[1]}+ 1\n$$\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(Tensor) The input tensor of convolution operator. The format of input tensor is NCHW, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Filter",
+   "comment" : "(Tensor) The filter tensor of convolution operator. The format of the filter tensor is MCHW, where M is the number of output image channels, C is the number of input image channels, H is the height of the filter, and W is the width of the filter. If the groups attribute is greater than 1, C equals the number of input image channels divided by the groups.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Output",
+   "comment" : "(Tensor) The output tensor of convolution operator. The format of output tensor is also NCHW.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector default:{1, 1}), the strides(h_stride, w_stride) of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector default:{0, 0}), the paddings(h_pad, w_pad) of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "groups",
+   "type" : "int",
+   "comment" : "(int default:1), the groups number of the convolution operator. According to grouped convolution in Alex Krizhevsky's Deep CNN paper: when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels.",
+   "generated" : 0
+ }, { 
+   "name" : "dilations",
+   "type" : "int array",
+   "comment" : "(vector default:{1, 1}), the dilations(h_dilation, w_dilation) of convolution operator.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "pool3d",
+ "comment" : "\nPool3d Operator.\n\nThe pooling3d operation calculates the output based on\nthe input, pooling_type, ksize, strides, and paddings parameters.\nInput(X) and output(Out) are in NCDHW format, where N is batch\nsize, C is the number of channels, and D, H and W are the depth, height and\nwidth of the feature, respectively. Parameters(ksize, strides, paddings) \nare three elements. These three elements represent depth, height and \nwidth, respectively. The input(X) size and output(Out) size may be different.\n\nExample:\n  Input:\n       X shape: $(N, C, D_{in}, H_{in}, W_{in})$\n  Output:\n       Out shape: $(N, C, D_{out}, H_{out}, W_{out})$\n  Where\n  $$\n       D_{out} = \\frac{(D_{in} - ksize[0] + 2 * paddings[0])}{strides[0]} + 1 \\\\\n       H_{out} = \\frac{(H_{in} - ksize[1] + 2 * paddings[1])}{strides[1]} + 1 \\\\\n       W_{out} = \\frac{(W_{in} - ksize[2] + 2 * paddings[2])}{strides[2]} + 1\n  $$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor of pooling operator. The format of input tensor is NCDHW, where N is batch size, C is the number of channels, and D, H and W is the depth, height and width of the feature, respectively.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of pooling operator.The format of output tensor is also NCDHW, where N is batch size, C is the number of channels, and D, H and W is the depth, height and width of the feature, respectively.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "pooling_type",
+   "type" : "string",
+   "comment" : "(string) Pooling type, can be \"max\" for max-pooling and \"avg\" for average-pooling.",
+   "generated" : 0
+ }, { 
+   "name" : "ksize",
+   "type" : "int array",
+   "comment" : "(vector) The pooling window size(depth, height, width) of pooling operator. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "global_pooling",
+   "type" : "bool",
+   "comment" : "(bool, default false) Whether to use the global pooling. If global_pooling = true, ksize and paddings wille be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector, default {1,1,1}) Strides(depth, height, width) of the pooling operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector, default {0,0,0}), paddings(depth, height, width) of pooling operator. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "pool2d",
+ "comment" : "\nPool2d Operator.\n\nThe pooling2d operation calculates the output based on\nthe input, pooling_type and ksize, strides, paddings parameters.\nInput(X) and output(Out) are in NCHW format, where N is batch size, C is the\nnumber of channels, H is the height of the feature, and W is the width of the feature.\nParameters(ksize, strides, paddings) are two elements.\nThese two elements represent height and width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:   \n  Input:\n       X shape: $(N, C, H_{in}, W_{in})$\n  Output:\n       Out shape: $(N, C, H_{out}, W_{out})$\n  Where\n       $$ \n       H_{out} = \\frac{(H_{in} - ksize[0] + 2 * paddings[0])}{strides[0]} + 1 \\\\\n       W_{out} = \\frac{(W_{in} - ksize[1] + 2 * paddings[1])}{strides[1]} + 1\n       $$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor of pooling operator. The format of input tensor is NCHW, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of pooling operator. The format of output tensor is also NCHW, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "pooling_type",
+   "type" : "string",
+   "comment" : "(string), pooling type, can be \"max\" for max-pooling and \"avg\" for average-pooling.",
+   "generated" : 0
+ }, { 
+   "name" : "ksize",
+   "type" : "int array",
+   "comment" : "(vector) The pooling window size(height, width) of the pooling operator. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "global_pooling",
+   "type" : "bool",
+   "comment" : "(bool, default false) Whether to use the global pooling. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector, default {1, 1}), strides(height, width) of pooling operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector, default {0,0}), paddings(height, width) of pooling operator.If global_pooling = true, paddings and ksize will be ignored.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "max_pool3d_with_index",
+ "comment" : "\nMaxPool3d Operator.\n\nThe maxpooling3d with index operation calculates the output and the mask\nbased on the input and ksize, strides, paddings parameters.\nInput(X) and output(Out, Mask) are in NCDHW format, where N is batch\nsize, C is the number of channels, and D, H and W are the depth, height and\nwidth of the feature, respectively. \nParameters(ksize, strides, paddings) are three elements.\nThese three elements represent depth, height and width, respectively.\nThe input(X) size and output(Out, Mask) size may be different.\n\nExample:\n  Input:\n       X shape: $(N, C, D_{in}, H_{in}, W_{in})$\n  Output:\n       Out shape: $(N, C, D_{out}, H_{out}, W_{out})$\n       Mask shape: $(N, C, D_{out}, H_{out}, W_{out})$\n  Where\n       $$\n       D_{out} = \\frac{(D_{in} - ksize[0] + 2 * paddings[0])}{strides[0]} + 1 \\\\\n       H_{out} = \\frac{(H_{in} - ksize[1] + 2 * paddings[1])}{strides[1]} + 1 \\\\\n       W_{out} = \\frac{(W_{in} - ksize[2] + 2 * paddings[2])}{strides[2]} + 1\n       $$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor of pooling operator. The format of input tensor is NCDHW, where N is batch size, C is the number of channels, and D, H and W are the depth, height and width of the image, respectively",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of pooling operator. The format of output tensor is also NCDHW, where N is the batch size, C is the number of channels, and D, H and W are the depth, height and width of the image, respectively.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Mask",
+   "comment" : "(Tensor) The Mask tensor of pooling operator. The format of output tensor is also NCDHW, where N is the batch size, C is the number of channels, and D, H and W are the depth, height and width of the image, respectively. It represents the index in the current feature map.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "ksize",
+   "type" : "int array",
+   "comment" : "(vector) The pooling window size(depth, height, width) of pooling operator. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "global_pooling",
+   "type" : "bool",
+   "comment" : "(bool, default false) Whether to use the global pooling. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector, default {1,1,1}), strides(depth, height, width) of pooling operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector, default {0,0,0}), paddings(depth, height, width) of pooling operator. If global_pooling = true, paddings and ksize will be ignored.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "lod_rank_table",
+ "comment" : "Create LoDRanTable by LoDTensor\n\nLoD Rank Table stores the `level` of `lod` which is ordered by sequence\nlength in descending order. It is useful when implement dynamic RNN and is\nshared by dynamic RNN memory, dynamic RNN slice input and dynamic RNN slice\noutput operators.\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) input lod tensor, must contain lod information.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDRankTable) The rank table of specific level.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "level",
+   "type" : "int",
+   "comment" : "(int) the specific lod level to rank.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "array_to_lod_tensor",
+ "comment" : "This Op build a big LoDTensor from a std::vector \n          and a LoDRankTable. It is supposed to be used in getting dynamic RNN's\n          outputs back to a normal LoDTensor. The std::vector \n          would be the output of RNN Op and the LoDRankTable would be build \n          with RNN's input.",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(std::vector) A vector of tensors that is going to be casted to a big LoDTensor.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "RankTable",
+   "comment" : "(LoDRankTable) RankTable provides the coarse lod infomation to build the output LoDTensor. See 'paddle/framework/lod_rank_table.h' for more details.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) The LoDTensor formed by input tensor array.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "sequence_conv",
+ "comment" : "\nSequence Conv Operator.\n\nSequenceConvOp performs convolution operation on features of contextLength\ntime-steps of each instance. The convolution operation calculates the output\nbased on the input, filter, strides and paddings parameters.\nThe size of each dimension of the parameters is checked during infer-shape.\nIn order to ensure the equal length of sequence before and after convolution,\nit is necessary to fill the top and bottom of each sequence based on\ncontext_length, context_stride and context_start.\n\n    ",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) the input(X) is a LodTensor, which supports variable-time length input sequence. The underlying tensor in this LoDTensor is a matrix with shape (T, N), where T is the total time steps in this mini-batch and N is the input_hidden_size.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "PaddingData",
+   "comment" : "(Tensor, optional) the input(PaddingData) is an optional parameter, and it is learnable. This is a tensor with shape (P, N), where P is the top_pad + bottom_pad, N is the input_hidden_size. In order to ensure the equal length of sequence before and after convolution, it is necessary to fill the top and bottom of each sequence according to context_length, context_stride and context_start",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Filter",
+   "comment" : "(Tensor) the input(Filter) is an learnable parameter.This is a tensor with shape (K, M), where K is the context_length * input_hidden_size, M is the output feature size.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) the output(Out) is a LodTensor, which support variable-time length output sequence. The underlying tensor in this LoDTensor is a matrix with shape (T, M), where, T is the total time steps in this mini-batch, M is the output feature size.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "paddingTrainable",
+   "type" : "bool",
+   "comment" : "(bool, default:false) the padding data of SequenceConvOp is trainable or not.",
+   "generated" : 0
+ }, { 
+   "name" : "contextLength",
+   "type" : "int",
+   "comment" : "(int) the contextLength of SequenceConvOp is the height of the convolution kernel.",
+   "generated" : 0
+ }, { 
+   "name" : "contextStart",
+   "type" : "int",
+   "comment" : "(int, default:0) the contextStart of SequenceConvOp represents the beginning of the convolution of the number of rows of sequence, which can be negative. The negative number means to pad contextStart time-steps of zeros or learnable parameters at the beginning of each instance. The positive number means to skip contextStart time-steps of each instance.",
+   "generated" : 0
+ }, { 
+   "name" : "contextStride",
+   "type" : "int",
+   "comment" : "(int, default:1) the contextStride of SequenceConvOp represents the stride length of convolution kernel. Currently, SequenceConvOp only supportscontextStride=1.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "sequence_pool",
+ "comment" : "\nSequence Pool Operator.\n\nThe SequencePoolOp pools features of all time-steps of each instance.\nIt supports six pooling types:\n1. AVERAGE: $$Out[i] = \\frac{\\sum_i X_i}{N}$$\n2. SUM:     $$Out[i] = \\sum_jX_{ij}$$\n3. SQRT:    $$Out[i] = \\frac{\\sum_jX_{ij}}{\\sqrt{len(X_i)}}$$\n4. LAST:    Out[i] = last instance in i-th sequence X[i]\n5. FIRST:   Out[i] = first instance in i-th sequence X[i]\n6. MAX:     $$Out[i] = max(X_i)$$\n\nThe following example explains how this works:\nFor a mini-batch of 3 variable-length sentences,\ncontaining 2, 3, and 2 time-steps:\n\nAssume X is a [7,M,N] LoDTensor, and X->lod()[0] = [0, 2, 5, 7], 7=2+3+2.\nBesides, for the sake of simplicity, we assume M=1 and N=1,\nand the value of X = [[1, 3], [2, 4, 6], [5, 1]].\n\nThus, Out is a [3,1,1] Tensor without LoD infomation.\nAnd for different pooltype, the value of Out is as follows:\n\n- AVERAGE: [2, 4, 3], where 2=(1+3)/2, 4=(2+4+6)/3, 3=(5+1)/2\n- SUM: [4, 12, 6], where 4=1+3, 12=2+4+6, 6=5+1\n- SQRT: [2.82, 6.93, 4.24], where 2.82=(1+3)/sqrt(2),\n           6.93=(2+4+6)/sqrt(3), 4.24=(5+1)/sqrt(2)\n- MAX: [3, 6, 5], where 3=max(1,3), 6=max(2,4,6), 5=max(5,1)\n- LAST: [3, 6, 1], where 3=last(1,3), 6=last(2,4,6), 1=last(5,1)\n- FIRST: [1, 2, 5], where 1=first(1,3), 2=first(2,4,6), 5=first(5,1)\n\n    ",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) The variable-length input of SequencePoolOp",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output of SequencePoolOp does not contain LoD infomation.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "MaxIndex",
+   "comment" : "(Tensor) This tensor is used for the sequence max-pooling to record the max indexes.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "pooltype",
+   "type" : "string",
+   "comment" : "(int, default AVERAGE) the pooling pooltype of SequencePoolOp.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "lstm",
+ "comment" : "\nLong-Short Term Memory (LSTM) Operator.\n\nThe defalut implementation is diagonal/peephole connection\n(https://arxiv.org/pdf/1402.1128.pdf), the formula is as follows:\n\n$$\ni_t = \\sigma(W_{ix}x_{t} + W_{ih}h_{t-1} + W_{ic}c_{t-1} + b_i) \\\\\n\nf_t = \\sigma(W_{fx}x_{t} + W_{fh}h_{t-1} + W_{fc}c_{t-1} + b_f) \\\\\n\n\\tilde{c_t} = act_g(W_{cx}x_t + W_{ch}h_{t-1} + b_c) \\\\\n\no_t = \\sigma(W_{ox}x_{t} + W_{oh}h_{t-1} + W_{oc}c_t + b_o) \\\\\n\nc_t = f_t \\odot c_{t-1} + i_t \\odot \\tilde{c_t} \\\\\n\nh_t = o_t \\odot act_h(c_t)\n$$\n\nwhere the W terms denote weight matrices (e.g. $W_{xi}$ is the matrix\nof weights from the input gate to the input), $W_{ic}, W_{fc}, W_{oc}$\nare diagonal weight matrices for peephole connections. In our implementation,\nwe use vectors to reprenset these diagonal weight matrices. The b terms\ndenote bias vectors ($b_i$ is the input gate bias vector), $\\sigma$\nis the non-line activations, such as logistic sigmoid function, and\n$i, f, o$ and $c$ are the input gate, forget gate, output gate,\nand cell activation vectors, respectively, all of which have the same size as\nthe cell output activation vector $h$.\n\nThe $\\odot$ is the element-wise product of the vectors. $act_g$ and $act_h$\nare the cell input and cell output activation functions and `tanh` is usually\nused for them. $\\tilde{c_t}$ is also called candidate hidden state,\nwhich is computed based on the current input and the previous hidden state.\n\nSet `use_peepholes` False to disable peephole connection. The formula\nis omitted here, please refer to the paper\nhttp://www.bioinf.jku.at/publications/older/2604.pdf for details.\n\nNote that these $W_{xi}x_{t}, W_{xf}x_{t}, W_{xc}x_{t}, W_{xo}x_{t}$\noperations on the input $x_{t}$ are NOT included in this operator.\nUsers can choose to use fully-connect operator before LSTM operator.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(LoDTensor) the first input is a LodTensor, which support variable-time length input sequence. The underlying tensor in this LoDTensor is a matrix with shape (T X 4D), where T is the total time steps in this mini-batch, D is the hidden size.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "H0",
+   "comment" : "(Tensor, optional) the initial hidden state is an optional input. This is a tensor with shape (N x D), where N is the batch size and D is the hidden size.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "C0",
+   "comment" : "(Tensor, optional) the initial cell state is an optional input. This is a tensor with shape (N x D), where N is the batch size. `H0` and `C0` can be NULL but only at the same time",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Weight",
+   "comment" : "(Tensor) the learnable hidden-hidden weights. - The shape is (D x 4D), where D is the hidden size.  - Weight = {W_ch, W_ih, W_fh, W_oh}",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Bias",
+   "comment" : "(Tensor) the learnable weights, which contains two parts: input-hidden bias weight and peephole connections weight if setting `use_peepholes` True. 1. `use_peepholes = False`  - The shape is (1 x 4D).  - Bias = {b_c, b_i, b_f, b_o}.2. `use_peepholes = True`  - The shape is (1 x 7D).  - Bias = {b_c, b_i, b_f, b_o, W_ic, W_fc, W_oc}.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Hidden",
+   "comment" : "(LoDTensor) the hidden state of LSTM operator. The shape is (T x D), and lod is the same with the `Input`.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Cell",
+   "comment" : "(LoDTensor) the cell state of LSTM operator. The shape is (T x D), and lod is the same with the `Input`.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "BatchGate",
+   "comment" : "(LoDTensor) This LoDTensor contains input gate, forget gate and output gate after the nonlinear computation. This LoDTensor has the same shape as the reorganized input, which is also be called batch input. The LoD size is 2. The first LoD is the batch offsets and the second LoD contains the indexes, which denote the position of reorganized sequence in the raw input.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "BatchCellPreAct",
+   "comment" : "(LoDTensor) This LoDTensor is obtained in the forward and used in the backward.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "use_peepholes",
+   "type" : "bool",
+   "comment" : "(bool, defalut: True) whether to enable diagonal/peephole connections.",
+   "generated" : 0
+ }, { 
+   "name" : "is_reverse",
+   "type" : "bool",
+   "comment" : "(bool, defalut: False) whether to compute reversed LSTM.",
+   "generated" : 0
+ }, { 
+   "name" : "gate_activation",
+   "type" : "string",
+   "comment" : "(string, default: sigmoid)The activation for input gate, forget gate and output gate, `sigmoid` by default.",
+   "generated" : 0
+ }, { 
+   "name" : "cell_activation",
+   "type" : "string",
+   "comment" : "(string, default: tanh)The activation for cell output, `tanh` by defalut.",
+   "generated" : 0
+ }, { 
+   "name" : "candidate_activation",
+   "type" : "string",
+   "comment" : "(string, default: tanh)The activation for candidate hidden state, `tanh` by default.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "conv3d_transpose",
+ "comment" : "\nConvolution3D Transpose Operator.\n\nThe convolution transpose operation calculates the output based on the input, filter\nand strides, paddings, groups parameters. The size of each dimension of the\nparameters is checked in the infer-shape.\nInput(Input) and output(Output) are in NCDHW format. Where N is batch size, C is the\nnumber of channels, D is the depth of the feature, H is the height of the feature,\nand W is the width of the feature.\nFilter(Input) is in MCDHW format. Where M is the number of input feature channels,\nC is the number of output feature channels, D is the depth of the filter,H is the\nheight of the filter, and W is the width of the filter.\nParameters(strides, paddings) are three elements. These three elements represent\ndepth, height and width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:   \n  Input:\n       Input shape: $(N, C_{in}, D_{in}, H_{in}, W_{in})$\n       Filter shape: $(C_{in}, C_{out}, D_f, H_f, W_f)$\n  Output:\n       Output shape: $(N, C_{out}, D_{out}, H_{out}, W_{out})$\n  Where\n  $$\n       D_{out} = (D_{in} - 1) * strides[0] - 2 * paddings[0] + D_f \\\\\n       H_{out} = (H_{in} - 1) * strides[1] - 2 * paddings[1] + H_f \\\\\n       W_{out} = (W_{in} - 1) * strides[2] - 2 * paddings[2] + W_f\n  $$\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(Tensor) The input tensor of convolution transpose operator.The format of input tensor is NCDHW. Where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Filter",
+   "comment" : "(Tensor) The filter tensor of convolution transpose operator.The format of the filter tensor is MCDHW, where M is the number of input feature channels, C is the number of output feature channels, D is the depth of the filter, H is the height of the filter, and W is the width of the filter.We enforce groups number == 1 and padding == 0 in the convolution3d transpose scenario.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Output",
+   "comment" : "(Tensor) The output tensor of convolution transpose operator.The format of output tensor is also NCDHW.Where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector default:{1, 1, 1}), the strides{d_stride, h_stride, w_stride} of convolution transpose operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector default:{0, 0, 0}), paddings(d_pad, h_pad, w_pad) of convolution transpose operator.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "conv2d_transpose",
+ "comment" : "\nConvolution2D Transpose Operator.\n\nThe convolution transpose operation calculates the output based on the input, filter\nand strides, paddings, groups parameters. The size of each dimension of the\nparameters is checked in the infer-shape.\nInput(Input) and output(Output) are in NCHW format. Where N is batchsize, C is the\nnumber of channels, H is the height of the feature, and W is the width of the feature.\nFilter(Input) is in MCHW format. Where M is the number of input feature channels,\nC is the number of output feature channels, H is the height of the filter,\nand W is the width of the filter.\nParameters(strides, paddings) are two elements. These two elements represent height\nand width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:\n  Input:\n       Input shape: $(N, C_{in}, H_{in}, W_{in})$\n       Filter shape: $(C_{in}, C_{out}, H_f, W_f)$\n  Output:\n       Output shape: $(N, C_{out}, H_{out}, W_{out})$\n  Where\n  $$\n       H_{out} = (H_{in} - 1) * strides[0] - 2 * paddings[0] + H_f \\\\\n       W_{out} = (W_{in} - 1) * strides[1] - 2 * paddings[1] + W_f\n  $$\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(Tensor) The input tensor of convolution transpose operator. The format of input tensor is NCHW. Where N is batch size, C is the number of input channels, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Filter",
+   "comment" : "(Tensor) The filter tensor of convolution transpose operator. The format of the filter tensor is MCHW, where M is the number of input feature channels, C is the number of output feature channels,H is the height of the filter, and W is the width of the filter. We enforce groups number == 1 in the convolution transpose scenario.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Output",
+   "comment" : "(Tensor) The output tensor of convolution transpose operator. The format of output tensor is also NCHW.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector default:{1, 1}), the strides(h_stride, w_stride) of convolution transpose operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector default:{0, 0}), the paddings(h_pad, w_pad) of convolution transpose operator.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "gru",
+ "comment" : "\nGRU Operator implements part calculations of the complete GRU as following:\n\n\\f[\nupdate \\ gate: u_t = actGate(xu_t + W_u * h_{t-1} + b_u) \\\\\nreset \\ gate: r_t = actGate(xr_t + W_r * h_{t-1} + b_r)  \\\\\noutput \\ candidate: {h}_t = actNode(xc_t + W_c * dot(r_t, h_{t-1}) + b_c) \\\\\noutput: h_t = dot((1 - u_t), h_{t-1}) + dot(u_t, {h}_t)\n\\f]\n\n@note To implement the complete GRU, fully-connected operator must be used  \nbefore to feed xu, xr and xc as the Input of GRU operator.\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(LoDTensor) The first input is a LodTensor, which supports variable-time length input sequence. The underlying tensor in this LoDTenosr is a matrix with shape (T X 3D), where, T is the total time steps in this mini-batch, D is the hidden size.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "H0",
+   "comment" : "(Tensor, optional) The initial hidden state is an optional input. This is a tensor with shape (N x D), where N is the batch size, D is the hidden size.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Weight",
+   "comment" : "(Tensor) The learnable hidden-hidden weight matrix with shape (D x 3D), where D is the hidden size. The elements continuous in memory can be divided into two parts. The first part are weights of the update gate and reset gate with shape (D x 2D), and the second part are weights of output candidate with shape (D x D).",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Bias",
+   "comment" : "(Tensor, optional) Bias vector with shape (1 x 3D) concating bias of the update gate, reset gate and output candidate.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "BatchGate",
+   "comment" : "(LoDTensor) To compute with batches, sequence data will be reorganized into several successive batches each containing data from the same time step. The LoDTensor BatchGate contains the update gate, reset gate and output candidate values organized in batches. The LoD size is 2. The first LoD contains the batch offsets and the second LoD contains the indexes in the raw sequence data.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "BatchResetHiddenPrev",
+   "comment" : "(LoDTensor) The reseted hidden state LoDTensor organized in batches. This LoDTensor is a matrix with shape (T X D) and has the same LoD with `BatchGate`.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "BatchHidden",
+   "comment" : "(LoDTensor) The hidden state LoDTensor organized in batches.  This LoDTensor is a matrix with shape (T X D) and has the same LoD with `BatchGate`.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "Hidden",
+   "comment" : "(LoDTensor) the hidden state LoDTensor organized in sequences. This LoDTensor is a matrix with shape (T X D) and has the same LoD with `BatchGate`.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "activation",
+   "type" : "string",
+   "comment" : "(string, default tanh) The activation type used for output candidate {h}_t.",
+   "generated" : 0
+ }, { 
+   "name" : "gate_activation",
+   "type" : "string",
+   "comment" : "(string, default sigmoid) The activation type used in update gate and reset gate.",
+   "generated" : 0
+ }, { 
+   "name" : "is_reverse",
+   "type" : "bool",
+   "comment" : "(bool, defalut: False) whether to compute reversed GRU.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "recurrent",
+ "comment" : "\nStatic Length Recurrent Operator.\n\nThe static length recurrent operator can only operate on fixed size sequence\ndata, i.e. in each mini-batch, the sequence length of all inputs are the same.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "inputs",
+   "comment" : "rnn inputs",
+   "duplicable" : 1,
+   "intermediate" : 0
+ }, { 
+   "name" : "initial_states",
+   "comment" : "rnn initial states",
+   "duplicable" : 1,
+   "intermediate" : 0
+ }, { 
+   "name" : "parameters",
+   "comment" : "Parameters are used by step block as its input. However, the input is not a sequence tensor. Every time step, each operator in step block just use the parameter directly.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "outputs",
+   "comment" : "The output sequence of RNN. The sequence length must be same.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ }, { 
+   "name" : "step_scopes",
+   "comment" : "StepScopes contain all local variables in each time step.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "ex_states",
+   "type" : "string array",
+   "comment" : "The ex-state variable names.\nThe ex-state means the state value in the ex-timestep or the previous time step\n[ex_states, states, initial_states@GRAD] must be the same order",
+   "generated" : 0
+ }, { 
+   "name" : "states",
+   "type" : "string array",
+   "comment" : "The state variable names. [ex_states, states, initial_states@GRAD] must be the same order",
+   "generated" : 0
+ }, { 
+   "name" : "step_block",
+   "type" : "block id",
+   "comment" : "The step block inside RNN",
+   "generated" : 0
+ }, { 
+   "name" : "reverse",
+   "type" : "bool",
+   "comment" : "Calculate RNN reversely or not.\nBy default reverse=False\n\nAssume the input data is [A, B, C, D]\n\nif reverse is False:\n  the computation of RNN is like\n      A          B          C         D\n      |          |          |         |\n      v          v          v         v\n     rnn -----> rnn -----> rnn ----> rnn\n      |          |          |         |\n      v          v          v         v\n      o          o          o         o\n\nif reverse is True\n  the computation of RNN is like\n      A          B          C         D\n      |          |          |         |\n      v          v          v         v\n     rnn <----- rnn <----- rnn <---- rnn\n      |          |          |         |\n      v          v          v         v\n      o          o          o         o\n",
+   "generated" : 0
+ }, { 
+   "name" : "is_train",
+   "type" : "bool",
+   "comment" : "",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "save",
+ "comment" : "\nSave operator\n\nThis operator will serialize and write a tensor variable to file on disk.\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor ) Input tensor to be saved",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [  ], 
+ "attrs" : [ 
+ { 
+   "name" : "overwrite",
+   "type" : "bool",
+   "comment" : "(boolean, default true)Overwrite the output file if exist",
+   "generated" : 0
+ }, { 
+   "name" : "file_path",
+   "type" : "string",
+   "comment" : "(string)The \"file_path\" where the variable will be saved.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "load",
+ "comment" : "\nLoad Operator.\n\nLoad operator will load a tensor variable from disk file.\n\n",
+ "inputs" : [  ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The tensor need to be loaded",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "file_path",
+   "type" : "string",
+   "comment" : "(string) Variable will be loaded from \"file_path\".",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "accuracy",
+ "comment" : "\nAccuracy Operator. \n\nIt will print accuracy rate for classification.\nThe accuracy is calculated as follows:\n\n$$accuracy = \\frac{NumOfCorrectPredicts}{NumOfAllSamples}$$\n\nBoth the input Out and Label can carry the LoD (Level of Details)\ninformation, or not. But the output only shares the LoD information \nwith the input Out(Inference).\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The network output of topk (inferences)",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Indices",
+   "comment" : "The the network output of topk (indices)",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Label",
+   "comment" : "Label of the training data",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Accuracy",
+   "comment" : "The accuracy of current batch",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Correct",
+   "comment" : "The correct samples count of current batch",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Total",
+   "comment" : "The samples count of current batch",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "hard_sigmoid",
+ "comment" : "\nHardSigmoid Activation Operator.\n\nSegment-wise linear approximation of sigmoid(https://arxiv.org/abs/1603.00391), \nwhich is much faster than sigmoid.\n\n$y = \\max(0, \\min(1, slope * x + shift))$\n\nThe slope should be positive. The offset can be either positive or negative.\nThe default slope and shift are set according to the above reference.\nIt is recommended to use the defaults for this activation.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of HardSigmoid operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of HardSigmoid operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "slope",
+   "type" : "float",
+   "comment" : "Slope for linear approximation of sigmoid",
+   "generated" : 0
+ }, { 
+   "name" : "offset",
+   "type" : "float",
+   "comment" : "Offset for linear approximation of sigmoid",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "cond",
+ "comment" : "\nSample Dependent Conditional Operator.\n\nGiven Cond[i] as a 1/0 vector to indicate true/false:\nOut[i] = subnet_true[i], if Cond[i] == true\nOut[i] = subnet_false[i], if Cond[i] == false\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Cond",
+   "comment" : "The condition, which is a bool vector",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Xs",
+   "comment" : "Inputs of Subnets",
+   "duplicable" : 1,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Outs",
+   "comment" : "Outputs of Cond_Op after merge",
+   "duplicable" : 1,
+   "intermediate" : 0
+ }, { 
+   "name" : "SubScopes",
+   "comment" : "sub scopes for true and false branches",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "IndexTensors",
+   "comment" : "Index Tensors contains indices for true/false",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "max_pool2d_with_index",
+ "comment" : "\nMaxPool2d Operator.\n\nThe maxPooling2d with index operation calculates the output and the mask\nbased on the input, ksize, strides, and paddings parameters. Input(X) and\noutput(Out, Mask) are in NCHW format, where N is batch size, C is the\nnumber of channels, H is the height of the feature, \nand W is the width of the feature.\nParameters(ksize, strides, paddings) are two elements.\nThese two elements represent height and width, respectively.\nThe input(X) size and output(Out, Mask) size may be different.\n\nExample:\n  Input:\n       X shape: $(N, C, H_{in}, W_{in})$\n  Output:\n       Out shape: $(N, C, H_{out}, W_{out})$\n       Mask shape: $(N, C, H_{out}, W_{out})$\n  Where\n       $$\n       H_{out} = \\frac{(H_{in} - ksize[0] + 2 * paddings[0])}{strides[0]} + 1 \\\\\n       W_{out} = \\frac{(W_{in} - ksize[1] + 2 * paddings[1])}{strides[1]} + 1\n       $$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor of pooling operator. The format of input tensor is NCHW, where N is batch size, C is the number of channels, H is the height of the image, and W is the width of the image.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of pooling operator. The format of output tensor is also NCHW, where N is batch size, C is the number of channels, H is the height of the image and W is the width of the image.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Mask",
+   "comment" : "(Tensor) The Mask tensor of pooling operator.The format of output tensor is also NCHW, where N is batch size, C is the number of channels, H is the height of the image, and W is the width of the image. It represents the index in the current feature map.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "ksize",
+   "type" : "int array",
+   "comment" : "(vector) The pooling window size(height, width) of pooling operator. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "global_pooling",
+   "type" : "bool",
+   "comment" : "(bool, default:false) Whether to use the global pooling. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector, default {1, 1}), strides(height, width) of pooling operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector, default:{0, 0}), paddings(height, width) of pooling operator. If global_pooling = true, paddings and will be ignored.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "thresholded_relu",
+ "comment" : "\nThresholdedRelu Activation Operator.\n\n$$\ny = \\begin{cases} \n    x, \\text{if } x > threshold \\\\\n    0,  \\text{otherwise}\n    \\end{cases}\n$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of ThresholdedRelu operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of ThresholdedRelu operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "threshold",
+   "type" : "float",
+   "comment" : "The threshold location of activation",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "hard_shrink",
+ "comment" : "\nHardShrink Activation Operator.\n\n$$\ny = \\begin{cases} \n    x, \\text{if } x > \\lambda \\\\\n    x, \\text{if } x < -\\lambda \\\\\n    0,  \\text{otherwise}\n    \\end{cases}\n$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of HardShrink operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of HardShrink operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "threshold",
+   "type" : "float",
+   "comment" : "The value of threshold for HardShrink",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "relu6",
+ "comment" : "\nRelu6 Activation Operator.\n\n$y = \\min(\\max(0, x), 6)$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Relu6 operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Relu6 operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "threshold",
+   "type" : "float",
+   "comment" : "The threshold value of Relu6",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "elu",
+ "comment" : "\nELU Activation Operator.\n\nApplies the following element-wise computation on the input according to\nhttps://arxiv.org/abs/1511.07289.\n\n$y = \\max(0, x) + \\min(0, \\alpha * (e^x - 1))$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of ELU operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of ELU operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "alpha",
+   "type" : "float",
+   "comment" : "The alpha value of ELU",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "leaky_relu",
+ "comment" : "\nLeakyRelu Activation Operator.\n\n$y = \\max(x, \\alpha * x)$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of LeakyRelu operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of LeakyRelu operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "alpha",
+   "type" : "float",
+   "comment" : "The small negative slope",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "logical_xor",
+ "comment" : "logical_xor Operator\n\nIt operates element-wise on X and Y, and returns the Out. X, Y and Out are N-dim boolean tensors.\nEach element of Out is calculated by $$Out = (X || Y) \\, \\&\\& \\, !(X \\&\\& Y)$$\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) Left hand operand of logical_xor operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(LoDTensor) Right hand operand of logical_xor operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) n-dim bool tensor. Each element is $$Out = (X || Y) \\, \\&\\& \\, !(X \\&\\& Y)$$",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "softmax",
+ "comment" : "\nSoftmax Operator.\n\nThe input of the softmax operator is a 2-D tensor with shape N x K (N is the\nbatch_size, K is the dimension of input feature). The output tensor has the\nsame shape as the input tensor.\n\nFor each row of the input tensor, the softmax operator squashes the\nK-dimensional vector of arbitrary real values to a K-dimensional vector of real\nvalues in the range [0, 1] that add up to 1.\nIt computes the exponential of the given dimension and the sum of exponential\nvalues of all the other dimensions in the K-dimensional vector input.\nThen the ratio of the exponential of the given dimension and the sum of\nexponential values of all the other dimensions is the output of the softmax\noperator.\n\nFor each row $i$ and each column $j$ in Input(X), we have:\n    $$Y[i, j] = \\frac{\\exp(X[i, j])}{\\sum_j(exp(X[i, j])}$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input tensor of softmax. 2-D with shape [batch_size, input_feature_dimensions].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "The normalized values with the same shape as X.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "lod_array_length",
+ "comment" : "\nLoDArrayLength Operator.\n\nThis operator obtains the length of lod tensor array:\n\n$$Out = len(X)$$\n\nNOTE: The output is a CPU Tensor since the control variable should be only in\nCPU and the length of LoDTensorArray should be used as control variables.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensorArray) The input tensor array.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) 1x1 CPU Tensor of length, int64_t",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "top_k",
+ "comment" : "\nTop K operator\n\nIf the input is a vector (1d tensor), this operator finds the k largest \nentries in the vector and outputs their values and indices as vectors. \nThus values[j] is the j-th largest entry in input, and its index is indices[j].\n\nFor matrices, this operator computes the top k entries in each row. ",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input of Topk op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of Topk op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Indices",
+   "comment" : "(Tensor) The indices of Topk elements of input",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "k",
+   "type" : "int",
+   "comment" : "(int, default 1) Number of top elements to look for along the last dimension (along each row for matrices).",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "clip",
+ "comment" : "\nClip Operator.\n\nThe clip operator limits the value of given input within an interval. The interval is\nspecified with arguments 'min' and 'max':\n\n$$\nOut = \\min(\\max(X, min), max)\n$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor)The input of clip op.The number of dimensions must be between [1, 9].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor)The output of clip op with shape as input(X)",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "min",
+   "type" : "float",
+   "comment" : "(float)Minimum value, under which element is replaced by min.",
+   "generated" : 0
+ }, { 
+   "name" : "max",
+   "type" : "float",
+   "comment" : "(float)Maximum value, above which element is replaced by max",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "margin_rank_loss",
+ "comment" : "\nMarginRankLoss Operator.\n\nThis operator measures the loss given a pair of training sample\n{`X1`, `X2`} and the `Label` with attribute `margin`, where `Label = +1` \nindicating X1 is ranked higher than `X2` and `Label = -1` otherwise. The loss \nis calculated as:\n\n$loss(X1, X2, Label) = \\max(0, -Label * (X1 - X2) + margin)$\n\nThe attribute `margin` here helps make the predictions more robust.\nDenote the item ranked higher as the positive sample, otherwise the negative \nsample. If the score of the two samples satisfies \n\n$positive sample - negative sample < margin$\n\nthe pair of samples will contribute to the final loss, which will backpropagate \nand train the ranking model to enlarge the difference between the two scores.\n\nFor batch input with size `batch_size`, `X1`, `X2` and `Label`\nall have the same shape [batch_size x 1].\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X1",
+   "comment" : "(2-D tensor with shape [batch_size x 1]) The score for one item X1 to be ranked, from pairwise ranking model.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "X2",
+   "comment" : "(2-D tensor with shape [batch_size x 1]) The score for another item X2 to be ranked, from pairwise ranking model.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Label",
+   "comment" : "(2-D tensor with shape [batch_size x 1]) The label indicating X1 ranked higher than X2 or not, can only be +1 or -1.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Activated",
+   "comment" : "(2-D tensor with shape [batch_size x 1]) Intermediate tensor to indicate whether each element of Output(Out) is activated.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "Out",
+   "comment" : "(2-D tensor with shape [batch_size x 1]) The output loss of MarginRankLoss operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "margin",
+   "type" : "float",
+   "comment" : "(scalar, default 0) Margin for MarginRankLossOp.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "mul",
+ "comment" : "\nMul Operator. \n\nThis operator is used to perform matrix multiplication for input X and Y.\n\nThe equation is:\n\n    $$Out = X * Y$$\n\nBoth the input `X` and `Y` can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD information with input `X`.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The first input of mul op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "The second input of mul op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of mul op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "x_num_col_dims",
+   "type" : "int",
+   "comment" : "(int, default 1) mul_op can take tensors with more than two dimensions as input `X`,\n            in that case, tensors will be reshaped to a matrix. The matrix's first\n            dimension(column length) will be the product of tensor's last\n            `num_col_dims` dimensions, and the matrix's second dimension(row length)\n            will be the product of tensor's first `rank - num_col_dims` dimensions.\n        ",
+   "generated" : 0
+ }, { 
+   "name" : "y_num_col_dims",
+   "type" : "int",
+   "comment" : "(int, default 1) mul_op can take tensors with more than two dimensions as input `Y`,\n             in that case, tensors will be reshaped to a matrix. Just like input `X`.\n        ",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "merge_lod_tensor",
+ "comment" : "\n        Merge True and False branches of LoDTensor into a single Output,\n        with a mask at certain lod level. X is used to obtain complete\n        lod information. Please refer to SplitLoDTensorOp.",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input LoDTensor, contains complete lod information to construct the output",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Mask",
+   "comment" : "A bool column vector which mask the input",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "InTrue",
+   "comment" : "The True branch to be merged",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "InFalse",
+   "comment" : "The False branch to be merged",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The merged output LoDTensor",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "level",
+   "type" : "int",
+   "comment" : "(int) the specific lod level to rank.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "minus",
+ "comment" : "\nMinus Operator.\n\nEquation:\n\n    $Out = X - Y$\n\nBoth the input `X` and `Y` can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD information with input `X`.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The left tensor of minus operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "The right tensor of minus operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output tensor of minus operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "proximal_adagrad",
+ "comment" : "\nProximal Adagrad Optimizer.\n\nOptimizer that implements the proximal adagrad algorithm:\n\n$$\nmoment = moment + grad * grad \\\\\nprox\\_param = param - learning\\_rate * grad * (1 / \\sqrt{moment}) \\\\\nparam = sign(prox\\_param) / (1 + learning\\_rate * l2) *\n        \\max(|prox\\_param| - learning\\_rate * l1 , 0)\n$$\n\nThe paper that proposed Proximal GD: \n(http://papers.nips.cc/paper/3793-efficient-learning-using-forward-backward-splitting.pdf)\nHere, we use the adagrad learning rate as specified here: \n(http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf)\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Param",
+   "comment" : "(Tensor, default Tensor) Input parameter that has to be updated.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Moment",
+   "comment" : "(Tensor, default Tensor) Moment parameter that has to be updated.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Grad",
+   "comment" : "(Tensor, default Tensor) Input gradient of the parameter.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LearningRate",
+   "comment" : "(Tensor, default Tensor) The learning rate should be a tensor of size 1.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ParamOut",
+   "comment" : "(Tensor) Output updated parameter value.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "MomentOut",
+   "comment" : "(Tensor) Output updated moment value.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "l1",
+   "type" : "float",
+   "comment" : "(float, default 0.0) L1 regularization strength.",
+   "generated" : 0
+ }, { 
+   "name" : "l2",
+   "type" : "float",
+   "comment" : "(float, default 0.0) L2 regularization strength.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "max_sequence_len",
+ "comment" : "Calculate the max sequence length through lod_rank_table.",
+ "inputs" : [ 
+ { 
+   "name" : "RankTable",
+   "comment" : "The lod_rank_table.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The max sequence length.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "multiplex",
+ "comment" : "\nMultiplex Operator.\n\nMultiplex multiple tensors according to the index provided by the index tensor.\n\nIds: the index tensor.\nX[0 : N - 1]: the candidate tensors for output (N >= 2).\nFor each index i from 0 to batchSize - 1, the output is the i-th row of the\nthe (Ids[i])-th tensor.\n\nFor i-th row of the output tensor:\n\n$$y[i] = x_{k}[i]$$\n\nwhere `y` is the output tensor, `x_{k}` is the k-th input tensor,\nand `k = Ids[i]`.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Ids",
+   "comment" : "The index tensor of multiplex operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "X",
+   "comment" : "The candidate tensors of multiplex operator.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output tensor of multiplex operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "pool3d_cudnn",
+ "comment" : "\nPool3d Operator.\n\nThe pooling3d operation calculates the output based on\nthe input, pooling_type, ksize, strides, and paddings parameters.\nInput(X) and output(Out) are in NCDHW format, where N is batch\nsize, C is the number of channels, and D, H and W are the depth, height and\nwidth of the feature, respectively. Parameters(ksize, strides, paddings) \nare three elements. These three elements represent depth, height and \nwidth, respectively. The input(X) size and output(Out) size may be different.\n\nExample:\n  Input:\n       X shape: $(N, C, D_{in}, H_{in}, W_{in})$\n  Output:\n       Out shape: $(N, C, D_{out}, H_{out}, W_{out})$\n  Where\n  $$\n       D_{out} = \\frac{(D_{in} - ksize[0] + 2 * paddings[0])}{strides[0]} + 1 \\\\\n       H_{out} = \\frac{(H_{in} - ksize[1] + 2 * paddings[1])}{strides[1]} + 1 \\\\\n       W_{out} = \\frac{(W_{in} - ksize[2] + 2 * paddings[2])}{strides[2]} + 1\n  $$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor of pooling operator. The format of input tensor is NCDHW, where N is batch size, C is the number of channels, and D, H and W is the depth, height and width of the feature, respectively.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of pooling operator.The format of output tensor is also NCDHW, where N is batch size, C is the number of channels, and D, H and W is the depth, height and width of the feature, respectively.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "pooling_type",
+   "type" : "string",
+   "comment" : "(string) Pooling type, can be \"max\" for max-pooling and \"avg\" for average-pooling.",
+   "generated" : 0
+ }, { 
+   "name" : "ksize",
+   "type" : "int array",
+   "comment" : "(vector) The pooling window size(depth, height, width) of pooling operator. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "global_pooling",
+   "type" : "bool",
+   "comment" : "(bool, default false) Whether to use the global pooling. If global_pooling = true, ksize and paddings wille be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector, default {1,1,1}) Strides(depth, height, width) of the pooling operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector, default {0,0,0}), paddings(depth, height, width) of pooling operator. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "pow",
+ "comment" : "\nPow Activation Operator.\n\n$y = x^{factor}$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Pow operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Pow operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "factor",
+   "type" : "float",
+   "comment" : "The exponential factor of Pow",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "l1_norm",
+ "comment" : "\nL1 Norm Operator.\n\nComputes the L1 norm of a tensor.\n\n$$Out = \\sum{|X|}$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input of l1_norm op.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Scalar) The output of l1_norm op.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "sqrt",
+ "comment" : "\nSqrt Activation Operator.\n\n$y = \\sqrt{x}$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Sqrt operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Sqrt operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "lookup_table",
+ "comment" : "\nLookup Table Operator.\n\nThis operator is used to perform lookups on the parameter W,\nthen concatenated into a dense tensor.\n\nThe input Ids can carry the LoD (Level of Details) information,\nor not. And the output only shares the LoD information with input Ids.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "W",
+   "comment" : "An input represents embedding tensors, which is a learnable parameter.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Ids",
+   "comment" : "An input with type int32 or int64 contains the ids to be looked up in W. Ids must be a column vector with rank = 2. The 2nd dimension size must be 1.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The lookup results, which have the same type as W.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "is_sparse",
+   "type" : "bool",
+   "comment" : "(boolean, default false) Sparse update",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "positive_negative_pair",
+ "comment" : "\n        PositiveNegativePairOp can be used to evaluate Learning To Rank(LTR) \n        model performance. \n        Within some context, e.g. the \"query\", a LTR model generates scores\n        for a list of items, which gives a partial order of the items.\n        PositiveNegativePairOp takes a list of reference rank order \n        (Input(\"Label\")) and the model generated scores (Input(Score)) as \n        inputs and counts the pairs that ranked correctly and incorrectly.\n",
+ "inputs" : [ 
+ { 
+   "name" : "Score",
+   "comment" : "(Tensor, float) Model Score on an item (with respect to QueryID). It's a 2-D tensor with shape [batch_size, depth], where the column specified by the attribute \"column\" is used as item score.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Label",
+   "comment" : "(Tensor, float) Label of an item (with repsect to QueryId). It's a 2-D tensor with shape [batch_size, 1].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "QueryID",
+   "comment" : "(Tensor, int64) Query ID that indicates the context. Its shape should be the same as Label.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "AccumulatePositivePair",
+   "comment" : "(float) Optional. The accumulated number of positive pairs over a stream of data. If provided, the output PositivePair will be initialized with this number rather than 0. it won't be modified in place.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "AccumulateNegativePair",
+   "comment" : "(float) Optional. The accumulated number of negative pairs over a stream of data. If provided, the output NegativePair will be initialized with this number rather than 0. it won't be modified in place.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "AccumulateNeutralPair",
+   "comment" : "(float) Optional. The accumulated number of neutral pairs over a stream of data. If provided, the output NeutralPair will be initialized with this number rather than 0. it won't be modified in place.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Weight",
+   "comment" : "(float) Optional. Weight of current item. If specified, its shape should be the same as Label, and the meaning of the output changes from numbers of pairs to the total sum of pairs' weights. Weight of a pair of items is the average of their weights.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "PositivePair",
+   "comment" : "(float) Number of positive pairs, i.e. the pairs of items that are ranked correctly.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "NegativePair",
+   "comment" : "(float) Number of negative pairs, i.e. the pairs of items that are ranked incorrectly.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "NeutralPair",
+   "comment" : "(float) Number of neutral pairs, i.e. the pairs of items that have the same score.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "column",
+   "type" : "int",
+   "comment" : "(int, default -1) The column position of Score used to rank items in descending order. It must be in the range of [-rank(Score), rank(Score)). If `dim < 0`, the dim to reduce is `rank + dim`. Noting that reducing on the first dim will make the LoD info lost.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "proximal_gd",
+ "comment" : "\nProximalGD Operator.\n\nOptimizer that implements the proximal gradient descent algorithm:\n\n$$\nprox\\_param = param - learning\\_rate * grad \\\\\nparam = sign(prox\\_param) / (1 + learning\\_rate * l2) *\n        \\max(|prox\\_param| - learning\\_rate * l1, 0)\n$$        \n\nThe paper that proposed Proximal Gradient Descent:\n(http://papers.nips.cc/paper/3793-efficient-learning-using-forward-backward-splitting.pdf)\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Param",
+   "comment" : "(Tensor, default Tensor) Input parameter value that has to be updated.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Grad",
+   "comment" : "(Tensor, default Tensor) Input gradient of the parameter.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LearningRate",
+   "comment" : "(Tensor, default Tensor) The learning rate should be a tensor of size 1.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ParamOut",
+   "comment" : "(Tensor) Output updated parameter value.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "l1",
+   "type" : "float",
+   "comment" : "(float, default 0.0) L1 regularization strength.",
+   "generated" : 0
+ }, { 
+   "name" : "l2",
+   "type" : "float",
+   "comment" : "(float, default 0.0) L2 regularization strength.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "prelu",
+ "comment" : "\nPRelu Operator.\n\nThe equation is:\n\n$$\nf(x) =\n\\begin{cases}\n\\alpha * x, \\quad  \\text{if} \\ x < 0 \\\\\nx,         \\qquad  \\text{if} \\ x >= 0\n\\end{cases}\n$$\n\nThe input `X` can carry the LoD (Level of Details) information,\nor not. And the output shares the LoD information with input `X`.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input tensor of prelu operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Alpha",
+   "comment" : "The alpha weight of prelu operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output tensor of prelu operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "precision_recall",
+ "comment" : "\nPrecision Recall Operator.\n\nWhen given Input(Indices) and Input(Labels), this operator can be used\nto compute various metrics including:\n1. macro average precision\n2. macro average recall\n3. macro f1 score\n4. micro average precision\n5. micro average recall\n6. micro f1 score\n\nTo compute the above metrics, we need to do statistics for true positives,\nfalse positives and false negatives. Here the count of true negatives is not\nnecessary, but counting it may provide potential usage and the cost is\ntrivial, so the operator also provides the count of true negatives.\n\nWe define state as a 2-D tensor with shape [class_number, 4]. Each row of a\nstate contains statistic variables for corresponding class. Layout of each row\nis: TP(true positives), FP(false positives), TN(true negatives),\nFN(false negatives). If Input(Weights) is provided, TP, FP, TN, FN will be\ncalculated by given weight instead of the instance count.\n\nThis operator also supports metrics computing for cross-batch situation. To\nachieve this, Input(StatesInfo) should be provided. State of current batch\ndata will be accumulated to Input(StatesInfo) and Output(AccumStatesInfo)\nis the accumulation state.\n\nOutput(BatchMetrics) is metrics of current batch data while\nOutput(AccumStatesInfo) is metrics of accumulation data.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "MaxProbs",
+   "comment" : "(Tensor, default Tensor) A 2-D tensor with shape N x 1, where N is the batch size. Each row contains the max probability of an instance which computed by the previous top_k (k=1) operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Indices",
+   "comment" : "(Tensor, default Tensor) A 2-D tensor with shape N x 1, where N is the batch size. Each row contains the corresponding index which computed by the previous top_k (k=1) operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Labels",
+   "comment" : "(Tensor, default Tensor) A 2-D tensor with shape N x 1, where N is the batch size. Each element is a label and the value should be in [0, class_number - 1].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Weights",
+   "comment" : "(Tensor, default Tensor) A 2-D tensor with shape N x 1, where N is the batch size. This input is optional. If provided, weight of instance would be considered when computing metrics.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "StatesInfo",
+   "comment" : "(Tensor, default Tensor) A 2-D tensor with shape D x 4, where D is the number of classes. This input is optional. If provided, current state will be accumulated to this state and the accumulation state will be the output state.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "BatchMetrics",
+   "comment" : "(Tensor, default Tensor) A 1-D tensor with shape {6}. This output tensor contains metrics for current batch data. The layout is [macro average precision, macro average recall, macro f1 score, micro average precision, micro average recall, micro f1 score].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "AccumMetrics",
+   "comment" : "(Tensor, default Tensor) A 1-D tensor with shape {6}. This output tensor contains metrics for accumulated data. The layout is [macro average precision, macro average recall, macro f1 score, micro average precision, micro average recall, micro f1 score].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "AccumStatesInfo",
+   "comment" : "(Tensor, default Tensor) A 2-D tensor with shape D x 4, where D is equal to class number. This output tensor contains accumulated state variables used to compute metrics. The layout for each class is [true positives, false positives, true negatives, false negatives].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "class_number",
+   "type" : "int",
+   "comment" : "(int) Number of classes to be evaluated.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "conv3d_cudnn",
+ "comment" : "\nConvolution3D Operator.\n\nThe convolution operation calculates the output based on the input, filter\nand strides, paddings, dilations, groups parameters. The size of each dimension of the\nparameters is checked in the infer-shape.\nInput(Input) and output(Output) are in NCDHW format, where N is batch\nsize, C is the number of channels,D is the depth of the feature, H is the height of\nthe feature, and W is the width of the feature.\nFilters(Input) is MCDHW format, where M is the number of output image channels,\nC is the number of input image channels, D is the depth of the filter,\nH is the height of the filter, and W is the width of the filter.\nParameters(strides, paddings, dilations) are three elements. These three elements\nrepresent depth, height and width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:\n  Input:\n       Input shape: $(N, C_{in}, D_{in}, H_{in}, W_{in})$\n       Filter shape: $(C_{out}, C_{in}, D_f, H_f, W_f)$\n  Output:\n       Output shape: $(N, C_{out}, D_{out}, H_{out}, W_{out})$\n  Where\n  $$\n       D_{out}= \\frac{(D_{in} + 2 * paddings[0] - (dilations[0] * (D_f - 1) + 1))}{ strides[0]}+ 1 \\\\\n       H_{out}= \\frac{(H_{in} + 2 * paddings[1] - (dilations[1] * (H_f - 1) + 1))}{ strides[1]}+ 1 \\\\\n       W_{out}= \\frac{(W_{in} + 2 * paddings[2] - (dilations[2] * (W_f - 1) + 1))}{ strides[2]}+ 1\n  $$\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(Tensor) The input tensor of convolution operator. The format of input tensor is NCDHW. Where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Filter",
+   "comment" : "(Tensor) The filter tensor of convolution operator. The format of the filter tensor is MCDHW, where M is the number of output image channels, C is the number of input image channels, D is the depth of the filter, H is the height of the filter, and W is the width of the filter.If the groups attribute is greater than 1, C equals the number of input image channels divided by the groups.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Output",
+   "comment" : "(Tensor) The output tensor of convolution operator.The format of output tensor is also NCDHW.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector, default:{1, 1, 1}), the strides(d_stride, h_stride, w_stride) of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector, default:{0, 0, 0}), the paddings(d_pad, h_pad, w_pad) of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "groups",
+   "type" : "int",
+   "comment" : "(int default:1), the groups number of the convolution operator. According to grouped convolution in Alex Krizhevsky's Deep CNN paper: when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels.",
+   "generated" : 0
+ }, { 
+   "name" : "dilations",
+   "type" : "int array",
+   "comment" : "(vector default:{1, 1, 1}), the dilations(d_dilation, h_dilation, w_dilation) of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "workspace_size_MB",
+   "type" : "int",
+   "comment" : "workspace size for cudnn, in MB, workspace is a section of GPU memory which will be allocated/freed each time the operator runs, larger workspace size can increase performance but also requires better hardware. This size should be chosen carefully.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "sigmoid_cross_entropy_with_logits",
+ "comment" : "\nSigmoidCrossEntropyWithLogits Operator.\n\nThis measures the element-wise probability error in classification tasks\nin which each class is independent. This can be thought of as predicting labels\nfor a data-point, where labels are not mutually exclusive.\nFor example, a news article can be about politics, technology or sports\nat the same time or none of these.\n\nThe logistic loss is given as follows:\n\n       $$loss = -Labels * \\log(\\sigma(X)) - (1 - Labels) * \\log(1 - \\sigma(X))$$\n\nWe know that $$\\sigma(X) = (1 / (1 + \\exp(-X)))$$. By substituting this we get:\n\n       $$loss = X - X * Labels + \\log(1 + \\exp(-X))$$\n\nFor stability and to prevent overflow of $$\\exp(-X)$$ when X < 0,\nwe reformulate the loss as follows:\n\n       $$loss = \\max(X, 0) - X * Labels + \\log(1 + \\exp(-|X|))$$\n\nBoth the input `X` and `Labels` can carry the LoD (Level of Details) information.\nHowever the output only shares the LoD with input `X`.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor, default Tensor), a 2-D tensor with shape N x D, where N is the batch size and D is the number of classes. This input is a tensor of logits computed by the previous  operator. Logits are unscaled log probabilities given as log(p/(1-p)).",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Label",
+   "comment" : "(Tensor, default Tensor), a 2-D tensor of the same type and shape as X. This input is a tensor of probabalistic labels for each logit",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor, default Tensor), a 2-D tensor with shape N x D  of elementwise logistic losses.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "rnn_memory_helper",
+ "comment" : "",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "dtype",
+   "type" : "int",
+   "comment" : "(int, default 5 (FP32)) Output data type",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "pad",
+ "comment" : "\nPad Operator.\n\nPad input into output, as specified by paddings and pad_value. \nThe input should be a k-D tensor(k > 0 and k < 7). As an example:\n\nGiven:\n\nX = [[1, 2],\n     [3, 4]],\n\npaddings = [0, 1, 1, 2],\n\nand\n\npad_value = 0,\n\nwe have:\n\nOut = [[0, 1, 2, 0, 0]\n       [0, 3, 4, 0, 0]\n       [0, 0, 0, 0, 0]]\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input of pad op. The input should be a k-D tensor(k > 0 and k < 7)",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of pad op. A tensor with the same shape as X.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector) A list to describe the padding rules for each dimension. For 2-D image tensor, paddings=[0, 1, 2, 3] means padding 0 row to top, 1 row to bottom, 2 columns to left and 3 columns to right. Size of paddings should be equal to 2 * dimension size of the input tensor.",
+   "generated" : 0
+ }, { 
+   "name" : "pad_value",
+   "type" : "float",
+   "comment" : "(float, default 0.0) The value to fill the padded areas.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "lstm_unit",
+ "comment" : "\nLstm Unit Operator\n\nEquation:\n\n$$\ni, f, o, j = split(X) \\\\\nC = C_{prev} * sigm(f + forget\\_bias) + sigm(i) * tanh(j) \\\\\nH = C * sigm(o)\n$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "FC input before the non-linear activation.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "C_prev",
+   "comment" : "The cell state tensor of last time-step in the Lstm Unit operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "C",
+   "comment" : "The cell tensor of Lstm Unit operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "H",
+   "comment" : "The hidden state tensor of Lstm Unit operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "forget_bias",
+   "type" : "float",
+   "comment" : "(float, default 0.0) The forget bias of Lstm Unit.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "squared_l2_norm",
+ "comment" : "\nSquaredL2Norm Operator.\n\nComputes the squared L2 norm of a tensor.\n\n$$Out = \\sum_{i} X_{i}^2$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input of squared_l2_norm op.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Scalar) The output of squared_l2_norm op.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "rmsprop",
+ "comment" : "\nRmsprop Optimizer. \n\n$$\nMeanSquareOut = decay * MeanSquare + (1 - decay) * Grad * Grad \\\\\nMomentOut = momentum * Moment +\n            \\frac{LearningRate * Grad}{\\sqrt{MeanSquareOut + epsilon}} \\\\\nParamOut = Param -  MomentOut\n$$\n\nThe original slides that proposed Rmsprop: Slide 29 of\nhttp://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf)\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Param",
+   "comment" : "(Tensor, default Tensor) Input parameter value that has to be updated.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "MeanSquare",
+   "comment" : "(Tensor, default Tensor) The mean square value that gets updated.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LearningRate",
+   "comment" : "(Tensor, default Tensor) The learning rate should be a tensor of size 1.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Grad",
+   "comment" : "(Tensor, default Tensor) Input gradient of the parameter.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Moment",
+   "comment" : "(Tensor, default Tensor) The moment that gets updated.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ParamOut",
+   "comment" : "(Tensor) Output updated parameter value.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "MomentOut",
+   "comment" : "(Tensor) Output updated moment.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "MeanSquareOut",
+   "comment" : "(Tensor) Output Mean squared updated value.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "epsilon",
+   "type" : "float",
+   "comment" : "(float, default 1e-10) Constant for numerical stability.",
+   "generated" : 0
+ }, { 
+   "name" : "decay",
+   "type" : "float",
+   "comment" : "(float, default 0.9) Discounting factor for coming gradient.",
+   "generated" : 0
+ }, { 
+   "name" : "momentum",
+   "type" : "float",
+   "comment" : "(float, default 0.0) Constant value.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "elementwise_mul",
+ "comment" : "\nLimited Elementwise Mul Operator.\n\nThe equation is:\n\n$Out = X \\odot\\ Y$\n\nX is a tensor of any dimension and the dimensions of tensor Y must be smaller than\nor equal to the dimensions of X. \n\nThere are two cases for this operator:\n1. The shape of Y is same with X;\n2. The shape of Y is a subset of X.\n\nFor case 2:\nY will be broadcasted to match the shape of X and axis should be \nthe starting dimension index for broadcasting Y onto X.\n\nexample:\n  shape(X) = (2, 3, 4, 5), shape(Y) = (,)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (5,)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1\n  shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0\n\nBoth the input X and Y can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD information with input X.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The first input tensor of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(Tensor) The second input tensor of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "axis",
+   "type" : "int",
+   "comment" : "(int, default -1) The starting dimension index for broadcasting Y onto X",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "uniform_random",
+ "comment" : "\nUniform random operator.\n\nThis operator initializes a tensor with random values sampled from a \nuniform distribution.\n\n",
+ "inputs" : [  ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of uniform random op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "shape",
+   "type" : "int array",
+   "comment" : "(vector) The shape of the output tensor",
+   "generated" : 0
+ }, { 
+   "name" : "min",
+   "type" : "float",
+   "comment" : "(float, default -1.0) Minimum value of uniform random",
+   "generated" : 0
+ }, { 
+   "name" : "max",
+   "type" : "float",
+   "comment" : "(float, default 1.0) Maximun value of uniform random",
+   "generated" : 0
+ }, { 
+   "name" : "seed",
+   "type" : "int",
+   "comment" : "(int, default 0) Random seed used for generating samples. 0 means use a seed generated by the system.",
+   "generated" : 0
+ }, { 
+   "name" : "dtype",
+   "type" : "int",
+   "comment" : "(int, default 5(FP32)) Output tensor data type",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "sign",
+ "comment" : "\nSign operator\n\n$$Out = X.sign()$$\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) Input tensor of sign operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) Output tensor of sign operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "roi_pool",
+ "comment" : "\nROIPool operator\n\nROI Pooling for Faster-RCNN. The link below is a further introduction: \nhttps://stackoverflow.com/questions/43430056/what-is-roi-layer-in-fast-rcnn\n    ",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor), the input of ROIPoolOp. The format of input tensor is NCHW. Where N is batch size, C is the number of input channels, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "ROIs",
+   "comment" : "(Tensor), ROIs (Regions of Interest) to pool over. should be a 2-D tensor of shape (num_rois, 5)given as [[batch_id, x1, y1, x2, y2], …]. Where batch_id is the id of the data, (x1, y1) is the top left coordinates, and (x2, y2) is the bottom right coordinates.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor), The output of ROIPoolOp is a 4-D tensor with shape (num_rois, channels, pooled_h, pooled_w).",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Argmax",
+   "comment" : "(Tensor), Argmaxes corresponding to indices in X used for gradient computation. Only output if arg “is_test” is false.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "spatial_scale",
+   "type" : "float",
+   "comment" : "(float, default 1.0), Multiplicative spatial scale factor to translate ROI coords from their input scale to the scale used when pooling.",
+   "generated" : 0
+ }, { 
+   "name" : "pooled_height",
+   "type" : "int",
+   "comment" : "(int, default 1), The pooled output height.",
+   "generated" : 0
+ }, { 
+   "name" : "pooled_width",
+   "type" : "int",
+   "comment" : "(int, default 1), The pooled output width.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "conv2d_transpose_cudnn",
+ "comment" : "\nConvolution2D Transpose Operator.\n\nThe convolution transpose operation calculates the output based on the input, filter\nand strides, paddings, groups parameters. The size of each dimension of the\nparameters is checked in the infer-shape.\nInput(Input) and output(Output) are in NCHW format. Where N is batchsize, C is the\nnumber of channels, H is the height of the feature, and W is the width of the feature.\nFilter(Input) is in MCHW format. Where M is the number of input feature channels,\nC is the number of output feature channels, H is the height of the filter,\nand W is the width of the filter.\nParameters(strides, paddings) are two elements. These two elements represent height\nand width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:\n  Input:\n       Input shape: $(N, C_{in}, H_{in}, W_{in})$\n       Filter shape: $(C_{in}, C_{out}, H_f, W_f)$\n  Output:\n       Output shape: $(N, C_{out}, H_{out}, W_{out})$\n  Where\n  $$\n       H_{out} = (H_{in} - 1) * strides[0] - 2 * paddings[0] + H_f \\\\\n       W_{out} = (W_{in} - 1) * strides[1] - 2 * paddings[1] + W_f\n  $$\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(Tensor) The input tensor of convolution transpose operator. The format of input tensor is NCHW. Where N is batch size, C is the number of input channels, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Filter",
+   "comment" : "(Tensor) The filter tensor of convolution transpose operator. The format of the filter tensor is MCHW, where M is the number of input feature channels, C is the number of output feature channels,H is the height of the filter, and W is the width of the filter. We enforce groups number == 1 in the convolution transpose scenario.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Output",
+   "comment" : "(Tensor) The output tensor of convolution transpose operator. The format of output tensor is also NCHW.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector default:{1, 1}), the strides(h_stride, w_stride) of convolution transpose operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector default:{0, 0}), the paddings(h_pad, w_pad) of convolution transpose operator.",
+   "generated" : 0
+ }, { 
+   "name" : "dilations",
+   "type" : "int array",
+   "comment" : "dilations of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "workspace_size_MB",
+   "type" : "int",
+   "comment" : "workspace size for cudnn, in MB, workspace is a section of GPU memory which will be allocated/freed each time the operator runs, larger workspace size can increase performance but also requires better hardward. This size should be carefully setted.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "reduce_min",
+ "comment" : "\n{ReduceOp} Operator.\n\nThis operator computes the min of input tensor along the given dimension. \nThe result tensor has 1 fewer dimension than the input unless keep_dim is true.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor. Tensors with rank at most 6 are supported.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The result tensor.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "dim",
+   "type" : "int",
+   "comment" : "(int, default 0) The dimension to reduce. Must be in the range [-rank(input), rank(input)). If `dim < 0`, the dim to reduce is `rank + dim`. Note that reducing on the first dim will make the LoD info lost.",
+   "generated" : 0
+ }, { 
+   "name" : "keep_dim",
+   "type" : "bool",
+   "comment" : "(bool, default false) If true, retain the reduced dimension with length 1.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "smooth_l1_loss",
+ "comment" : "\nSmooth L1 Loss Operator.\n\nThis operator computes the smooth l1 loss for X and Y.\nThe operator takes the first dimension of X and Y as batch size.\nFor each instance, it computes the smooth l1 loss element by element first\nand then sums all the losses. So the shape of Out is [batch_size, 1].\n\nThe equation is:\n$$\nOut_{\\sigma}(X, Y)_i = \\begin{cases}\n0.5 * (\\sigma * (X_i - Y_i)) ^ 2\n\\quad |X_i - Y_i| \\lt \\frac{1} {{\\sigma} ^ 2} \\\\\n\\frac{|X_i - Y_i| - 0.5}{{\\sigma}^2},\n\\quad otherwise\n\\end{cases}\n$$\n\nIn the above equation, $Out_{\\sigma}(X, Y)_i$, $X_i$ and $Y_i$ represent the ith\nelement of Out, X and Y.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor, default Tensor) A tensor with rank at least 2. The input value of smooth l1 loss op with shape [batch_size, dim1, ..., dimN].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(Tensor, default Tensor) A tensor with rank at least 2. The target value of smooth l1 loss op with same shape as X.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "InsideWeight",
+   "comment" : "(Tensor, default Tensor) A tensor with rank at least 2. This input is optional and should have same shape with X. If provided, the result of (X - Y) will be multiplied by this tensor element by element.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "OutsideWeight",
+   "comment" : "(Tensor, default Tensor) A tensor with rank at least 2. This input is optional and should have same shape with X. If provided, the out smooth l1 loss will be multiplied by this tensor element by element.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Diff",
+   "comment" : "Intermediate variable to cache InsideWeight * (X - Y).",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "Out",
+   "comment" : "(Tensor, default Tensor) A tensor with rank be 2. The output smooth l1 loss with shape [batch_size, 1].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "sigma",
+   "type" : "float",
+   "comment" : "Hyper parameter of smooth l1 loss op.A float scalar with default value 3.0.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "log_loss",
+ "comment" : "\nLogLoss Operator.\n\nLog loss is a loss function used for binary classification. Log Loss quantifies\nthe accuracy of a classifier by penalising false classifications. Minimising the\nLog Loss is equivalent to maximising the accuracy of the classifier. We define\nPredicted as the values predicted by our model and Labels as the target ground\ntruth value. Log loss can evaluate how close the predicted values are to the\ntarget. The shapes of Predicted and Labels are both [batch_size, 1].\nThe equation is:\n\n$$\nLoss = - Labels * log(Predicted + \\epsilon) -\n        (1 - Labels) * log(1 - Predicted + \\epsilon)\n$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Predicted",
+   "comment" : "The input value (Predicted) of Log loss op.Predicted is a 2-D tensor with shape [batch_size, 1].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Labels",
+   "comment" : "The target value (Labels) of Log loss op.Labels is a 2-D tensor with shape [batch_size, 1].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Loss",
+   "comment" : "The output tensor with shape [batch_size, 1] which represents the log loss.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "epsilon",
+   "type" : "float",
+   "comment" : "Epsilon in log loss.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "unpool",
+ "comment" : "\n        \"Input shape: $(N, C_{in}, H_{in}, W_{in})$\n        Output shape: $(N, C_{out}, H_{out}, W_{out})$\n        Where\n          $$\n            H_{out} = (H_{in}−1) * strides[0] − 2 * paddings[0] + ksize[0] \\\\\n            W_{out} = (W_{in}−1) * strides[1] − 2 * paddings[1] + ksize[1]\n          $$\n        Paper: http://www.matthewzeiler.com/wp-content/uploads/2017\n        /07/iccv2011.pdf\n        ",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor of unpool operator. The format of input tensor is NCHW. Where N is batch size, C is the number of channels, H and W is the height and width of feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Indices",
+   "comment" : "(Tensor) The input tensor of the indices given out by MaxPool2d. The format of input tensor is NCHW. Where N is batch size, C is the number of channels, H and W is the height and width of feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of unpool operator.The format of output tensor is also NCHW.Where N is batch size, C is the number of channels, H and W is the height and width of feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "ksize",
+   "type" : "int array",
+   "comment" : "(vector), the unpooling window size(height, width) of unpooling operator.",
+   "generated" : 0
+ }, { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector, default:{1, 1}), strides (height, width) of unpooling operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector defalut:{0,0}), paddings (height, width) of unpooling operator.",
+   "generated" : 0
+ }, { 
+   "name" : "unpooling_type",
+   "type" : "string",
+   "comment" : "(string), unpooling type, can be \"max\" for max-unpooling ",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "transpose",
+ "comment" : "\nTranspose Operator.\n\nThe input tensor will be permuted according to the axis values given.\nThe op functions similar to how numpy.transpose works in python.\nFor example:\n >> input = numpy.arange(6).reshape((2,3))\n >> input\n array([[0, 1, 2],\n        [3, 4, 5]])\n >> axis = [1, 0]\n >> output = input.transpose(axis)\n >> output\n array([[0, 3],\n        [1, 4],\n\t\t[2, 5]])\nSo, given a input tensor of shape(N, C, H, W) and the axis is {0, 2, 3, 1},\nthe output tensor shape will be (N, H, W, C)\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor)The input tensor, tensors with rank at most 6 are supported",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor)The output tensor",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "axis",
+   "type" : "int array",
+   "comment" : "(vector)A list of values, and the size of the list should be the same with the input tensor rank, the tensor will permute the axes according the the values given",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "rnn_memory_helper_grad",
+ "comment" : "",
+ "inputs" : [ 
+ { 
+   "name" : "Out@GRAD",
+   "comment" : "",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "X",
+   "comment" : "",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Out",
+   "comment" : "",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "X@GRAD",
+   "comment" : "",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "dtype",
+   "type" : "int",
+   "comment" : "(int, default 5 (FP32)) Output data type",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "swish",
+ "comment" : "\nSwish Activation Operator.\n\n$$y = \\frac{x}{1 + e^{- \\beta x}}$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Swish operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Swish operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "beta",
+   "type" : "float",
+   "comment" : "Constant beta of swish operator",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "is_empty",
+ "comment" : "\nIsEmpty Operator which checks whether a tensor is empty.\n\nIt will just return product(tensor.ddims()) > 0;\n              ",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) Tensor which is to be checked.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) a boolean Tensor that indicate empty or not.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "reduce_max",
+ "comment" : "\n{ReduceOp} Operator.\n\nThis operator computes the max of input tensor along the given dimension. \nThe result tensor has 1 fewer dimension than the input unless keep_dim is true.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor. Tensors with rank at most 6 are supported.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The result tensor.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "dim",
+   "type" : "int",
+   "comment" : "(int, default 0) The dimension to reduce. Must be in the range [-rank(input), rank(input)). If `dim < 0`, the dim to reduce is `rank + dim`. Note that reducing on the first dim will make the LoD info lost.",
+   "generated" : 0
+ }, { 
+   "name" : "keep_dim",
+   "type" : "bool",
+   "comment" : "(bool, default false) If true, retain the reduced dimension with length 1.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "sequence_softmax",
+ "comment" : "\nSequence Softmax Operator.\n\nSequenceSoftmaxOp computes the softmax activation among all time-steps for each\nsequence. The dimension of each time-step should be 1. Thus, the shape of\ninput Tensor can be either [N, 1] or [N], where N is the sum of the length\nof all sequences.\n\nThe algorithm works as follows:\n    for i-th sequence in a mini-batch:\n        $$Out(X[lod[i]:lod[i+1]], :) =\n            \\frac{\\exp(X[lod[i]:lod[i+1], :])}\n            {\\sum(\\exp(X[lod[i]:lod[i+1], :]))}$$\n\nFor example, for a mini-batch of 3 sequences with variable-length,\neach containing 2, 3, 2 time-steps, the lod of which is [0, 2, 5, 7],\nthen softmax will be computed among X[0:2, :], X[2:5, :], X[5:7, :]\nand N turns out to be 7.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) 1-D or 2-D input LoDTensor with the 2-nd dimension of length 1.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) 1-D or 2-D output LoDTensor with the 2-nd dimension of length 1.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "momentum",
+ "comment" : "\nMomentum Optimizer.\n\nThis optimizer has a flag for Nestrov Momentum.\nThe update equations are as follows:\n\n$$\nvelocity = mu * velocity + gradient \\\\\nif (use\\_nesterov):   \\\\\n  param = param - gradient * learning\\_rate + mu * velocity * learning\\_rate \\\\\nelse:   \\\\\n  param = param - learning\\_rate * velocity. \\\\\n$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Param",
+   "comment" : "(Tensor, default Tensor) Input parameter that has to be updated",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Grad",
+   "comment" : "(Tensor, default Tensor) Input gradient of the parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Velocity",
+   "comment" : "(Tensor, default Tensor) Input velocity (corresponding to the parameter) that has to be updated",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LearningRate",
+   "comment" : "(Tensor, default Tensor) Input learning rate",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ParamOut",
+   "comment" : "(Tensor) This output is updated parameter. It shared memory with Input(Param).",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "VelocityOut",
+   "comment" : "(Tensor) This output is updated velocity. It shared memory with Input(Velocity).",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "mu",
+   "type" : "float",
+   "comment" : "(float) Momentum coefficient",
+   "generated" : 0
+ }, { 
+   "name" : "use_nesterov",
+   "type" : "bool",
+   "comment" : "(bool, default false) Use Nesterov Momentum",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "scatter",
+ "comment" : "\nScatter Operator.\n\nThis operator obtains output by updating the input on selected indices on the first axis:\n\n$$\nOut = Ref \\\\\nOut[Index] = Ref[Index] + Updates\n$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Ref",
+   "comment" : "The source input of scatter op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Index",
+   "comment" : "The index input of scatter op where Ref will be updated",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Updates",
+   "comment" : "The updated value of updates op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of add op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "shrink_rnn_memory",
+ "comment" : "\n        In dynamic RNN, we are able to handle sequences of different lengths. \n        Because of the multiple lengths, the size of each step input can be \n        different, which may lead to a mismatching between the input of\n        the current step and the memory generated by the previous one. This \n        operator shrinks memory according to the size of the next step input, \n        to make sure that they can match each other.\n        ",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) The RNN step memory to be shrinked.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "RankTable",
+   "comment" : "(LoDRankTable) The lod_rank_table of dynamic RNN.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "I",
+   "comment" : "(LoDTensor) The step index. The RNN step memory 'X' will be shrinked to match the size of the input of the index'th step.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) The shrinked RNN step memory.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "lod_reset",
+ "comment" : "LoDReset operator\n\nReset LoD of Input(X) into a new one specified by Input(TargetLoD) or\nAttr(target_lod), or set LoD for Input(X) if it doesn't have one.\nCurrently the lod_reset operator only supports the reset of level 0 LoD.\nAt least one of Input(TargetLoD) and Attr(target_lod) must be set,\nand if both of them are set, Input(TargetLoD) will be chosen as the\ntarget LoD.\n\nAn example:\nGiven a float LoDTensor X with shape (6, 1), its transpose form represents\n\n    [1.0, 2.0, 3.0, 4.0, 5.0, 6.0],\n\nwith LoD = [[0, 2, 5, 6]] and the three (transposed) sequences look like\n\n    [1.0, 2.0], [3.0, 4.0, 5.0], [6.0].\n\nIf target LoD = [0, 4, 6], the lod_reset operator will reset the LoD and\nthe sequences that the LoDTensor Output(Out) contains becomes:\n\n    [1.0, 2.0, 3.0, 4.0], [5.0, 6.0].\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) The input tensor of lod_reset operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "TargetLoD",
+   "comment" : "(Tensor, optional) The target level 0 LoD from Input().",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) The output tensor of lod_reset operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "target_lod",
+   "type" : "int array",
+   "comment" : "The target level 0 LoD from Attr().",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "write_to_array",
+ "comment" : "\nWriteToArray Operator.\n\nThis operator writes a LoDTensor to a LoDTensor array.\n\nAssume $T$ is LoDTensor, $i$ is the subscript of the array, and $A$ is the array. The\nequation is\n\n$$A[i] = T$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) the tensor will be written to tensor array",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "I",
+   "comment" : "(Tensor) the subscript index in tensor array. The number of element should be 1",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(TensorArray) the tensor array will be written",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "increment",
+ "comment" : "\nIncrement Operator.\n\nThe equation is: \n$$Out = X + step$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor of increment operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of increment operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "step",
+   "type" : "float",
+   "comment" : "(float, default 1.0) The step size by which the input tensor will be incremented.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "logical_or",
+ "comment" : "logical_or Operator\n\nIt operates element-wise on X and Y, and returns the Out. X, Y and Out are N-dim boolean tensors.\nEach element of Out is calculated by $$Out = X || Y$$\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) Left hand operand of logical_or operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(LoDTensor) Right hand operand of logical_or operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) n-dim bool tensor. Each element is $$Out = X || Y$$",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "seq_expand",
+ "comment" : "\nSeq Expand Operator.\n\nThis operator expands input(X) according to LOD of input(Y).\nFollowing are cases to better explain how this works:\nCase 1:\n\nGiven 2-level a LoDTensor input(X)\n    X.lod = [[0,       2, 3],\n             [0, 1,    3, 4]]\n    X.data = [a, b, c, d]\n    X.dims = [4, 1]\nand input(Y)\n    Y.lod = [[0,    2,    4],\n             [0, 3, 6, 7, 8]]\nwith condition len(Y.lod[-1]) -1 == X.dims[0]\nthen we get 2-level LoDTensor\n    Out.lod = [[0,                2,    4],\n               [0,       3,       6, 7, 8]]\n    Out.data = [a, a, a, b, b, b, c, d]\n    Out.dims = [8, 1]\n\nCase 2:\n\nGiven a 0-level LoDTensor input(X)\n    X.data = [a, b, c]\n    X.lod = NULL\n    X.dims = [3, 1]\nand input(Y)\n    Y.lod = [[0, 2, 3, 6]]\nwith condition len(Y.lod[-1]) -1 == X.dims[0]\nthen we get 1-level LoDTensor\n    Out.lod = [[0,    2, 3,      6]]\n    Out.data = [a, a, b, c, c, c]\n    Out.dims = [6, 1]\n\nCase 3:\n\nGiven a 0-level LoDTensor input(X)\n    X.data = [[a, b], [c, d], [e, f]]\n    X.lod = NULL\n    X.dims = [3, 2]\nand input(Y)\n    Y.lod = [[0, 2, 3, 6]]\nwith condition len(Y.lod[-1]) -1 == X.dims[0]\nthen we get 1-level LoDTensor\n    Out.lod = [[0,           2,     3,                     6]]\n    Out.data = [[a,b], [a,b] [c,d], [e, f], [e, f], [e, f]]\n    Out.dims = [6, 2]\n\nCase 4:\n\nGiven 2-level a LoDTensor input(X)\n    X.lod = [[0,       2, 3],\n             [0, 1,    3, 4]]\n    X.data = [a, b, c, d]\n    X.dims = [4, 1]\nand input(Y)\n    Y.lod = [[0,    2,    4],\n             [0, 3, 6, 6, 8]]\nwith condition len(Y.lod[-1]) -1 == X.dims[0]\nthen we get 2-level LoDTensor\n    Out.lod = [[0,                2,    4],\n               [0,       3,       6, 6, 8]]\n    Out.data = [a, a, a, b, b, b, d, d]\n    Out.dims = [8, 1]\n\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor or LoDTensor) The input(X) of this operator can be a LoDTensor or a base Tensor.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(LoDTensor)The reference input(Y) of seq_expand op.It must be a LoDTensor with k-level(k>0).The input(X) will be expanded according to LOD of input(Y).The element numbers of last level in input(Y) must be equal to dims[0] of input(X).",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LodTensor)The output of seq_expand op.The lod of output will be as same as input(Y)'s lod.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "scale",
+ "comment" : "\nScale operator\n\n$$Out = scale*X$$\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) Input tensor of scale operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) Output tensor of scale operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "scale",
+   "type" : "float",
+   "comment" : "(float, default 0)The scaling factor of the scale operator.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "mean",
+ "comment" : "\nMean Operator.\n\nOut is a scalar which is the mean of all elements in X. \n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input of mean op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of mean op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "reduce_mean",
+ "comment" : "\n{ReduceOp} Operator.\n\nThis operator computes the mean of input tensor along the given dimension. \nThe result tensor has 1 fewer dimension than the input unless keep_dim is true.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor. Tensors with rank at most 6 are supported.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The result tensor.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "dim",
+   "type" : "int",
+   "comment" : "(int, default 0) The dimension to reduce. Must be in the range [-rank(input), rank(input)). If `dim < 0`, the dim to reduce is `rank + dim`. Note that reducing on the first dim will make the LoD info lost.",
+   "generated" : 0
+ }, { 
+   "name" : "keep_dim",
+   "type" : "bool",
+   "comment" : "(bool, default false) If true, retain the reduced dimension with length 1.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "reduce_sum",
+ "comment" : "\n{ReduceOp} Operator.\n\nThis operator computes the sum of input tensor along the given dimension. \nThe result tensor has 1 fewer dimension than the input unless keep_dim is true.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor. Tensors with rank at most 6 are supported.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The result tensor.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "dim",
+   "type" : "int",
+   "comment" : "(int, default 0) The dimension to reduce. Must be in the range [-rank(input), rank(input)). If `dim < 0`, the dim to reduce is `rank + dim`. Note that reducing on the first dim will make the LoD info lost.",
+   "generated" : 0
+ }, { 
+   "name" : "keep_dim",
+   "type" : "bool",
+   "comment" : "(bool, default false) If true, retain the reduced dimension with length 1.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "stanh",
+ "comment" : "\nSTanh Activation Operator.\n\n$$y = b * \\frac{e^{a * x} - e^{-a * x}}{e^{a * x} + e^{-a * x}}$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of STanh operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of STanh operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "scale_a",
+   "type" : "float",
+   "comment" : "The scale parameter of a for the input",
+   "generated" : 0
+ }, { 
+   "name" : "scale_b",
+   "type" : "float",
+   "comment" : "The scale parameter of b for the input",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "adamax",
+ "comment" : "\nAdamax Optimizer.\n\nWe implement the Adamax optimizer from Section 7 of the Adam\npaper: https://arxiv.org/abs/1412.6980. Adamax is a variant of the\nAdam algorithm based on the infinity norm.\n\nAdamax updates:\n\n$$\n  momentOut = \\beta_{1} * moment + (1 - \\beta_{1}) * grad \\\\\n  infNormOut = max(\\beta_{2} * infNorm + \\epsilon, |grad|) \\\\\n  learningRate = \\frac{learningRate}{1 - \\beta_{1}^{Beta1Pow}} \\\\\n  paramOut = param - learningRate * \\frac{momentOut}{infNormOut}\n$$\n\nThe original paper does not have an epsilon attribute.\nHowever, it is added here for numerical stability to prevent the\ndivision by 0 error.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Param",
+   "comment" : "(Tensor) Input parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Grad",
+   "comment" : "(Tensor) Input gradient",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LearningRate",
+   "comment" : "(Tensor) Learning rate",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Moment",
+   "comment" : "(Tensor) First moment",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "InfNorm",
+   "comment" : "(Tensor) Input exponentially weighted infinity norm",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Beta1Pow",
+   "comment" : "(Tensor) Input beta1 power accumulator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ParamOut",
+   "comment" : "(Tensor) Output parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "MomentOut",
+   "comment" : "(Tensor) Output first moment",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "InfNormOut",
+   "comment" : "(Tensor) Output exponentially weighted infinity norm",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "beta1",
+   "type" : "float",
+   "comment" : "(float, default 0.9) Exponential decay rate for the 1st moment estimates.",
+   "generated" : 0
+ }, { 
+   "name" : "beta2",
+   "type" : "float",
+   "comment" : "(float, default 0.999) exponential decay rate for the weighted infinity norm estimates.",
+   "generated" : 0
+ }, { 
+   "name" : "epsilon",
+   "type" : "float",
+   "comment" : "(float, default 1.0e-8) Constant for numerical stability",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "tanh_shrink",
+ "comment" : "\nTanhShrink Activation Operator.\n\n$$y = x - \\frac{e^{x} - e^{-x}}{e^{x} + e^{-x}}$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of TanhShrink operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of TanhShrink operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "greater_than",
+ "comment" : "greater_than Operator\n\nIt operates element-wise on X and Y, and returns the Out. Each of them is a\nN-dim tensor. X and Y could be any type.  The each element of the Out tensor is\ncalculated by Out = X > Y\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) the left hand operand of greater_than operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(LoDTensor) the right hand operand of greater_than operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) n-dim bool tensor. Each element is Out = X > Y",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "rank_loss",
+ "comment" : "\nRankLoss Operator.\n\nRankLoss operator for RankNet\n(http://icml.cc/2015/wp-content/uploads/2015/06/icml_ranking.pdf). \nRankNet is a pairwise ranking model with\none training sample consisting of a pair of doc A and B, and the label P\nindicating that A is ranked higher than B or not:\n\nP = {0, 1} or {0, 0.5, 1}, where 0.5 means no information about the rank of\nthe input pair.\n\nThe RankLoss operator takes three inputs: Left (o_i), Right (o_j) and Label\n(P_{i,j}), which represent the output score of RankNet for the two docs and \nthe label respectively, and yields the rank loss C_{i,j} using the following \nequation:\n\n$$\n  C_{i,j} = -\\tilde{P_{ij}} * o_{i,j} + \\log(1 + e^{o_{i,j}}) \\\\\n  o_{i,j} =  o_i - o_j  \\\\\n  \\tilde{P_{i,j}} = \\left \\{0, 0.5, 1 \\right \\} \\ or \\ \\left \\{0, 1 \\right \\}\n$$\n\nThe operator can take batch inputs with size batch_size (batch_size >= 1).\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Label",
+   "comment" : "(2-D Tensor with shape [batch_size x 1]) The label indicating A ranked higher than B or not.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Left",
+   "comment" : "(2-D Tensor with shape [batch_size x 1]) The output of RankNet for doc A.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Right",
+   "comment" : "(2-D Tensor with shape [batch_size x 1]) The output of RankNet for doc B.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(2-D Tensor with shape [batch_size x 1]) The output loss of RankLoss operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "reciprocal",
+ "comment" : "\nReciprocal Activation Operator.\n\n$$y = \\frac{1}{x}$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Reciprocal operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Reciprocal operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "pool2d_cudnn",
+ "comment" : "\nPool2d Operator.\n\nThe pooling2d operation calculates the output based on\nthe input, pooling_type and ksize, strides, paddings parameters.\nInput(X) and output(Out) are in NCHW format, where N is batch size, C is the\nnumber of channels, H is the height of the feature, and W is the width of the feature.\nParameters(ksize, strides, paddings) are two elements.\nThese two elements represent height and width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:   \n  Input:\n       X shape: $(N, C, H_{in}, W_{in})$\n  Output:\n       Out shape: $(N, C, H_{out}, W_{out})$\n  Where\n       $$ \n       H_{out} = \\frac{(H_{in} - ksize[0] + 2 * paddings[0])}{strides[0]} + 1 \\\\\n       W_{out} = \\frac{(W_{in} - ksize[1] + 2 * paddings[1])}{strides[1]} + 1\n       $$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor of pooling operator. The format of input tensor is NCHW, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of pooling operator. The format of output tensor is also NCHW, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "pooling_type",
+   "type" : "string",
+   "comment" : "(string), pooling type, can be \"max\" for max-pooling and \"avg\" for average-pooling.",
+   "generated" : 0
+ }, { 
+   "name" : "ksize",
+   "type" : "int array",
+   "comment" : "(vector) The pooling window size(height, width) of the pooling operator. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "global_pooling",
+   "type" : "bool",
+   "comment" : "(bool, default false) Whether to use the global pooling. If global_pooling = true, ksize and paddings will be ignored.",
+   "generated" : 0
+ }, { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector, default {1, 1}), strides(height, width) of pooling operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector, default {0,0}), paddings(height, width) of pooling operator.If global_pooling = true, paddings and ksize will be ignored.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "lod_tensor_to_array",
+ "comment" : "",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "RankTable",
+   "comment" : "",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "reshape",
+ "comment" : "\nReshape Operator.\n\nReshape Input(X) into the shape specified by Attr(shape).\n\nAn example:\nGiven a 2-D tensor X with 2 rows and 2 columns\n\n    [[1, 2], [3, 4]]\n\nand target shape = [1, 4], the reshape operator will transform\nthe tensor X into a 1-D tensor:\n\n    [1, 2, 3, 4]\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input tensor of reshape operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output tensor of reshape operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "shape",
+   "type" : "int array",
+   "comment" : "(vector) Target shape of reshape operator.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "modified_huber_loss",
+ "comment" : "\nModified Huber Loss Operator.\n\nThis operator is used in binary classification problem. The shape of\ninput X and target Y are both [N, 1] and so is the shape of the output loss.\nSince target Y is not differentiable, calculating gradient for Y is illegal.\nThe formula of modified huber loss is:\n\n$$\nL(y, f(x)) = \n\\begin{cases}\n(\\max(0, 1 - yf(x)))^2,  \\text{if} \\  yf(x) >= -1    \\\\\n             -4yf(x),    \\quad \\text{otherwise}\n\\end{cases}\n$$\n\nMake sure the values of target label Y are in {0, 1} here. This operator will\nscale values of Y to {-1, +1} when computing losses and gradients.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input tensor of modified huber loss op. X is 2-D tensor with shape [batch_size, 1].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "The target labels of modified huber loss op. The shape of Y is the same as X. Values of Y must be 0 or 1.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "IntermediateVal",
+   "comment" : "Variable to save intermediate result which will be reused in backward processing.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "Out",
+   "comment" : "Classification loss for X.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "elementwise_sub",
+ "comment" : "\nLimited Elementwise Sub Operator.\n\nThe equation is:\n\n$Out = X - Y$\n\nX is a tensor of any dimension and the dimensions of tensor Y must be smaller than\nor equal to the dimensions of X. \n\nThere are two cases for this operator:\n1. The shape of Y is same with X;\n2. The shape of Y is a subset of X.\n\nFor case 2:\nY will be broadcasted to match the shape of X and axis should be \nthe starting dimension index for broadcasting Y onto X.\n\nexample:\n  shape(X) = (2, 3, 4, 5), shape(Y) = (,)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (5,)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1\n  shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0\n\nBoth the input X and Y can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD information with input X.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The first input tensor of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(Tensor) The second input tensor of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "axis",
+   "type" : "int",
+   "comment" : "(int, default -1) The starting dimension index for broadcasting Y onto X",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "logical_and",
+ "comment" : "logical_and Operator\n\nIt operates element-wise on X and Y, and returns the Out. X, Y and Out are N-dim boolean tensors.\nEach element of Out is calculated by $$Out = X \\&\\& Y$$\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) Left hand operand of logical_and operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(LoDTensor) Right hand operand of logical_and operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) n-dim bool tensor. Each element is $$Out = X \\&\\& Y$$",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "logical_not",
+ "comment" : "logical_not Operator\n\nIt operates element-wise on X, and returns the Out. X and Out are N-dim boolean tensors.\nEach element of Out is calculated by $$Out = !X$$\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) Operand of logical_not operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) n-dim bool tensor. Each element is $$Out = !X$$",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "abs",
+ "comment" : "\nAbs Activation Operator.\n\n$y = |x|$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Abs operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Abs operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "square",
+ "comment" : "\nSquare Activation Operator.\n\n$y = x^2$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Square operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Square operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "huber_loss",
+ "comment" : "\nHuberLoss Operator.\n\nHuber loss is a loss function used in robust regression. We define X as the\ninput value and Y as the target value. Huber loss can evaluate the fitness of\nX to Y. Different from MSE loss, Huber loss is more robust for outliers. The\nshape of X and Y are [batch_size, 1]. The equation is:\n\n$$\nOut_{\\delta}(X, Y)_i =\n\\begin{cases}\n0.5 * (Y_i - X_i)^2,\n\\quad |Y_i - X_i| \\leq \\delta \\\\\n\\delta * (|Y_i - X_i| - 0.5 * \\delta),\n\\quad otherwise\n\\end{cases}\n$$\n\nIn the above equation, $Out_\\delta(X, Y)_i$, $X_i$ and $Y_i$ represent the ith\nelement of Out, X and Y.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input value of huber loss op.X is a 2-D tensor with shape [batch_size, 1].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "The target value of huber loss op.Y is a 2-D tensor with shape [batch_size, 1].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Residual",
+   "comment" : "Intermediate tensor to cache residual value between Y and X.The shape is same as Input(X) and will be reused in backward.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "Out",
+   "comment" : "The output tensor with shape [batch_size, 1] which represents the huber loss.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "delta",
+   "type" : "float",
+   "comment" : "Hyper parameter in huber loss.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "sequence_slice",
+ "comment" : "\nSequence slice operator\n\nThe operator crops a subsequence from given sequence with given start offset and subsequence length.\nIt only supports sequence (LoD Tensor with level number is 1).\n- Case:\n    X = [[a1, a2;\n        b1, b2;\n        c1, c2]\n       [d1, d2;\n        e1, e2]]\n    LoD(X) = {{0, 3, 5}}; Dims(X) = (5, 2)\n    Offset = [[0], [1]]; Length = [[2], [1]]\n\n    Out = [[a1, a2;\n            b1, b2]\n            [e1, e2]]\n    LoD(Out) = {{0, 2, 3}}; Dims(Out) = (3, 2)\nNOTE: The first dimension size of input, the size of offset and Length, should be equal. The offset start from 0.\n    ",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor), the input of SequenceSliceOp.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Offset",
+   "comment" : "(Tensor), a vector to describe the offset of every input sequence for sub sequence item.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Length",
+   "comment" : "(Tensor), a vector to describe the length of every input sequence for sub sequence item.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor), the output of SequenceSliceOp.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "hinge_loss",
+ "comment" : "\nHingeLoss Operator.\n\nLet x be a logit (prediction) and y be the actual label. The logit can\ntake any values from (-inf, inf), but the labels should be either -1 or 1.\nThen, the hinge loss is computed as follows:\n\n$$\nL_(x, y) = max(1 - y.x, 0) \n$$\n\nNote that the labels passed as input will have values as either 0 or 1.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Logits",
+   "comment" : "The input value (Logits) of Hinge loss op.Logits is a 2-D tensor with shape [batch_size, 1].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Labels",
+   "comment" : "The target value (Labels) of Hinge loss op.Labels is a 2-D tensor with shape [batch_size, 1].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Loss",
+   "comment" : "The output tensor with shape [batch_size, 1] which represents the hinge loss.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "less_than",
+ "comment" : "less_than Operator\n\nIt operates element-wise on X and Y, and returns the Out. Each of them is a\nN-dim tensor. X and Y could be any type.  The each element of the Out tensor is\ncalculated by Out = X < Y\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) the left hand operand of less_than operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(LoDTensor) the right hand operand of less_than operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) n-dim bool tensor. Each element is Out = X < Y",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "gru_unit",
+ "comment" : "\nGRUUnit Operator implements partial calculations of the GRU unit as following:\n\n$$\nupdate \\ gate: u_t = actGate(xu_t + W_u * h_{t-1} + b_u) \\\\\nreset \\ gate: r_t = actGate(xr_t + W_r * h_{t-1} + b_r)  \\\\\noutput \\ candidate: {h}_t = actNode(xc_t + W_c * dot(r_t, h_{t-1}) + b_c) \\\\\noutput: h_t = dot((1 - u_t), h_{t-1}) + dot(u_t, {h}_t)\n$$\n\nwhich is same as one time step of GRU Operator.\n\n@note To implement the complete GRU unit, fully-connected operator must be \nused before to feed xu, xr and xc as the Input of GRUUnit operator.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(Tensor) Matrix with shape [batch_size, frame_size * 3] for the input.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "HiddenPrev",
+   "comment" : "(Tensor) Matrix with shape [batch_size, frame_size] for the states of previous time step.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Weight",
+   "comment" : "(Tensor) Weight matrix with shape [frame_size, frame_size * 3]. The elements continuous in memory can be divided into two parts. The first part are weights of the update gate and reset gate with shape [frame_size, frame_size * 2], and the second part are weights of output candidate with shape [frame_size, frame_size].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Bias",
+   "comment" : "(Tensor) Bias vector with shape [1, frame_size * 3] concatenating bias of the update gate, reset gate and output candidate.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Gate",
+   "comment" : "(Tensor) Matrix with shape [batch_size, frame_size * 3] for the output of update gate, reset gate and output candidate.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "ResetHiddenPrev",
+   "comment" : "(Tensor) Matrix with shape [batch_size, frame_size] for the reseted hidden state of previous time step.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "Hidden",
+   "comment" : "(Tensor) The GRU hidden state of the current time step with shape [batch_size, frame_size].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "activation",
+   "type" : "int",
+   "comment" : "(enum int, default tanh) The activation type used for output candidate {h}_t.",
+   "generated" : 0
+ }, { 
+   "name" : "gate_activation",
+   "type" : "int",
+   "comment" : "(enum int, default sigmoid) The activation type used in update gate and reset gate.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "gaussian_random",
+ "comment" : "\nGaussianRandom Operator.\n\nUsed to initialize tensors with gaussian random generator.\n\n",
+ "inputs" : [  ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "Output matrix of gaussian random op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "shape",
+   "type" : "int array",
+   "comment" : "(vector) The dimension of random tensor.",
+   "generated" : 0
+ }, { 
+   "name" : "mean",
+   "type" : "float",
+   "comment" : "(float, default 0.0) mean of random tensor.",
+   "generated" : 0
+ }, { 
+   "name" : "std",
+   "type" : "float",
+   "comment" : "(float, default 1.0) std of random tensor.",
+   "generated" : 0
+ }, { 
+   "name" : "seed",
+   "type" : "int",
+   "comment" : "(int, default 0) Random seed of generator.0 means use system wide seed.",
+   "generated" : 0
+ }, { 
+   "name" : "dtype",
+   "type" : "int",
+   "comment" : "(int, default 5(FP32)) Output data type.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "fill_constant",
+ "comment" : "\nFillConstantBatchSizeLike Operator.\n\nFill up a variable with specified constant value.\n\n",
+ "inputs" : [  ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) Tensor of specified shape will be filled with the specified value",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "dtype",
+   "type" : "int",
+   "comment" : "(int, default 5 (FP32)) Output data type",
+   "generated" : 0
+ }, { 
+   "name" : "shape",
+   "type" : "int array",
+   "comment" : "(vector) The shape of the output",
+   "generated" : 0
+ }, { 
+   "name" : "value",
+   "type" : "float",
+   "comment" : "(float, default 0) The value to be filled",
+   "generated" : 0
+ }, { 
+   "name" : "force_cpu",
+   "type" : "bool",
+   "comment" : "(bool, default false) Force fill output variable to cpu memory. Otherwise, fill output variable to the running device",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "fill_zeros_like",
+ "comment" : "\nFillZerosLike Operator.\n\nFill up a variable with zeros.\nThe output will have the same size as the input.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input of fill-zeros-like op.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "The variable will be filled up with zeros.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "softmax_with_cross_entropy",
+ "comment" : "\nSoftmax With Cross Entropy Operator.\n\nCross entropy loss with softmax is used as the output layer extensively. This\noperator computes the softmax normalized values for each row of the input\ntensor, after which cross-entropy loss is computed. This provides a more\nnumerically stable gradient.\n\nBecause this operator performs a softmax on logits internally, it expects\nunscaled logits. This operator should not be used with the output of\nsoftmax operator since that would produce incorrect results.\n\nWhen the attribute soft_label is set false, this operators expects mutually\nexclusive hard labels, each sample in a batch is in exactly one class with a\nprobability of 1.0. Each sample in the batch will have a single label.\n\nThe equation is as follows:\n\n1) Hard label (one-hot label, so every sample has exactly one class)\n\n$$Loss_j =  -\\text{Logit}_{Label_j} +\n\\log\\left(\\sum_{i=0}^{K}\\exp(\\text{Logit}_i)\\right),\nj = 1,..., K$$\n\n2) Soft label (each sample can have a distribution over all classes)\n\n$$Loss_j =  -\\sum_{i=0}^{K}\\text{Label}_i \\left(\\text{Logit}_i -\n\\log\\left(\\sum_{i=0}^{K}\\exp(\\text{Logit}_i)\\right)\\right),\nj = 1,...,K$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Logits",
+   "comment" : "(Tensor, default: Tensor), The unscaled log probabilities which is a 2-D tensor with shape [N x K]. N is the batch_size, and K is the class number.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Label",
+   "comment" : "(Tensor) The ground truth which is a 2-D tensor. If soft_label is set to false, Label is a Tensor with shape [N x 1]. If soft_label is set to true, Label is a Tensor with shape [N x K].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Softmax",
+   "comment" : "(Tensor, default: Tensor), A 2-D tensor with shape [N x K]. The outputs value of softmax activation by given the input batch, which will be used in backward calculation.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "Loss",
+   "comment" : "(Tensor, default: Tensor), A 2-D tensor. The cross entropy loss with shape [N x 1].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "soft_label",
+   "type" : "bool",
+   "comment" : "(bool, default: false), A flag to indicate whether to interpretate the given labels as soft labels.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "fill_constant_batch_size_like",
+ "comment" : "\nFillConstantBatchSizeLike Operator.\n\nFill up a variable with specified constant value.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(Tensor) Tensor whose dim_idx th dimension is used to specify the batch_size",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) Tensor of specified shape will be filled with the specified value",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "dtype",
+   "type" : "int",
+   "comment" : "(int, default 5 (FP32)) Output data type",
+   "generated" : 0
+ }, { 
+   "name" : "shape",
+   "type" : "int array",
+   "comment" : "(vector) The shape of the output",
+   "generated" : 0
+ }, { 
+   "name" : "input_dim_idx",
+   "type" : "int",
+   "comment" : "(int, default 0) The index of input's batch size dimension",
+   "generated" : 0
+ }, { 
+   "name" : "output_dim_idx",
+   "type" : "int",
+   "comment" : "(int, default 0) The index of output's batch size dimension",
+   "generated" : 0
+ }, { 
+   "name" : "value",
+   "type" : "float",
+   "comment" : "(float, default 0) The value to be filled",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "tanh",
+ "comment" : "\nTanh Activation Operator.\n\n$$y = \\frac{e^{x} - e^{-x}}{e^{x} + e^{-x}}$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Tanh operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Tanh operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "feed",
+ "comment" : "\nFeed Operator.\n\nIt should not be configured by users directly.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input of feed op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of feed op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "col",
+   "type" : "int",
+   "comment" : "(int) The column of feed",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "expand",
+ "comment" : "\nExpand operator tiles the input by given times number. You should set times\nnumber for each dimension by providing attribute 'expand_times'. The rank of X\nshould be in [1, 6]. Please notice that size of 'expand_times' must be same with\nX's rank. Following is a using case:\n\nInput(X) is a 3-D tensor with shape [2, 3, 1]:\n\n        [\n           [[1], [2], [3]],\n           [[4], [5], [6]]\n        ]\n\nAttr(expand_times):  [1, 2, 2]\n\nOutput(Out) is a 3-D tensor with shape [2, 6, 2]:\n\n        [\n            [[1, 1], [2, 2], [3, 3], [1, 1], [2, 2], [3, 3]],\n            [[4, 4], [5, 5], [6, 6], [4, 4], [5, 5], [6, 6]]\n        ]\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor, default Tensor) A tensor with rank in [1, 6].X is the input tensor to be expanded.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor, default Tensor) A tensor with rank in [1, 6].The rank of Output(Out) is same as Input(X) except that each dimension size of Output(Out) is equal to corresponding dimension size of Input(X) multiplying corresponding value of Attr(expand_times).",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "expand_times",
+   "type" : "int array",
+   "comment" : "Expand times number for each dimension.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "elementwise_div",
+ "comment" : "\nLimited Elementwise Div Operator.\n\nThe equation is:\n\n$Out = X / Y$\n\nX is a tensor of any dimension and the dimensions of tensor Y must be smaller than\nor equal to the dimensions of X. \n\nThere are two cases for this operator:\n1. The shape of Y is same with X;\n2. The shape of Y is a subset of X.\n\nFor case 2:\nY will be broadcasted to match the shape of X and axis should be \nthe starting dimension index for broadcasting Y onto X.\n\nexample:\n  shape(X) = (2, 3, 4, 5), shape(Y) = (,)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (5,)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1\n  shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0\n\nBoth the input X and Y can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD information with input X.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The first input tensor of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(Tensor) The second input tensor of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "axis",
+   "type" : "int",
+   "comment" : "(int, default -1) The starting dimension index for broadcasting Y onto X",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "elementwise_add",
+ "comment" : "\nLimited Elementwise Add Operator.\n\nThe equation is:\n\n$Out = X + Y$\n\nX is a tensor of any dimension and the dimensions of tensor Y must be smaller than\nor equal to the dimensions of X. \n\nThere are two cases for this operator:\n1. The shape of Y is same with X;\n2. The shape of Y is a subset of X.\n\nFor case 2:\nY will be broadcasted to match the shape of X and axis should be \nthe starting dimension index for broadcasting Y onto X.\n\nexample:\n  shape(X) = (2, 3, 4, 5), shape(Y) = (,)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (5,)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5)\n  shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1\n  shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0\n\nBoth the input X and Y can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD information with input X.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The first input tensor of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(Tensor) The second input tensor of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of elementwise op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "axis",
+   "type" : "int",
+   "comment" : "(int, default -1) The starting dimension index for broadcasting Y onto X",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "cross_entropy",
+ "comment" : "\nCrossEntropy Operator.\n\nIt supports both standard cross-entropy and soft-label cross-entropy loss\ncomputation.\n1) One-hot cross-entropy:\n    soft_label = false, Label[i, 0] indicates the class index for sample i:\n\n                $Y[i] = -\\log(X[i, Label[i]])$\n\n2) Soft-label cross-entropy:\n    soft_label = true, Label[i, j] indicates the soft label of class j\n    for sample i:\n\n                $Y[i] = \\sum_j{-Label[i, j] * log(X[i, j])}$\n\n   Please make sure that in this case the summuation of each row of Label\n   equals one.\n\n3) One-hot cross-entropy with vecterized Input(Label):\n     As a special case of 2), when each row of Input(Label) has only one\n     non-zero element (equals 1), soft-label cross-entropy degenerates to a\n     one-hot cross-entropy with one-hot label representation.\n\nBoth the input X and Label can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD information with input X.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor, default Tensor), a 2-D tensor with shape N x D, where N is the batch size and D is the number of classes. This input is a probability computed by the previous operator, which is almost always the result of a softmax operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Label",
+   "comment" : "(Tensor), the ground truth which is a 2-D tensor. When soft_label is set to false, Label is a Tensor with shape [N x 1]. When soft_label is set to true, Label is a Tensor with shape [N x K].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "(Tensor, default Tensor), a 2-D tensor with shape [N x 1]. The cross entropy loss.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "soft_label",
+   "type" : "bool",
+   "comment" : "(bool, default false), a flag indicating whether to interpretate the given labels as soft labels.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "matmul",
+ "comment" : "\nMatMul Operator.\n\n\nThis operator is used to perform (batched) matrix multiplication\nover the last two dimensions of the input tensors `X` and `Y`.\n\nIf a transpose flag is specified, the last two dimensions of the\ntensor are transposed. If the tensor is rank-1 of shape [D], then\nfor `X` it is treated as [1, D] in nontransposed form and as [D, 1]\nin transposed form, whereas for `Y` it is the opposite: It is treated\nas [D, 1] in nontransposed form and as [1, D] in transposed form.\n\nExamples without transpose:\n- X: [K], Y: [K] => Out: [1]\n- X: [K], Y: [K, N] => Out: [N]\n- X: [B, M, K], Y: [K] => Out: [B, M]\n- X: [M, K], Y: [B, K, N] => Out: [B, M, N]\n- X: [B, M, K], Y: [B, K, N] => Out: [B, M, N]\n\nThe behavior is designed to be similar to the `numpy.matmul` function.\nThe differences are:\n- Currently only rank 1 to rank 3 input tensors are supported.\n- We add `transpose_X` and `transpose_Y` flags.\n\nBoth the input `X` and `Y` can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD information with input `X`.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The first input of MatMul op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "The second input of MatMul op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of MatMul op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "transpose_X",
+   "type" : "bool",
+   "comment" : "If true, use the transpose of `X`.\n        ",
+   "generated" : 0
+ }, { 
+   "name" : "transpose_Y",
+   "type" : "bool",
+   "comment" : "If true, use the transpose of `Y`.\n        ",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "dropout",
+ "comment" : "\nDropout Operator.\n\nDropout refers to randomly dropping out units in a nerual network. It is a\nregularization technique for reducing overfitting by preventing neuron\nco-adaption during training. The dropout operator randomly set (according to\nthe given dropout probability) the outputs of some units to zero, while others\nare set equal to their corresponding inputs.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input of dropout op.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of dropout op.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Mask",
+   "comment" : "The random sampled dropout mask.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "dropout_prob",
+   "type" : "float",
+   "comment" : "Probability of setting units to zero.",
+   "generated" : 0
+ }, { 
+   "name" : "is_test",
+   "type" : "bool",
+   "comment" : "True if in test phase.",
+   "generated" : 0
+ }, { 
+   "name" : "seed",
+   "type" : "int",
+   "comment" : "Dropout random seed.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "fetch",
+ "comment" : "\nFetch Operator.\n\nIt should not be configured by users directly.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input of fetch op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of fetch op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "col",
+   "type" : "int",
+   "comment" : "(int) The column of fetch",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "squared_l2_distance",
+ "comment" : "\nSquaredL2Distance operator\n\nThis operator will cacluate the squared L2 distance for the input and \nthe target. Number of distance value will be equal to the first dimension \nof input. First dimension of the target could be equal to the input or to 1. \nIf the first dimension of target is 1, the operator will broadcast target's \nfirst dimension to input's first dimension. During backward propagation, \nthe user can decide whether to calculate the gradient of the input or \nthe target or both.\n\nBoth the input X and Y can carry the LoD (Level of Details) information. \nHowever, the output only shares the LoD information with input X.\n    ",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) Input of SquaredL2DistanceOp.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(Tensor) Target of SquaredL2DistanceOp.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "sub_result",
+   "comment" : "(Tensor) Buffering subtraction result which will be reused in backward.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "Out",
+   "comment" : "(Tensor) Squared l2 distance between input and target.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "while",
+ "comment" : "\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "A set of variables, which are required by operators inside the block of While Op.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ }, { 
+   "name" : "Condition",
+   "comment" : "(Bool) An scalar. When it's False, the While Op will be terminated.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "A set of variables, which will be assigned with values generated by the operators inside the block of While Op.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ }, { 
+   "name" : "StepScopes",
+   "comment" : "(StepScopeVar) A vector of local scope, which size equals the step number of While Op. The i'th scope storages temporary variables generated in the i'th step.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "step_block",
+   "type" : "block id",
+   "comment" : "The step block inside WhileOp",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "relu",
+ "comment" : "\nRelu Activation Operator.\n\n$y = \\max(x, 0)$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Relu operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Relu operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "decayed_adagrad",
+ "comment" : "\nDecayed Adagrad Optimizer.\n\nThe update is done as follows:\n\n$$\nmoment\\_out = decay * moment + (1 - decay) * grad * grad \\\\\nparam\\_out = param - \\frac{learning\\_rate * grad}{\\sqrt{moment\\_out} + epsilon}\n$$\n\nThe original paper(http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf)\ndoes not have an epsilon attribute. It is added here for numerical\nstability to avoid the division by zero error.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Param",
+   "comment" : "(Tensor) Input parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Grad",
+   "comment" : "(Tensor) Input gradient",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Moment",
+   "comment" : "(Tensor) Second moment",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LearningRate",
+   "comment" : "(Tensor) Learning rate",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ParamOut",
+   "comment" : "(Tensor) Output parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "MomentOut",
+   "comment" : "(Tensor) Output second moment",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "decay",
+   "type" : "float",
+   "comment" : "(float, default 0.95) Discounting factor for coming gradient",
+   "generated" : 0
+ }, { 
+   "name" : "epsilon",
+   "type" : "float",
+   "comment" : "(float, default 1.0e-6) Constant for numerical stability",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "beam_search",
+ "comment" : "This is a beam search operator that help to generate sequences.",
+ "inputs" : [ 
+ { 
+   "name" : "pre_ids",
+   "comment" : "ids in previous step",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "ids",
+   "comment" : "a LoDTensor of shape of [None,k]",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "scores",
+   "comment" : "a LoDTensor that has the same shape and LoD with `ids`",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "selected_ids",
+   "comment" : "a LoDTensor that stores the IDs selected by beam search",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "selected_scores",
+   "comment" : "a LoDTensor that has the same shape and LoD with `selected_ids`",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "level",
+   "type" : "int",
+   "comment" : "the level of LoDTensor",
+   "generated" : 0
+ }, { 
+   "name" : "beam_size",
+   "type" : "int",
+   "comment" : "beam size for beam search",
+   "generated" : 0
+ }, { 
+   "name" : "end_id",
+   "type" : "int",
+   "comment" : "the token id which indicates the end of a sequence",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "split_lod_tensor",
+ "comment" : "\n        Split a LoDTensor with a Mask at certain level. The input LoDTensor\n        has 3 sequence at certain lod level. The Mask is a bool column vector,\n        such as [0, 1, 0] at the same level. The first and third sequence will\n        be send to False Output LoDTensor; whereas the second sequence will\n        be send to True Output LoDTensor. Please refer to MergeLoDTensorOp.",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input LoDTensor",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Mask",
+   "comment" : "A bool column vector which mask the input",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "OutTrue",
+   "comment" : "True branch of input LoDTensor",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "OutFalse",
+   "comment" : "False branch of input LoDTensor",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "level",
+   "type" : "int",
+   "comment" : "(int) the specific lod level to split.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "greater_equal",
+ "comment" : "greater_equal Operator\n\nIt operates element-wise on X and Y, and returns the Out. Each of them is a\nN-dim tensor. X and Y could be any type.  The each element of the Out tensor is\ncalculated by Out = X >= Y\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) the left hand operand of greater_equal operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(LoDTensor) the right hand operand of greater_equal operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) n-dim bool tensor. Each element is Out = X >= Y",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "crop",
+ "comment" : "\nCrop Operator.\n\nCrop input into output, as specified by offsets and shape.\n\nThere are two ways to set shape:\n1. reference input: crop input X into the same shape as reference input.\n                    The dimension of reference input should\n                    be the same as the dimension of input X.\n2. shape list: crop input X into the shape described by a list.\n               The size of shape list should be the same as\n               the dimension size of input X.\n\nThe input should be a k-D tensor(k > 0 and k < 7). As an example:\n\nGiven:\n\n    X = [[0, 1, 2, 0, 0]\n         [0, 3, 4, 0, 0]\n         [0, 0, 0, 0, 0]],\n\nand\n\n    offsets = [0, 1],\n\nand\n\n    shape = [2, 2],\n\nwe get:\n\n    Out = [[1, 2],\n           [3, 4]].\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input of pad op. The input should be a k-D tensor(k > 0 and k < 7).",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "The input used as reference for cropping, which is of the same dimensions as X.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of crop op, which is of the same dimensions as X.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "offsets",
+   "type" : "int array",
+   "comment" : "A list describing offsets to be cropped. The size of offsets list should be the same as the dimension size of input X.",
+   "generated" : 0
+ }, { 
+   "name" : "shape",
+   "type" : "int array",
+   "comment" : "A list describing the shape of output. The size of shape list should be the same as the dimension size of input X.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "brelu",
+ "comment" : "\nBRelu Activation Operator.\n\n$y = \\max(\\min(x, t_{min}), t_{max})$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of BRelu operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of BRelu operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "t_min",
+   "type" : "float",
+   "comment" : "The min marginal value of BRelu",
+   "generated" : 0
+ }, { 
+   "name" : "t_max",
+   "type" : "float",
+   "comment" : "The max marginal value of BRelu",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "crf_decoding",
+ "comment" : "\nThe crf_decoding operator reads the emission feature weights and the transition\nfeature weights learned by the linear_chain_crf operator. It implements the\nViterbi algorithm which is a dynamic programming algorithm for finding the most\nlikely sequence of hidden states, called the Viterbi path, that results in a\nsequence of observed tags.\n\nThe output of this operator changes according to whether Input(Label) is given:\n\n1. Input(Label) is given:\n\nThis happens in training. This operator is used to co-work with the chunk_eval\noperator.\n\nWhen Input(Label) is given, the crf_decoding operator returns a row vector\nwith shape [N x 1] whose values are fixed to be 0, indicating an incorrect\nprediction, or 1 indicating a tag is correctly predicted. Such an output is the\ninput to chunk_eval operator.\n\n2. Input(Label) is not given:\n\nThis is the standard decoding process.\n\nThe crf_decoding operator returns a row vector with shape [N x 1] whose values\nrange from 0 to maximum tag number - 1. Each element indicates an index of a\npredicted tag.\n",
+ "inputs" : [ 
+ { 
+   "name" : "Emission",
+   "comment" : "(LoDTensor, default: LoDTensor). A LoDTensor with shape [N x D] where N is the size of the mini-batch and D is the total tag number. This input is the unscaled emission weight matrix of the linear_chain_crf operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Transition",
+   "comment" : "(Tensor, default: Tensor). A Tensor with shape [(D + 2) x D]. This input is the transition weights learned by the linear_chain_crf operator, denoted as w. The 1st row of w are transition weights for the start mask. The 2nd row of w are transition weights for the end mask. Transition weights between other tags begin from the 3rd row of w. See more details in comments of the linear_chain_crf operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Label",
+   "comment" : "(LoDTensor,  LoDTensor). The ground truth with shape [N x 1]. This input is optional. See more details in the operator's comments.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ViterbiPath",
+   "comment" : "(LoDTensor, LoDTensor). The decoding results. What to return changes depending on whether the Input(Label) (the ground truth) is given. See more details in the operator's comment.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "cos_sim",
+ "comment" : "\nCosine Similarity Operator.\n\n$Out = X^T * Y / (\\sqrt{X^T * X} * \\sqrt{Y^T * Y})$\n\nThe input X and Y must have the same shape, except that the 1st dimension\nof input Y could be just 1 (different from input X), which will be\nbroadcasted to match the shape of input X before computing their cosine\nsimilarity.\n\nBoth the input X and Y can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD information with input X.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The 1st input of cos_sim op.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "The 2nd input of cos_sim op.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of cos_sim op.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "XNorm",
+   "comment" : "Norm of the first input, reduced along the 1st dimension.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "YNorm",
+   "comment" : "Norm of the second input, reduced along the 1st dimension.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "conv3d_transpose_cudnn",
+ "comment" : "\nConvolution3D Transpose Operator.\n\nThe convolution transpose operation calculates the output based on the input, filter\nand strides, paddings, groups parameters. The size of each dimension of the\nparameters is checked in the infer-shape.\nInput(Input) and output(Output) are in NCDHW format. Where N is batch size, C is the\nnumber of channels, D is the depth of the feature, H is the height of the feature,\nand W is the width of the feature.\nFilter(Input) is in MCDHW format. Where M is the number of input feature channels,\nC is the number of output feature channels, D is the depth of the filter,H is the\nheight of the filter, and W is the width of the filter.\nParameters(strides, paddings) are three elements. These three elements represent\ndepth, height and width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:   \n  Input:\n       Input shape: $(N, C_{in}, D_{in}, H_{in}, W_{in})$\n       Filter shape: $(C_{in}, C_{out}, D_f, H_f, W_f)$\n  Output:\n       Output shape: $(N, C_{out}, D_{out}, H_{out}, W_{out})$\n  Where\n  $$\n       D_{out} = (D_{in} - 1) * strides[0] - 2 * paddings[0] + D_f \\\\\n       H_{out} = (H_{in} - 1) * strides[1] - 2 * paddings[1] + H_f \\\\\n       W_{out} = (W_{in} - 1) * strides[2] - 2 * paddings[2] + W_f\n  $$\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(Tensor) The input tensor of convolution transpose operator.The format of input tensor is NCDHW. Where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Filter",
+   "comment" : "(Tensor) The filter tensor of convolution transpose operator.The format of the filter tensor is MCDHW, where M is the number of input feature channels, C is the number of output feature channels, D is the depth of the filter, H is the height of the filter, and W is the width of the filter.We enforce groups number == 1 and padding == 0 in the convolution3d transpose scenario.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Output",
+   "comment" : "(Tensor) The output tensor of convolution transpose operator.The format of output tensor is also NCDHW.Where N is batch size, C is the number of channels, D is the depth of the feature, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector default:{1, 1, 1}), the strides{d_stride, h_stride, w_stride} of convolution transpose operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector default:{0, 0, 0}), paddings(d_pad, h_pad, w_pad) of convolution transpose operator.",
+   "generated" : 0
+ }, { 
+   "name" : "dilations",
+   "type" : "int array",
+   "comment" : "dilations of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "workspace_size_MB",
+   "type" : "int",
+   "comment" : "workspace size for cudnn, in MB, workspace is a section of GPU memory which will be allocated/freed each time the operator runs, larger workspace size can increase performance but also requires better hardward. This size should be carefully setted.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "conv_shift",
+ "comment" : "\nConvShift Operator.\n\nA layer for circular convolution of two vectors,\nas used in the Neural Turing Machine: https://arxiv.org/abs/1410.5401\n\nThe equation is:\n\n$$Out[i] = \\sum_{j=-(N-1)/2}^{(N-1)/2} X_{i+j} * Y_{j}$$\n\nwhere X's index is computed modulo M, and Y's index is computed modulo N.\n\nBoth inputs X and Y can carry LoD (Level of Details) information.\nHowever, the output only shares the LoD information with input X.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor, default Tensor), a 2-D tensor with shape B x M, where B is the batch size and M is the data dimension.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(Tensor, default Tensor), a 2-D tensor with shape B x N, where B is the batch size and N is the data dimension. N must be odd.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor, default Tensor), a 2-D tensor with shape B x M, i.e., the same shape as X.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "conv2d_cudnn",
+ "comment" : "\nConvolution Operator.\n\nThe convolution operation calculates the output based on the input, filter\nand strides, paddings, dilations, groups parameters. The size of each dimension of the\nparameters is checked in the infer-shape.\nInput(Input) and Output(Output) are in NCHW format. Where N is batch\nsize, C is the number of channels, H is the height of the feature, and W is\nthe width of the feature.\nFilters(Input) is MCHW format. Where M is the number of output image channels, C is\nthe number of input image channels, H is the height of the filter, and W\nis the width of the filter.\nParameters(strides, paddings, dilations) are two elements. These two elements represent\nheight and width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:\n  Input:\n       Input shape: $(N, C_{in}, H_{in}, W_{in})$\n       Filter shape: $(C_{out}, C_{in}, H_f, W_f)$\n  Output:\n       Output shape: $(N, C_{out}, H_{out}, W_{out})$\n  Where\n$$\n       H_{out}= \\frac{(H_{in} + 2 * paddings[0] - (dilations[0] * (H_f - 1) + 1))}{strides[0]}+ 1 \\\\\n       W_{out}= \\frac{(W_{in} + 2 * paddings[1] - (dilations[1] * (W_f - 1) + 1))}{strides[1]}+ 1\n$$\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(Tensor) The input tensor of convolution operator. The format of input tensor is NCHW, where N is batch size, C is the number of channels, H is the height of the feature, and W is the width of the feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Filter",
+   "comment" : "(Tensor) The filter tensor of convolution operator. The format of the filter tensor is MCHW, where M is the number of output image channels, C is the number of input image channels, H is the height of the filter, and W is the width of the filter. If the groups attribute is greater than 1, C equals the number of input image channels divided by the groups.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Output",
+   "comment" : "(Tensor) The output tensor of convolution operator. The format of output tensor is also NCHW.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "strides",
+   "type" : "int array",
+   "comment" : "(vector default:{1, 1}), the strides(h_stride, w_stride) of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "paddings",
+   "type" : "int array",
+   "comment" : "(vector default:{0, 0}), the paddings(h_pad, w_pad) of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "groups",
+   "type" : "int",
+   "comment" : "(int default:1), the groups number of the convolution operator. According to grouped convolution in Alex Krizhevsky's Deep CNN paper: when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels.",
+   "generated" : 0
+ }, { 
+   "name" : "dilations",
+   "type" : "int array",
+   "comment" : "(vector default:{1, 1}), the dilations(h_dilation, w_dilation) of convolution operator.",
+   "generated" : 0
+ }, { 
+   "name" : "workspace_size_MB",
+   "type" : "int",
+   "comment" : "workspace size for cudnn, in MB, workspace is a section of GPU memory which will be allocated/freed each time the operator runs, larger workspace size can increase performance but also requires better hardware. This size should be chosen carefully.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "conditional_block",
+ "comment" : "Conditional block operator\n\nRun the sub-block if X is not empty. Params is the other inputs and Out is the\noutputs of the sub-block.\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The conditional variable of this operator. If X is empty, the whole sub-block will not be executed.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ }, { 
+   "name" : "Params",
+   "comment" : "The input variables of the sub-block.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output variables of the sub-block.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ }, { 
+   "name" : "Scope",
+   "comment" : "(std::vector) The step scope of conditional block. To unify the conditional block, rnn and while op, the type of scope is std::vector",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "block",
+   "type" : "block id",
+   "comment" : "The step block of conditional block operator",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "sum",
+ "comment" : "\nSum operator.\n\nThis operators sums the input tensors. All the inputs can carry the \nLoD (Level of Details) information. However, the output only shares \nthe LoD information with the first input.\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(vector) The input tensors of sum operator.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of sum operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "concat",
+ "comment" : "\nConcat Operator.\n\nConcatenate the input tensors along dimension axis.\nExamples:\n  Input[0] = [[1,2],[3,4]]\n  Input[1] = [[5,6]]\n  axis = 0\n  Output = [[1,2],\n            [3,4],\n            [5,6]]\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input tensors of concat operator.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "Output tensor of concat operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "axis",
+   "type" : "int",
+   "comment" : "The axis along which the input tensors will be concatenated.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "less_equal",
+ "comment" : "less_equal Operator\n\nIt operates element-wise on X and Y, and returns the Out. Each of them is a\nN-dim tensor. X and Y could be any type.  The each element of the Out tensor is\ncalculated by Out = X <= Y\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) the left hand operand of less_equal operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(LoDTensor) the right hand operand of less_equal operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) n-dim bool tensor. Each element is Out = X <= Y",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "equal",
+ "comment" : "equal Operator\n\nIt operates element-wise on X and Y, and returns the Out. Each of them is a\nN-dim tensor. X and Y could be any type.  The each element of the Out tensor is\ncalculated by Out = X == Y\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor) the left hand operand of equal operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "(LoDTensor) the right hand operand of equal operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) n-dim bool tensor. Each element is Out = X == Y",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "gather",
+ "comment" : "\nGather Operator.\n\n$Out = X[Index]$\n\nOut is obtained by gathering entries of the outer-most dimension \nof X indexed by Index and concatenate them together.\n\nExample:\n\nX = [[1, 2],\n     [3, 4],\n     [5, 6]]\n\nIndex = [[1, 2]]\n\nThen:\n\nOut = [[3, 4],\n       [5, 6]]\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The source input of gather op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Index",
+   "comment" : "The index input of gather op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of gather op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "clip_by_norm",
+ "comment" : "\nClipByNorm Operator.\n\nThis operator limits the L2 norm of the input $X$ within $max\\_norm$.\nIf the L2 norm of $X$ is less than or equal to $max\\_norm$, $Out$ will be\nthe same as $X$. If the L2 norm of $X$ is greater than $max\\_norm$, $X$ will\nbe linearly scaled to make the L2 norm of $Out$ equal to $max\\_norm$, as\nshown in the following formula:\n\n$$\nOut = \\frac{max\\_norm * X}{norm(X)},\n$$\n\nwhere $norm(X)$ represents the L2 norm of $X$.\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input of clip_by_norm op.The number of dimensions must be between [1, 9].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output of clip_by_norm op with shape as input(X)",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "max_norm",
+   "type" : "float",
+   "comment" : "(float) The maximum norm value.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "chunk_eval",
+ "comment" : "\nFor some basics of chunking, please refer to\n‘Chunking with Support Vector Machines ’.\n\n\nCheckEvalOp computes the precision, recall, and F1-score of chunk detection,\nand supports IOB, IOE, IOBES and IO (also known as plain) tagging schemes.\nHere is a NER example of labeling for these tagging schemes:\n\n \t     Li     Ming    works  at  Agricultural   Bank   of    China  in  Beijing.\n  IO:    I-PER  I-PER   O      O   I-ORG          I-ORG  I-ORG I-ORG  O   I-LOC\n  IOB:   B-PER  I-PER   O      O   B-ORG          I-ORG  I-ORG I-ORG  O   B-LOC\n  IOE:   I-PER  E-PER   O      O   I-ORG          I-ORG  I-ORG E-ORG  O   E-LOC\n  IOBES: B-PER  E-PER   O      O   I-ORG          I-ORG  I-ORG E-ORG  O   S-LOC\n\nThere are three chunk types(named entity types) including PER(person), ORG(organization)\nand LOC(LOCATION), and we can see that the labels have the form -.\n\nSince the calculations actually use label ids rather than labels, extra attention\nshould be paid when mapping labels to ids to make CheckEvalOp work. The key point\nis that the listed equations are satisfied by ids.\n\n    tag_type = label % num_tag_type\n    chunk_type = label / num_tag_type\n\nwhere `num_tag_type` is the num of tag types in the tagging scheme, `num_chunk_type`\nis the num of chunk types, and `tag_type` get its value from the following table.\n\n    Scheme Begin Inside End   Single\n     plain   0     -      -     -\n     IOB     0     1      -     -\n     IOE     -     0      1     -\n     IOBES   0     1      2     3\n\nStill use NER as example, assuming the tagging scheme is IOB while chunk types are ORG,\nPER and LOC. To satisfy the above equations, the label map can be like this:\n\n    B-ORG  0\n    I-ORG  1\n    B-PER  2\n    I-PER  3\n    B-LOC  4\n    I-LOC  5\n    O      6\n\nIt’s not hard to verify the equations noting that the num of chunk types\nis 3 and the num of tag types in IOB scheme is 2. For example, the label\nid of I-LOC is 5, the tag type id of I-LOC is 1, and the chunk type id of\nI-LOC is 2, which consistent with the results from the equations.\n",
+ "inputs" : [ 
+ { 
+   "name" : "Inference",
+   "comment" : "(Tensor, default: Tensor). Predictions from the network.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Label",
+   "comment" : "(Tensor, default: Tensor). The true tag sequences.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Precision",
+   "comment" : "(float). The evaluated precision (called positive predictive value) of chunks on the given mini-batch.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Recall",
+   "comment" : "(float). The evaluated recall (true positive rate or sensitivity) of chunks on the given mini-batch.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "F1-Score",
+   "comment" : "(float). The evaluated F1-Score on the given mini-batch.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "num_chunk_types",
+   "type" : "int",
+   "comment" : "(int). The number of chunk type. See below for details.",
+   "generated" : 0
+ }, { 
+   "name" : "chunk_scheme",
+   "type" : "string",
+   "comment" : "(string, default IOB). The labeling scheme indicating how to encode the chunks. Must be IOB, IOE, IOBES or plain. See below for details.",
+   "generated" : 0
+ }, { 
+   "name" : "excluded_chunk_types",
+   "type" : "int array",
+   "comment" : "(list) A list including chunk type ids indicating chunk types that are not counted. See below for details.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "sigmoid",
+ "comment" : "\nSigmoid Activation Operator\n\n$$y = \\frac{1}{1 + e^{-x}}$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Sigmoid operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Sigmoid operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "sequence_concat",
+ "comment" : "\nThe sequence_concat operator concatenates multiple LoDTensors. \nIt only supports sequence (LoD Tensor with level number is 1) \nor a nested sequence (LoD tensor with level number is 2) as its input.\n- Case1:\n  If the axis is other than 0(here, axis is 1 and level is 1),\n  each input should have the same LoD information and the LoD \n  information of the output keeps the same as the input.\n\n  LoD(x0) = {{0,2,4}, {0,1,2,3,4}}; Dims(x0) = (4,3,4)\n  LoD(x1) = {{0,2,4}, {0,1,2,3,4}}; Dims(x1) = (4,4,4)\n  LoD(Out) = {{0,2,4}, {0,1,2,3,4}}; Dims(Out) = (4,7,4)\n\n- Case2:\n  If the axis is 0(here, leve is 0), the inputs are concatenated along \n  time steps, the LoD information of the output need to re-compute.\n  The LoD information of level-1 should be same.\n\n  LoD(x0) = {{0,2,4}, {0,1,2,3,4}}; Dims(x0) = (4,3,4)\n  LoD(x1) = {{0,2,4}, {0,1,3,5,7}}; Dims(x1) = (7,3,4)\n  LoD(Out) = {{0,2,4}, {0,2,5,8,11}}; Dims(Out) = (11,3,4)\n\n- Case3:\n  If the axis is 0(here, level is 1).\n\n  LoD(x0) = {{0,2,4}, {0,1,2,3,4}}; Dims(x0) = (4,3,4)\n  LoD(x1) = {{0,3,4}, {0,1,3,5,7}}; Dims(x1) = (7,3,4)\n  LoD(Out) = {{0,5,8}, {0,1,2,3,5,7,8,9,11}}; Dims(Out) = (11,3,4)\n\n- Case4:\n  If the LoD number is 1, axis is 0, level is 0\n\n  LoD(x0) = {{0,1,2,3,4}}; Dims(x0) = (4,3,4)\n  LoD(x1) = {{0,1,3,5,7}}; Dims(x1) = (7,3,4)\n  LoD(Out) = {{0,2,5,8,11}}; Dims(Out) = (11,3,4)\n\nNOTE: The levels of all the inputs should be the same.\n    ",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LodTensorArray) Input is a vector of LoDTensor, each of which is a variable-length sequence or nested sequence.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor), Variable-length output of sequence_concat Op.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "axis",
+   "type" : "int",
+   "comment" : "(int, default 0) The axis along which the inputs will be joined. If axis is 0, the inputs will be joined with LoD index.",
+   "generated" : 0
+ }, { 
+   "name" : "level",
+   "type" : "int",
+   "comment" : "(int, default 0) The level at which the inputs will be joined. If the level is 0, the inputs will be joined at the nested sequence level. If the level is 1, the inputs will be joined at the sequence level. The level should be less than the level number of inputs.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "floor",
+ "comment" : "\nFloor Activation Operator.\n\n$y = floor(x)$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Floor operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Floor operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "cast",
+ "comment" : "\nCast Operator.\n\nThis Operator casts the input tensor to another data type and\nreturns tha Output Tensor.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input tensor of cast op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output tensor of cast op",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "out_dtype",
+   "type" : "int",
+   "comment" : "output data type",
+   "generated" : 0
+ }, { 
+   "name" : "in_dtype",
+   "type" : "int",
+   "comment" : "input data type",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "ceil",
+ "comment" : "\nCeil Activation Operator.\n\n$y = ceil(x)$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Ceil operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Ceil operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "lrn",
+ "comment" : "\nLocal Response Normalization Operator.\n\nThis operator comes from the paper:\n<>.\n\nThe original formula is:\n\n$$\nOutput(i, x, y) = Input(i, x, y) / \\left(\nk + \\alpha \\sum\\limits^{\\min(C, c + n/2)}_{j = \\max(0, c - n/2)}\n(Input(j, x, y))^2\n\\right)^{\\beta}\n$$\n\nFunction implementation:\n\nInputs and outpus are in NCHW format, while input.shape.ndims() equals 4.\nAnd dimensions 0 ~ 3 represent batch size, feature maps, rows,\nand columns, respectively.\n\nInput and Output in the formula above is for each map(i) of one image, and\nInput(i, x, y), Output(i, x, y) represents an element in an image.\n\nC is the number of feature maps of one image. n is a hyper-parameter\nconfigured when operator is initialized. The sum in the denominator\nis the sum of the same positions in the neighboring maps.\n    \n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input of LRN operator. It must be a 4D tenor with NCHW format.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output of LRN operator, which is also the 4D tensor with NCHW format.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "MidOut",
+   "comment" : "(Tensor) Middle result of LRN operator. It's computed in forward process and also used in backward process.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "n",
+   "type" : "int",
+   "comment" : "(int default 5) n is the \"adjacent\" kernel that maps at the same spatial position.",
+   "generated" : 0
+ }, { 
+   "name" : "k",
+   "type" : "float",
+   "comment" : "(float, default 2.0) k is the bias.",
+   "generated" : 0
+ }, { 
+   "name" : "alpha",
+   "type" : "float",
+   "comment" : "(float, default 0.0001) alpha is the scale number.",
+   "generated" : 0
+ }, { 
+   "name" : "beta",
+   "type" : "float",
+   "comment" : "(float, default 0.75) beta is the power number.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "bilinear_tensor_product",
+ "comment" : "\nBilinear Tensor Product operator.\nGiven input X and Y, a 3D tensor Weight and a Bias. Each column of the\nOutput is computed by one slice $i = 1, . . . , k$ of the tensor:\n\n$$\nM =  (X W_i) * Y \\\\\nOut_i = \\sum_j {M_j} + Bias_i\n$$\n\nWhere $W_i$ is the $i$-th slice of Input(Weight);\n      $M_j$ is the $j$-th column of $M$;\n      $Out_i$ is the $i$-th column of Output(Out);\n      $Bias_i$ is a column vector, each element of it is equal to\n        the $i$-th element of $Bias$;\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The first input of bilinear_tensor_product operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Y",
+   "comment" : "The second input of bilinear_tensor_product operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Weight",
+   "comment" : "The learnable parameters of bilinear_tensor_product operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Bias",
+   "comment" : "The learnable bias of bilinear_tensor_product operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "The output of bilinear_tensor_product operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "batch_norm",
+ "comment" : "\nBatch Normalization.\n\nBatch Norm has been implemented as discussed in the paper:\nhttps://arxiv.org/pdf/1502.03167.pdf\nCan be used as a normalizer function for conv2d and fully_connected operations.\nThe required data format for this layer is one of the following:\n1. NHWC `[batch, in_height, in_width, in_channels]`\n2. NCHW `[batch, in_channels, in_height, in_width]`\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "The input tensor",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Scale",
+   "comment" : "Scale is a 1-dimensional tensor of size C that is applied to the output",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Bias",
+   "comment" : "Bias is a 1-dimensional tensor of size C that is applied to the output",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Mean",
+   "comment" : "The global mean (for training) or estimated mean (for testing)",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Variance",
+   "comment" : "The global variance (for training) or estimated Variance (for testing)",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "result after normalization",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "MeanOut",
+   "comment" : "Share memory with Mean. Store the global mean when training",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "VarianceOut",
+   "comment" : "Share memory with Variance. Store the global Variance when training",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "SavedMean",
+   "comment" : "Mean of the current mini batch, will apply to output when training",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "SavedVariance",
+   "comment" : "Variance of the current mini batch, will apply to output when training",
+   "duplicable" : 0,
+   "intermediate" : 1
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "is_test",
+   "type" : "bool",
+   "comment" : "",
+   "generated" : 0
+ }, { 
+   "name" : "momentum",
+   "type" : "float",
+   "comment" : "",
+   "generated" : 0
+ }, { 
+   "name" : "epsilon",
+   "type" : "float",
+   "comment" : "",
+   "generated" : 0
+ }, { 
+   "name" : "tensor_format",
+   "type" : "string",
+   "comment" : "",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "auc",
+ "comment" : "\nArea Under The Curve (AUC) Operator.\n\nThis implementation computes the AUC according to forward output and label.\nIt is used very widely in binary classification evaluation. As a note:\nIf input label contains values other than 0 and 1, it will be cast\nto bool. You can find the relevant definitions here:\nhttps://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve\n\nThere are two types of possible curves:\n1. ROC: Receiver operating characteristic\n2. PR: Precision Recall\n",
+ "inputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "A floating point 2D tensor, values are in the range [0, 1].Each row is sorted in descending order. This input should be theoutput of topk.Typically, this tensor indicates the probability of each label",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Indices",
+   "comment" : "An int 2D tensor, indicating the indices of originaltensor before sorting. Typically, this tensor indicates which label the probability stands for.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Label",
+   "comment" : "A 2D int tensor indicating the label of the training data.The height is batch size and width is always 1.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "AUC",
+   "comment" : "A scalar representing the current area-under-the-curve.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "curve",
+   "type" : "string",
+   "comment" : "Curve type, can be 'ROC' or 'PR'.",
+   "generated" : 0
+ }, { 
+   "name" : "num_thresholds",
+   "type" : "int",
+   "comment" : "The number of thresholds to use when discretizing the roc curve.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "read_from_array",
+ "comment" : "\nReadFromArray Operator.\n\nRead a LoDTensor from a LoDTensor Array.\n\nAssume $T$ is LoDTensor, $i$ is the subscript of the array, and $A$ is the array. The\nequation is\n\n$$T = A[i]$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(TensorArray) the array will be read from.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "I",
+   "comment" : "(Tensor) the subscript index in tensor array. The number of element should be 1",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor) the tensor will be read from.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "softplus",
+ "comment" : "\nSoftplus Activation Operator.\n\n$y = \\ln(1 + e^{x})$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Softplus operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Softplus operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "split",
+ "comment" : "\nSplit operator\n\nThis operator splits the input tensor into multiple sub-tensors.\n\nExample:\n  Input = [[1,2],\n           [3,4],\n           [5,6]]\n  sections = [2,1]\n  axis = 0\n  Output[0] = [[1,2],\n               [3,4]]\n  Output[1] = [[5,6]]\n\n    ",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) Input tensor of the split operator.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) Output tensors of the split operator.",
+   "duplicable" : 1,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "sections",
+   "type" : "int array",
+   "comment" : "(vector) the length of each output along the specified axis.",
+   "generated" : 0
+ }, { 
+   "name" : "num",
+   "type" : "int",
+   "comment" : "(int, default 0)Number of sub-tensors. This must evenly divide Input.dims()[axis]",
+   "generated" : 0
+ }, { 
+   "name" : "axis",
+   "type" : "int",
+   "comment" : "(int, default 0) The axis which the input will be splited on.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "beam_search_decode",
+ "comment" : "\nPack the result of Beam search op into SentenceIds and SentenceScores.\n",
+ "inputs" : [ 
+ { 
+   "name" : "Ids",
+   "comment" : "(LodTensorArray)score of the candidate words in each step",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Scores",
+   "comment" : "(LodTensorArray)score of the candidate words in each step",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "SentenceIds",
+   "comment" : "(LodTensor)All possible result sentences of word ids",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "SentenceScores",
+   "comment" : "(LodTensor)All possible result sentences of word scores",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "assign",
+ "comment" : "Assign Operator\n\nOut = X,  when type in [LoDTensor/SelectedRows/LoDTensorArray]\nraise error if the type is not listed above.\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(LoDTensor, SelectedRows or LoDTensorArray) The input variable could be LoDTensor, SelectedRows or LoDTensorArray.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(LoDTensor, SelectedRows or LoDTensorArray) The type of output is the same as input X.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "adam",
+ "comment" : "\nAdam Optimizer.\n\nThis implements the Adam optimizer from Section 2 of the Adam\npaper : https://arxiv.org/abs/1412.6980.\nAdam is a first-order gradient-based optimization method based on\nadaptive estimates of lower-order moments.\n\nAdam updates:\n\n$$moment_1_{out} = \\beta_1 * moment_1 + (1 - \\beta_1) * grad \\break\nmoment_2_{out} = \\beta_2 * moment_2 + (1 - \\beta_2) * grad * grad \\break\nlearningRate = learningRate *\n                  $\\sqrt{(1 - \\beta_2_{pow})}$ / (1 - \\beta_1_{pow}) \\break\nparamOut = param - learningRate * moment_1/ ($\\sqrt{(moment_2)} + \\epsilon)$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Param",
+   "comment" : "(Tensor) Input parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Grad",
+   "comment" : "(Tensor) Input gradient",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LearningRate",
+   "comment" : "(Tensor) Learning rate",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Moment1",
+   "comment" : "(Tensor) Input first moment",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Moment2",
+   "comment" : "(Tensor) Input second moment",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Beta1Pow",
+   "comment" : "(Tensor) Input beta1 power accumulator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Beta2Pow",
+   "comment" : "(Tensor) Input beta2 power accumulator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ParamOut",
+   "comment" : "(Tensor) Output parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Moment1Out",
+   "comment" : "(Tensor) Output first moment",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Moment2Out",
+   "comment" : "(Tensor) Output second moment",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "beta1",
+   "type" : "float",
+   "comment" : "(float, default 0.9) Exponential decay rate for the first moment estimates.",
+   "generated" : 0
+ }, { 
+   "name" : "beta2",
+   "type" : "float",
+   "comment" : "(float, default 0.999) exponential decay rate for the second moment estimates.",
+   "generated" : 0
+ }, { 
+   "name" : "epsilon",
+   "type" : "float",
+   "comment" : "(float, default 1.0e-8) Constant for numerical stability",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "adadelta",
+ "comment" : "\nAdadelta Optimizer.\n\nAdadelta optimizer is implemented as explained in:\nhttps://arxiv.org/abs/1212.5701\nAdadelta is a per-dimension adaptive learning rate method used\nfor gradient descent.\n\nAdadelta updates are as follows:\n\n$$avgSquaredGradOut = \\rho * avgSquaredGrad + (1 - \\rho) * grad * grad \\break\nparamUpdate =  - $\\sqrt{((avgSquaredUpdate + \\epsilon) /\n                       (avgSquaredGrad_out + \\epsilon))}$ * grad \\break\navgSquaredUpdateOut = \\rho * avgSquaredUpdate + (1 - \\rho) *\n                                  {(paramUpdate)}^2 \\break\nparamOut = param + paramUpdate$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Param",
+   "comment" : "(Tensor) Input parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Grad",
+   "comment" : "(Tensor) Input gradient",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "AvgSquaredGrad",
+   "comment" : "(Tensor) Input average of squared gradient",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "AvgSquaredUpdate",
+   "comment" : "(Tensor) Input average of squared parameter updates",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ParamOut",
+   "comment" : "(Tensor) Output parameter",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "AvgSquaredGradOut",
+   "comment" : "(Tensor) Output average of squared gradient",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "AvgSquaredUpdateOut",
+   "comment" : "(Tensor) Output average of squared parameter updates",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "rho",
+   "type" : "float",
+   "comment" : "(float, default 0.95) Exponential decay rate for squared gradients.",
+   "generated" : 0
+ }, { 
+   "name" : "epsilon",
+   "type" : "float",
+   "comment" : "(float, default 1.0e-6) Constant for numerical stability",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "log",
+ "comment" : "\nLog Activation Operator.\n\n$y = \\ln(x)$\n\nNatural logarithm of x.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Log operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Log operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "nce",
+ "comment" : "\nCompute and return the noise-contrastive estimation training loss.\nSee [Noise-contrastive estimation: A new estimation principle for unnormalized statistical models](http://www.jmlr.org/proceedings/papers/v9/gutmann10a/gutmann10a.pdf).\nBy default this operator uses a uniform distribution for sampling.\n",
+ "inputs" : [ 
+ { 
+   "name" : "Input",
+   "comment" : "(Tensor) A tensor of shape [batch_size, dim].",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Label",
+   "comment" : "(Tensor) A tensor of shape [batch_size, num_true_class]. 'num_true_class' is the number of target classes in each sample.The number of target classes per sample should be same. If you have a variable number of target classes, you can pad them out to a constant number by either repeating them or by padding with an otherwise unused class.)",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Weight",
+   "comment" : "(Tensor) A tensor of shape [num_class, dim]. 'num_class' is the total number of class.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Bias",
+   "comment" : "(Tensor) A tensor of shape [num_class, 1]. 'num_class' is the total number of class. It is a dispensable input.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "SampleWeight",
+   "comment" : "(Tensor) A tensor of shape [batch_size, 1] storing a weight for each sample. And it is a dispensable input. The default value of sample is 1.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Cost",
+   "comment" : "(Tensor) A tensor of shape [batch_size, 1]. Cost of samples.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "SampleLogits",
+   "comment" : "An intermediate tensor of shape[batch_size, num_neg_samples + num_pos_samples].This tensor is output of forward kernel and used in backward kernel to compute grads.Given X is  the dot product of input tensor and sampled labels' weights.Then 'SampleLogits' is sigmoid(X).",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "SampleLabels",
+   "comment" : "An intermediate tensor of shape[batch_size, num_neg_samples + num_pos_samples].This tensor is output of forward kernel and used in backward kernel to compute grads.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "num_total_classes",
+   "type" : "int",
+   "comment" : "Total number of classes in all samples.",
+   "generated" : 0
+ }, { 
+   "name" : "num_neg_samples",
+   "type" : "int",
+   "comment" : "The number of negative classes. The default value is 10.",
+   "generated" : 0
+ }, { 
+   "name" : "custom_neg_classes",
+   "type" : "int array",
+   "comment" : "This attribute only be used in unitest. Classes in this list wiil be used as negative classes for every samples. Under normal conditions, user should avoid setting this attribute.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "linear_chain_crf",
+ "comment" : "\nLinearChainCRF Operator.\n\nConditional Random Field defines an undirected probabilistic graph with nodes\ndenoting random variables and edges denoting dependencies between these\nvariables. CRF learns the conditional probability $P(Y|X)$, where\n$X = (x_1, x_2, ... , x_n)$ are structured inputs and\n$Y = (y_1, y_2, ... , y_n)$ are labels for the inputs.\n\nLinear chain CRF is a special case of CRF that is useful for sequence labeling\ntask. Sequence labeling tasks do not assume a lot of conditional\nindependences among inputs. The only constraint they impose is that the input\nand output must be linear sequences. Thus, the graph of such a CRF is a simple\nchain or a line, which results in the linear chain CRF.\n\nThis operator implements the Forward-Backward algorithm for the linear chain\nCRF. Please refer to http://www.cs.columbia.edu/~mcollins/fb.pdf and\nhttp://cseweb.ucsd.edu/~elkan/250Bwinter2012/loglinearCRFs.pdf for details.\n\nEquation:\n1. Denote Input(Emission) to this operator as $x$ here.\n2. The first D values of Input(Transition) to this operator are for starting\nweights, denoted as $a$ here.\n3. The next D values of Input(Transition) of this operator are for ending\nweights, denoted as $b$ here.\n4. The remaning values of Input(Transition) are for transition weights,\ndenoted as $w$ here.\n5. Denote Input(Label) as $s$ here.\n\nThe probability of a sequence $s$ of length $L$ is defined as:\n$$P(s) = (1/Z) \\exp(a_{s_1} + b_{s_L}\n                + \\sum_{l=1}^L x_{s_l}\n                + \\sum_{l=2}^L w_{s_{l-1},s_l})$$\n\nwhere $Z$ is a normalization value so that the sum of $P(s)$ over\nall possible sequences is 1, and $x$ is the emission feature weight\nto the linear chain CRF.\n\nFinally, the linear chain CRF operator outputs the logarithm of the conditional\nlikelihood of each training sample in a mini-batch.\n\nNOTE:\n1. The feature function for a CRF is made up of the emission features and the\ntransition features. The emission feature weights are NOT computed in\nthis operator. They MUST be computed first before this operator is called.\n\n2. Because this operator performs global normalization over all possible\nsequences internally, it expects UNSCALED emission feature weights.\nPlease do not call this op with the emission feature being output of any\nnonlinear activation.\n\n3. The 2nd dimension of Input(Emission) MUST be equal to the tag number.\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Emission",
+   "comment" : "(LoDTensor, default LoDTensor) A 2-D LoDTensor with shape [N x D], where N is the size of the mini-batch and D is the total tag number. The unscaled emission weight matrix for the linear chain CRF. ",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Transition",
+   "comment" : "(Tensor, default Tensor) A 2-D Tensor with shape [(D + 2) x D]. The learnable parameter for the linear_chain_crf operator. See more details in the operator's comments.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Label",
+   "comment" : "(LoDTensor, default LoDTensor) A LoDTensor with shape [N x 1], where N is the total element number in a mini-batch. The ground truth.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Alpha",
+   "comment" : "(Tensor, default Tensor) A 2-D Tensor with shape [N x D]. The forward vectors for the entire batch. Denote it as $lpha$. $lpha$ is a memo table used to calculate the normalization factor in CRF. $lpha[k, v]$ stores the unnormalized probabilites of all possible unfinished sequences of tags that end at position $k$ with tag $v$. For each $k$, $lpha[k, v]$ is a vector of length $D$ with a component for each tag value $v$. This vector is called a forward vecotr and will also be used in backward computations.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "EmissionExps",
+   "comment" : "(Tensor, default Tensor) A 2-D Tensor with shape [N x D]. The exponentials of Input(Emission). This is an intermediate computational result in forward computation, and will be reused in backward computation.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "TransitionExps",
+   "comment" : "(Tensor, default Tensor) A 2-D Tensor with shape [(D + 2) x D]. The exponentials of Input(Transition). This is an intermediate computational result in forward computation, and will be reused in backward computation.",
+   "duplicable" : 0,
+   "intermediate" : 1
+ }, { 
+   "name" : "LogLikelihood",
+   "comment" : "(Tensor, default Tensor) The logarithm of the conditional likelihood of each training sample in a mini-batch. This is a 2-D tensor with shape [S x 1], where S is the sequence number in a mini-batch. Note: S is equal to the sequence number in a mini-batch. The output is no longer a LoDTensor.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "logsigmoid",
+ "comment" : "\nLogsigmoid Activation Operator\n\n$$y = \\log \\frac{1}{1 + e^{-x}}$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of LogSigmoid operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of LogSigmoid operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "exp",
+ "comment" : "\nExp Activation Operator.\n\n$y = e^x$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Exp operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Exp operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "soft_relu",
+ "comment" : "\nSoftRelu Activation Operator.\n\n$y = \\ln(1 + \\exp(\\max(\\min(x, threshold), threshold))$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of SoftRelu operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of SoftRelu operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "threshold",
+   "type" : "float",
+   "comment" : "The threshold value of SoftRelu",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "softshrink",
+ "comment" : "\nSoftshrink Activation Operator.\n\n$$\ny = \\begin{cases} \n    x - \\lambda, \\text{if } x > \\lambda \\\\\n    x + \\lambda, \\text{if } x < -\\lambda \\\\\n    0,  \\text{otherwise}\n    \\end{cases}\n$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Softshrink operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Softshrink operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "lambda",
+   "type" : "float",
+   "comment" : "non-negative offset",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "maxout",
+ "comment" : "\nMaxOut Operator.\n\nAssumed the input shape is (N, Ci, H, W).\nThe output shape is (N, Co, H, W).\nThen $Co = Ci / groups$ and the operator formula is as follows:\n\n$$\ny_{si+j} = \\max_k x_{gsi + sk + j} \\\\\ng = groups \\\\\ns = \\frac{input.size}{num\\_channels} \\\\\n0 \\le i < \\frac{num\\_channels}{groups} \\\\\n0 \\le j < s \\\\\n0 \\le k < groups\n$$\n\nPlease refer to Paper:\n  - Maxout Networks: http://www.jmlr.org/proceedings/papers/v28/goodfellow13.pdf\n  - Multi-digit Number Recognition from Street View \\\n    Imagery using Deep Convolutional Neural Networks: \\\n    https://arxiv.org/pdf/1312.6082v4.pdf\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "(Tensor) The input tensor of maxout operator. The format of input tensor is NCHW. Where N is batch size, C is the number of channels, H and W is the height and width of feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Out",
+   "comment" : "(Tensor) The output tensor of maxout operator.The format of output tensor is also NCHW.Where N is batch size, C is the number of channels, H and W is the height and width of feature.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "groups",
+   "type" : "int",
+   "comment" : "\"Specifies how many groups the input tensor will be split\"\n        \"in the channel dimension. And the number of output channel is \"\n        \"the number of channels divided by groups..\"\n        ",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "ftrl",
+ "comment" : "\nFTRL (Follow The Regularized Leader) Operator.\n\nOptimizer that implements the FTRL algorithm:\n\n$$\nnew\\_accum = squared\\_accum + grad^2 \\\\\nif (lr\\_power == -0.5) {\n   linear\\_accum += grad - (\\surd(new\\_accum) - \\surd(squared\\_accum)) /\n                   (learning\\_rate * param) \\\\\n} else {\n   linear\\_accum += grad -\n                  (new\\_accum^{-lr\\_power} - accum^{-lr\\_power}) /\n                  (learning\\_rate * param) \\\\\n}\n\nx = (l1 * sign(linear\\_accum) - linear\\_accum)\nif (lr\\_power == -0.5) {\n   y = \\frac{\\surd(new\\_accum)}{learning\\_rate} + (2 * l2) \\\\\n   pre\\_shrink = \\frac{x}{y} \\\\\n   param = (abs(linear\\_accum) > l1).select(pre\\_shrink, 0.0) \\\\\n} else {\n   y = \\frac{new\\_accum^{-lr\\_power}}{learning\\_rate} + (2 * l2) \\\\\n   pre\\_shrink = \\frac{x}{y} \\\\\n   param = (abs(linear\\_accum) > l1).select(pre\\_shrink, 0.0) \\\\\n}\nsquared\\_accum += grad^2;\n$$\n\nThe paper that proposed Follow The Regularized Leader (FTRL):\n(https://www.eecs.tufts.edu/~dsculley/papers/ad-click-prediction.pdf)\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "Param",
+   "comment" : "(Tensor, default Tensor) Input parameter value that has to be updated.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "SquaredAccumulator",
+   "comment" : "(Tensor, default Tensor) Accumulator that accumulates squared gradients.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LinearAccumulator",
+   "comment" : "(Tensor, default Tensor) Accumulator that accumulates linear gradients.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "Grad",
+   "comment" : "(Tensor, default Tensor) Input gradient of the parameter.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LearningRate",
+   "comment" : "(Tensor, default Tensor) The learning rate should be a tensor of size 1.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "ParamOut",
+   "comment" : "(Tensor) Output updated parameter value.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "SquaredAccumOut",
+   "comment" : "(Tensor) Output accumulated squared gradients.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ }, { 
+   "name" : "LinearAccumOut",
+   "comment" : "(Tensor) Output accumulated linear gradients.",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [ 
+ { 
+   "name" : "l1",
+   "type" : "float",
+   "comment" : "(float, default 0.0) L1 regularization strength.",
+   "generated" : 0
+ }, { 
+   "name" : "l2",
+   "type" : "float",
+   "comment" : "(float, default 0.0) L2 regularization strength.",
+   "generated" : 0
+ }, { 
+   "name" : "lr_power",
+   "type" : "float",
+   "comment" : "(float, default -0.5f) Learning Rate Power.",
+   "generated" : 0
+ } ] 
+},{
+ "type" : "round",
+ "comment" : "\nRound Activation Operator.\n\n$y = [x]$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Round operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Round operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+},{
+ "type" : "softsign",
+ "comment" : "\nSoftsign Activation Operator.\n\n$$y = \\frac{x}{1 + |x|}$$\n\n",
+ "inputs" : [ 
+ { 
+   "name" : "X",
+   "comment" : "Input of Softsign operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "outputs" : [ 
+ { 
+   "name" : "Y",
+   "comment" : "Output of Softsign operator",
+   "duplicable" : 0,
+   "intermediate" : 0
+ } ], 
+ "attrs" : [  ] 
+}]
diff --git a/doc/py-modindex.html b/doc/py-modindex.html
index 40422411bb9b6fc275484a9459da3716312d541e..ca5f08ca39487590ce064fa96bc48aa3e4306a11 100644
--- a/doc/py-modindex.html
+++ b/doc/py-modindex.html
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,7 +87,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +111,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -138,17 +143,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -198,12 +226,12 @@
      
        
           
-       paddle.trainer_config_helpers.attrs
+       paddle.v2.fluid.regularizer
        
      
        
           
-       paddle.trainer_config_helpers.data_sources
+       paddle.v2.image
        
    
 
diff --git a/doc/search.html b/doc/search.html
index 09f082299f10449f0c952e3a54debe593cb8153f..04aec6bfe459ff3a0fe0026a63d79b4093fc1174 100644
--- a/doc/search.html
+++ b/doc/search.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,7 +84,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
diff --git a/doc/searchindex.js b/doc/searchindex.js
index 74b7196a783856d992a2bea8008bdaa5ff0f03e9..8121ba1852c23c8cff600c7bef8fcc94c153ce6b 100644
--- a/doc/searchindex.js
+++ b/doc/searchindex.js
@@ -1 +1 @@
-Search.setIndex({docnames:["about/index_en","api/index_en","api/v1/data_provider/dataprovider_en","api/v1/data_provider/pydataprovider2_en","api/v1/index_en","api/v1/predict/swig_py_paddle_en","api/v1/trainer_config_helpers/activations","api/v1/trainer_config_helpers/attrs","api/v1/trainer_config_helpers/data_sources","api/v1/trainer_config_helpers/evaluators","api/v1/trainer_config_helpers/layers","api/v1/trainer_config_helpers/networks","api/v1/trainer_config_helpers/optimizers","api/v1/trainer_config_helpers/poolings","api/v2/config/activation","api/v2/config/attr","api/v2/config/layer","api/v2/config/networks","api/v2/config/optimizer","api/v2/config/pooling","api/v2/data","api/v2/model_configs","api/v2/run_logic","design/api","design/dist/README","design/multi_language_interface/why_plain_c","design/reader/README","getstarted/basic_usage/index_en","getstarted/build_and_install/build_from_source_en","getstarted/build_and_install/docker_install_en","getstarted/build_and_install/index_en","getstarted/build_and_install/ubuntu_install_en","getstarted/index_en","howto/deep_model/rnn/index_en","howto/deep_model/rnn/rnn_config_en","howto/dev/contribute_to_paddle_en","howto/dev/new_layer_en","howto/index_en","howto/optimization/gpu_profiling_en","howto/usage/cluster/cluster_train_en","howto/usage/cmd_parameter/arguments_en","howto/usage/cmd_parameter/detail_introduction_en","howto/usage/cmd_parameter/index_en","howto/usage/cmd_parameter/use_case_en","howto/usage/k8s/k8s_aws_en","howto/usage/k8s/k8s_en","howto/usage/k8s/src/k8s_data/README","howto/usage/k8s/src/k8s_train/README","index_en","tutorials/embedding_model/index_en","tutorials/gan/index_en","tutorials/image_classification/index_en","tutorials/imagenet_model/resnet_model_en","tutorials/index_en","tutorials/quick_start/index_en","tutorials/rec/ml_dataset_en","tutorials/rec/ml_regression_en","tutorials/semantic_role_labeling/index_en","tutorials/sentiment_analysis/index_en","tutorials/text_generation/index_en"],envversion:50,filenames:["about/index_en.rst","api/index_en.rst","api/v1/data_provider/dataprovider_en.rst","api/v1/data_provider/pydataprovider2_en.rst","api/v1/index_en.rst","api/v1/predict/swig_py_paddle_en.rst","api/v1/trainer_config_helpers/activations.rst","api/v1/trainer_config_helpers/attrs.rst","api/v1/trainer_config_helpers/data_sources.rst","api/v1/trainer_config_helpers/evaluators.rst","api/v1/trainer_config_helpers/layers.rst","api/v1/trainer_config_helpers/networks.rst","api/v1/trainer_config_helpers/optimizers.rst","api/v1/trainer_config_helpers/poolings.rst","api/v2/config/activation.rst","api/v2/config/attr.rst","api/v2/config/layer.rst","api/v2/config/networks.rst","api/v2/config/optimizer.rst","api/v2/config/pooling.rst","api/v2/data.rst","api/v2/model_configs.rst","api/v2/run_logic.rst","design/api.md","design/dist/README.md","design/multi_language_interface/why_plain_c.md","design/reader/README.md","getstarted/basic_usage/index_en.rst","getstarted/build_and_install/build_from_source_en.md","getstarted/build_and_install/docker_install_en.rst","getstarted/build_and_install/index_en.rst","getstarted/build_and_install/ubuntu_install_en.rst","getstarted/index_en.rst","howto/deep_model/rnn/index_en.rst","howto/deep_model/rnn/rnn_config_en.rst","howto/dev/contribute_to_paddle_en.md","howto/dev/new_layer_en.rst","howto/index_en.rst","howto/optimization/gpu_profiling_en.rst","howto/usage/cluster/cluster_train_en.md","howto/usage/cmd_parameter/arguments_en.md","howto/usage/cmd_parameter/detail_introduction_en.md","howto/usage/cmd_parameter/index_en.rst","howto/usage/cmd_parameter/use_case_en.md","howto/usage/k8s/k8s_aws_en.md","howto/usage/k8s/k8s_en.md","howto/usage/k8s/src/k8s_data/README.md","howto/usage/k8s/src/k8s_train/README.md","index_en.rst","tutorials/embedding_model/index_en.md","tutorials/gan/index_en.md","tutorials/image_classification/index_en.md","tutorials/imagenet_model/resnet_model_en.md","tutorials/index_en.md","tutorials/quick_start/index_en.md","tutorials/rec/ml_dataset_en.md","tutorials/rec/ml_regression_en.rst","tutorials/semantic_role_labeling/index_en.md","tutorials/sentiment_analysis/index_en.md","tutorials/text_generation/index_en.md"],objects:{"paddle.trainer.PyDataProvider2":{provider:[3,0,1,""]},"paddle.trainer_config_helpers":{attrs:[7,1,0,"-"],data_sources:[8,1,0,"-"]},"paddle.trainer_config_helpers.attrs":{ExtraAttr:[7,2,1,""],ExtraLayerAttribute:[7,3,1,""],ParamAttr:[7,2,1,""],ParameterAttribute:[7,3,1,""]},"paddle.trainer_config_helpers.attrs.ParameterAttribute":{set_default_parameter_name:[7,4,1,""]},"paddle.trainer_config_helpers.data_sources":{define_py_data_sources2:[8,0,1,""]}},objnames:{"0":["py","function","Python function"],"1":["py","module","Python module"],"2":["py","attribute","Python attribute"],"3":["py","class","Python class"],"4":["py","method","Python method"]},objtypes:{"0":"py:function","1":"py:module","2":"py:attribute","3":"py:class","4":"py:method"},terms:{"0000x":54,"00186201e":5,"00m":38,"02595v1":[10,16],"03m":38,"0424m":38,"0473v3":[11,17],"055ee37d":44,"05d":51,"0630u":38,"06u":38,"0810u":38,"08823112e":5,"0957m":38,"0ab":[10,16],"0rc2":29,"0th":59,"10007_10":58,"10014_7":58,"100gb":38,"100gi":44,"10m":38,"1150u":38,"11e6":45,"12194102e":5,"124n":38,"13m":45,"1490u":38,"15501715e":5,"1550u":38,"15mb":54,"1636k":59,"16mb":54,"16u":38,"173m":52,"173n":38,"1770u":38,"18ad":44,"18e457ce3d362ff5f3febf8e7f85ffec852f70f3b629add10aed84f930a68750":45,"197u":38,"1gb":38,"1st":[49,52,58,59],"202mb":59,"210u":38,"211839e770f7b538e2d8":[11,17],"215n":38,"228u":38,"234m":52,"2520u":38,"252kb":54,"25639710e":5,"25k":54,"2680u":38,"27787406e":5,"279n":38,"27m":38,"285m":38,"2863m":38,"28m":38,"28x28":3,"2977m":38,"2cbf7385":44,"2nd":[10,16,58,59],"302n":38,"30u":38,"32777140e":5,"328n":38,"32u":38,"32x32":[20,51],"331n":38,"3320u":38,"36540484e":5,"365e":44,"36u":38,"3710m":38,"3768m":38,"387u":38,"38u":38,"3920u":38,"39u":38,"3rd":[56,58,59],"4035m":38,"4090u":38,"4096mb":41,"4279m":38,"43630644e":5,"43u":38,"448a5b355b84":45,"4560u":38,"4563m":38,"45u":38,"4650u":38,"4726m":38,"473m":45,"48565123e":5,"48684503e":5,"49316648e":5,"4gb":41,"50bd":44,"50gi":44,"51111044e":5,"514u":38,"525n":38,"526u":38,"53018653e":5,"536u":38,"5460u":38,"5470u":38,"54u":38,"55g":59,"5690m":38,"573u":38,"578n":38,"5798m":38,"586u":38,"58s":45,"5969m":38,"6080u":38,"6082v4":[10,16],"6140u":38,"6305m":38,"639u":38,"655u":38,"6780u":38,"6810u":38,"682u":38,"6970u":38,"6ce9":44,"6node":39,"6th":59,"704u":38,"70634608e":5,"7090u":38,"72296313e":5,"72u":38,"73u":38,"75u":38,"760u":38,"767u":38,"783n":38,"784u":38,"78m":38,"7eamaa":20,"7kb":45,"8250u":38,"8300u":38,"830n":38,"849m":38,"85625684e":5,"861u":38,"864k":59,"8661m":38,"892m":38,"901n":38,"90u":38,"918u":38,"9247m":38,"924n":38,"9261m":38,"93137714e":5,"9330m":38,"94u":38,"9530m":38,"96644767e":5,"983m":38,"988u":38,"997u":38,"99982715e":5,"99m":52,"99u":38,"9f18":45,"\u4e0d\u4f7f\u7528\u9759\u6001\u5e93":25,"\u4e0d\u4f7f\u7528c":25,"\u4e0d\u4f7f\u7528swig":25,"\u4e0d\u540c\u7248\u672c\u7684\u7f16\u8bd1\u5668\u4e4b\u95f4":25,"\u4e0d\u540c\u8bed\u8a00\u7684\u63a5\u53e3\u9002\u5e94\u4e0d\u540c\u8bed\u8a00\u7684\u7279\u6027":25,"\u4e0d\u5d4c\u5165\u5176\u4ed6\u8bed\u8a00\u89e3\u91ca\u5668":25,"\u4e0d\u5d4c\u5165python\u89e3\u91ca\u5668":25,"\u4e0d\u663e\u793a\u7684\u5199\u6bcf\u4e2a\u7c7b\u5177\u4f53\u5305\u542b\u4ec0\u4e48":25,"\u4e14\u589e\u52a0\u4e00\u4e2a\u7b2c\u4e09\u65b9\u8bed\u8a00":25,"\u4e14c99\u652f\u6301bool\u7c7b\u578b\u548c\u5b9a\u957f\u6574\u6570":25,"\u4e14c99\u76f8\u5bf9\u4e8ec11\u4f7f\u7528\u66f4\u52a0\u5e7f\u6cdb":25,"\u4e2d":25,"\u4e2d\u5b8c\u5168\u4e00\u81f4":25,"\u4e5f\u4e0d\u4f7f\u7528\u5176\u4ed6\u52a8\u6001\u5e93":25,"\u4e66\u5199":25,"\u4ec5\u4ec5\u4f7f\u7528":25,"\u4ed6\u7684\u76ee\u6807\u662f\u4f7f\u7528c":25,"\u4ee3\u7801\u751f\u6210\u7684\u7b26\u53f7\u53ef\u80fd\u4e0d\u4e00\u81f4":25,"\u4f1a\u5bfc\u81f4\u4e0d\u540c\u7248\u672cpython\u5728\u4e00\u4e2a\u8fdb\u7a0b\u91cc\u7684bug":25,"\u4f1a\u76f4\u63a5\u62a5\u9519\u9000\u51fa":25,"\u4f46\u662f\u89e3\u91ca\u6027\u8bed\u8a00":25,"\u4f5c\u4e3a\u7c7b\u53e5\u67c4":25,"\u4f7f\u7528\u52a8\u6001\u5e93":25,"\u4f7f\u7528\u9759\u6001\u5e93\u548c\u52a8\u6001\u5e93\u96be\u5ea6\u5dee\u4e0d\u591a":25,"\u4f7f\u7528c99\u505a\u63a5\u53e3":25,"\u4f7f\u7528c99\u800c\u4e0d\u4f7f\u7528c11\u7684\u539f\u56e0\u662f":25,"\u4f7f\u7528c99\u800c\u4e0d\u4f7f\u7528c89":25,"\u4f7f\u7528swig\u53ea\u652f\u6301cpython\u89e3\u91ca\u5668":25,"\u4f7f\u7528swig\u9700\u8981\u591a\u8bed\u8a00\u7ed1\u5b9a\u7684\u5f00\u53d1\u4eba\u5458\u719f\u7ec3\u638c\u63e1swig\u914d\u7f6e":25,"\u4f7f\u7528void":25,"\u4f8b\u5982":25,"\u4f8b\u5982\u5bf9\u4e8ejava\u6216\u8005python":25,"\u4f8b\u5982\u5bf9\u4e8ejava\u6765\u8bf4":25,"\u4f8b\u5982\u5bf9\u4e8epython":25,"\u4f8b\u5982c":25,"\u4f8b\u5982java\u4e0epython\u7684\u9519\u8bef\u5904\u7406\u662f\u76f4\u63a5\u6254\u51fa\u6765except":25,"\u4f8b\u5982python\u53ef\u4ee5\u4f7f\u7528":25,"\u4f8b\u5982python\u7684":25,"\u4fbf\u662f\u5c06\u9759\u6001\u5e93\u52a0\u5165jvm\u4e2d":25,"\u505a\u63a5\u53e3":25,"\u5176\u4e2d":25,"\u5185\u90e8\u9a71\u52a8python\u89e3\u91ca\u5668\u8fdb\u884c\u6a21\u578b\u914d\u7f6e\u89e3\u6790\u548c\u6570\u636e\u8bfb\u53d6":25,"\u518d\u5728\u6bcf\u4e00\u4e2aapi\u4e2d\u81ea\u5df1\u68c0\u67e5\u7c7b\u578b":25,"\u5199\u4ee3\u7801":25,"\u51fd\u6570\u547d\u540d":25,"\u52a8\u6001\u5e93":25,"\u5373\u8fd9\u4e2a\u52a8\u6001\u5e93\u662f\u4e0d\u4f9d\u8d56\u4e8e\u5176\u4ed6\u4efb\u4f55\u6587\u4ef6\u7684":25,"\u53c2\u6570":25,"\u53ea\u80fd\u8c03\u7528paddle\u7684\u52a8\u6001\u5e93":25,"\u53ef\u4ee5\u5728\u4efb\u4f55\u673a\u5668\u4e0a\u6267\u884c\u7684":25,"\u540d\u5b57\u4fee\u9970":25,"\u5426\u5219\u5f97\u628apaddle\u9759\u6001\u5e93\u94fe\u63a5\u5230\u89e3\u91ca\u5668\u91cc":25,"\u548c":25,"\u56e0\u4e3aswig\u5728\u7b2c\u4e09\u65b9\u8bed\u8a00\u4e2d\u66b4\u9732\u7684\u51fd\u6570\u540d":25,"\u5728\u8fd9\u4e2a\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165\u4efb\u4f55\u5176\u4ed6\u8bed\u8a00\u7684\u89e3\u91ca\u5668":25,"\u5728c":25,"\u5728c\u7684\u5934\u6587\u4ef6":25,"\u5927\u591a\u6570\u8bed\u8a00\u90fd\u652f\u6301\u4f7f\u7528c\u8bed\u8a00api":25,"\u5982\u679c\u4f7f\u7528swig\u6211\u4eec\u9700\u8981\u5c06\u5728interface\u6587\u4ef6\u91cc":25,"\u5982\u679c\u7528\u6237\u8981\u628apaddle\u7684\u9759\u6001\u5e93":25,"\u5982\u679c\u8c03\u7528\u9759\u6001\u5e93\u53ea\u80fd\u5c06\u9759\u6001\u5e93\u4e0e\u89e3\u91ca\u5668\u94fe\u63a5":25,"\u5b66\u4e60\u6210\u672c\u9ad8":25,"\u5b9e\u73b0\u7b80\u5355":25,"\u5bf9\u4e8e\u4e0d\u540c\u8bed\u8a00":25,"\u5bf9\u4e8e\u540c\u4e00\u6bb5c":25,"\u5bf9\u4e8e\u591a\u8bed\u8a00\u63a5\u53e3":25,"\u5bf9\u4e8e\u5927\u591a\u6570\u8bed\u8a00":25,"\u5bf9\u6bd4":25,"\u5c06\u5927\u91cf\u7684":25,"\u5c31\u9700\u8981\u5bf9\u8fd9\u4e2a\u7b2c\u4e09\u65b9\u8bed\u8a00\u589e\u52a0\u4e00\u4e9b\u5b9a\u4e49":25,"\u5e76\u4e14\u5728\u5e38\u89c1\u7684\u5e73\u53f0\u4e0a":25,"\u5e76\u4e14\u8ba9\u63a5\u53e3\u8131\u79bb\u5b9e\u73b0\u7ec6\u8282":25,"\u5e76\u6ca1\u6709paddle\u7279\u522b\u9700\u8981\u7684\u7279\u6027":25,"\u5f88\u96be\u4fdd\u8bc1\u591a\u8bed\u8a00\u4ee3\u7801\u98ce\u683c\u7684\u4e00\u81f4\u6027":25,"\u5f97\u4f7f\u7528":25,"\u6211\u4eec\u4f7f\u7528\u52a8\u6001\u5e93\u6765\u5206\u53d1paddl":25,"\u6211\u4eec\u6700\u7ec8\u7684\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165python\u6216\u8005\u5176\u4ed6\u4efb\u4f55\u8bed\u8a00\u7684\u89e3\u91ca\u5668":25,"\u6216\u8005":25,"\u624b\u5199\u591a\u8bed\u8a00\u7ed1\u5b9a":25,"\u63a5\u53e3":25,"\u6570\u636e\u8bfb\u53d6\u5747\u4ea4\u7531\u5176\u4ed6\u8bed\u8a00\u5b8c\u6210":25,"\u6587\u4ef6":25,"\u6587\u4ef6\u5185\u5bb9\u4e3a":25,"\u65e0\u6cd5\u505a\u5230\u5bf9\u4e8e\u5404\u79cd\u8bed\u8a00\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u7684\u9002\u914d":25,"\u662f\u4e00\u4e2a\u591a\u8bed\u8a00\u63a5\u53e3\u7684\u4ee3\u7801\u751f\u6210\u5668":25,"\u662f\u4e0d\u5e38\u89c1\u7684\u505a\u6cd5":25,"\u662f\u56e0\u4e3ac99\u652f\u6301":25,"\u6700\u5e38\u89c1\u7684\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u662fexcept":25,"\u6709\u6807\u51c6\u7684":25,"\u6709\u7684\u65f6\u5019":25,"\u6765\u786e\u4fdd\u628a":25,"\u6765\u8868\u793apaddle\u5185\u90e8\u7c7b":25,"\u6a21\u578b\u914d\u7f6e\u89e3\u6790":25,"\u73b0\u9636\u6bb5paddle\u6709\u4e00\u4e2a\u95ee\u9898\u662f":25,"\u751f\u6210\u5404\u79cd\u8bed\u8a00\u7684\u7ed1\u5b9a\u4ee3\u7801":25,"\u751f\u6210\u6587\u6863":25,"\u751f\u6210api\u6587\u6863":25,"\u7531\u4e8ec":25,"\u7684\u547d\u540d\u98ce\u683c\u5e76\u4e0d\u80fd\u9002\u5e94\u5176\u4ed6\u7b2c\u4e09\u65b9\u8bed\u8a00":25,"\u7684\u5934\u6587\u4ef6":25,"\u7684\u63a5\u53e3\u6837\u5f0f":25,"\u7684\u6e90\u7801\u91cc\u4f7f\u7528\u4e86":25,"\u7684\u89c4\u8303":25,"\u76ee\u524d\u5d4c\u5165python\u89e3\u91ca\u5668":25,"\u76ee\u524dpaddle\u7684\u8fdb\u7a0b\u6a21\u578b\u662fc":25,"\u76f4\u63a5\u4f7f\u7528c\u8bed\u8a00\u7684":25,"\u76f4\u63a5\u5bfc\u51fa\u5230c\u7684\u63a5\u53e3\u6bd4\u8f83\u56f0\u96be":25,"\u793e\u533a\u53c2\u4e0e\u56f0\u96be":25,"\u793e\u533a\u8d21\u732e\u4ee3\u7801\u5b66\u4e60\u6210\u672c\u9ad8":25,"\u7c7b\u540d\u548cc":25,"\u7c7b\u578b":25,"\u7ea2\u697c\u68a6":49,"\u7ed3\u8bba":25,"\u7f16\u8bd1\u5668\u6ca1\u6709":25,"\u7f16\u8bd1\u578b\u8bed\u8a00":25,"\u800c\u4e0d\u652f\u6301pypy\u89e3\u91ca\u5668":25,"\u800c\u5728cpp\u91cc\u9762\u5b9e\u73b0\u8fd9\u4e2ac\u7684\u63a5\u53e3":25,"\u800c\u591a\u8bed\u8a00\u63a5\u53e3\u9700\u8981\u76f4\u63a5\u8bfb\u53d6\u751f\u6210\u7684\u4e8c\u8fdb\u5236":25,"\u800c\u5bf9\u4e8egolang":25,"\u800c\u5bf9\u4e8egolang\u9519\u8bef\u5904\u7406\u5e94\u8be5\u4f7f\u7528\u8fd4\u56de\u503c":25,"\u800cswig\u53ea\u80fd\u7b80\u5355\u7684\u66b4\u9732c":25,"\u826f\u597d\u7684\u6587\u6863":25,"\u89e3\u91ca\u578b\u8bed\u8a00\u53ea\u80fd\u8c03\u7528\u52a8\u6001\u5e93":25,"\u89e3\u91ca\u6027\u8bed\u8a00\u5b9e\u9645\u8fd0\u884c\u7684\u4e8c\u8fdb\u5236\u662f\u89e3\u91ca\u5668\u672c\u8eab":25,"\u8fd9\u4e2a\u63a5\u53e3\u9700\u8981\u505a\u5230":25,"\u8fd9\u4e2a\u6587\u4ef6\u5177\u6709\u72ec\u7279\u7684\u8bed\u6cd5":25,"\u8fd9\u5bf9\u4e8e\u901a\u5e38\u7684java\u7684\u5f00\u53d1\u8005\u6765\u8bf4":25,"\u8fd9\u662f\u56e0\u4e3a":25,"\u8fd9\u90fd\u9700\u8981\u8fd9\u4e2a\u63a5\u53e3\u6309\u7167\u7ea6\u5b9a\u4fd7\u6210\u7684\u89c4\u5219\u6765\u6ce8\u91ca\u5b8c\u5907":25,"\u90fd\u662fabi\u8c03\u7528\u6807\u51c6\u7684":25,"\u91cc\u6240\u6709\u7684\u7b26\u53f7\u90fd\u5199\u5165\u81ea\u5df1\u7684\u7a0b\u5e8f\u7684\u4e8c\u8fdb\u5236\u6587\u4ef6\u91cc":25,"\u91cd\u547d\u540d\u6210":25,"\u94fe\u63a5\u5230\u81ea\u5df1\u7684\u7a0b\u5e8f\u91cc":25,"\u9519\u8bef\u5904\u7406":25,"\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u662f\u8fd4\u56de\u503c":25,"\u9519\u8bef\u5904\u7406\u7684\u65b9\u5f0f\u4e5f\u4e0d\u5c3d\u76f8\u540c":25,"\u9700\u8981\u6709\u7a33\u5b9a\u7684\u5bfc\u51fa\u7b26\u53f7":25,"\ufb01xed":59,"abstract":[36,41],"api\u4e2d\u4f7f\u7528":25,"boolean":[10,16,25],"break":54,"c99\u662f\u76ee\u524dc\u6700\u5e7f\u6cdb\u7684\u4f7f\u7528\u6807\u51c6":25,"c\u6709\u6807\u51c6\u7684abi":25,"c\u8bed\u8a00\u662f\u6709\u5bfc\u51fa\u7b26\u53f7\u7684\u6807\u51c6\u7684":25,"case":[10,16,26,27,34,35,36,38,42,44,50,54],"char":56,"class":[5,7,10,12,14,15,16,17,18,19,20,22,23,25,40,51,58],"const":36,"default":[3,7,9,10,11,12,15,16,17,19,20,22,23,29,39,41,43,44,45,54,56,58,59],"export":[28,51],"final":[11,17,27,28,36,56,58],"float":[3,7,9,10,12,15,16,18,20,27,36,38,43,49,52,56],"function":[3,5,8,10,11,12,16,17,18,20,23,26,27,34,36,38,39,41,50,51,54,57,58,59],"golang\u53ef\u4ee5\u4f7f\u7528":25,"golang\u7684":25,"h\u5e76\u4e0d\u56f0\u96be":25,"import":[3,5,9,10,16,23,27,34,38,44,49,50,51,52,54,56,58,59],"int":[3,7,9,10,11,12,15,16,17,20,25,26,36,43,54,56,57],"interface\u6587\u4ef6\u7684\u5199\u6cd5\u975e\u5e38":25,"long":[2,10,11,16,17,20,29,38,57,58],"new":[3,10,16,20,24,26,35,37,44,45,50,54,57,58],"null":[10,16,36,41,56],"paddle\u4e00\u4e2a\u52a8\u6001\u5e93\u53ef\u4ee5\u5728\u4efb\u4f55linux\u7cfb\u7edf\u4e0a\u8fd0\u884c":25,"paddle\u5185\u5d4c\u7684python\u89e3\u91ca\u5668\u548c\u5916\u90e8\u4f7f\u7528\u7684python\u5982\u679c\u7248\u672c\u4e0d\u540c":25,"paddle\u5185\u90e8\u7684\u7c7b\u4e3ac":25,"paddle\u7684\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0\u5305\u62ec\u4e00\u4e0b\u51e0\u4e2a\u65b9\u9762":25,"paddle\u7684\u94fe\u63a5\u65b9\u5f0f\u6bd4\u8f83\u590d\u6742":25,"paddle\u9700\u8981\u4e00\u4e2a\u591a\u8bed\u8a00\u63a5\u53e3":25,"paddle\u9759\u6001\u5e93\u94fe\u63a5\u590d\u6742":25,"public":[20,36,39,44,45,58],"return":[3,8,9,10,11,16,17,19,20,22,23,27,34,36,44,50,52,54,55,56,59],"short":[10,11,16,17,27,56,57,58],"static":[10,44],"super":36,"swig\u652f\u6301\u7684\u8bed\u8a00\u6216\u8005\u89e3\u91ca\u5668\u6709\u5c40\u9650":25,"swig\u66b4\u9732\u7684\u63a5\u53e3\u4fdd\u7559\u4e86c":25,"swig\u751f\u6210\u7684\u4ee3\u7801\u4e0d\u80fd\u4fdd\u8bc1\u591a\u8bed\u8a00\u4ee3\u7801\u98ce\u683c\u7684\u4e00\u81f4\u6027":25,"swig\u76f4\u63a5\u8bfb\u53d6c":25,"swig\u9700\u8981\u5199\u4e00\u4e2ainterface\u6587\u4ef6":25,"switch":[44,58],"throw":44,"true":[3,7,9,10,11,12,15,16,17,19,20,22,23,26,27,34,36,41,43,44,52,56,57,58,59],"try":[12,18,24,26,38,50,56],"void":[25,36],"while":[2,3,7,9,15,20,26,29,34,41,50,54,58,59],AGE:[44,45],AND:56,ARE:56,AWS:[37,46,47],Abs:6,Age:55,And:[3,9,10,12,16,18,20,26,31,35,43,44,45,49,52,56,58,59],But:[3,10,11,16,17],EOS:[10,16],For:[2,3,8,9,10,12,16,18,20,23,26,27,28,29,34,36,38,39,40,41,43,49,51,52,54,58,59],Going:58,Has:3,IDs:[20,54],Ids:54,Into:44,Its:[3,34,44,56],Not:[23,24,39],ONE:3,One:[9,10,11,17,22,34,36,41,50,54,58,59],QoS:45,THE:3,TLS:[23,44],That:[10,16,20,26,29,41,43],The:[2,3,5,7,8,9,10,11,12,14,15,16,17,18,20,22,23,24,26,27,28,29,30,31,34,35,36,38,39,41,43,44,45,49,50,51,52,54,55,56,57,58,59],Their:[3,10,16,24],Then:[5,10,16,28,29,34,35,36,38,44,45,49,51,56,57,58],There:[9,10,16,20,22,23,24,27,29,31,38,44,50,51,52,53,54,56,59],These:[39,43,51,57],USE:56,USING:56,Use:[3,23,26,29,36,38,41,42,44,56],Used:[11,17],Useful:3,Using:[45,58],VPS:44,WITH:35,Will:[20,22],With:[3,10,11,16,17,27,50,57],Yes:29,___fc_layer_0__:44,__init__:36,__list_to_map__:56,__main__:52,__meta__:56,__name__:52,__rnn_step__:34,_error:50,_link:[11,17],_proj:[10,16],_res2_1_branch1_bn:52,_source_language_embed:[34,49],_target_language_embed:[34,49],aaaaaaaaaaaaa:44,abc:[10,16],abl:[10,16,23,50,58],about:[5,10,11,16,17,27,38,40,41,44,48,57,58,59],abov:[3,5,10,16,23,24,27,29,38,44,45,50,52,54,57],abs:[11,17,50],absolut:[2,39],academ:55,acceler:43,accept:[3,5,20,23,26,54,57],acceptor:57,access:[2,10,11,16,17,23,34,59],accessmod:44,accident:55,accomplish:29,accord:[2,3,9,10,16,34,35,39,40,41,43],accordingli:[5,36],accordingto:57,accrod:[11,17],accumul:24,accuraci:[9,36,54,55,58],achiev:[38,51],ack:41,acl:58,aclimdb:58,across:[10,16],act:[10,11,16,17,27,34,54],act_typ:54,action:[44,55],activ:[0,4,5,10,11,16,17,21,27,28,36,41,54,58],activi:[11,17],actual:[3,10,16,27],adadelta:[12,54],adagrad:[12,54],adam:[12,23,54,58,59],adamax:[12,54],adamoptim:[49,54,58,59],adapt:[9,12,18,27,58,59],add:[3,10,11,16,17,20,27,28,35,36,38,43,54,56],add_input:36,add_test:36,add_to:[10,16],add_unittest_without_exec:36,addbia:36,added:[3,9,36],adding:52,addit:[10,11,16,17,29,54],address:[24,29,38,41],addrow:36,addtion:39,addto:10,addtolay:[10,16],adject:58,adjust:27,admin:55,adopt:57,advanc:[34,38,41],advantag:58,adventur:55,adverb:58,adversari:26,advic:38,affect:[10,16],afi:3,aforement:39,after:[10,16,20,28,31,34,36,39,41,43,44,45,50,51,52,54,56,57,58,59],again:[23,24,38],against:44,age:[20,56],agg_level:[10,16],aggreg:44,aggregatelevel:[10,16],aid:38,aim:[58,59],aircraft:59,airplan:51,aistat:[10,16],alex:[10,16,58],alexnet_pass1:43,alexnet_pass2:43,algorithm:[10,12,16,18,27,34,49,51,58,59],alia:[6,7,13,14,15],align:[10,11,16,17,20,59],all:[0,3,7,9,10,12,15,16,18,23,24,27,29,34,35,36,38,39,40,41,43,44,45,49,50,52,54,55,56,57,58,59],alloc:[7,15,36,43],allow:[23,29,35,36,38,41,44,54],allow_only_one_model_on_one_gpu:[40,41,43],almost:[11,17,27,39,49],along:58,alreadi:[24,29,38,39,41,44,45,58],alreali:[40,59],also:[2,3,9,10,11,16,17,20,23,26,28,29,34,36,38,39,45,50,51,52,54,57,58],although:27,alwai:[5,10,11,16,17,22,26,27,41,44,59],amaz:51,amazon:[44,45,54,58],amazonaw:44,amazonec2fullaccess:44,amazonelasticfilesystemfullaccess:44,amazonroute53domainsfullaccess:44,amazonroute53fullaccess:44,amazons3fullaccess:44,amazonvpcfullaccess:44,ambigu:[26,57],amd64:44,amend:35,american:51,among:[44,58],amount:[38,58],analysi:[27,38,53,57],analyz:[54,58],andd:44,ani:[2,3,10,11,16,17,20,23,24,26,34,35,38,44,54,56,59],anim:55,annot:57,annual:57,anoth:[3,10,16,23,29,41,44,57,58],ans:44,answer:[27,44,57],anyth:[20,26,35,44,57],api:[20,22,23,28,36,38,44,48,50,54,56,58],apiserv:44,apivers:[44,45],apo:59,appar:59,appear:57,append:[3,22,26,34,36,39,56],append_gradient_machin:22,appleclang:28,appleyard:38,appli:[0,10,11,16,17,34,36,51,54],applic:[29,38,44,45,58],appreci:[35,58],approach:[10,16],apt:[28,31,51],arbitrari:[10,16],architectur:[49,57,58,59],architecur:58,archiv:[20,25],arg:[3,8,9,10,11,12,16,17,20,27,40,50,51,52,54,56,57,58],arg_nam:[10,16],argu:57,argument:[3,5,8,10,16,20,34,36,41,42,49,50,51,52,56,57,58,59],argv:52,arn:44,around:[3,10,16,44],arrai:[5,10,16,20,22,26,27,52],art:[27,57],articl:[39,45],artifact:44,artifici:50,artist:55,arxiv:[10,11,16,17,50,58],ask:24,aspect:58,assign:[10,16,41,44],associ:[57,58,59],assum:[10,16,34,43,49],assur:2,astyp:[26,50],async:[12,24,40],async_count:41,async_lagged_grad_discard_ratio:41,async_lagged_ratio_default:[40,41],async_lagged_ratio_min:[40,41],asynchron:[24,41],atla:28,atlas_root:28,attenion:[11,17],attent:[10,11,16,17,29,59],attitud:58,attr:[7,11,15,16,17],attribut:[3,4,10,11,16,17,21,36,49,57],auc:[9,40],aucvalidationlay:41,authent:44,author:[44,52],authorized_kei:39,autmot:35,auto:[25,36,38,53,56],autom:[44,59],automak:28,automat:[10,16,23,28,34,36,39,40,41,44,56,57,59],automaticli:[10,16],automobil:51,avail:[24,28,44],availabel:28,averag:[9,10,12,16,19,41,52,54,56,57,58,59],average_test_period:[40,41,57],average_window:58,averagepool:[10,16],avg:[13,38,54],avgcost:[9,54,56,58,59],avgpool:[10,16,54],avoid:[24,38],avx:[28,29,31],await:45,awar:[23,24,29,44],aws_account_id:44,awsaccountid:44,awskeymanagementservicepowerus:44,b2t:49,b363:45,b8561f5c79193550d64fa47418a9e67ebdd71546186e840f88de5026b8097465:45,ba5f:44,back:[3,24,29],backward:[10,11,14,16,17,34,36,41,43],backward_first:34,backwardactiv:36,bag:[54,58],baidu:[0,10,16,27,31,35,45,49],baik:49,balanc:[41,44,50],balasubramanyan:58,bank:57,bardward:[11,17],bare:45,barrier:41,barrierstatset:38,base:[6,12,16,17,19,20,23,27,31,34,35,36,38,39,41,44,49,50,54,56,58,59],baseactiv:[10,11,16],baseev:22,basematrix:36,basenam:9,basepool:13,basepoolingtyp:[10,11,16,17],baseregular:12,basestr:[7,8,9,10,11,15,16,17,19,22,56],bash:[29,44,45],bashrc:28,basic:[3,10,16,22,29,35,36,54,55,58],batch:[3,9,10,11,12,16,17,18,20,22,23,24,36,39,41,44,45,50,51,52,54,56,57,58,59],batch_0:52,batch_id:22,batch_norm:[10,17],batch_norm_lay:11,batch_norm_typ:[10,16],batch_read:26,batch_siz:[3,12,20,22,27,39,49,50,51,54,56,58,59],batchsiz:[10,16,36],bcd:[10,16],bcebo:20,beam:[10,16,34,41,57,59],beam_gen:[10,16,34],beam_search:[22,34],beam_siz:[10,16,34,40,41,43],beamsiz:59,becaus:[5,10,16,20,23,24,26,34,35,36,43,44,51,54,57],becom:[35,38],been:[3,28,35,51,54,57,58,59],befor:[5,10,11,16,17,24,26,29,35,39,44,51,56,58,59],begin:[5,9,10,16,36],beginiter:[22,23],beginn:34,beginpass:[22,23],begintrain:23,behavior:38,being:[24,26,50],belong:[10,16,59],below:[3,10,16,20,24,26,34,36,38,39,44,50,51,54,56],benefit:[11,17],bengio:[10,16],bertolami:58,besid:[2,10,16,20,59],best:[8,10,16,28,29,41,54,56,58,59],best_model_path:57,besteffort:45,beta1:[12,18],beta2:[12,18],beta:52,better:[10,11,16,17,27,39,44,50,56],between:[10,12,16,18,24,27,35,44,50,54,55,58,59],bgr:52,bi_lstm:[11,17],bia:[10,11,12,16,17,18,34,36,52],bias:[10,16,36],bias_attr:[10,11,16,17,27,34],bias_param_attr:[11,17],biases_:36,biasparameter_:36,biassiz:36,bidi:45,bidirect:[11,17,34,57,59],bidirectional_lstm_net:58,big:38,bigger:24,biggest:58,bilinear:[10,16],bilinear_interpol:[10,16],bilinearfwdbwd:38,bin:[28,29,39,44,45,56],binari:[3,9,10,16,20,38,44,49,54,58],bird:51,bison:28,bit:54,bitext:59,bla:28,blank:[10,16,44],block:[10,16,27,36,38,41,52,58],block_expand:10,block_i:[10,16],block_x:[10,16],blog:58,bn_attr:17,bn_bias_attr:[11,17],bn_layer_attr:11,bn_param_attr:[11,17],bollen:58,book:20,bool:[3,7,9,10,11,12,15,16,17,19,20,36,41,43,54,56,58],boot:[10,16,34],boot_bia:[10,16],boot_bias_active_typ:[10,16],boot_lay:[10,16,34],boot_with_const_id:[10,16],bootstrap:28,bos_id:[10,16,34],both:[0,7,10,11,14,15,16,17,23,24,29,34,36,38,44,50,52,54],bottleneck:[38,52],bottom:58,bow:[54,58],box:38,branch:[10,16,23,35],breadth:[41,59],brelu:6,brendan:58,brew:28,briefli:38,broadcast:24,brows:29,browser:[29,44],bryan:58,bucket_nam:44,buf_siz:20,buffer:[3,20,26,41],buffered_read:26,bug:44,bui:58,build:[0,20,27,29,32,41,44,46,47,49,51,52,54,56,58,59],build_dict:20,built:[0,28,29,50,57],bunch:[38,54],bunk:58,button:[35,44],c11:25,c99e:44,cach:[54,56,57],cache_pass_in_mem:[3,54,56,57],cachetyp:[3,54,56,57],calc_batch_s:[3,57],calcul:[3,9,10,11,12,16,17,18,24,34,36,38,41,43,50,56],call:[3,10,11,16,17,23,27,29,34,36,38,41,44,51,52,54,58,59],callabl:[3,10,16,20],callback:36,caller:44,caltech:51,can:[2,3,5,7,8,9,10,11,15,16,17,20,23,24,26,27,28,29,31,34,35,36,38,39,40,41,43,44,45,49,50,51,52,54,56,57,58,59],can_over_batch_s:[3,57],candid:[10,16],cannot:36,caoi:59,capabl:[28,58],capac:44,caption:[27,59],captur:[27,39],card:39,care:[11,17,26,40,41,55],carefulli:[39,41,52],cat:[29,51,52,58],categor:57,categori:[10,16,20,24,54,58],categorig:20,categoryfil:45,caus:24,caution:[44,45],ccb2_pc30:59,cde:[10,16],cdn:20,ceil:[10,16],ceil_mod:[10,16],cell:[10,11,16,17,58],center:3,ceph:45,certain:[2,40,57],certif:[23,44],cffi:25,cfg:45,cgo:25,chain:[20,36],challeng:24,chanc:[23,36,54],chang:[10,16,20,26,27,29,34,35,36,38,41,44,54,58],channel:[10,16,38,39,52],channl:[39,52],char_bas:56,charact:[54,56],character:27,characterist:[43,51],check:[3,20,27,28,29,35,41,43,44,55],check_align:20,check_eq:36,check_fail_continu:3,check_l:36,check_sparse_distribution_batch:[40,41],check_sparse_distribution_in_pserv:[40,41],check_sparse_distribution_ratio:[40,41],check_sparse_distribution_unbalance_degre:[40,41],checkgrad:41,checkgrad_ep:41,checkout:35,children:55,china:29,chines:53,chmod:[28,44],choic:[29,55],choos:[41,54,56],chosen:[2,55,59],chunk:[9,50,57],chunk_schem:9,chunktyp:9,cifar:[50,51],cifar_vgg_model:51,claim:44,claimnam:44,clang:[25,28,29,35],class1:58,class2:58,class_dim:58,classfic:[52,58],classfiic:51,classic:[10,16,27],classif:[3,5,10,16,43,52,53,54,58,59],classifc:58,classifi:[9,50,51,52,54,58],classification_cost:[51,54],classification_error_evalu:[50,54,58,59],classification_threshold:9,claster:44,clean:[5,56],cleric:55,cli:44,click:[35,38,44],client:35,clip:[7,12,15,41,54,58],clock:[10,16],clone:[28,29],close:[3,26],closer:27,cloud:24,cls:54,cludform:44,cluster:[23,24,40,41,45,54,59],cluster_train:39,cm469:44,cmake3:28,cmake:[28,36,38],cmakelist:36,cmd:45,cna:[10,16],cname:44,cnn:[45,52,54],code:[0,3,5,20,23,26,27,28,29,30,34,36,37,38,39,44,45,50,54,55],coeff:[10,16],coeffici:[10,16],collabor:24,collect:[10,16,20,22,27,55],collectbia:36,colleg:55,color:[51,52],colour:20,column:[9,10,16,26,36,49,59],colunm:59,com:[10,11,16,17,20,28,29,31,35,44,45,52],combin:[10,11,16,17,20,22,50,56,58],come:58,comedi:55,comma:[41,49],command:[2,5,27,28,29,31,35,36,37,38,39,44,45,46,47,49,50,51,52,56,57,58],commandlin:[38,58],commenc:54,comment:[11,17,35,54,58],commnun:39,common:[34,36,40],common_util:[39,56],commonli:[34,38,43],commun:[0,24,36,39,44],compani:58,compar:[36,50,54],compat:3,compet:58,competit:50,compil:[28,29,35,36],complet:[0,5,10,11,16,17,20,22,24,36,44,45,54],complex:[2,3,11,17,26,34,38,54],complic:[10,16],compon:36,compos:[20,23,50,57],composenotalign:20,comput:[10,11,16,17,23,24,27,28,29,34,36,38,43,44,54,56,57,58],computation:34,conat:16,conat_lay:10,concat:[10,59],concat_lay:34,concaten:[11,17],concept:[3,23,29,34],concern:23,concurr:24,concurrentremoteparameterupdat:41,condit:[10,16,34,39,45,59],conduct:38,conf:[5,10,16,39,49,50,52,59],conf_paddle_gradient_num:44,conf_paddle_n:44,conf_paddle_port:44,conf_paddle_ports_num:44,conf_paddle_ports_num_spars:44,confid:58,config:[3,7,10,11,15,16,17,27,36,39,40,41,44,45,49,50,51,52,54,58,59],config_:41,config_arg:[40,41,43,52,54,57,58],config_bas:[16,17,22],config_fil:57,config_gener:[39,56],config_lay:36,config_pars:[5,36],configur:[1,2,3,5,8,10,16,27,35,36,38,41,49,51,52,58,59],conflict:35,confront:59,congest:41,conll05st:57,conll:[20,57],connect:[2,11,17,27,36,44,45,50,51,52,54,56,58],connectionist:[10,16,58],connor:58,consequ:[10,11,16,17],consid:[9,10,12,16,18,28,29,38,43,51],consider:[3,11,17],consist:[10,16,20,26,51,52,54,57,59],consol:[38,44],constant:36,construct:[3,5,23,34,56],construct_featur:56,constructor:36,consum:[24,58],contact:24,contain:[3,8,9,10,11,16,17,19,20,22,23,30,31,34,35,39,44,51,52,54,55,58,59],containerport:44,contemporan:58,content:[45,57,58],context:[10,11,16,17,20,34,49,54,56,57,58,59],context_attr:[11,17],context_len:[10,11,16,17,54,56],context_proj_layer_nam:11,context_proj_nam:17,context_proj_param_attr:[11,17],context_project:[11,17,56],context_start:[10,11,16,17,54],contibut:35,contin:44,continu:[3,24,31,41],contrast:[10,16,59],contribut:[0,30,37,58],contributor:0,control:[7,15,29,41,44,45,59],conv:[11,17],conv_act:[11,17],conv_attr:17,conv_batchnorm_drop_r:[11,17],conv_bias_attr:[11,17],conv_filter_s:[11,17],conv_layer_attr:11,conv_num_filt:[11,17],conv_op:[10,16],conv_pad:[11,17],conv_param_attr:[11,17],conv_shift:10,conv_strid:[11,17],conv_with_batchnorm:[11,17],conveni:[23,39],converg:[39,50,58],convert:[3,5,20,26,34,49,51,52,54,56],convlay:[10,16],convolut:[10,11,16,17,50,52,56],convoper:[10,16],convtran:[10,16],convtranslay:[10,16],cool:[3,35],copi:[22,23,29,44,50,56],copy_shared_paramet:50,copytonumpymat:50,core:[3,7,15,41,59],coreo:44,corespond:57,corpora:59,corpu:[20,57],correct:[3,9,10,16,36,44],correctli:[9,20,36,50],correl:[27,51,58],correspoind:23,correspond:[3,5,23,27,34,36,51,55,57,58,59],corss_entropi:23,cos:[10,16],cos_sim:56,cosin:[10,16,56],cost:[5,12,18,22,23,27,41,50,54,56,58,59],cost_id:[10,16],could:[3,5,9,10,16,20,22,23,26,29,38,39,44,54,56],count:[24,26,38,41,43,45,49,56,57,58,59],counter:24,coupl:27,coverag:28,coveral:28,coveralls_uploadpackag:28,cpickl:[52,56],cpp:[25,35,36,38,54,56,59],cpu:[2,3,7,10,15,16,28,31,38,41,45,50,57,58,59],cpuinfo:29,craftsman:55,crash:[24,38,39,41],crazi:39,creat:[5,7,10,15,16,20,22,23,24,27,28,29,36,39,41,49,50,51,59],create_bias_paramet:36,create_input_paramet:36,createargu:50,createfromconfigproto:[5,50],createstack:44,creation:44,creationd:44,creator:20,credit:50,cretor:20,crf:[10,57],crf_decod:10,crime:55,critic:58,crop:52,crop_siz:52,cross:[10,16,54,57],cross_entropi:[16,23,50],cross_entropy_with_selfnorm:16,csc:36,cslm:59,csr:36,csv:55,ctc:10,ctc_layer:9,ctest:29,ctrl:[39,56],ctx:57,ctx_0:57,ctx_0_slot:57,ctx_n1:57,ctx_n1_slot:57,ctx_n2:57,ctx_n2_slot:57,ctx_p1:57,ctx_p1_slot:57,ctx_p2:57,ctx_p2_slot:57,cub:51,cuda:[28,29,31,38,39,41],cuda_dir:[40,41],cudaconfigurecal:38,cudadevicegetattribut:38,cudaeventcr:38,cudaeventcreatewithflag:38,cudafre:38,cudagetdevic:38,cudagetdevicecount:38,cudagetdeviceproperti:38,cudagetlasterror:38,cudahostalloc:38,cudalaunch:38,cudamalloc:38,cudamemcpi:38,cudaprofilerstart:38,cudaprofilerstop:38,cudaruntimegetvers:38,cudasetdevic:38,cudasetupargu:38,cudastreamcr:38,cudastreamcreatewithflag:38,cudastreamsynchron:38,cudeviceget:38,cudevicegetattribut:38,cudevicegetcount:38,cudevicegetnam:38,cudevicetotalmem:38,cudnn:[10,16,19,28,31,41],cudnn_batch_norm:[10,16],cudnn_conv:[10,16],cudnn_conv_workspace_limit_in_mb:[40,41],cudnn_convt:[10,16],cudnn_dir:[40,41],cudrivergetvers:38,cuinit:38,cumul:[10,16],curl:[28,44],current:[3,10,12,16,24,27,29,34,35,36,39,41,44,54,58,59],current_word:34,currentcost:[9,54,56,58,59],currentev:[9,54,56,58,59],curv:[23,51,57],custom:[2,3,23,36,44,55,58],custom_batch_read:26,cutoff:20,cycl:24,cyclic:[10,16],cython:25,d3e0:44,dai:59,daili:58,dalla:3,dan:57,danger:3,darwin:44,dat:[20,39,56],data:[1,2,3,5,8,11,12,17,18,22,23,24,28,29,36,38,39,40,41,43,46,52,55],data_batch_gen:50,data_dir:[49,51,58,59],data_feed:20,data_fil:27,data_initialz:54,data_lay:[3,9,27,34,50,51,54,56,57],data_nam:20,data_provid:8,data_read:[20,26],data_reader_creator_random_imag:26,data_sourc:[8,50],data_typ:[16,20],databas:[20,58],datadim:[10,16],datalay:[10,16],dataprovid:[2,8,27,34,39,56,57],dataprovider_bow:54,dataprovider_emb:54,dataproviderconvert:5,datasci:[10,16],dataset:[1,3,26,27,41,49,51,52,54,57,58],datasourc:[4,56],date:57,db_lstm:57,dcgan:50,dcmake_install_prefix:28,dead:24,deal:[35,50],deb:[30,31],debian:[29,30],debug:3,decai:[12,18,51],decid:[23,26],declar:[10,11,16,56],decod:[10,11,16,17,34,57,59],decoder_boot:34,decoder_group_nam:34,decoder_input:34,decoder_mem:34,decoder_prev:[11,17],decoder_s:34,decoder_st:[11,17,34],deconv:[10,16],deconvolut:[10,16],decor:[3,20,36],decreas:27,decrypt:44,deep:[0,10,16,27,29,32,38,50,51,52,54,57],deeper:[27,29,52],deer:51,def:[3,10,16,20,23,26,27,34,36,50,52,54,56,57],defalut:[10,16,41,43],default_devic:43,default_valu:43,defferenct:3,defin:[2,3,8,9,10,11,16,17,20,23,26,27,34,36,39,41,49,50,51,56,57],define_py_data_sources2:[3,8,27,51,52,54,56],defini:59,definit:[3,20,24,27,29,49,54,58],degre:[10,16],del:56,delai:41,delar:54,delet:24,deletestack:44,delimit:[9,55,56],demand:24,demo:[10,16,20,34,39,45,46,49,50,51,52,53,54,55,56,57,58,59],demograph:55,demolish:45,demonstr:[27,34,50,56],denot:[43,54,55,57],dens:[3,10,16,20,36,44,54,56],dense_vector:[3,5,16,20,27,56],dense_vector_sequ:20,depend:[24,27,29,31,39,43,51,55],deploi:[39,43],deploy:[39,44],deriv:[14,23],descent:[10,12,16,24],describ:[23,27,36,44,45,50,54,57],describestack:44,describestackev:44,describestackresourc:44,descript:[5,28,34,42,44,51,56],deseri:22,design:[3,10,16,20,25,58],desir:[24,44,45,49],destructor:36,detail:[3,5,7,10,11,12,15,16,17,18,34,35,36,38,39,42,43,44,45,49,50,52,54,56,58,59],detect:9,determin:[3,10,16,20,36,50],dev:[28,29,51,56,59],devel:28,develop:[0,28,35,40,41,59],deverlop:41,deviat:[7,15],devic:[7,15,41,59],deviceid:43,devid:[10,16,41],dez:58,dfs:11,diagnos:39,diagram:52,dict:[3,8,20,22,54,56,58,59],dict_dim:58,dict_fil:[9,34,54,57],dict_nam:8,dict_siz:20,dictionai:54,dictionari:[3,8,9,10,16,20,22,23,34,43,52,54,56,57,58,59],dictsiz:59,did:3,differ:[3,8,9,10,16,24,27,29,34,35,36,39,41,44,45,49,51,52,54,58,59],difficult:27,dig:[29,38,44],digit:[3,10,16],dim:[20,36,49,52,54,58],dimens:[10,14,16,19,20,36,43,49,54,56,58],dimension:[3,27,34,36,50,54],dimenst:49,dimes:[10,16],din:56,dir:[39,52,54,56,57,58,59],dirctori:29,direct:[10,11,16,17,29,52,57],directli:[2,3,11,17,27,29,39,45,58],directori:[2,28,29,35,38,39,41,45,51,52,54,56,57,58,59],diretcoti:52,dis_conf:50,dis_train:50,dis_training_machin:50,disabl:3,discard:[20,24,41],discount:[10,16],discov:[24,57],discoveri:44,discrep:38,discrimin:50,discriminator_train:50,discuss:23,disk:45,dispatch:[24,39,41],disput:59,dist_train:23,distanc:9,distibut:49,distinguish:[39,50,59],distribut:[10,16,28,37,45,46,47,50,54,57],distribute_test:[40,41],distributedli:36,disucss:23,divid:[12,18,40,51,59],diy_beam_search_prob_so:[40,41],dmkl_root:28,dns:44,do_forward_backward:26,doc:[5,11,17,20,28,29,39],docker:[30,44,46,47],docker_build:23,docker_push:23,dockerfil:29,dockerhub:29,doctor:55,document:[3,5,11,17,28,35,43,51,54,56,57,58],documentari:[3,55],doe:[3,5,11,17,24,26,27,31,34,36,38,54,56,57],doesn:[7,10,15,16,20,23,26,35,38,45,59],dog:[51,52],doing:38,domain:44,don:[11,17,23,26,27,44,58],done:[10,11,16,17,24,34,38,44,50,58],dopenblas_root:28,dot:[41,52,59],dot_period:[41,43,50,51,56,58,59],dotmuloper:[10,16],dotmulproject:[10,16],doubl:[3,28,41],down:[38,54],download:[20,24,29,31,50,51,54,57,58],download_cifar:51,downsampl:51,doxygen:[28,35],dpkg:31,drama:55,drop:3,drop_rat:[7,15],dropout:[7,10,15,16,36,54],dropout_lay:10,dropout_r:[11,17],drwxr:45,dtoh:38,dtype:[5,27,52],dubai:59,due:[55,56],duplic:55,durat:38,dure:[2,3,10,16,24,27,35,36,40,41,44,54,56,57,59],durn:3,dwith_doc:28,dwith_profil:38,dwith_tim:38,dynam:[2,3,26,28,38,41],dynamic_cast:36,each:[2,3,5,9,10,16,19,20,22,24,26,27,29,34,35,36,39,41,43,44,49,51,52,54,55,56,57,58,59],each_feature_vector:14,each_meta:56,each_pixel_str:3,each_sequ:[10,16],each_time_step_output:14,each_timestep:[10,16],each_word:3,eaqual:[10,16],eas:[20,26,52],easi:[0,26,29,36,39,54],easier:[23,26,36],easili:[23,26,27],echo:[29,56,58],edit:[9,29,44],editor:[29,35],edu:[20,44,45,51],educ:55,eeoi3ezpr86c:44,effect:[3,41,44],effici:[0,2,3,34,36],efg:[10,16],efs:44,efs_dns_nam:44,efsvol:44,eight:57,either:[10,16,20,22,23,38,54,56],elb:44,elbapis:44,elec:54,electron:[45,54],elem_dim:[10,16],element:[3,5,9,10,11,16,17,20,22,26,54,58,59],elif:[23,56],elimin:57,els:[10,16,23,29,36,52,54,56],emac:[29,35],emb:[45,54],embed:[10,23,34,53,56,58],embedd:57,embedding_lay:[34,54,56],embedding_nam:34,embedding_s:34,emphas:38,empir:[10,16],emplace_back:36,emploi:[34,55],empti:[9,20,24,27],emul:59,enabl:[3,7,15,38,39,41,44],enable_grad_shar:[40,41],enable_parallel_vector:41,enc_proj:[11,17,34],enc_seq:[11,17],enc_vec:34,encod:[11,17,34,59],encoded_proj:[11,17,34],encoded_sequ:[11,17,34],encoded_vector:34,encoder_last:[10,16],encoder_proj:34,encoder_s:34,encrypt:44,encrypt_decrypt:44,end:[3,9,10,16,26,27,34,41,49,57,58,59],end_pass:23,enditer:[22,23],endpass:[22,23],endpoint:44,endtrain:23,engin:[0,38,55],english:[3,10,16,59],enjoi:29,enough:27,ensembl:[11,17],ensur:[3,24,36],enter:[29,55],entir:[10,11,16,17,58],entri:[20,36,44,55],entropi:[10,16,54,57],enumer:[10,14,54,56],enumerate_data_types_of_data_lay:20,env:[35,44],environ:[23,28,29,31,38,39,40,41,44,45,50,51,56],eol:35,eos:10,eos_id:[10,16,34],epel:28,epoch:55,epsilon:[12,18],equal:[10,11,12,16,17,24,41],equat:[10,11,12,16,17,18,29],equilibrium:50,equip:[28,34],equival:[10,16,23],error:[7,9,10,12,15,16,18,23,27,31,36,39,41,44,51,52,54,55,56,58,59],error_clipping_threshold:[7,15],errorr:9,especi:[3,11,17,57],essenc:23,essenti:[10,16,23,28,57,59],estat:27,estim:[10,16,23],eta:45,etc:[12,20,26,29,39,40,43,44,58,59],etcd:24,eth0:[39,44],ethternet:39,eval:[9,54,56,58,59],eval_bleu:59,evalu:[2,4,10,16,22,38,39,54,58,59],evaluate_pass:58,evaluator_bas:9,evalut:[27,59],even:[23,26,38,41,58],evenli:44,event:45,event_handl:[22,23],everi:[2,3,9,10,11,16,17,20,23,24,34,35,36,41,54,57,58,59],everyth:[27,35],exactli:[3,9,10,11,16,17,29,44,57],exampl:[2,3,8,9,10,11,12,16,17,18,20,22,26,27,28,29,34,36,38,39,40,41,43,44,45,51,52,53,54,58,59],exceed:[10,16],except:[3,20,43,49,56,58],excluded_chunk_typ:9,exconv:[10,16],exconvt:[10,16],exdb:20,exec:[29,41],execut:[24,36,38,44,55,57,58],exist:[23,24,26,36,41,44,55,58],exit:[41,45],exp:6,expand:[10,36,57,58,59],expand_a:[10,16],expand_level:[10,16],expandconvlay:[10,16],expandlevel:[10,16],expect:[10,16,38,58],expens:59,experi:43,expir:24,explain:[3,9,24,39,50,58],explan:[10,16,54,59],explanatori:[27,29],explicit:36,explicitli:[3,23],exploit:51,explor:[10,16],exponenti:14,expos:[29,44],express:[23,44,58],extend:[0,56],extens:[12,55,56,59],extern:[3,25],extra:[10,11,15,16,17,27],extra_input:16,extra_lay:22,extraattr:[7,15,43],extraattribut:[16,17],extraattributenon:16,extract:[10,16,44,51,57,58],extract_fea_c:52,extract_fea_pi:52,extract_para:49,extralayerattribut:[7,10,11,15,16],extralayeroutput:11,extrapaddl:17,extrem:[10,16,38],extremli:2,f120da72:45,f7e3:44,fa0wx:45,fabric:39,facotr:[10,16],fact:52,factor:[7,10,12,15,16,18],factori:25,fail:[3,41,43,45,51],failur:24,fake:50,fake_imag:26,fals:[3,7,9,10,11,12,15,16,17,18,20,26,27,34,36,41,43,45,49,54,56,57,58,59],false_label:26,false_read:26,famili:59,familiar:[3,27],fanscin:3,fantasi:55,fantast:54,far:0,farmer:55,fascinatingli:2,fast:[10,16,35,38],faster:[10,11,16,17,34,38,58],favorit:35,favourit:29,fbd1f2bb71f4:45,fc1:[36,43],fc2:43,fc3:43,fc4:43,fc8a365:44,fc8a:44,fc_act:[11,17],fc_attr:[11,17],fc_bias_attr:[11,17],fc_layer:[16,27,36,43,54,56],fc_layer_nam:11,fc_mat:22,fc_name:17,fc_param_attr:[11,17],fclayer:36,fdata:57,fea:52,fea_output:52,feat:58,featur:[3,10,14,16,20,35,41,51,54,58,59],feature_map:56,feed:[11,17,20,22,23,27,58],feedback:0,feeder:20,feedforward:51,femal:55,fernan:58,festiv:3,fetch:[20,34,36],few:[3,24,26,29],fewer:[10,16],fg0:[10,16],field:[10,16,22,38,44],figur:[23,34,36,38,49,50,51,52,57,58,59],file1:59,file2:59,file:[2,3,5,9,10,16,20,22,23,24,26,27,28,29,34,35,36,39,41,49,51,52,57,58,59],file_list:3,file_nam:[3,27,52,54,57],filenam:[3,56],filer:[10,16],filesystem:[29,44],fill:[10,16,24,44,54],film:55,filter:[10,16,52],filter_s:[10,11,16,17],filter_size_i:[10,16],finali:39,find:[10,12,16,18,24,29,38,51,58,59],fine:[7,15,56],fingerprint:44,finish:[3,24,29,39,44,45,51],finit:36,first:[3,10,16,20,23,24,27,29,31,34,35,36,38,41,43,44,49,50,51,52,54,56,57,58,59],first_seq:34,firstn:20,firstseen:45,fit:[2,20,35],five:[38,54],fix:[3,7,15,25,59],flag:[20,41,50,51,57],flexiabl:26,flexibl:[0,2,10,11,16,17,23,34],flight:59,float32:[5,20,26,27,50,52],floor:[10,16],flow:35,fly:[27,54],fnt03:44,focu:[3,38],folder:[28,44,51,58,59],follow:[2,3,9,10,11,12,16,17,18,20,23,24,26,28,29,31,34,35,36,38,39,43,44,45,46,47,49,50,51,52,54,55,56,57,58,59],fool:50,forbid:23,force_load:25,forecast:58,forget:[12,18,23,58],form:[2,3,11,12,17,18,38,57],format:[2,3,9,27,35,36,41,44,49,51,55,56,58],former:[23,59],formula:[10,11,16,17],formular:[10,16],forward:[11,14,17,34,35,36,43,50,57,58],forwardactiv:36,forwardtest:5,found:[3,5,10,16,28,34,50,51,54,58],four:[3,31,49,52,54,56,57,58],frame:9,framework:[23,36,52,54,58],free:[20,59],french:59,frequenc:[20,38,49,54,58],frequent:[26,39,59],frog:51,from:[0,3,5,10,11,16,17,20,22,24,26,27,29,34,35,36,38,39,41,43,44,45,49,50,51,52,54,55,56,57,58,59],from_timestep:[10,16],fromfil:[26,27,52],fulfil:38,full:[10,16,24,29,34,36],full_matrix_project:[11,17,34],fulli:[27,35,36,38,50,51,52,54,56,58],fullmatrixproject:[10,16],fully_matrix_project:[11,17],fullyconnect:49,fullyconnectedlay:36,func:20,fundament:27,further:[10,16],fusion:56,gain:[10,16],game:50,gamma:52,gan:23,gan_train:50,gap:41,gate:[10,11,16,17,58],gate_act:[10,11,16,17],gate_recurr:[10,16],gather:[10,16,36,56],gauss:[7,15],gaussian:50,gcc:[25,28,29],gdebi:31,gen:[10,16,59],gen_conf:[50,59],gen_data:59,gen_result:59,gen_train:50,gen_training_machin:50,gen_trans_fil:34,gender:[20,55,56],gener:[2,3,5,9,10,11,16,17,20,22,23,24,26,27,28,29,38,39,41,43,44,49,52,53,54,56,58],generatedinput:34,generator_conf:50,generator_machin:50,generator_train:50,genert:3,genr:[55,56],gereat:9,get:[3,10,11,16,17,20,22,27,28,31,34,36,38,39,44,48,51,52,54,56,57,58],get_batch_s:57,get_best_pass:58,get_config_arg:[43,54,56,58],get_data:[45,54,57],get_dict:20,get_embed:20,get_imdb:58,get_input_lay:36,get_mnist_data:50,get_model:52,get_movie_title_dict:20,get_nois:50,get_output_attr:17,get_output_layer_attr:11,get_shap:22,get_training_loss:50,get_word_dict:20,getbatchs:36,getenv:23,getinput:36,getinputgrad:36,getinputvalu:36,getoutputgrad:36,getoutputvalu:36,getparameterptr:36,getsiz:36,getslotvalu:50,gettempl:44,gettranspos:36,getw:36,getweight:36,getwgrad:36,gfortran:28,gildea:57,gist:[11,17],git:[28,29,35],github:[10,11,16,17,28,29,31,52],give:[3,24,27,29,36,38,44,54],given:[3,20,22,26,36,41,50,54,57,58,59],global:[3,7,12,15,23,24,38,41,44,56,58],global_learning_r:[7,15],globalstat:38,globalstatinfo:38,globe:3,goal:[38,57],godoc:25,goe:[10,11,16,17,24,27],going:[54,58],good:[10,16,26,38,58,59],goodfellow13:[10,16],googl:23,googleapi:44,gpg2:44,gpg:44,gpu:[2,3,7,10,12,15,16,19,28,31,37,39,50,51,52,56,57,58,59],gpu_id:[41,43,50],gpugpu_id:40,grab:[24,58],grad:[41,55],grad_share_block_num:[40,41],gradient:[7,9,10,12,15,16,18,22,24,41,54,58],gradient_clipping_threshold:[7,12,15,54,58],gradient_machin:22,gradientmachin:[5,22,50,56,59],gradual:[27,38],grai:51,gram:[49,58],grant:44,graph:[10,16,22,24,49],graphviz:52,grave:58,grayscal:3,greater:[10,16],grep:[29,58],groudtruth:34,ground:[9,10,16,54,59],group:[11,17,58],group_id:56,group_input:34,grouplen:[20,55],gru:[10,16,34,54,59],gru_attr:17,gru_bias_attr:[11,17],gru_decod:34,gru_decoder_with_attent:34,gru_encoder_decod:[49,59],gru_layer_attr:11,gru_memori:[11,17],gru_siz:54,gru_step:[17,34],gru_step_lay:[11,34],grumemori:[11,17,34],gserver:[10,16,36],gsizex:38,guarante:36,guess:[27,58],gui:38,guid:[30,34,35,36,38,44,45,49,51,58,59],guidenc:27,gur_group:[11,17],gzip:45,hack:[30,39],hadoop:23,half:44,hand:[55,56,58],handl:[23,26,39,56,58],handler:22,handwrit:[3,58],hard:[44,54],hardwar:[29,38],has:[3,5,10,11,12,16,17,18,20,23,24,29,34,36,38,44,45,49,51,54,55,56,57,58,59],has_kei:22,have:[2,3,5,9,10,11,16,17,20,23,24,26,27,28,29,34,35,36,38,39,41,43,44,49,51,54,55,56,58,59],hdf:2,head:[35,49,58],header:[27,36,49,52,56],health:55,heavi:39,height:[10,16,20,25,26,36,51],held:24,hello:23,help:[3,5,35,39],helper:[8,10,11,16,17,36],here:[3,5,7,10,11,15,16,17,20,23,26,27,28,34,39,40,43,44,45,49,51,52,53,54,55,56,57,58,59],heurist:[10,16,41,59],hidden:[10,11,16,17,34,44,54,56,58],hidden_s:[11,17,56],hierarch:[10,16,34],high:[7,15,36,50],higher:2,highest:[20,59],highli:[2,3,20,34,43,56,58],him:23,hint:27,histor:58,hl_get_sync_flag:36,hold:[23,24,44],home:[39,44,45],homemak:55,hook:[3,56,57],hope:0,horizont:[10,16,52],horror:55,hors:51,horst:58,host:[28,29,39,44,45],hostnam:[39,44],hostpath:45,hostport:44,hot:56,hour:59,hous:[3,20,27,49],how:[2,3,7,10,15,16,23,24,27,34,39,41,44,45,48,51,52,54,56],howev:[3,11,17,26,27,34,35,40,41,44,58,59],hpp:25,html:[20,29,51],htod:38,http:[10,11,16,17,20,28,29,31,35,44,45,50,51,52,59],huber:[10,16],huge:[10,16,35],huina:58,human:59,hyper:[10,16,36],hyperplan:20,i0601:56,i0706:59,i0719:59,i1117:38,iamfullaccess:44,iamusersshkei:44,ib0:39,ics:20,icwsm:58,id_input:[9,34],idea:[10,16,26],ident:[27,29,44,55],identifi:[34,36],identityoffsetproject:[10,16],identityproject:[10,16],ids:[9,10,16,36,54,56],idx:36,ieee:58,ignor:[3,9,10,16,41,49],ijcnlp:58,illustr:[3,24,34,36,38,54],ilsvrc:52,imag:[3,19,20,23,26,27,30,43,44,46,47,50,52,53,59],image_a:26,image_b:26,image_classif:51,image_fil:26,image_lay:26,image_list_provid:52,image_nam:23,image_path:26,image_provid:51,image_reader_cr:26,image_s:52,imagenet:53,imagepullpolici:44,imageri:[10,16],images_reader_cr:26,imdb:55,imdber:58,img:[3,10,16,51],img_conv:17,img_conv_lay:11,img_featur:3,img_norm_typ:10,img_pool:17,img_pool_lay:11,img_siz:51,imgsiz:38,imgsizei:38,imgsizex:38,immedi:44,immutable_paramet:23,implement:[3,10,11,12,16,17,18,20,34,54,57],importerror:56,improv:[0,38,44,58,59],inbound:44,includ:[2,3,10,11,16,17,20,23,25,28,29,34,36,38,41,44,45,49,54,55,57,59],inconsist:55,incorrect:[10,16],increas:[24,41,59],increment:41,incupd:36,inde:[20,26,29],independ:[10,16,54],index:[3,9,10,16,19,20,22,24,34,39,44,56],indexslot:[10,16,57],indic:[3,9,10,16,27,39,44,57],individu:[27,44],industri:24,infer:[1,23,24,28],infiniband:39,info:[9,10,16,20,36,39],infom:35,inform:[5,9,20,36,38,41,44,55,56,57,58,59],infrastructur:[44,50],ingor:41,ininst:23,init:[7,15,36,43,44,50,54,56,57],init_hook:[54,56,57],init_hook_wrapp:8,init_model_path:[40,41,43,49,54,57],initi:[3,5,7,10,15,16,20,34,36,41,49,50,54,57],initial_max:[7,15],initial_mean:[7,10,15,16],initial_min:[7,15],initial_std:[7,10,15,16],initpaddl:[5,50],inlcud:[11,17],inlin:44,inner:36,inner_param_attr:[11,17],input1:[10,11,16,17],input2:[10,16],input:[3,5,9,10,11,14,16,17,19,20,22,26,27,34,36,43,49,50,51,52,54,56,57,58,59],input_data:36,input_data_target:36,input_featur:14,input_fil:[27,57],input_hassub_sequence_data:36,input_id:[10,16],input_imag:[11,17,51],input_index:36,input_label:36,input_lay:[10,36],input_nam:23,input_sequence_data:36,input_sequence_label:36,input_sparse_float_value_data:36,input_sparse_non_value_data:36,input_t:36,input_typ:[27,34,54,56],inputdef:36,inputlayers_:36,inputtyp:[3,20],insid:[9,10,16,24,26,29,44],inspir:49,instal:[29,32,35,39,45,51,52,56,57,58],instanc:[10,12,16,24,34,36,38,41,57],instance_ip:44,instanti:24,instead:[10,16,19,26,29,35,39,54,59],instruct:[29,31,38,54],int32:41,integ:[3,9,10,16,20,25,34,36,54,58],integer_valu:[3,20,54],integer_value_sequ:[3,20,34,54,57],integr:[28,57],intend:0,inter:[10,16,39],interact:[29,44],intercept:[10,16],interest:[38,58],interfac:[1,5,7,10,11,15,16,17,39,44,51,56,58],interg:54,intergr:[10,16],intermedi:57,intern:[10,11,16,17,20,22,44],internet:[24,58],interpol:10,interpret:[3,9,28,38],interv:58,intrins:28,introduc:[3,24,45,56,58],introduct:[4,50],invalid:26,invari:51,invok:[3,10,16,22,38,44,56],involv:50,iob:9,ioe:9,ips:44,ipt:[10,16,34],ipython:23,is_async:12,is_discriminator_train:50,is_gener:[10,16,49,50,59],is_generator_train:50,is_kei:56,is_layer_typ:10,is_predict:[54,56,58],is_seq:[10,16,34,56],is_sequ:56,is_stat:[7,15],is_test:[52,57,58],is_train:3,isn:38,isol:29,isspars:36,issu:[28,29,38],item:[10,16,20,22,26],iter:[10,11,12,16,17,18,20,22,23,24,26,51,57,58],its:[3,9,10,11,16,17,23,24,36,38,41,44,49,50,51,54,58,59],itself:[11,17,24],java:25,jeremi:38,jie:[57,58],jmlr:[10,16],job:[5,9,20,40,41,43,52,54,56,57,58,59],job_dispatch_packag:39,job_id:20,job_mod:49,job_nam:44,job_namespac:44,job_path:44,job_workspac:39,jobpath:44,jobport0:44,jobport1:44,jobport2:44,jobport3:44,johan:58,join:24,joint:[49,59],jointli:[11,17,59],journal:[57,58],journei:29,jpeg:51,jpg:52,json:[39,44,45,56],jth:[11,17],judg:59,jupyt:29,just:[3,9,10,11,14,16,17,20,27,35,39,43,44,49,51,56,57,58],jx4xr:44,jypyt:23,k8s_data:44,k8s_job:23,k8s_token:23,k8s_train:44,k8s_user:23,kaim:[10,16],kaimingh:52,kebilinearinterpbw:38,kebilinearinterpfw:38,keep:[3,10,16,24],kei:[3,20,22,24,38,39,56,58],kernel:[10,16,38,54],key1:41,key2:41,key_pair_nam:44,keyid:44,keymetadata:44,keypair:44,keyserv:44,keystat:44,keyusag:44,keyword:3,kill:[24,44],kind:[2,3,23,24,27,44,45,50,54,56],kingsburi:57,kms:44,know:[3,11,17,23,27,36,38,44,56],knowledg:58,known:[50,58,59],kriz:[20,51],ksimonyan:[11,17],kube_cluster_tl:23,kube_ctrl_start_job:23,kube_list_containers_in_job_and_return_current_containers_rank:23,kubeconfig:44,kubectl:45,kuberent:44,kubernet:[23,24,37,39,46,47],kubernetes_service_host:23,kwarg:[3,9,10,11,12,16,17,18,20,54,56,57],l1_rate:[7,15],l2_rate:[7,15],l2regular:[51,54,58],label:[3,5,9,10,12,16,18,20,22,26,27,34,45,50,51,52,53,54,56,58],label_dict:57,label_dim:[10,16,54],label_fil:[26,57],label_lay:[10,26],label_list:57,label_path:26,label_slot:57,labeledbow:58,labl:58,lag:41,lake:3,lambdacost:[10,16],lambdarank:[10,16],languag:[10,16,20,43,49,57,58,59],larg:[19,20,57,58,59],larger:[3,7,9,10,12,15,16,39],last:[9,10,11,16,17,27,34,39,41,54,58,59],last_time_step_output:[10,16],lastseen:45,late:58,latenc:[39,44],later:[28,35,44,54],latest:[10,16,24,29,35,45,58],latter:59,launch:[41,44,58],launcher:23,lawyer:55,layer1:[10,11,16,17],layer2:[10,16],layer3:[10,16],layer:[4,5,7,9,11,15,17,19,20,21,22,26,27,34,37,40,41,49,50,51,52,54,56,57,58],layer_0:36,layer_attr:[10,16,34,43],layer_num:[43,52],layer_s:[10,16],layer_typ:[10,16],layerbas:36,layerconfig:36,layergradutil:36,layermap:36,layeroutout:[10,16],layeroutput:[9,11,16,56],lbl:[9,51],ld_library_path:[28,31,39],lead:38,learn:[0,7,9,10,11,12,15,16,17,18,20,23,26,27,29,32,34,36,38,51,52,54,57,58,59],learnabl:[10,16],learning_method:[12,27,49,51,54,56,58,59],learning_r:[7,12,15,27,49,51,54,56,58,59],leas:24,least:[9,10,16,24,28,55],leav:[3,44],lecun:20,left:[10,16,27,52],leman:59,len:[3,10,16,34,36,54,56,57],length:[10,11,16,17,20,34,41,45,58,59],less:[10,16,23,39,59],less_than:23,let02:45,let:[5,10,16,23,27,44,56],level:[7,10,15,16,39,41,50,56,58,59],lib64:[28,39,41],libcudnn:28,libjpeg:51,libpaddl:25,libpython:28,librari:[10,16,28,29,39,41,56],licens:57,life:24,like:[3,9,10,16,20,24,26,27,28,34,38,39,40,43,44,49,52,54,56,58,59],limit:[10,16,20,38,41],line:[2,3,5,9,20,27,35,37,38,39,43,44,49,51,52,56,57,58,59],linear:[6,10,16],linear_comb:10,linearactiv:[10,16,27],linguist:57,link:[10,11,16,17,28,44,54,58],linux:[28,29,31,44,59],lipeng:49,lipton:58,list:[2,3,8,9,10,11,16,20,22,23,27,29,34,36,39,41,43,44,51,52,54,56,57,58,59],listen:41,literatur:58,littl:[2,3,41,54,58],lium:59,live:[24,29],liwicki:58,load:[2,3,5,10,16,23,24,27,41,44,52,56,57,58,59],load_featur:52,load_feature_c:52,load_feature_pi:52,load_missing_parameter_strategi:[40,41,43,49,57],load_uniform_data:50,loadparamet:5,loadsave_parameters_in_pserv:[40,41],local:[7,15,24,28,29,35,39,40,41,45,51,58],localhost:29,locat:[34,36,54,57],lock:24,log:[3,6,35,36,39,41,44,45,51,56,57,58,59],log_barrier_abstract:41,log_barrier_lowest_nod:[40,41],log_barrier_show_log:[40,41],log_clip:[40,41],log_error_clip:[40,41],log_period:[41,43,45,50,51,54,56,57,58,59],log_period_serv:[40,41],logarithm:14,logger:3,logic:[3,39],login:29,longer:59,look:[3,9,27,39,40,44,45,50,54],lookup:54,loop:26,loss:[10,16,36,50,54,58,59],lot:40,low:[10,16],lower:39,lowest:41,lst:56,lstm:[10,16,34,45,54],lstm_attr:17,lstm_bias_attr:[11,17],lstm_cell_attr:[11,17],lstm_group:[11,17],lstm_layer_attr:11,lstm_size:54,lstm_step:[11,17],lstmemori:[11,17,34],lstmemory_group:[10,16],ltr:[10,16],lucki:27,mac:[28,29],machan:[11,17],machin:[10,11,12,16,17,20,22,27,35,36,40,41,43,44,45,54,56,58,59],made:[3,24,27,34,55],mai:[3,8,9,10,16,26,29,35,38,44,55],main:[3,5,35,44,51,57,58],mainli:41,maintain:[10,16,44],major:[29,35,50,52,58,59],make:[3,10,23,24,26,28,29,35,36,38,39,44,51,54,56,58],male:55,malloc:36,manag:[24,35,39],manageri:55,mandarin:[10,16],mani:[0,10,11,16,17,27,29,41,54,55,56,58],mannal:39,manual:35,manufactur:59,mao:58,map:[3,10,16,20,22,23,41,51,52,56],map_read:20,mapreduc:23,marcu:58,mark:[3,34,57],mark_slot:57,market:[27,55,58],martha:57,mask:[7,10,15,16],master:[23,35,41,58],mat_param_attr:[11,17],match:38,math:[11,17,25,36,38],matirx:[10,16],matplotlib:51,matric:[5,34,36],matrix:[9,10,11,16,17,20,22,25,34,36,40,43,52,57],matrixptr:36,matter:3,max:[3,7,10,13,15,16,20,38,41,43,51,54,56],max_id:[16,22,54],max_job_id:20,max_length:[10,16,34],max_movie_id:20,max_sort_s:[10,16],max_user_id:20,maxid:[9,10,54],maxid_lay:[9,54],maxim:[10,16,59],maximum:[9,20,34,38,41,54,57,58],maxinum:19,maxout:10,maxpool:[10,16],mayb:[10,11,16,17,51],md5:20,mean:[3,7,9,10,11,12,15,16,17,18,19,20,22,26,27,34,38,39,41,43,44,49,50,51,52,54,56,57,58,59],mean_img_s:51,mean_meta:52,mean_meta_224:52,mean_valu:52,measur:[27,38],mechan:[10,11,16,17,34,44,58],media:58,meet:57,mem:[10,16],member:23,memcpi:38,memor:58,memori:[2,3,11,17,34,36,38,41,43,45,54,57,58,59],memory_nam:[10,16],memory_threshold_on_load_data:41,mere:[11,17],merg:[35,41,49,59],mergedict:[49,59],messag:[27,41,45,56,58,59],meta:[39,51,52,54],meta_config:[39,56],meta_fil:56,meta_gener:[39,56],meta_path:51,meta_to_head:56,metadata:[44,45],metaplotlib:23,method:[3,8,10,11,12,16,18,22,29,36,38,41,43,54,56,58,59],might:[10,16,29,36,44],mileag:38,million:[20,43,55],min:[7,15,38,43,44,56],min_pool_s:3,mind:39,mini:[3,10,16,20,22,24],mini_batch:26,minibatch:[10,16],minibatch_data:20,minim:[3,12,18,27,41],minimum:[10,16],minimun:41,minst:3,minut:[24,44,59],mirror:29,miss:[41,49,57],mit:44,mix:[11,17,34,57],mixed_attr:17,mixed_bias_attr:[11,17],mixed_lay:[11,16,34,57],mixed_layer_attr:11,mixedlayertyp:[10,16],mkdir:[28,29,44],mkl:28,mkl_path:28,mkl_root:28,ml_data:[39,56],mnist:[3,5,26],mnist_provid:3,mnist_random_image_batch_read:26,mnist_train:[3,26],mnist_train_batch_read:26,mod:57,modal:57,mode:[10,16,41,50,51,52,56,58,59],model:[1,2,5,8,10,11,12,16,17,20,24,35,36,37,41,44,56,57,58],model_averag:12,model_config:[5,50],model_list:[41,43,57,58],model_output:58,model_path:43,model_zoo:[49,52],modelaverag:12,modifi:[5,34,35,36,39,44],modul:[2,3,5,8,11,17,20,22,27,28,51,52,54,56,57],modulo:[10,16],momentum:[7,12,15,27,54],momentumoptim:[27,51],mon:45,monitor:[54,58],mono:[10,16],month:[54,59],mood:58,more:[2,3,5,9,10,11,16,17,20,23,24,26,27,29,34,36,38,39,43,45,51,54,57,58,59],morin:[10,16],mose:[58,59],moses_bleu:59,mosesdecod:58,most:[3,5,10,16,20,23,26,27,34,36,38,40,56,57,58,59],mostli:[51,55],mount:[29,44,45],mountpath:[44,45],move:[10,16,24,38,44,56,58],movement:[38,58],movi:[3,20,58],movie_categori:20,movie_featur:56,movie_head:56,movie_id:56,movie_info:20,movie_meta:56,movie_nam:56,movie_review:20,movieid:55,movieinfo:20,movielen:53,moving_average_fract:[10,16],mpi:39,mse:10,mse_cost:[27,56],much:[10,16,24,26,38],mul:36,mulit:39,multi:[10,16,36,40,41,52,59],multi_binary_label_cross_entropi:16,multi_crop:52,multinomi:[10,16],multipl:[9,10,11,16,17,20,23,29,34,36,41,43,44,50,54,56,58],multipli:[9,10,16,36,51],multithread:3,music:55,must:[3,9,10,11,14,16,17,26,28,29,34,35,36,39,41,43,44,59],my_cluster_nam:44,my_cool_stuff_branch:35,my_external_dns_nam:44,mypaddl:45,mysteri:55,name:[3,7,8,9,10,11,15,16,17,19,20,22,23,24,27,29,34,36,38,39,41,43,45,46,47,49,50,51,52,54,56,58,59],namespac:[25,29,36,45],nano:35,nativ:[10,16],natur:[43,57,58],nchw:[10,16],ndarrai:22,ndarri:22,ndcg:[10,16],ndcg_num:[10,16],nearest:54,necessari:[3,10,16,28,36,39,54,58],necessarili:36,need:[3,10,11,16,17,20,23,27,28,29,31,34,35,36,39,40,41,43,44,45,50,51,52,54,56,57,58,59],neg:[3,9,10,16,54,57,58],neg_distribut:[10,16],negat:57,neighbor:54,nest:[3,20],net:[10,11,16,17],net_conf:58,net_diagram:52,network:[2,3,4,5,7,9,10,12,15,16,18,20,21,22,23,26,27,29,36,38,39,41,49,58,59],network_config:43,networkadministr:44,neural:[3,5,10,11,12,16,17,18,20,22,23,27,38,41,49,50,52,58,59],neuralnetwork:[10,16],neuron:[5,36,54,58],never:[20,26,44,45],newest:35,newtork:58,next:[10,16,20,24,34,36,38,41,44,45,57,58,59],nfs4:44,nfs:44,nfsver:44,nginx:29,nic:[39,40,41],nine:[20,57],nlp:[3,10,16],nltk:20,nmt:59,nnz:36,no_cach:3,no_sequ:[3,56],noah:58,noavx:[29,31],node:[10,16,36,39,41,44,45,58,59],node_0:44,node_1:44,node_2:44,nodefil:39,noir:55,nois:[10,16,50],noise_dim:50,non:[10,16,24,36,41,44],none:[2,3,5,7,8,9,10,11,12,15,16,17,18,19,20,22,23,27,34,52,54],nonlinear:36,norm:50,norm_by_tim:[10,16],normal:[3,5,10,11,16,17,20,31,34,36,39,41,45,49,50,52],normzal:52,north:51,notat:[10,16],note:[3,5,7,10,11,12,15,16,17,19,22,23,26,28,38,41,43,44,49,51,56,58],notebook:29,noth:[14,22,41],notic:[34,36],novel:58,now:[0,3,10,16,24,27,29,35,41,44,50,56,57],np_arrai:20,nproc:28,ntst1213:59,ntst14:59,nullptr:36,num:[10,16,39,41,54,57,58,59],num_channel:[10,11,16,17,51],num_chunk_typ:9,num_class:[10,11,16,17,51],num_filt:[10,11,16,17],num_gradient_serv:[40,41],num_group:[10,16],num_neg_sampl:[10,16],num_parameter_serv:23,num_pass:[22,27,40,41,43,45,54,56,57,58,59],num_repeat:[10,16],num_result:9,num_results_per_sampl:[10,16],number:[3,9,10,16,20,24,26,27,36,39,41,44,49,51,52,54,57,58,59],numchunktyp:9,numdevices_:43,numlogicaldevices_:43,numofallsampl:9,numofwrongpredict:9,numpi:[20,22,26,27,28,50,52],numsampl:38,numtagtyp:9,nvcc:29,nvidia:[28,29,38,41],obj:[3,8,27,51,52,54,56],object:[3,5,7,8,9,10,11,12,15,16,17,18,20,22,23,25,38,50,51,52,54,57],observ:[12,18,27,36,38,59],obtain:[54,57,58],occup:[55,56],occur:[20,22,35],oct:45,odd:[10,16],off:29,offer:[5,57],offici:[29,44,51],offlin:24,offset:[10,16,56],often:[9,39,54,59],ograd:36,old:[29,35,41],omit:54,on_init:3,on_travisexclud:28,onc:[3,10,16,24,29,35,36,44,54],one:[3,8,9,10,11,12,14,16,17,18,19,20,23,24,26,27,29,35,36,39,41,43,44,45,49,50,51,52,54,56,57,58,59],one_host_dens:56,one_hot_dens:56,onli:[2,3,5,9,10,11,16,17,19,20,22,23,27,28,34,35,36,38,40,41,43,44,45,49,52,54,55,58,59],onlin:[12,18,24,26],onto:44,open:[0,3,10,16,23,26,27,29,44,52,54,56,57],openbla:28,openblas_path:28,openblas_root:28,oper:[10,11,12,16,17,18,29,34,36,38,41,44,49,51,56],opinion:58,opt:[23,28],optim:[3,4,7,15,21,22,27,36,38,58],option:[3,9,10,16,23,27,35,36,39,43],order:[3,10,11,16,17,20,26,36,41,44,45,50,52,54,58,59],ordinari:58,oregon:44,org:[10,11,16,17,20,28,29,50],organ:[10,16,51,58,59],origin:[0,2,3,10,16,20,35,50,57,59],other:[3,9,10,11,12,16,17,20,28,29,31,34,35,43,44,45,49,50,51,52,54,55,56,57,58,59],otherchunktyp:9,otherwis:[2,8,10,16,20,23,24,26,34,39,43,56,59],our:[23,29,34,36,44,45,49,51,54,57,58,59],out:[10,16,22,23,27,34,38,41,44,45,51,58],out_dir:44,out_left:[10,16],out_mem:34,out_right:[10,16],out_size_i:[10,16],out_size_x:[10,16],outlin:42,outperform:57,output:[5,7,9,10,14,15,16,17,19,20,22,23,26,27,34,36,38,41,43,45,49,50,51,52,54,56,57,58,59],output_:[10,16,36],output_dir:52,output_fil:57,output_id:[10,16],output_lay:[22,52],output_max_index:19,output_mem:[10,16,34],outputh:[10,16],outputw:[10,16],outsid:[3,10,11,16,17],outter_kwarg:3,outv:36,over:[2,10,11,16,17,23,35,36,38,54,57,58],overcom:58,overhead:38,overlap:36,overrid:[24,36],owe:0,own:[29,35,39,44],pacakg:31,pack:29,packag:[3,20,29,30,44],pad:[10,34,54],pad_c:[10,16],pad_h:[10,16],pad_w:[10,16],paddepaddl:2,padding_attr:[10,16],padding_i:[10,16],padding_x:[10,16],paddl:[3,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,24,25,27,28,29,30,31,35,36,37,38,39,41,43,44,50,51,54,56,57,58,59],paddle_error:25,paddle_matrix:25,paddle_matrix_shap:25,paddle_n:39,paddle_output:45,paddle_port:39,paddle_ports_num:39,paddle_ports_num_for_spars:39,paddle_pserver2:39,paddle_root:49,paddle_source_root:49,paddle_train:39,paddledev:[44,45],paddlepaddl:[0,2,3,5,10,11,12,16,17,20,24,26,27,28,31,34,35,36,37,38,39,46,47,52,54,56,57,58],paddlepadl:3,paddlpaddl:0,paddpepaddl:3,page:[35,44,56],pai:29,pair:[9,57],palceholder_just_ignore_the_embed:49,palmer:57,paper:[10,16,49,50,52,57,58,59],paraconvert:49,paragraph:58,parallel:[38,41,43,44,45,59],parallel_nn:[7,15,40,41],param:[7,10,15,16,56],param_attr:[10,11,16,17,27,34],paramattr:[7,10,15,16,27,34],paramet:[2,3,4,5,8,9,10,11,12,16,17,18,19,20,21,26,27,36,37,43,50,51,54,56,57,58,59],parameter_attribut:[10,16],parameter_block_s:[40,41],parameter_block_size_for_spars:[40,41],parameter_learning_r:[7,15],parameter_nam:[22,23],parameter_serv:23,parameterattribut:[7,10,11,15,16,17],parametermap:36,parameters_:36,parameterset:23,parametris:[12,18],paramt:[44,49],paramutil:56,paraphras:59,paraphrase_data:49,paraphrase_model:49,paraspars:36,parent:[10,36],pars:[5,20,43,44,50,56,57],parse_config:[5,50],parser:56,part:[3,27,34,35,36,38,50,54,56,57,58,59],parti:[38,56],partial:[10,16,50],participl:49,particular:38,partit:[24,44],pass:[3,8,10,16,20,22,24,26,27,35,36,38,39,41,44,45,50,51,54,56,57,58,59],pass_id:22,pass_idx:26,pass_test:50,passtyp:36,password:39,past:[23,29,44],path:[2,3,9,20,22,24,26,27,28,34,39,41,43,44,45,49,51,52,54,57,58,59],pattern:[20,24,25,27,44,56,58],paul:57,paus:24,pave:59,pdf:[10,11,16,17],pem:[23,44],pend:24,penn:57,per:[10,16,20,26,41,51,54],perfom:[41,43],perform:[2,10,11,16,17,27,34,35,36,37,39,40,50,51,54,58,59],period:[2,24,41,54,56,57,58,59],perl:[58,59],permiss:44,peroid:[10,16],persist:44,persistentvolum:44,persistentvolumeclaim:44,person:23,perspect:38,perturb:36,pgp:44,phase:27,photo:51,pick:[3,44],pickl:56,picklabl:8,pictur:54,piec:[10,11,16,17,27],pillow:51,pip:[28,35,39,51,56],pipe:55,pipelin:57,pixel:[3,10,16,20],pixels_float:3,pixels_str:3,place:[2,3,24,36,38,39,52,59],placehold:[27,49],plai:[57,58],plain:[2,9,10,16,22],plan:[24,36],platform:[0,27,44],pleas:[3,5,7,10,11,12,15,16,17,18,23,24,26,28,29,30,34,35,36,44,49,51,54,56,57],plot:[23,51],plotcurv:51,png:[51,52],pnpairvalidationlay:41,pnpairvalidationpredict_fil:40,pod:[44,45],pod_nam:44,point:[27,38],polar:[20,58],polici:44,polit:58,poll:58,poo:51,pool3:36,pool:[3,4,11,17,21,51,54,56],pool_attr:[11,17],pool_bias_attr:[11,17],pool_layer_attr:11,pool_pad:[11,17],pool_siz:[3,10,11,16,17],pool_size_i:[10,16],pool_strid:[11,17],pool_typ:[10,11,16,17],pooling_lay:[11,54,56],pooling_typ:[10,16,54],poolingtyp:19,popular:[27,52],port:[29,39,40,41,44,45],port_num:40,ports_num:41,ports_num_for_spars:[40,41,43],pos:[56,58],pose:24,posit:[3,9,10,16,20,54,57,58,59],positive_label:9,possibl:[23,35,38,50],post1:28,potenti:38,power:[10,54,59],practic:[8,10,16,27,34,36],pre:[3,10,11,16,17,20,23,29,44,45,49,51,57,58,59],pre_dictandmodel:49,precis:[9,28],pred:[54,57],predefin:58,predetermin:[10,16,41,59],predic:[20,57],predicate_dict:57,predicate_dict_fil:57,predicate_slot:57,predict:[3,4,9,10,12,16,18,22,27,34,39,41,49,54,59],predict_fil:41,predict_output_dir:[40,41,54],predict_sampl:5,predicted_label_id:54,predictor:56,predin:51,prefer:58,prefetch:36,prefix:[24,44],pregrad:36,preinstal:28,premodel:49,prepar:[5,46,54],preprcess:58,preprocess:[20,34,39,45,58],prerequisit:28,present:[23,52,57,59],pretti:27,prev_batch_st:[40,41],prevent:[2,12,18,23,24],previou:[10,11,16,17,24,36,41,44,57,59],previous:[45,52],price:[20,27],primarili:58,principl:23,print:[7,15,22,23,27,34,41,49,54,56,57,58,59],printallstatu:38,printer:9,printstatu:38,prite:9,privileg:44,prob:[9,22,50],probabilist:[10,16,49],probability_of_label_0:54,probability_of_label_1:54,probabl:[9,10,16,22,34,35,52,54,57],problem:[5,10,12,16,18,23,54,57,58],proc:29,proc_from_raw_data:54,proce:[20,26,44],procedur:[49,57,59],proceed:[10,16,57],process:[2,3,5,7,8,10,11,12,15,16,17,23,27,29,34,39,41,43,44,45,49,51,52,54,56,57,58,59],process_pr:54,process_test:8,process_train:8,processdata:[51,52],processor:38,prod:29,produc:[11,17,20,24,26,29,52,54],product:[0,29,36,44,54,58],productgraph:45,profil:28,proflier:38,program:[2,20,23,26,29,38,39,41],programm:55,progress:[24,41],proivid:3,proj:[10,16],project:[10,11,16,17,28,34,36,56],promis:[10,11,16,17],prompt:35,prone:23,prop:57,propag:[12,18,41,43],properli:54,properti:[3,41],propos:59,proposit:57,protect:36,proto:19,protobuf:28,protocol:41,prove:54,proven:59,provid:[0,8,10,16,20,23,27,29,34,38,39,44,49,50,51,52,55,58],providermemory_threshold_on_load_data:40,provis:44,provod:3,prune:[10,16],ps_desir:24,pserver:[39,40,41,44],pserver_num_thread:[40,41],pserverstart_pserv:40,pseudo:23,psize:36,pull:[29,49,59],punctuat:58,purchas:54,purpos:[0,24,38],push_back:36,put:[24,29,36,39,45,54],pvc:44,pwd:29,py_paddl:[5,20,50],pydataprovid:[2,3,54],pydataprovider2:[4,5,27,34,54,56,58],pydataproviderwrapp:8,pyramid:[10,16],pyramid_height:[10,16],python:[2,3,4,8,22,23,25,27,28,35,39,49,50,51,57,58,59],pythonpath:51,pzo:58,qualifi:28,qualiti:54,queri:[10,16,44,59],question:[10,16,23,44,57],quick:[41,45,53,59],quick_start:[44,45,46,54],quick_start_data:45,quickli:27,quickstart:45,quit:38,quot:55,rac:[10,16],rais:20,ramnath:58,ran:38,rand:[38,41,43,50,57],random:[3,7,10,15,16,20,26,27,41,50,51,57],randomli:58,randomnumberse:40,rang:[3,10,16,20,26,41,43,51,55,57],rank:[10,16,23,44,52,54],rare:3,rate:[7,9,12,15,18,20,36,39,51,54,56,58,59],rather:[5,44,58],ratio:41,raw:[10,16,27,54,58],raw_meta:56,rdma:[28,41],rdma_tcp:[40,41],reach:[24,38,57],read:[2,3,20,22,23,24,26,27,34,39,44,52,54,56],read_from_realistic_imag:23,read_from_rng:23,read_mnist_imag:23,read_ranking_model_data:23,reader:[1,22,59],reader_creator_bool:26,reader_creator_random_imag:[20,26],reader_creator_random_image_and_label:[20,26],readi:[24,27,44,45,51],readm:[55,56,58],readonesamplefromfil:3,readwritemani:44,real:[3,26,27,50],realist:23,reason:[10,11,17,23,24,29,45],rebas:35,recal:9,receiv:[8,24],recent:59,reciev:41,recogn:51,recognit:[3,10,16,52,58],recommand:3,recommend:[2,11,17,23,29,34,36,39,41,56],recommonmark:28,recompil:38,record:[44,56,57],recordio:23,recov:[24,27,50],rectangular:[10,16],recurr:[57,58],recurrent_group:[11,17,34],recurrent_lay:11,recurrentgroup:9,recurrentlay:41,recv:44,reduc:[12,18,39,41,43],refer:[2,5,7,8,10,11,12,15,16,17,18,24,34,36,39,45,49,51,54,56,59],referenc:[10,16],regard:57,regardless:59,regex:56,region:[38,57],regist:[36,38],register_gpu_profil:38,register_lay:36,register_timer_info:38,registri:[29,45],regress:[9,53],regular:[7,12,15,36,44,51,54,58],rel:[2,11,17,39],relat:[3,8,24,29,31,45,56,58],relationship:[20,27,50],releas:[28,29,31,44,55,57],relev:[57,59],reli:28,reliabl:24,relu:[6,10,16,36],reluactiv:10,remain:54,rememb:[10,16],remot:[7,15,35,36,39,41,43,44],remoteparameterupdat:41,remov:[20,39,41,58],renam:59,reorgan:[10,16],repeat:10,replac:[29,58],repo:35,report:[38,39],repositori:35,repres:[3,5,10,12,16,20,34,36,44,51,54,55],represent:[54,58],reproduc:59,request:[24,44,45,49,59],requir:[2,9,10,16,23,24,36,39,44,45,50,51,54,56],requrest:35,res5_3_branch2c_bn:52,res5_3_branch2c_conv:52,res:57,research:[10,16,20,51,55,58],resembl:58,reserv:3,reserveoutput:36,reset:[10,16,24],reshap:26,reshape_s:[10,16],residu:52,resnet:53,resnet_101:52,resnet_152:52,resnet_50:52,resolv:[35,45],resourc:[29,44],respect:[3,27,34,36,41,51,52,57,59],respons:[10,16,44,45],rest:[3,10,16,27],restart:[24,44,45],restartpolici:[44,45],restrict:41,resu:26,result:[5,9,10,14,16,22,34,38,41,44,51,52,54,56,57,58],result_fil:[9,34],ret_val:56,retir:55,retran:44,retriev:[36,45],return_seq:[11,17],reus:[26,36],reveal:23,revers:[10,11,16,17,34,57,58],review:[20,35,45,54,58],reviews_electronics_5:45,revis:54,rewrit:59,rgb:[10,16],rgen:58,rho:[12,18],rich:27,right:[3,10,16,52],rmsprop:[12,54],rmspropoptim:56,rnn:[10,11,16,17,37,40,54,58],rnn_bias_attr:34,rnn_layer_attr:34,rnn_out:34,rnn_step:[10,16],rnn_use_batch:[40,41],rnnlm:20,robot:51,role:[20,23,34,44,53,58],roman:58,romanc:55,root:[12,18,19,39,44,45],root_dir:39,rot:[10,16],rotat:10,roughli:[3,50],routin:56,routledg:58,row:[5,9,10,16,20,36,52],row_id:[10,16],rsize:44,rtype:[10,16,56],rule:[36,44],run:[23,24,29,35,36,37,38,41,44,46,47,49,51,52,54,56,58,59],runinitfunct:38,runtim:[2,3,28,29,39],s_fusion:56,s_id:56,s_param:50,s_recurrent_group:34,sacrif:2,sai:[27,41,43],sake:36,sale:55,same:[3,5,8,9,10,11,16,17,23,34,39,43,44,49,54,56,57,58,59],samping_id:[10,16],sampl:[3,5,9,20,39,41,43,49,50,52,54,56,57,58,59],sample_dim:50,sample_id:9,sample_num:9,santiago:58,satisfi:[39,44,54],save:[3,10,16,20,24,27,41,43,44,45,51,52,54,56,57,58,59],save_dir:[27,41,43,45,50,51,54,56,57,58,59],save_only_on:[40,41],saving_period:[40,41],saving_period_by_batch:[40,41,43,54],saw:3,scalabl:0,scalar:[3,10,16],scale:[0,10,14,52,55,56],scalingproject:[10,16],scatter:[10,16],scenario:[27,40],scene:40,schdule:44,schedul:[44,50],scheduler_factor:[7,15],schema:49,scheme:[9,12,57],schmidhub:58,schwenk:59,sci:55,scienc:58,scientist:[0,55],score:[9,10,16,56,58,59],screen:56,scrip:54,script:[5,20,39,44,51,52,54,57,58,59],seaplane_s_000978:51,search:[10,16,24,28,34,41,57,59],seat:59,second:[3,10,16,20,23,26,27,29,35,39,49,52,54,55,56,58],secret:44,section:[3,34,36,39,44,54],sed:58,see:[3,5,10,11,16,17,23,27,29,35,38,44,49,50,52,54,56,58,59],seed:[38,41],segment:9,segmentor:49,sel_fc:[10,16],select:[10,16,35,44,55,59],selectiv:[10,16],selector:45,self:[27,36,55,58],selfnorm:[10,16],semant:[20,23,34,53,58],semat:23,sen_len:57,send:[24,41,44],sens:10,sent:[23,45],sent_id:34,sentenc:[3,10,16,20,34,54,57,58,59],sentiment:[3,27,53,54,57],sentiment_data:58,sentiment_net:58,sentimental_provid:3,separ:[3,9,41,49,54,55,56,57,59],seq:[10,16,56],seq_pool:[10,16],seq_text_print:9,seq_to_seq_data:[49,59],seq_typ:[3,20,56],seqtext_printer_evalu:34,seqtoseq:[10,16,34,49,59],seqtoseq_net:[10,16,34,49,59],sequel:3,sequenc:[3,9,10,11,14,16,17,19,20,36,49,54,56,57,58,59],sequence_conv_pool:54,sequence_layer_group:[10,16],sequence_nest_layer_group:[10,16],sequencesoftmax:6,sequencestartposit:[10,16],sequencetextprint:9,sequencetyp:3,sequenti:[8,10,16,34,54,57],seri:[11,17,58],serial:[3,22],serv:[29,38,44,50],server:[23,29,36,39,40],serverless:24,servic:[29,55],session:[29,38],set:[2,3,5,7,9,10,11,15,16,17,20,22,23,24,27,28,29,31,34,36,37,38,39,40,41,43,44,45,49,51,52,54,55,56,57,58,59],set_active_typ:36,set_default_parameter_nam:[7,15],set_drop_r:36,set_input:[10,16],set_siz:36,set_typ:36,setp:44,settup:36,setup:[3,29,36,54],sever:[3,10,16,39,43,44,53,54,56,57,58,59],sgd:[12,18,22,23,24,39,50,58,59],sgdasync_count:40,shallow:57,shape:[10,16,22,52],shard:[24,44],share:[10,16,28,29,38,41,45,57],shared_bia:[11,17],shared_bias:[10,16],shell:[44,52],shift:52,ship:51,shold:58,shop:58,shorten:[10,16],shorter:52,should:[3,5,9,10,12,16,20,22,23,26,27,31,34,35,39,44,51,54,56,57,58,59],should_be_fals:23,should_be_tru:23,should_shuffl:[3,57],shouldn:35,show:[5,12,18,20,24,27,35,41,44,45,49,52,54,56,57,58,59],show_check_sparse_distribution_log:[40,41],show_layer_stat:[40,41],show_parameter_stats_period:[40,41,43,45,54,57,58,59],shown:[3,9,10,16,23,34,36,38,44,50,51,52,54,56,58,59],shrink:36,shuf:56,shuffl:[3,20,56,58],sid:44,side:[10,16,22,52],sig:44,sigint:39,sigmoid:[6,10,16,17,36],sigmoidactiv:[10,11],sign:44,signal:39,signatur:44,signific:38,similar:[10,16,26,44,54,56],similarli:[10,16,57],simpl:[2,3,9,10,11,14,16,17,20,22,28,29,35,38,41,54,56,57,58],simple_attent:34,simple_gru:54,simple_lstm:[10,16,54],simple_rnn:[10,16,34],simplest:44,simpli:[2,10,16,23,28,29,34,35,38,49,52,56,58,59],simplifi:[23,36,45],simultan:44,sinc:[10,16,24,26,27,29,38,44,50,54,55,59],sincer:[35,58],singl:[3,9,11,12,17,20,24,29,36,39,45,52,54,57,59],site:44,six:[49,57,59],size:[3,9,10,11,12,16,17,18,20,24,26,27,34,36,39,41,50,51,52,54,55,56,57,58,59],size_a:[10,16],size_b:[10,16],size_t:36,sizeof:49,skill:59,skip:[26,27,39,44,52],slide:[10,12,16,18,20,24],slightli:51,slope:[10,16],slot:[56,57],slot_dim:56,slot_nam:56,slottyp:56,slow:[3,38],small:[3,20,36,39,41,51,59],small_messag:[40,41],small_vgg:51,smaller:[10,16,24],smith:58,snap:45,snapshot:44,snippet:[34,36,38,44,54],social:58,sock_recv_buf_s:[40,41],sock_send_buf_s:[40,41],socket:41,softmax:[6,10,11,16,17,23,34,36,49,54,57,58],softmax_param_attr:[11,17],softmax_selfnorm_alpha:[10,16],softmaxactiv:[34,54],softrelu:6,softwar:[29,38],solv:[23,57],solver:59,some:[3,7,10,12,15,16,20,22,23,27,28,35,36,38,40,41,43,44,50,54,55,56,57,58,59],some_python_class:25,somecppclass:25,somedata:22,somegotyp:25,someth:[3,10,16],sometim:[12,18,26,38,58],soon:24,sophist:[27,36,39],sort:[10,16,20,41,44,56,58,59],sourc:[0,8,10,16,20,26,27,29,34,35,44,45,49,54,56,59],source_dict_dim:34,source_language_word:34,space:[9,29,34,38],space_seperated_tokens_from_dictionary_according_to_seq:9,space_seperated_tokens_from_dictionary_according_to_sub_seq:9,spars:[3,7,10,12,15,16,18,20,36,39,41,44,54],sparse_binary_vector:[3,20,54],sparse_binary_vector_sequ:20,sparse_float_vector:3,sparse_non_value_slot:20,sparse_upd:[7,15],sparse_value_slot:20,sparse_vector:20,sparse_vector_sequ:20,sparseparam:36,sparseprefetchrowcpumatrix:36,spatial:[10,16,51],speak:[34,59],spec:[44,45],specfii:41,speci:51,special:[10,16,28,49,54,59],specif:[2,43,51,54,56],specifi:[2,3,9,10,16,20,23,27,28,34,36,41,44,50,51,52,54,55,56,58,59],speech:[10,16],speed:[11,17,29],spefici:52,sphinx:[25,28,29],sphinx_rtd_them:28,split:[3,10,16,39,43,44,49,52,54,57],split_count:44,spp:10,sql:2,squar:[6,10,12,16,18,19,27],squarerootn:13,squarerootnpool:[10,16],squash:59,srand:41,src:59,src_backward:34,src_dict:34,src_embed:34,src_forward:34,src_id:34,src_root:5,src_word_id:34,srl:[20,57],ssd:16,ssh:[29,39,44,45],sshd:29,ssl:28,sstabl:23,stabl:44,stack:[27,44,54,57],stacked_lstm_net:58,stacked_num:58,stackexchang:[10,16],stage:39,stake:59,stale:35,stamp:38,standard:[7,15,49,51,57,58,59],stanford:[20,45],stanh:6,star:55,start:[10,16,22,24,27,29,34,35,38,39,41,48,49,53,56,59],start_pass:[40,41],start_pserv:41,startup:[24,44],stat:[28,38,41,57,58,59],state:[10,11,16,17,24,27,34,41,45,50,57,59],state_act:[10,11,16,17],statement:[36,44],staticinput:[10,16,34],statist:[10,16,41,54,57,58,59],statset:38,statu:[9,35,38,44,45],status:45,std:[36,41],stderr:39,stdout:39,step:[5,10,11,12,16,17,19,24,29,34,36,38,39,44,45,54,56,57,58,59],still:52,stmt1482205552000:44,stmt1482205746000:44,stochast:[12,18,24],stock:58,stop:[10,16,29,39,41,45,56],storag:[44,45,51],store:[9,10,16,20,24,36,39,41,44,45,49,51,52,54,56,57,58,59],str:[22,43],straight:35,strategi:[3,19,24,41,57],street:[10,16,57],strength:50,strict:26,stride:[10,16],stride_i:[10,16],stride_x:[10,16],string:[2,3,8,9,10,16,36,41,44,58],strip:[54,56,57],structur:[20,39,44,49,51,54,56,57,58,59],sts:44,stub:[10,16],student:55,stuff:35,stun:3,style:[3,10,16,28,35],sub:[9,10,16,20,23,34,36,51,54,59],sub_sequ:3,subgradi:[12,18],submit:[35,40,41,44],subnet0:44,subnet:[23,44],subobjectpath:45,subsequenceinput:[10,16],subset:[36,59],substanti:52,substitut:59,succe:58,succeed:45,success:[44,45,52,57],successfulcr:45,successfuli:58,successfulli:[52,56,58],successor:[41,59],sucessfulli:59,sudo:[28,31,44,51],suffic:[26,27],suffici:41,suffix:59,suggest:[10,16,38],suitabl:[35,41,51],sum:[9,10,12,13,16,18,34,36],sum_:[10,16],sum_to_one_norm:10,summar:[54,58],sumpool:[10,16],support:[7,9,10,12,15,16,19,20,24,26,28,29,31,34,36,38,41,44,57],suppos:[27,36,54],sure:[35,36,44,51,58],survei:58,swap_channel:52,swig:[5,25,28],swig_paddl:[5,20,50],symbol:[10,16],sync:[24,35,41,50],syncflag:36,synchron:[12,18,24,39,41,44],syntact:57,syntax:[26,56],synthect:27,synthes:50,synthet:27,sys:52,system:[24,28,29,39,45,54,57,58,59],t2b:49,t_i:[10,16],tab:[29,54],tabl:[3,10,16,52,54,59],tableproject:[10,16],tag:[9,20,29,34],tagtyp:9,take:[3,5,9,10,11,16,17,23,34,36,38,44,45,50,57,59],taken:[3,57],tanh:[6,10,11,16,17,36],tanhactiv:[10,11,34],taobao:58,tar:[28,44],tarbal:44,target:[10,16,20,22,34,49,54,59],target_dict_dim:34,target_language_word:34,targetinlink:[10,16],task:[3,9,10,16,27,34,43,49,52,57,58,59],tbd:25,tconf:58,tcp:[41,44],teach:54,tear:38,technic:24,technician:55,techniqu:[34,36],tee:[45,51,56,57,58,59],tell:[24,29,38,56],tellig:58,templat:[45,57],tempor:[10,16,54,57],tensor:10,term:[10,11,16,17,57,58],termin:45,terminolog:27,tese:2,tesh:57,test100:20,test10:20,test:[2,3,8,9,10,16,20,22,23,26,28,29,31,35,38,39,40,49,51,52,54,55,59],test_all_data_in_one_period:[45,51,56,57,58],test_data:59,test_fcgrad:36,test_gpuprofil:38,test_layergrad:36,test_list:[3,8,27,51,54],test_part_000:58,test_pass:[40,41,43,59],test_period:[40,41,43],test_ratio:56,test_wait:[40,41],testa:23,testb:23,testbilinearfwdbwd:38,testconfig:36,tester:[56,59],testfcgrad:36,testfclay:36,testlayergrad:36,testmodel_list:40,testq:23,testresult:22,testsave_dir:40,testutil:36,text:[2,3,9,11,17,20,23,29,34,44,49,53,54,56,58],text_conv:54,text_conv_pool:56,text_fil:[20,58],tflop:38,tgz:[20,28],than:[3,5,7,9,10,11,12,15,16,17,24,28,29,34,36,39,44,52,57,58,59],thank:[0,49,59],thei:[3,23,24,27,29,34,36,38,39,40,44,52,58],them:[2,3,11,17,23,24,26,27,29,34,38,40,41,44,51,52,54,56,58,59],theori:38,therefor:28,therein:[10,16],therun:52,thi:[2,3,7,8,9,10,11,12,15,16,17,18,20,22,23,24,26,27,28,29,31,34,35,36,38,39,41,43,44,45,49,50,51,52,54,55,56,57,58,59],thing:[3,27,34,35,38,56,57],think:23,third:[10,16,24,38,52,58],those:[24,52,57],thought:38,thread:[36,38,41,43,56,57,58,59],thread_local_rand_use_global_se:[40,41],threadid:43,threadloc:38,three:[3,9,10,12,16,24,26,27,34,41,50,52,58,59],threshold:[7,9,12,15,24,41,58],thriller:55,through:[5,10,16,24,34,36,38,39,49,50,51,58,59],throughout:54,throughput:38,thu:[3,10,16,27,36,44,59],tier:45,tight:28,time:[3,10,11,16,17,19,20,23,24,26,27,34,38,41,43,45,54,55,57,58,59],timelin:[10,16,38],timeo:44,timeout:24,timer:28,timestamp:[10,16,55],timestep:[3,10,16],titil:56,titl:[20,35,55,56],tmall:58,todo:[9,11,17,20,24],toend:[10,16],togeth:[3,10,11,16,17,20,22,34],token:[9,10,16,23,34,49,58,59],too:[20,29,31],tool:[29,34,35,44,58],toolchain:28,toolkit:[28,31],top:[9,52,57],top_k:9,topolog:[20,23],topolopi:22,toronto:[20,51],total:[9,22,24,26,38,39,45,49,59],total_pass:26,touch:58,tourism:58,tourist:59,toward:27,tra:59,track:[10,24],tractabl:[10,16],tradesman:55,tradit:[10,16],trail:20,train100:20,train10:20,train:[1,2,3,5,7,8,9,10,12,15,16,18,20,34,36,37,38,40,46,47,52],train_conf:[49,59],train_config_dir:44,train_data:59,train_id:44,train_list:[3,8,27,51,52,54],train_part_000:58,trainabl:[10,16],traindot_period:40,trainer:[3,5,23,27,36,39,41,43,50,54,57,58,59],trainer_config:[2,3,27,39,44,45,54,56,58],trainer_config_help:[3,6,7,8,9,10,11,12,13,27,36,51,54,56],trainer_count:[40,41,43,44,45,56,57,58,59],trainer_id:[41,44],trainerintern:[54,56,59],training_machin:50,trainingtest_period:40,trainonedatabatch:50,tran:[10,36,41],trane:3,transact:[24,58],transfer:[2,3],transform:[10,16,34,36,50,51,54,57],transform_param_attr:[11,17],translat:[10,11,16,17,27,49,56,58,59],transpar:39,transport:41,transpos:[10,16,36,50],transposedfullmatrixproject:[10,16],travel:[3,11],travi:[28,35],treat:[10,16,34],tree:[10,16,35,41,59],trg:59,trg_dict:34,trg_dict_path:34,trg_embed:34,trg_id:34,trg_ids_next:34,triain:2,tricki:25,trivial:3,trn:54,truck:51,true_imag:26,true_label:26,true_read:26,truth:[9,10,16,54,59],tst:54,tune:[7,15,37,54,56,59],tuninglog_barrier_abstract:40,tupl:[3,8,10,11,16,20,22,26],ture:[10,16],turn:[10,16,26,50],tutori:[29,34,35,36,38,39,44,45,46,47,52,54],tweet:58,twelv:59,twitter:58,two:[2,3,10,11,16,17,23,26,27,29,34,38,39,43,44,49,50,51,52,54,56,57,58,59],txt:[3,36,39,44,54,56,58],type:[3,8,9,10,11,12,16,17,19,20,22,23,24,25,26,27,29,34,36,41,43,44,45,51,52,54,56,57],type_nam:[10,56],typedef:25,typic:[5,9,29,38,58],ubuntu:31,ubyt:26,uci:20,ufldl:[10,16],uid:45,uint64:25,uint64_t:25,unbalanc:41,unbound:34,unconstrain:58,under:[27,28,29,44,55,58],underli:27,understand:[29,38,49,51,58],understudi:59,undeterminist:38,unemploi:55,unexist:57,uniform:[7,10,15,16,20,26,41,50],uniqu:[23,24,35,41,44],unique_ptr:36,unit:[10,11,16,17,27,28,29,34,35,57],unittestcheckgrad_ep:40,univ:59,unix:39,unk:[49,59],unk_idx:[54,57],unknown:[10,16],unlabel:58,unlik:[57,58,59],unseg:[10,16],unsup:58,unsupbow:58,until:[24,39,44,57],unus:56,unzip:56,updat:[7,10,12,15,16,24,28,36,39,41,43,58],update_equ:22,updatecallback:36,updatestack:44,upload:24,upon:[0,57],upstream:35,uri:44,url:[20,31,58],urls_neg:58,urls_po:58,urls_unsup:58,usag:[2,3,9,10,11,16,17,20,22,27,38,49,50,56],use:[0,2,3,5,7,8,9,10,11,12,15,16,17,19,20,22,23,27,28,29,30,31,34,35,36,38,39,41,43,44,45,49,50,51,52,54,55,56,57,58,59],use_global_stat:[10,16],use_gpu:[40,41,43,45,50,51,52,54,56,57,58,59],use_jpeg:51,use_old_updat:[40,41],use_seq:[27,56],use_seq_or_not:56,used:[2,3,5,9,10,11,12,16,17,18,19,20,22,23,24,26,27,30,31,34,36,38,39,40,41,43,44,49,51,52,54,56,57,58,59],useful:[2,3,10,11,16,17,34,36,43,54,57,58],usegpu:[36,50],useless:39,user:[2,3,7,9,10,11,15,16,17,20,22,23,26,27,29,35,39,40,41,44,52,54,57],user_featur:56,user_head:56,user_id:56,user_info:20,user_meta:56,user_nam:56,userid:55,userinfo:20,usernam:35,uses:[3,24,34,35,36,41,44,51,52,54,56,59],using:[2,3,5,7,8,10,15,16,20,23,24,26,27,29,34,35,36,38,41,43,44,45,49,50,51,52,54,57,58],usr:[28,39,41,44],usrdict:49,usrmodel:49,usual:[10,16,20,22,27,28,38,41,43,44,58],utf:49,util:[5,28,34,36,38,51,56,58],v28:[10,16],valid:[26,44,52,58],valu:[3,5,7,9,10,12,15,16,18,19,20,22,24,27,34,36,41,43,44,50,51,52,57,58],value1:41,value2:41,value_rang:20,vanilla:34,vanish:58,vari:[38,44],variabl:[3,10,16,20,23,27,28,31,36,39,44,45,58],varianc:[10,16,52],variant:29,vast:35,vector:[3,10,11,16,17,20,23,34,36,49,54,56,58,59],vectorenable_parallel_vector:40,verb:[20,57],veri:[3,10,16,19,34,38,51,54,58],verifi:[35,36],versa:28,version:[10,11,16,17,28,29,31,36,38,39,40,41,44,45,49,51,55,57,58,59],versu:23,vertic:[10,16,52],vgg:[11,17,51],vgg_16_cifar:51,via:[24,26,28,38,39,44,54],vice:28,view:[10,16],vim:35,virtual:29,virtualenv:56,visibl:29,vision:51,visipedia:51,visual:[10,16,29,38],viterbi:57,voc_dim:54,vocab:58,volum:[29,45],volumemount:[44,45],volumn:44,voluntarili:55,vutbr:20,wai:[3,10,11,16,17,23,27,29,34,36,39,43,56,57,59],wait:[12,18,24,41],walk:[5,50],wall:57,want:[3,10,11,16,17,23,26,27,28,29,36,41,43,49,52,54,56,57,58],war:55,warn:[10,16],warp:[10,16,38],watch:24,wbia:[44,52],web:29,websit:[51,54,57,58],wei:[57,58],weight:[9,10,11,12,16,17,18,34,36,41,43,51,52],weight_act:[11,17],weightlist:36,weights_:36,weights_t:36,welcom:[56,58],well:[29,36,41,44,51,54],west:44,western:55,wether:[10,16],what:[7,10,11,12,15,16,17,18,27,39,54,56],wheel:28,when:[2,3,7,9,10,12,15,16,20,22,24,31,34,35,36,38,41,43,44,45,49,50,51,57,58,59],whenev:56,where:[3,10,11,12,16,17,18,23,24,27,34,36,38,39,41,43,49,52,57,59],whether:[9,10,11,16,17,26,36,41,50,51,56,58,59],which:[0,2,3,5,9,10,11,12,16,17,18,20,23,24,26,27,31,34,36,38,39,41,43,44,50,51,52,54,55,56,57,58,59],whichev:50,whl:28,who:[49,52,55],whole:[3,9,20,25,44,45,54,55,56,59],whole_cont:56,whose:[3,10,16,20,24,34,56,57],why:[11,17],wide:57,width:[9,10,16,20,25,26,36,51,59],wiki:[10,16],wikipedia:[10,16,20],wilder:3,window:[10,16,19,20,29,58],wise:[10,16],with_avx:29,with_avxcompil:28,with_coveragecompil:28,with_doccompil:28,with_doubl:36,with_doublecompil:28,with_dsocompil:28,with_gpu:29,with_gpucompil:28,with_profil:38,with_profilercompil:28,with_pythoncompil:28,with_rdmacompil:28,with_style_checkcompil:28,with_swig_pycompil:28,with_test:29,with_testingcompil:28,with_tim:38,with_timercompil:28,within:[10,16,27],without:[9,10,16,26,39,58],wmt14:59,wmt14_data:59,wmt14_model:59,wmt:59,wmt_shrinked_data:20,woboq:29,won:[38,52],wonder:3,word:[3,9,10,16,20,34,43,53,56,57,58,59],word_dict:[54,57],word_dim:54,word_id:3,word_idx:20,word_slot:57,word_vector:54,word_vector_dim:[34,49],words_freq_sort:20,work:[3,5,20,23,24,26,28,34,35,36,38,39,41,44,45,54,56],worker:44,workercount:44,workflow:[29,35,44],workspac:[29,41,56],worri:27,wors:50,would:[22,26,29,39,44,50,54,57],wrap:57,wrapper:[11,17,38],writ:56,write:[3,20,23,24,26,29,34,35,37,39,44,51,56,57,59],writelin:27,writer:[23,55],written:[56,58],wrong:[3,26],wsize:44,wsj:57,www:[10,16,20,51,59],x64:28,xarg:36,xgbe0:41,xgbe1:41,xiaojun:58,xrang:[26,27,36],xxbow:58,xxx:[23,52,59],xxxxxxxxx:44,xxxxxxxxxx:44,xxxxxxxxxxxxx:44,xxxxxxxxxxxxxxxxxxx:44,xzf:28,y_i:[10,16],y_predict:27,yaml:[44,56],yann:20,year:55,yeild:[22,51],yield:[3,20,23,26,27,34,54,56,57,58],you:[2,3,5,7,10,11,12,15,16,17,27,28,29,31,34,35,36,38,39,41,43,44,49,50,51,52,54,56,57,58,59],your:[3,10,16,23,28,29,36,38,39,43,44,54,58],your_access_key_id:44,your_secrete_access_kei:44,yum:28,yuyang18:[11,17,20],zachari:58,zeng:58,zero:[3,7,10,12,15,16,18,20,24,36,41,44,54],zhidao:49,zhou:[57,58],zip:[20,55],zone:44,zxvf:44},titles:["ABOUT","API","Introduction","PyDataProvider2","API","Python Prediction","Activations","Parameter Attributes","DataSources","Evaluators","Layers","Networks","Optimizers","Poolings","Activation","Parameter Attribute","Layers","Networks","Optimizer","Pooling","Data Reader Interface and DataSets","Model Configuration","Training and Inference","PaddlePaddle Design Doc","Design Doc: Distributed Training","Paddle\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0","Python Data Reader Design Doc","Simple Linear Regression","Installing from Sources","PaddlePaddle in Docker Containers","Install and Build","Debian Package installation guide","GET STARTED","RNN Models","RNN Configuration","Contribute Code","Write New Layers","HOW TO","Tune GPU Performance","Run Distributed Training","Argument Outline","Detail Description","Set Command-line Parameters","Use Case","Distributed PaddlePaddle Training on AWS with Kubernetes","Paddle On Kubernetes","<no title>","<no title>","PaddlePaddle Documentation","Chinese Word Embedding Model Tutorial","Generative Adversarial Networks (GAN)","Image Classification Tutorial","Model Zoo - ImageNet","TUTORIALS","Quick Start","MovieLens Dataset","Regression MovieLens Ratting","Semantic Role labeling Tutorial","Sentiment Analysis Tutorial","Text generation Tutorial"],titleterms:{"\u4e0d\u4f7f\u7528":25,"\u4e0d\u4f7f\u7528swig\u8fd9\u79cd\u4ee3\u7801\u751f\u6210\u5668":25,"\u4e0d\u5bfc\u51fapaddle\u5185\u90e8\u7684\u7ed3\u6784\u4f53":25,"\u4e0d\u5f15\u7528\u5176\u4ed6\u52a8\u6001\u5e93":25,"\u4ec5\u4ec5\u4f7f\u7528void":25,"\u4f7f\u7528\u52a8\u6001\u5e93\u6765\u5206\u53d1paddl":25,"\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165\u4efb\u4f55\u5176\u4ed6\u8bed\u8a00\u7684\u89e3\u91ca\u5668":25,"\u539f\u56e0":25,"\u539f\u56e0\u5217\u8868":25,"\u57fa\u672c\u8981\u6c42":25,"\u5bfc\u51fac":25,"\u6307\u9488\u4f5c\u4e3a\u7c7b\u578b\u7684\u53e5\u67c4":25,"\u7b26\u53f7":25,"\u7b80\u5355\u5b9e\u73b0":25,"\u7c7b":25,"\u800c\u662f\u624b\u5199\u591a\u8bed\u8a00\u7ed1\u5b9a":25,"\u80cc\u666f":25,"\u8fd9\u4e2a\u52a8\u6001\u5e93\u4f7f\u7528c99\u6807\u51c6\u7684\u5934\u6587\u4ef6\u5bfc\u51fa\u4e00\u4e9b\u51fd\u6570":25,"case":43,"class":36,"function":49,"new":36,"paddle\u52a8\u6001\u5e93\u4e2d":25,"paddle\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0":25,"return":26,AWS:44,Abs:14,DNS:44,EFS:44,For:45,KMS:44,Use:[43,45],Using:[29,35],With:29,about:0,absactiv:6,access:44,account:44,activ:[6,14],adadelta:18,adadeltaoptim:12,adagrad:18,adagradoptim:12,adam:18,adamax:18,adamaxoptim:12,adamoptim:12,add:44,address:44,addto:16,addto_lay:10,adversari:50,aggreg:[10,16],algorithm:[24,54],analysi:58,api:[1,4,29],appendix:54,applic:4,approach:38,architectur:[34,54],argument:[26,40,43,54],asset:44,associ:44,async:41,attent:34,attribut:[7,15],auc_evalu:9,avg:19,avgpool:13,aws:44,background:27,base:[9,10],baseactiv:6,basepool:19,basepoolingtyp:13,basesgdoptim:12,batch:26,batch_norm:16,batch_norm_lay:10,batch_siz:26,beam_search:[10,16],between:23,bidirect:58,bidirectional_lstm:[11,17],bilinear_interp:16,bilinear_interp_lay:10,bleu:59,block_expand:16,block_expand_lay:10,book:29,brelu:14,breluactiv:6,bucket:44,build:[28,30,45],built:38,cach:3,cento:28,check:[10,16,36,39],chines:49,choos:44,chunk_evalu:9,cifar:20,classif:[9,51],classification_error_evalu:9,classification_error_printer_evalu:9,clone:35,cloudform:44,cluster:[39,43,44],code:35,column_sum_evalu:9,command:[42,43,54,59],commit:[35,45],common:41,commun:41,compos:26,concat:16,concat_lay:10,concept:44,config:[4,43,56,57],configur:[21,34,37,39,44,54,56],conll05:20,connect:[10,16],contain:[29,45],content:[38,44],context_project:[10,16],contribut:35,conv:[10,16],conv_oper:[10,16],conv_project:[10,16],conv_shift:16,conv_shift_lay:10,convolut:[51,54],core:44,cos_sim:[10,16],cost:[10,16],cpu:[29,43],creat:[26,35,44,45],creator:26,credenti:44,credit:0,crf:16,crf_decod:16,crf_decoding_lay:10,crf_layer:10,cross_channel_norm:16,cross_entropi:10,cross_entropy_cost:16,cross_entropy_with_selfnorm:10,cross_entropy_with_selfnorm_cost:16,ctc:16,ctc_error_evalu:9,ctc_layer:10,cudnnavg:19,cudnnmax:19,custom:26,dat:55,data:[10,16,20,26,27,34,44,45,49,50,51,54,56,57,58,59],data_lay:10,datafeed:20,dataprovid:[3,4,41],dataset:[20,24,55,56,59],datasourc:8,datatyp:20,date:35,debian:31,decayedadagrad:18,decayedadagradoptim:12,decor:26,defin:[44,54,58,59],delet:44,delv:51,demo:44,depend:28,deriv:36,descript:[41,50,55,57],design:[23,24,26],destroi:44,detail:[41,51],develop:[29,37],devic:43,dictionari:[26,49],differ:43,directori:44,distribut:[23,24,39,41,44],doc:[23,24,26],docker:[29,45],document:[29,48],dotmul_oper:[10,16],dotmul_project:[10,16],down:44,download:[28,44,45,49,52,56,59],dropout_lay:[11,17],dynam:24,ec2:44,elast:44,embed:[16,49,54],embedding_lay:10,entri:26,eos:16,eos_lay:10,equat:36,evalu:[9,27,56],evalutaion:59,event:[22,23],exampl:[23,49,50],exercis:51,exp:14,expactiv:6,expand:16,expand_lay:10,extern:44,extract:[49,52,56,59],fault:24,fc_layer:10,featur:[52,55,56,57],field:56,file:[44,45,54,55,56],find:44,first_seq:[10,16],fork:35,format:[24,54],from:[23,28,30],full_matrix_project:[10,16],fulli:[10,16],gan:50,gate:34,gener:[34,50,59],get:[32,45],get_output:16,get_output_lay:10,github:35,gpu:[29,38,41,43],gradient:36,gradient_printer_evalu:9,group:[10,16,44],gru:[11,17,41],gru_group:[11,17],gru_step:16,gru_step_lay:10,gru_unit:[11,17],grumemori:[10,16],guid:31,hand:38,handler:[23,25],hook:35,how:[26,37,38],hsigmoid:[10,16],huber_cost:[10,16],iam:44,ident:14,identity_project:[10,16],identityactiv:6,imag:[10,11,16,17,29,45,51],imagenet:52,imdb:[20,58],img_cmrnorm:16,img_cmrnorm_lay:10,img_conv:16,img_conv_bn_pool:[11,17],img_conv_group:[11,17],img_conv_lay:10,img_pool:16,img_pool_lay:10,imikolov:20,implement:[26,36,50],infer:[22,54],info:52,ingredi:23,init_hook:3,initi:[43,44],input_typ:3,inspect:44,instal:[28,30,31,44,54],instanc:44,integr:44,interfac:[20,24,26,52],interpol:16,interpolation_lay:10,introduct:[2,49,52,58,59],isn:26,job:[24,39,44,45],join:[10,16],keep:35,kei:44,kill:39,kube:44,kubectl:44,kubernet:[44,45],label:57,lambda_cost:[10,16],last_seq:[10,16],lastest:35,launch:39,layer:[10,16,23,36,43],layeroutput:10,layertyp:10,line:[42,54],linear:[14,27],linear_comb:16,linear_comb_lay:10,linearactiv:6,list:26,local:[43,44],log:[14,54],logactiv:6,logist:54,lstm:[11,17,41,57,58],lstm_step:16,lstm_step_lay:10,lstmemori:[10,16],lstmemory_group:[11,17],lstmemory_unit:[11,17],map:26,master:24,math:[10,16],matrix:41,max:19,maxframe_printer_evalu:9,maxid:16,maxid_lay:10,maxid_printer_evalu:9,maxout:16,maxout_lay:10,maxpool:13,memori:[10,16],meta:56,mini:26,minibatch:20,misc:[11,17],mix:[10,16,43],mixed_lay:10,mnist:[20,50],model:[3,4,21,23,27,29,33,34,39,43,49,50,51,52,53,54,59],modifi:45,momentum:18,momentumoptim:12,movi:[55,56],movielen:[20,55,56],mse_cost:[10,16],multi_binary_label_cross_entropi:10,multi_binary_label_cross_entropy_cost:16,multipl:26,name:44,nce:16,nce_lay:10,need:[26,38],network:[11,17,34,43,50,51,52,54,56,57],neural:[34,51,54,56,57],neuralnetwork:27,nlp:[11,17,41],non:3,norm:[10,16],nvprof:38,nvvp:38,object:[24,56],observ:[49,52],onli:[26,29],optim:[12,18,24,37,54],option:[28,49],outlin:40,output:[11,39,44],overview:54,packag:31,pad:16,pad_lay:10,paddl:[26,45],paddlepaddl:[23,29,30,44,48,49,59],pair:44,parallel_nn:43,paramet:[7,15,22,23,24,41,42,44,49,52],paraphras:49,pass:43,perform:[38,41],pnpair_evalu:9,point:44,pool:[10,13,16,19],pooling_lay:10,power:16,power_lay:10,pre:35,precision_recall_evalu:9,predict:[5,51,52,56,57,58],prefetch:26,prepar:[27,34,39,44,49,50,51,56,58,59],preprocess:[49,51,54,56,59],prerequisit:39,pretrain:[49,59],print:9,privat:44,problem:27,process:24,profil:38,provid:[3,26,54,56,57],pull:35,push:35,pydataprovider2:3,python:[5,26,29,36,52,54,56],queue:24,quick:54,randomnumb:41,rank:9,rank_cost:[10,16],rat:56,rate:55,reader:[20,23,26],recoveri:24,recurr:[10,11,16,17,34,54],recurrent_group:[10,16],recurrent_lay:10,refer:[3,38,57,58],region:44,regress:[27,54,56],relu:14,reluactiv:6,render:44,repeat:16,repeat_lay:10,request:35,requir:[28,35],reshap:[10,16],resnet:52,result:[39,45,59],revis:[35,49],rmsprop:18,rmspropoptim:12,rnn:[33,34,41],role:57,rotat:16,rotate_lay:10,route53:44,run:[39,45,57],sampl:[10,16],sampling_id:16,sampling_id_lay:10,scale:[16,24],scaling_lay:10,scaling_project:[10,16],script:45,secur:44,selective_fc:16,selective_fc_lay:10,semant:57,sentiment:[20,58],seq_concat:16,seq_concat_lay:10,seq_reshap:16,seq_reshape_lay:10,seqtext_printer_evalu:9,sequenc:34,sequence_conv_pool:[11,17],sequencesoftmax:14,sequencesoftmaxactiv:6,sequenti:3,server:[24,41,44],servic:44,set:[12,42],setup:[28,44],sgd:41,share:23,shuffl:26,sigmoid:14,sigmoidactiv:6,simpl:[27,34],simple_attent:[11,17],simple_gru:[11,17],simple_img_conv_pool:[11,17],simple_lstm:[11,17],singl:26,slice:[10,16],slope_intercept:16,slope_intercept_lay:10,softmax:14,softmaxactiv:6,softrelu:14,softreluactiv:6,sourc:[28,30],span:28,spars:43,specifi:[43,49],split:56,spp:16,spp_layer:10,squar:14,squareactiv:6,squarerootn:19,squarerootnpool:13,stack:58,standard:54,stanh:14,stanhactiv:6,start:[23,32,44,45,54],startup:45,structur:50,suffici:26,sum:19,sum_cost:[10,16],sum_evalu:9,sum_to_one_norm:16,sum_to_one_norm_lay:10,summar:23,summari:54,sumpool:13,system:44,table_project:[10,16],take:26,tanh:14,tanhactiv:6,task:24,tear:44,templat:44,tensor:16,tensor_lay:10,test:[36,41,43,56,57,58],text:59,text_conv_pool:[11,17],timer:38,tip:38,toi:50,toler:24,tool:38,train:[22,23,24,26,27,29,39,41,43,44,45,49,50,51,54,56,57,58,59],trainer:[22,24,44,56],tran:16,trans_full_matrix_project:[10,16],trans_lay:10,transfer:54,tune:[38,41],tutori:[49,51,53,57,58,59],ubuntu:28,uci_h:20,unit:[36,41],updat:[23,35,44],usag:[26,29,37],use:26,user:[24,49,55,56,58,59],util:9,value_printer_evalu:9,vector:41,verifi:44,version:35,vgg_16_network:[11,17],visual:52,volum:44,vpc:44,warp_ctc:16,warp_ctc_lay:10,what:38,why:[26,38],wmt14:20,word:[49,54],work:29,workflow:59,workspac:39,wrapper:36,write:[36,54],yaml:45,your:35,zoo:[52,53]}})
\ No newline at end of file
+Search.setIndex({docnames:["api/index_en","api/v1/data_provider/dataprovider_en","api/v1/data_provider/pydataprovider2_en","api/v1/index_en","api/v1/predict/swig_py_paddle_en","api/v2/config/activation","api/v2/config/attr","api/v2/config/evaluators","api/v2/config/layer","api/v2/config/networks","api/v2/config/optimizer","api/v2/config/pooling","api/v2/data","api/v2/data/data_reader","api/v2/data/dataset","api/v2/data/image","api/v2/fluid","api/v2/fluid/data_feeder","api/v2/fluid/evaluator","api/v2/fluid/executor","api/v2/fluid/initializer","api/v2/fluid/layers","api/v2/fluid/nets","api/v2/fluid/optimizer","api/v2/fluid/param_attr","api/v2/fluid/profiler","api/v2/fluid/regularizer","api/v2/model_configs","api/v2/run_logic","design/api","design/auto_gradient_check","design/block","design/build_system/README","design/cluster_train/README","design/cluster_train/checkpointing","design/cluster_train/data_dispatch","design/cluster_train/large_model_dist_train","design/cluster_train/master_server","design/cluster_train/pserver_client","design/cluster_train/remote_parameter_updater","design/cluster_train/save_model","design/cluster_train/submit-job","design/evaluator","design/executor","design/file_manager/README","design/file_manager/pfs/pfsclient","design/float16","design/functions_operators_layers","design/gan_api","design/graph","design/graph_survey","design/if_else_op","design/infer_var_type","design/model_format","design/multi_language_interface/00.why_plain_c","design/multi_language_interface/01.inference_implementation","design/ops/rnn","design/ops/sequence_decoder","design/optimizer","design/parameter_average","design/parameters_in_cpp","design/program","design/prune","design/python_api","design/reader/README","design/refactor/distributed_architecture","design/refactor/parameter_server","design/refactor/session","design/refactorization","design/register_grad_op","design/regularization","design/releasing_process","design/scope","design/selected_rows","design/simple_op_design","design/tensor_array","design/var_desc","getstarted/build_and_install/build_from_source_en","getstarted/build_and_install/docker_install_en","getstarted/build_and_install/index_en","getstarted/build_and_install/pip_install_en","getstarted/index_en","howto/deep_model/rnn/index_en","howto/deep_model/rnn/rnn_config_en","howto/dev/build_en","howto/dev/contribute_to_paddle_en","howto/dev/new_layer_en","howto/dev/new_op_en","howto/dev/use_eigen_en","howto/dev/write_docs_en","howto/index_en","howto/optimization/cpu_profiling","howto/optimization/gpu_profiling_en","howto/usage/cluster/cluster_train_en","howto/usage/cmd_parameter/arguments_en","howto/usage/cmd_parameter/detail_introduction_en","howto/usage/cmd_parameter/index_en","howto/usage/cmd_parameter/use_case_en","howto/usage/k8s/k8s_aws_en","howto/usage/k8s/k8s_en","howto/usage/k8s/src/k8s_data/README","howto/usage/k8s/src/k8s_train/README","index_en","mobile/cross_compiling_for_android_en","mobile/cross_compiling_for_raspberry_en","mobile/index_en","survey/cluster_bootstrapping_tools","v1_api_tutorials/README","v1_api_tutorials/embedding_model/index_en","v1_api_tutorials/gan/index_en","v1_api_tutorials/imagenet_model/resnet_model_en","v1_api_tutorials/quick_start/index_en"],envversion:50,filenames:["api/index_en.rst","api/v1/data_provider/dataprovider_en.rst","api/v1/data_provider/pydataprovider2_en.rst","api/v1/index_en.rst","api/v1/predict/swig_py_paddle_en.rst","api/v2/config/activation.rst","api/v2/config/attr.rst","api/v2/config/evaluators.rst","api/v2/config/layer.rst","api/v2/config/networks.rst","api/v2/config/optimizer.rst","api/v2/config/pooling.rst","api/v2/data.rst","api/v2/data/data_reader.rst","api/v2/data/dataset.rst","api/v2/data/image.rst","api/v2/fluid.rst","api/v2/fluid/data_feeder.rst","api/v2/fluid/evaluator.rst","api/v2/fluid/executor.rst","api/v2/fluid/initializer.rst","api/v2/fluid/layers.rst","api/v2/fluid/nets.rst","api/v2/fluid/optimizer.rst","api/v2/fluid/param_attr.rst","api/v2/fluid/profiler.rst","api/v2/fluid/regularizer.rst","api/v2/model_configs.rst","api/v2/run_logic.rst","design/api.md","design/auto_gradient_check.md","design/block.md","design/build_system/README.md","design/cluster_train/README.md","design/cluster_train/checkpointing.md","design/cluster_train/data_dispatch.md","design/cluster_train/large_model_dist_train.md","design/cluster_train/master_server.md","design/cluster_train/pserver_client.md","design/cluster_train/remote_parameter_updater.md","design/cluster_train/save_model.md","design/cluster_train/submit-job.md","design/evaluator.md","design/executor.md","design/file_manager/README.md","design/file_manager/pfs/pfsclient.md","design/float16.md","design/functions_operators_layers.md","design/gan_api.md","design/graph.md","design/graph_survey.md","design/if_else_op.md","design/infer_var_type.md","design/model_format.md","design/multi_language_interface/00.why_plain_c.md","design/multi_language_interface/01.inference_implementation.md","design/ops/rnn.md","design/ops/sequence_decoder.md","design/optimizer.md","design/parameter_average.md","design/parameters_in_cpp.md","design/program.md","design/prune.md","design/python_api.md","design/reader/README.md","design/refactor/distributed_architecture.md","design/refactor/parameter_server.md","design/refactor/session.md","design/refactorization.md","design/register_grad_op.md","design/regularization.md","design/releasing_process.md","design/scope.md","design/selected_rows.md","design/simple_op_design.md","design/tensor_array.md","design/var_desc.md","getstarted/build_and_install/build_from_source_en.rst","getstarted/build_and_install/docker_install_en.rst","getstarted/build_and_install/index_en.rst","getstarted/build_and_install/pip_install_en.rst","getstarted/index_en.rst","howto/deep_model/rnn/index_en.rst","howto/deep_model/rnn/rnn_config_en.rst","howto/dev/build_en.md","howto/dev/contribute_to_paddle_en.md","howto/dev/new_layer_en.rst","howto/dev/new_op_en.md","howto/dev/use_eigen_en.md","howto/dev/write_docs_en.rst","howto/index_en.rst","howto/optimization/cpu_profiling.md","howto/optimization/gpu_profiling_en.rst","howto/usage/cluster/cluster_train_en.md","howto/usage/cmd_parameter/arguments_en.md","howto/usage/cmd_parameter/detail_introduction_en.md","howto/usage/cmd_parameter/index_en.rst","howto/usage/cmd_parameter/use_case_en.md","howto/usage/k8s/k8s_aws_en.md","howto/usage/k8s/k8s_en.md","howto/usage/k8s/src/k8s_data/README.md","howto/usage/k8s/src/k8s_train/README.md","index_en.rst","mobile/cross_compiling_for_android_en.md","mobile/cross_compiling_for_raspberry_en.md","mobile/index_en.rst","survey/cluster_bootstrapping_tools.md","v1_api_tutorials/README.md","v1_api_tutorials/embedding_model/index_en.md","v1_api_tutorials/gan/index_en.md","v1_api_tutorials/imagenet_model/resnet_model_en.md","v1_api_tutorials/quick_start/index_en.md"],objects:{"paddle.trainer.PyDataProvider2":{provider:[2,0,1,""]},"paddle.v2":{image:[15,2,0,"-"]},"paddle.v2.fluid":{regularizer:[26,2,0,"-"]},"paddle.v2.fluid.evaluator.Evaluator":{metrics:[18,1,1,""],states:[18,1,1,""]},"paddle.v2.fluid.regularizer":{L1DecayRegularizer:[26,3,1,""]},"paddle.v2.image":{batch_images_from_tar:[15,0,1,""],center_crop:[15,0,1,""],left_right_flip:[15,0,1,""],load_and_transform:[15,0,1,""],load_image:[15,0,1,""],load_image_bytes:[15,0,1,""],random_crop:[15,0,1,""],resize_short:[15,0,1,""],simple_transform:[15,0,1,""],to_chw:[15,0,1,""]}},objnames:{"0":["py","function","Python function"],"1":["py","attribute","Python attribute"],"2":["py","module","Python module"],"3":["py","class","Python class"]},objtypes:{"0":"py:function","1":"py:attribute","2":"py:module","3":"py:class"},terms:{"0000x":111,"00186201e":4,"00m":92,"01852v1":8,"03m":92,"0424m":92,"0473v3":9,"055ee37d":98,"0630u":92,"06u":92,"08083v2":8,"0810u":92,"08823112e":4,"0957m":92,"0_cudnn5":77,"0_cudnn5_avx_mkl":80,"0_cudnn7_avx_mkl":80,"0ab":8,"0rc":93,"0rc1":71,"0rc2":71,"0x10f256d50":50,"0x7ffe4de00110":50,"100gb":92,"100gi":98,"10g":41,"10m":92,"1150u":92,"11\u5b9e\u73b0\u4e86c":55,"11e6":99,"12194102e":4,"124n":92,"13m":99,"1490u":92,"15501715e":4,"1550u":92,"15mb":111,"16mb":111,"16u":92,"173m":110,"173n":92,"1770u":92,"18ad":98,"18e457ce3d362ff5f3febf8e7f85ffec852f70f3b629add10aed84f930a68750":99,"197u":92,"1gb":92,"1st":[108,110],"210u":92,"211839e770f7b538e2d8":9,"215n":92,"228u":92,"234m":110,"2520u":92,"252kb":111,"25639710e":4,"25k":111,"2680u":92,"27787406e":4,"279n":92,"27m":92,"285m":92,"2863m":92,"28m":92,"28x28":2,"2977m":92,"2cbf7385":98,"302n":92,"30u":92,"32777140e":4,"328n":92,"32u":92,"32x32":14,"331n":92,"3320u":92,"36540484e":4,"365e":98,"36u":92,"3710m":92,"3768m":92,"387u":92,"38u":92,"3920u":92,"39u":92,"4035m":92,"4090u":92,"4096mb":95,"4279m":92,"43630644e":4,"43u":92,"448a5b355b84":99,"4560u":92,"4563m":92,"45u":92,"4650u":92,"4726m":92,"473m":99,"48565123e":4,"48684503e":4,"49316648e":4,"4gb":95,"4yf":8,"50bd":98,"50gi":98,"51111044e":4,"514u":92,"525n":92,"526u":92,"53018653e":4,"536u":92,"5460u":92,"5470u":92,"54u":92,"5690m":92,"573u":92,"578n":92,"5798m":92,"586u":92,"58s":99,"5969m":92,"5_cudnn5_avx_mkl":80,"5delta":8,"6080u":92,"6082v4":8,"6140u":92,"6305m":92,"639u":92,"64m":53,"655u":92,"6780u":92,"6810u":92,"682u":92,"6970u":92,"6ce9":98,"704u":92,"70634608e":4,"7090u":92,"72296313e":4,"72u":92,"73u":92,"75u":92,"760u":92,"767u":92,"783n":92,"784u":92,"78m":92,"7eamaa":14,"7kb":99,"8250u":92,"8300u":92,"830n":92,"849m":92,"85625684e":4,"861u":92,"8661m":92,"892m":92,"901n":92,"90u":92,"918u":92,"9247m":92,"924n":92,"9261m":92,"93137714e":4,"9330m":92,"94u":92,"9530m":92,"96644767e":4,"983m":92,"988u":92,"997u":92,"99982715e":4,"99m":110,"99u":92,"9f18":99,"\u4e00\u4e2a\u5178\u578b\u7684chunk\u5982\u4e0b\u6240\u793a":44,"\u4e00\u4e2a\u795e\u7ecf\u7f51\u7edc\u7684\u6a21\u578b\u7531\u5927\u91cf\u7684\u53c2\u6570\u7ec4\u6210":34,"\u4e00\u4e2achunk\u7531\u6240\u5728\u7684\u6587\u4ef6\u504f\u79fb":44,"\u4e00\u4e2aposix\u517c\u5bb9\u7684\u6587\u4ef6\u7cfb\u7edf":44,"\u4e00\u822c\u4e0d\u5141\u8bb8\u518d\u4ece":71,"\u4e0a\u4f20\u5230cloud\u6216\u8005\u4e0b\u8f7d\u5230\u672c\u5730\u7684\u65f6\u95f4\u53ef\u80fd\u6bd4\u8f83\u957f":44,"\u4e0a\u4f20\u65b9\u6cd5":71,"\u4e0a\u6ce8\u518c\u4e00\u4e0b":44,"\u4e0b\u5b58\u653e\u516c\u5171\u6570\u636e\u96c6\u5408":35,"\u4e0b\u8f7d":44,"\u4e0b\u8f7d\u5230\u672c\u5730":44,"\u4e0b\u9762\u5206\u522b\u4ecb\u7ecd\u67d0\u4e00\u7c7b\u6587\u4ef6\u7684\u5b9e\u73b0\u65b9\u5f0f":55,"\u4e0d\u4e00\u81f4\u7684\u7531pfsclient\u4e0b\u8f7d\u6216\u8005\u4f20\u8f93chunk\u5b8c\u6210":44,"\u4e0d\u4f7f\u7528\u9759\u6001\u5e93":54,"\u4e0d\u4f7f\u7528c":54,"\u4e0d\u4f7f\u7528swig":54,"\u4e0d\u540c\u7248\u672c\u7684\u7f16\u8bd1\u5668\u4e4b\u95f4":54,"\u4e0d\u540c\u8bed\u8a00\u7684\u63a5\u53e3\u9002\u5e94\u4e0d\u540c\u8bed\u8a00\u7684\u7279\u6027":54,"\u4e0d\u5728":55,"\u4e0d\u5bb9\u6613\u51fa\u9519":44,"\u4e0d\u5d4c\u5165\u5176\u4ed6\u8bed\u8a00\u89e3\u91ca\u5668":54,"\u4e0d\u5d4c\u5165python\u89e3\u91ca\u5668":54,"\u4e0d\u663e\u793a\u7684\u5199\u6bcf\u4e2a\u7c7b\u5177\u4f53\u5305\u542b\u4ec0\u4e48":54,"\u4e0d\u7528mount\u7684\u65b9\u5f0f\u6765\u8bbf\u95ee\u6570\u636e":35,"\u4e0e\u4e4b\u76f8\u5bf9\u7684\u662flocal":44,"\u4e0e\u529f\u80fd\u5206\u652f\u4e0d\u540c\u7684\u662f":71,"\u4e0e\u53ef\u80fd\u6709\u7684":71,"\u4e14\u589e\u52a0\u4e00\u4e2a\u7b2c\u4e09\u65b9\u8bed\u8a00":54,"\u4e14\u8c03\u7528\u65f6\u4e0d\u80fd\u629b\u51fa\u5f02\u5e38\u6216\u51fa\u73b0\u8fd0\u884c\u65f6\u9519\u8bef":55,"\u4e14c99\u652f\u6301bool\u7c7b\u578b\u548c\u5b9a\u957f\u6574\u6570":54,"\u4e14c99\u76f8\u5bf9\u4e8ec11\u4f7f\u7528\u66f4\u52a0\u5e7f\u6cdb":54,"\u4e25\u683c\u7684\u547d\u540d\u89c4\u8303pep":71,"\u4e2a\u6027\u5316\u63a8\u8350":71,"\u4e2d":[54,55],"\u4e2d\u5199\u5165json\u5185\u5bb9":34,"\u4e2d\u5b8c\u5168\u4e00\u81f4":54,"\u4e2d\u5b9e\u73b0\u7684\u7ed3\u6784\u4f53":55,"\u4e2d\u7684\u7248\u672c\u4fe1\u606f":71,"\u4e2d\u8fd0\u884c\u4efb\u52a1\u7684\u89d2\u5ea6":35,"\u4e3a\u4e86\u5e94\u5bf9\u4ee5\u4e0a\u7684\u95ee\u9898":44,"\u4e3a\u4e86\u66b4\u9732\u7684\u63a5\u53e3\u5c3d\u91cf\u7b80\u5355":55,"\u4e3b\u8981\u529f\u80fd\u5305\u62ec":44,"\u4e4b\u5916\u7684\u6240\u6709\u5934\u6587\u4ef6":55,"\u4e5f\u4e0d\u4f7f\u7528\u5176\u4ed6\u52a8\u6001\u5e93":54,"\u4e5f\u4e0d\u5e94\u8be5\u62a5\u9519":55,"\u4e5f\u4e0d\u751f\u6210":55,"\u4e66\u5199":54,"\u4eba\u8138\u8bc6\u522b":35,"\u4ec5\u4ec5\u4f7f\u7528":54,"\u4ece":71,"\u4ece\u78c1\u76d8\u6587\u4ef6\u4e2d\u52a0\u8f7duuid\u6587\u4ef6\u540d\u7684\u68c0\u67e5\u70b9\u5feb\u7167\u6587\u4ef6":34,"\u4eceetcd\u4e2d\u8bfb\u53d6\u8282\u70b9":34,"\u4ed6\u4e3b\u8981\u5305\u542b\u4e86\u5b9e\u9645\u66b4\u9732\u7684\u7c7b\u578b\u7ed3\u6784":55,"\u4ed6\u662f\u5c06":55,"\u4ed6\u7684\u76ee\u6807\u662f\u4f7f\u7528c":54,"\u4ee3\u7801\u751f\u6210\u7684\u7b26\u53f7\u53ef\u80fd\u4e0d\u4e00\u81f4":54,"\u4ee3\u8868\u8fd9\u4e2ashard\u7684\u6700\u5927index":35,"\u4ee3\u8868shard\u7684index":35,"\u4ee5\u4e0a\u4ee3\u7801\u7684reader\u8f93\u51fa\u7684data":35,"\u4ee5\u4e0a\u547d\u4ee4\u4f1a\u5728\u5f53\u524d\u76ee\u5f55\u4e0b\u751f\u6210100\u4e2a\u6587\u4ef6":35,"\u4ee5\u4e0b":35,"\u4ee5\u4fbf\u6211\u4eec\u53ef\u4ee5\u628a\u66f4\u591a\u7684\u7cbe\u529b\u653e\u5230\u903b\u8f91\u672c\u8eab\u4e0a":44,"\u4ee5\u53canumpi":35,"\u4efb\u610f\u65f6\u523b\u53ea\u53ef\u80fd\u540c\u65f6\u6709\u4e00\u53f0\u670d\u52a1\u5668\u6545\u969c":34,"\u4f1a\u5bfc\u81f4\u4e0d\u540c\u7248\u672cpython\u5728\u4e00\u4e2a\u8fdb\u7a0b\u91cc\u7684bug":54,"\u4f1a\u76f4\u63a5\u62a5\u9519\u9000\u51fa":54,"\u4f1a\u88abpickle\u5e8f\u5217\u5316\u6210\u5b57\u7b26\u4e32":35,"\u4f20\u5165":35,"\u4f46":55,"\u4f46\u4e0d\u66b4\u9732":55,"\u4f46\u5e76\u6ca1\u6709\u7ecf\u8fc7\u56de\u5f52\u6d4b\u8bd5":71,"\u4f46\u6240\u6709fork\u7684\u7248\u672c\u5e93\u7684\u6240\u6709\u5206\u652f\u90fd\u76f8\u5f53\u4e8e\u7279\u6027\u5206\u652f":71,"\u4f46\u662f\u53c8\u8fc7\u4e8e\u7410\u788e":55,"\u4f46\u662f\u89e3\u91ca\u6027\u8bed\u8a00":54,"\u4f5c\u4e3a\u5b58\u50a8\u7cfb\u7edf":35,"\u4f5c\u4e3a\u7c7b\u53e5\u67c4":54,"\u4f7f\u7528":[55,71],"\u4f7f\u7528\u4e0b\u9762\u547d\u4ee4":35,"\u4f7f\u7528\u52a8\u6001\u5e93":54,"\u4f7f\u7528\u540c\u6837\u7684\u8bad\u7ec3\u6570\u636eblock":34,"\u4f7f\u7528\u667a\u80fd\u6307\u9488\u7684\u539f\u56e0\u662f":55,"\u4f7f\u7528\u76f8\u5bf9\u8def\u5f84\u7684\u5f15\u7528\u65b9\u5f0f":55,"\u4f7f\u7528\u8fd9\u4e2a\u795e\u7ecf\u7f51\u7edc\u53ef\u4ee5\u5b8c\u6210\u5bf9\u65b0\u6570\u636e\u7684\u9884\u6d4b":34,"\u4f7f\u7528\u9759\u6001\u5e93\u548c\u52a8\u6001\u5e93\u96be\u5ea6\u5dee\u4e0d\u591a":54,"\u4f7f\u7528c":55,"\u4f7f\u7528c99\u505a\u63a5\u53e3":54,"\u4f7f\u7528c99\u800c\u4e0d\u4f7f\u7528c11\u7684\u539f\u56e0\u662f":54,"\u4f7f\u7528c99\u800c\u4e0d\u4f7f\u7528c89":54,"\u4f7f\u7528regress":71,"\u4f7f\u7528swig\u53ea\u652f\u6301cpython\u89e3\u91ca\u5668":54,"\u4f7f\u7528swig\u9700\u8981\u591a\u8bed\u8a00\u7ed1\u5b9a\u7684\u5f00\u53d1\u4eba\u5458\u719f\u7ec3\u638c\u63e1swig\u914d\u7f6e":54,"\u4f7f\u7528void":54,"\u4f8b\u5982":[35,54,55,71],"\u4f8b\u5982\u5bf9\u4e8ejava\u6216\u8005python":54,"\u4f8b\u5982\u5bf9\u4e8ejava\u6765\u8bf4":54,"\u4f8b\u5982\u5bf9\u4e8epython":54,"\u4f8b\u5982c":54,"\u4f8b\u5982java\u4e0epython\u7684\u9519\u8bef\u5904\u7406\u662f\u76f4\u63a5\u6254\u51fa\u6765except":54,"\u4f8b\u5982python\u53ef\u4ee5\u4f7f\u7528":54,"\u4f8b\u5982python\u7684":54,"\u4f9d\u6b21\u7c7b\u63a8":71,"\u4fbf\u662f\u5c06\u9759\u6001\u5e93\u52a0\u5165jvm\u4e2d":54,"\u4fee\u590d\u6240\u6709bug\u540e":71,"\u4fee\u590ddocker\u7f16\u8bd1\u955c\u50cf\u95ee\u9898":71,"\u4fee\u590dubuntu":71,"\u4fee\u6539":71,"\u4fee\u6539\u6210":71,"\u505a\u53ea\u8bfb\u6302\u8f7d":35,"\u505a\u5982\u4e0b\u51e0\u4e2a\u64cd\u4f5c":71,"\u505a\u63a5\u53e3":54,"\u505c\u6b62\u4fdd\u5b58\u68c0\u67e5\u70b9\u7684\u7ebf\u7a0b":34,"\u5148\u5b9e\u73b0\u6a21\u578b\u63a8\u65ad\u7684api":55,"\u5176\u4e2d":[54,71],"\u5176\u4ed6\u51fd\u6570\u5747\u8fd4\u56de":55,"\u5176\u4ed6\u7528\u6237\u7684fork\u7248\u672c\u5e93\u5e76\u4e0d\u9700\u8981\u4e25\u683c\u9075\u5b88":71,"\u5177\u4f53\u4f7f\u7528\u65b9\u6cd5\u4e3a":55,"\u5177\u4f53\u539f\u56e0\u53c2\u8003":55,"\u5177\u4f53\u8bf7\u53c2\u8003":55,"\u5185\u90e8\u9a71\u52a8python\u89e3\u91ca\u5668\u8fdb\u884c\u6a21\u578b\u914d\u7f6e\u89e3\u6790\u548c\u6570\u636e\u8bfb\u53d6":54,"\u518d\u5728\u6bcf\u4e00\u4e2aapi\u4e2d\u81ea\u5df1\u68c0\u67e5\u7c7b\u578b":54,"\u518d\u57fa\u4e8e":71,"\u5199\u4ee3\u7801":54,"\u5199\u5165\u5feb\u7167\u6570\u636e":34,"\u51fd\u6570\u5373\u53ef\u5b8c\u6210\u8f6c\u6362":35,"\u51fd\u6570\u540d\u4e3a":55,"\u51fd\u6570\u547d\u540d":54,"\u5206\u5e03\u5f0f\u5b58\u50a8\u670d\u52a1":34,"\u5206\u652f":71,"\u5206\u652f\u4e00\u65e6\u5efa\u7acb":71,"\u5206\u652f\u4e2d":71,"\u5206\u652f\u4e3a\u5f00\u53d1":71,"\u5206\u652f\u4e3a\u6bcf\u4e00\u6b21release\u65f6\u5efa\u7acb\u7684\u4e34\u65f6\u5206\u652f":71,"\u5206\u652f\u4e3a\u7a33\u5b9a":71,"\u5206\u652f\u529f\u80fd\u7684\u5c01\u95ed":71,"\u5206\u652f\u5408\u5165":71,"\u5206\u652f\u5408\u5165master\u5206\u652f":71,"\u5206\u652f\u540c\u6b65\u4e3b\u7248\u672c\u5e93\u7684":71,"\u5206\u652f\u540d\u4e3a":71,"\u5206\u652f\u5b58\u5728\u7684\u65f6\u5019":71,"\u5206\u652f\u6d3e\u751f\u51fa\u65b0\u7684\u5206\u652f":71,"\u5206\u652f\u7684\u7248\u672c\u90fd\u662f\u7ecf\u8fc7\u5355\u5143\u6d4b\u8bd5\u548c\u56de\u5f52\u6d4b\u8bd5\u7684\u7248\u672c":71,"\u5206\u652f\u7684\u7248\u672c\u90fd\u7ecf\u8fc7\u5355\u5143\u6d4b\u8bd5":71,"\u5206\u7247":34,"\u5219\u4f7f\u7528\u542f\u52a8\u53c2\u6570\u5b9a\u4e49\u7684\u521d\u59cb\u5316\u65b9\u6cd5\u521d\u59cb\u5316\u53c2\u6570":34,"\u5219\u5ffd\u7565":34,"\u5219\u628a\u53e6\u4e00\u4e2a\u6162\u901f\u7684kill\u6389":34,"\u5219\u76f4\u63a5\u5f15\u5165\u53e6\u4e00\u79cd\u7c7b\u578b\u7684\u5934\u6587\u4ef6":55,"\u5219\u9700\u8981\u56de\u6eda\u5230\u4e0a\u4e00\u4e2a\u68c0\u67e5\u70b9":34,"\u5220\u9664\u78c1\u76d8\u76ee\u5f55\u4e2d\u4e0d\u662f\u5f53\u524duuid\u7684\u5feb\u7167\u6587\u4ef6":34,"\u5230":34,"\u529f\u80fd":44,"\u529f\u80fd\u7684\u6b63\u786e\u6027\u5305\u62ec\u9a8c\u8bc1paddlepaddle\u76ee\u524d\u7684":71,"\u52a8\u6001\u5e93":54,"\u5305\u542b\u4e86\u67d0\u79cd\u7c7b\u578b\u7684\u7c7b\u578b\u5b9a\u4e49\u548c\u66b4\u9732\u7684\u5168\u90e8\u51fd\u6570":55,"\u5305\u62ec":35,"\u5305\u62ec\u6743\u91cdw\u548c\u504f\u7f6eb":34,"\u534f\u540c\u5b8c\u6210releas":71,"\u5355\u4e2a\u503c":35,"\u5355\u70b9\u6545\u969c":34,"\u5373":55,"\u5373\u4f7f\u7528":55,"\u5373\u4f7f\u7528\u6237\u76f4\u63a5\u5f15\u7528\u67d0\u79cd\u7c7b\u578b\u7684\u5934\u6587\u4ef6":55,"\u5373\u4f7fc":55,"\u5373\u4f8b\u5982":55,"\u5373\u4fbfpaddl":55,"\u5373\u5b8c\u6210\u67d0\u4e00\u4e2a\u4efb\u52a1\u7684\u6700\u5c11\u51fd\u6570":55,"\u5373\u66b4\u9732":55,"\u5373\u8fd9\u4e2a\u52a8\u6001\u5e93\u662f\u4e0d\u4f9d\u8d56\u4e8e\u5176\u4ed6\u4efb\u4f55\u6587\u4ef6\u7684":54,"\u53c2\u6570":54,"\u53c2\u8003":[44,54],"\u53cc\u5411\u9a8c\u8bc1":44,"\u53d1\u5e03\u5230dockerhub":71,"\u53d1\u5e03\u5230github":71,"\u53ea\u5bf9\u7279\u6b8a\u5728\u7ebf\u7cfb\u7edf\u8003\u8651\u4e24\u53f0\u4ee5\u4e0a\u540c\u65f6\u6545\u969c\u7684\u5bb9\u707e":34,"\u53ea\u66b4\u9732\u6982\u5ff5\u7684\u63a5\u53e3":55,"\u53ea\u80fd\u8c03\u7528paddle\u7684\u52a8\u6001\u5e93":54,"\u53ea\u9700\u8981\u6062\u590d\u8fd9\u53f0\u8282\u70b9":34,"\u53ef\u4ee5\u51cf\u5c0f\u7cfb\u7edf\u590d\u6742\u6027":34,"\u53ef\u4ee5\u5728\u4efb\u4f55\u673a\u5668\u4e0a\u6267\u884c\u7684":54,"\u53ef\u4ee5\u628a\u672c\u5730\u7684\u6570\u636e\u4e0a\u4f20\u5230\u5b58\u50a8\u96c6\u7fa4\u4e2d":35,"\u53ef\u4ee5\u6709\u6548\u7684\u907f\u514dparamet":34,"\u53ef\u4ee5\u7528":44,"\u53ef\u4ee5\u7528\u4ee5\u4e0b\u6307\u4ee4":35,"\u53ef\u4ee5\u7ee7\u7eed\u5728\u81ea\u5df1\u7684\u529f\u80fd\u5206\u652f\u63d0\u4ea4\u4ee3\u7801":71,"\u53ef\u4ee5\u901a\u8fc7\u9636\u6bb5\u6027\u7684\u4fdd\u5b58\u6bcf\u4e2aparamet":34,"\u53ef\u80fd\u4f1a\u9020\u6210\u7f51\u7edc\u62e5\u585e":34,"\u540c\u65f6\u518d\u5c06":71,"\u540c\u65f6\u63d0\u8d77":71,"\u540d\u5b57\u4fee\u9970":54,"\u5411\u6307\u5b9a\u7684\u76ee\u5f55\u4e2d\u4e00\u4e2a\u65b0\u7684\u6587\u4ef6":34,"\u5411paddlepaddle\u7684\u4e3b\u7248\u672c\u5e93\u63d0\u4ea4":71,"\u5426\u5219\u5f97\u628apaddle\u9759\u6001\u5e93\u94fe\u63a5\u5230\u89e3\u91ca\u5668\u91cc":54,"\u542f\u52a8\u4e00\u4e2a\u65b0\u7684\u7ebf\u7a0b\u5f00\u59cb\u4fdd\u5b58\u68c0\u67e5\u70b9":34,"\u548c":[35,54,55,71],"\u548c\u79bb\u7ebf\u6570\u636e\u7684\u65b9\u5f0f":35,"\u54ea\u4e2atrainer\u5148\u5b8c\u6210block\u7684\u8bad\u7ec3":34,"\u56e0\u4e3a\u8fd9\u6837\u505a\u4e5f\u6ca1\u6cd5\u4fdd\u8bc1\u6d88\u9664\u968f\u673a\u6027":34,"\u56e0\u4e3aswig\u5728\u7b2c\u4e09\u65b9\u8bed\u8a00\u4e2d\u66b4\u9732\u7684\u51fd\u6570\u540d":54,"\u56fe\u50cf\u5206\u7c7b":71,"\u5728":[55,71],"\u5728\u4e00\u4e2a\u4e0d\u53ef\u4e2d\u65ad\u5e76\u7f3a\u5c11\u5907\u4efd\u7684\u8bad\u7ec3\u4efb\u52a1\u4e2d":34,"\u5728\u4e0a\u56fe\u4e2d\u663e\u793a\u4e86\u5728\u4e00\u4e2a\u5b9e\u9645\u751f\u4ea7\u73af\u5883\u4e2d\u7684\u5e94\u7528":35,"\u5728\u4f7f\u7528twine\u4e0a\u4f20\u4e4b\u524d":71,"\u5728\u51fa\u73b0\u5355\u70b9\u6545\u969c\u65f6":34,"\u5728\u5b9e\u73b0\u8fc7\u7a0b\u4e2d":55,"\u5728\u5f00\u59cb\u8bad\u7ec3\u4e4b\u524d":35,"\u5728\u5f02\u6784\u96c6\u7fa4\u4e2d":34,"\u5728\u5f15\u5165\u5176\u4ed6\u7c7b\u578b\u7684\u5934\u6587\u4ef6\u65f6":55,"\u5728\u5feb\u7167\u5199\u5165\u5b8c\u6210\u540e":34,"\u5728\u60a8\u7684\u5b9e\u9645\u73af\u5883\u4e2d":34,"\u5728\u672c\u6587\u6863\u4e2d":44,"\u5728\u673a\u7fa4\u4e0a\u8fd0\u884c\u8f6c\u6362\u7a0b\u5e8f":35,"\u5728\u6837\u4f8b\u4e2d":55,"\u5728\u7528\u6237\u4f7f\u7528c":55,"\u5728\u7ebf\u6a21\u578b\u9884\u6d4b\u670d\u52a1":35,"\u5728\u8bc4\u5ba1\u8fc7\u7a0b\u4e2d":71,"\u5728\u8fd9\u4e2a":71,"\u5728\u8fd9\u4e2a\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165\u4efb\u4f55\u5176\u4ed6\u8bed\u8a00\u7684\u89e3\u91ca\u5668":54,"\u5728\u8fd9\u4e2a\u9636\u6bb5\u7684\u4ee3\u7801\u6b63\u5728\u7ecf\u5386\u56de\u5f52\u6d4b\u8bd5":71,"\u5728\u8fd9\u4e9b\u5934\u6587\u4ef6\u4e2d":55,"\u5728\u8fd9\u4e9b\u6587\u4ef6\u4e2d":55,"\u5728c":54,"\u5728c\u7684\u5934\u6587\u4ef6":54,"\u5728paddle\u4e4b\u4e0a\u8fd0\u884c\u7684\u6df1\u5ea6\u5b66\u4e60\u8bad\u7ec3\u8f93\u51fa\u7684\u6a21\u578b\u4f1a\u63d0\u4f9b\u7ed9\u5728\u7ebf\u4eba\u8138\u8bc6\u522b\u7684\u5e94\u7528\u4f7f\u7528":35,"\u5728paramet":34,"\u5747\u4f1a\u88ab\u5b89\u88c5\u5230includ":55,"\u5747\u662f\u5728":55,"\u57fa\u4e8e\u7c98\u6027\u4f1a\u8bdd\u7684\u8d1f\u8f7d\u5747\u8861\u529f\u80fd":44,"\u591a\u4e2a\u503c":35,"\u591a\u4e2aparamet":34,"\u5927\u591a\u6570\u8bed\u8a00\u90fd\u652f\u6301\u4f7f\u7528c\u8bed\u8a00api":54,"\u5982\u56fe\u4e2dtrainer":34,"\u5982\u679c\u4e0a\u9762\u4e24\u6b65\u51fa\u73b0\u9519\u8bef":34,"\u5982\u679c\u4f7f\u7528swig\u6211\u4eec\u9700\u8981\u5c06\u5728interface\u6587\u4ef6\u91cc":54,"\u5982\u679c\u5931\u8d25":71,"\u5982\u679c\u5b58\u5728\u67d0\u4e9btrainer\u6267\u884c\u901f\u5ea6\u8fc7\u6162\u4f1a\u5f71\u54cd\u6574\u4f53\u96c6\u7fa4\u7684\u901f\u5ea6":34,"\u5982\u679c\u5df2\u7ecf\u6b63\u5728\u6267\u884c\u4fdd\u5b58\u68c0\u67e5\u70b9\u7684\u7ebf\u7a0b":34,"\u5982\u679c\u662f\u5176\u5b83\u7c7b\u578b":35,"\u5982\u679c\u6709bugfix\u7684\u884c\u4e3a":71,"\u5982\u679c\u67d0\u4e00\u4e2a\u7c7b\u578b\u9700\u8981\u5f15\u7528\u53e6\u4e00\u4e2a\u7c7b\u578b":55,"\u5982\u679c\u67d0\u4e00\u4e2apaddl":55,"\u5982\u679c\u67d0\u4e00\u4e2apaddle\u6982\u5ff5\u5fc5\u987b\u8981\u66b4\u9732":55,"\u5982\u679c\u6ee1\u8db3\u6761\u4ef6":34,"\u5982\u679c\u7528\u6237\u8981\u628apaddle\u7684\u9759\u6001\u5e93":54,"\u5982\u679c\u8981\u4e0a\u4f20gpu\u7248\u672c\u7684\u5305":71,"\u5982\u679c\u8c03\u7528\u9759\u6001\u5e93\u53ea\u80fd\u5c06\u9759\u6001\u5e93\u4e0e\u89e3\u91ca\u5668\u94fe\u63a5":54,"\u5982\u679cparamet":34,"\u5b57\u6bb5\u8bbe\u4e3a":71,"\u5b57\u7b26\u4e32":35,"\u5b58\u50a8":35,"\u5b66\u4e60\u6210\u672c\u9ad8":54,"\u5b83\u4eec\u7684\u6587\u4ef6\u540d\u662f":35,"\u5b89\u88c5\u540e\u7684\u76ee\u5f55\u7ed3\u6784\u4e3a":55,"\u5b8c\u6210\u4e00\u4e2a\u4f20\u8f93\u52a8\u4f5c\u5b8c\u6210\u7684\u65f6\u95f4\u4e5f\u6bd4\u8f83\u77ed":44,"\u5b8c\u6210\u6570\u636e\u7684\u9884\u5904\u7406":35,"\u5b9e\u73b0\u7b80\u5355":54,"\u5bf9\u4e8e\u4e0d\u540c\u8bed\u8a00":54,"\u5bf9\u4e8e\u540c\u4e00\u6bb5c":54,"\u5bf9\u4e8e\u591a\u8bed\u8a00\u63a5\u53e3":54,"\u5bf9\u4e8e\u5927\u591a\u6570\u8bed\u8a00":54,"\u5bf9\u4e8e\u6bcf\u79cd\u7c7b\u578b":55,"\u5bf9\u4e8e\u6bcf\u79cdc":55,"\u5bf9\u6bd4":54,"\u5bf9\u8f93\u5165\u53c2\u6570\u7684\u5b89\u5168\u6027\u8fdb\u884c\u4e86\u5fc5\u8981\u7684\u5224\u65ad":55,"\u5bf9\u8fd9\u4e2a\u7248\u672c\u7684\u63d0\u4ea4":71,"\u5bfc\u51fa\u8fd9\u4e9b\u63a5\u53e3":55,"\u5c06":71,"\u5c06\u4e00\u4e2a\u795e\u7ecf\u7f51\u7edc\u53c2\u6570\u62c6\u5206\u6210\u591a\u4efd":34,"\u5c06\u5927\u91cf\u7684":54,"\u5c06\u65b0\u5206\u652f\u7684\u7248\u672c\u6253\u4e0atag":71,"\u5c06master\u5206\u652f\u7684\u5408\u5165commit\u6253\u4e0atag":71,"\u5c31\u9700\u8981\u5bf9\u8fd9\u4e2a\u7b2c\u4e09\u65b9\u8bed\u8a00\u589e\u52a0\u4e00\u4e9b\u5b9a\u4e49":54,"\u5e73\u5747\u6545\u969c\u4fee\u590d\u65f6\u95f4":34,"\u5e73\u5747\u6545\u969c\u7387":34,"\u5e76\u4e14\u4f7f\u7528":55,"\u5e76\u4e14\u5728\u5e38\u89c1\u7684\u5e73\u53f0\u4e0a":54,"\u5e76\u4e14\u628a\u7cfb\u7edf\u751f\u6210\u7684ca":44,"\u5e76\u4e14\u628a\u7ed3\u679c\u8fd4\u56depfsclient\u7aef":44,"\u5e76\u4e14\u8ba9\u63a5\u53e3\u8131\u79bb\u5b9e\u73b0\u7ec6\u8282":54,"\u5e76\u5220\u9664":71,"\u5e76\u5220\u9664\u66f4\u65e9\u7684\u5feb\u7167":34,"\u5e76\u52a0\u8f7d\u5176\u4e2d\u7684\u53c2\u6570":34,"\u5e76\u53d1\u5e03\u5230pypi":71,"\u5e76\u5728\u96c6\u7fa4\u4e2d\u8fd0\u884c\u591a\u4e2a\u5206\u5e03\u5f0f\u6570\u636e\u5904\u7406\u4efb\u52a1":35,"\u5e76\u5c06":71,"\u5e76\u5c06c":55,"\u5e76\u628a\u5feb\u7167\u4fdd\u5b58\u5230\u8fd9\u4e2a\u76ee\u5f55\u4e0b":34,"\u5e76\u6ca1\u6709paddle\u7279\u522b\u9700\u8981\u7684\u7279\u6027":54,"\u5e76\u88ab\u5b58\u50a8\u5728\u8bf8\u5982hadoop":35,"\u5e76\u9002\u5e94github\u7684\u7279\u6027\u505a\u4e86\u4e00\u4e9b\u533a\u522b":71,"\u5e76\u91cd\u65b0\u6253\u5305wheel\u5305":71,"\u5efa\u8bae":71,"\u5f00\u53d1\u4e86\u6a21\u578b\u9884\u6d4b\u7684\u6837\u4f8b\u4ee3\u7801":55,"\u5f00\u53d1\u8005\u4fee\u6539\u81ea\u5df1\u7684\u4ee3\u7801":71,"\u5f00\u53d1\u8005fork\u7684\u7248\u672c\u5e93\u4e2d":71,"\u5f00\u53d1\u8005fork\u7684\u7248\u672c\u5e93\u4f7f\u7528":71,"\u5f00\u59cb\u63d0\u4f9b\u670d\u52a1":34,"\u5f15\u5165\u4e86\u7c7b\u578b\u7684\u5934\u6587\u4ef6":55,"\u5f53\u529f\u80fd\u5206\u652f\u5f00\u53d1\u5b8c\u6bd5\u540e":71,"\u5f53\u7528\u6237\u4f7f\u7528\u5b8c\u8fd9\u4e2a\u53c2\u6570\u540e":55,"\u5f53destination\u6587\u4ef6\u4e0d\u5b58\u5728\u6216\u8005\u5927\u5c0f\u548csource\u6587\u4ef6\u4e0d\u4e00\u81f4\u65f6":44,"\u5f88\u96be\u4fdd\u8bc1\u591a\u8bed\u8a00\u4ee3\u7801\u98ce\u683c\u7684\u4e00\u81f4\u6027":54,"\u5f97\u4f7f\u7528":54,"\u5fc5\u8981":55,"\u60c5\u611f\u5206\u6790":71,"\u6211\u4eec\u4e5f\u53ef\u4ee5\u786e\u5b9a\u6bcf\u4e00\u4e2a\u53c2\u6570\u7684\u7c7b\u578b":55,"\u6211\u4eec\u4f7f\u7528\u52a8\u6001\u5e93\u6765\u5206\u53d1paddl":54,"\u6211\u4eec\u63d0\u4f9b\u4e24\u4e2a\u8f6c\u6362\u65b9\u5f0f":35,"\u6211\u4eec\u63d0\u51fa\u4e86chunk\u7684\u6982\u5ff5":44,"\u6211\u4eec\u6700\u7ec8\u7684\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165python\u6216\u8005\u5176\u4ed6\u4efb\u4f55\u8bed\u8a00\u7684\u89e3\u91ca\u5668":54,"\u6211\u4eec\u8bbe\u8ba1\u8bf4\u660e\u4e86\u540d\u4e3afilemanager\u7cfb\u7edf":44,"\u6211\u4eec\u9009\u62e9":35,"\u6211\u4eec\u90fd\u63d0\u4f9bpython\u7684\u8f6c\u6362\u5e93":35,"\u6216\u8005":[54,55],"\u6216\u8005\u5c06\u8fd9\u53f0\u8282\u70b9\u8fc1\u79fb\u5230\u53e6\u4e00\u4e2a\u8282\u70b9\u5e76\u542f\u52a8\u5373\u53ef\u6062\u590d\u8bad\u7ec3\u4efb\u52a1":34,"\u6216\u8005\u7528tuple\u8868\u793a\u7684\u591a\u4e2a\u503c":35,"\u6216\u8005\u7531\u5b83\u4eec\u7ec4\u6210\u7684list":35,"\u6240\u4ee5\u5728\u5199\u5165\u5feb\u7167\u7684\u8fc7\u7a0b\u4e2d":34,"\u6240\u4ee5\u7528\u6237\u9700\u8981\u9996\u5148\u5728":44,"\u6240\u6709\u4e0e\u7c7b\u578b\u76f8\u5173\u7684\u51fd\u6570":55,"\u6240\u6709\u7684\u63a5\u53e3\u5747\u4e3ac\u63a5\u53e3":55,"\u6240\u6709\u7c7b\u578b\u540d\u4e3a":55,"\u624b\u5199\u591a\u8bed\u8a00\u7ed1\u5b9a":54,"\u6253\u5f00\u8fd9\u4e2a\u7f16\u8bd1\u9009\u9879":55,"\u628a":35,"\u628a\u4e4b\u524d\u793a\u4f8b\u4e2d\u8f6c\u6362\u5b8c\u6bd5\u7684random":35,"\u6307\u6df1\u5ea6\u5b66\u4e60\u8bad\u7ec3\u4e4b\u540e\u5f97\u5230\u7684\u6240\u6709\u53c2\u6570":34,"\u63a5\u53e3":[54,55],"\u63a5\u53e3\u5c42\u505a\u8fc7\u591a\u5c01\u88c5":55,"\u63a5\u53e3\u662f":35,"\u63a5\u6536\u5904\u7406pfsclient\u7aef\u7684\u6587\u4ef6\u7ba1\u7406\u8bf7\u6c42":44,"\u63a7\u5236\u7528\u6237\u6743\u9650":35,"\u63d0\u4f9b\u4e03\u5c42\u534f\u8bae\u7684\u53cd\u5411\u4ee3\u7406":44,"\u63d0\u4f9b\u5e38\u7528\u7684\u547d\u4ee4\u884c\u7ba1\u7406\u547d\u4ee4\u7ba1\u7406\u6587\u4ef6\u548c\u76ee\u5f55":44,"\u63d0\u4f9b\u7528\u6237\u7ba1\u7406\u6587\u4ef6\u7684\u547d\u4ee4":44,"\u63d0\u4f9b\u7ed9paddle\u4f5c\u4e3a\u8bad\u7ec3\u6570\u636e":35,"\u652f\u6301\u5927\u6587\u4ef6\u7684\u65ad\u70b9\u4e0a\u4f20":44,"\u6570\u636e":44,"\u6570\u636e\u8bfb\u53d6\u5747\u4ea4\u7531\u5176\u4ed6\u8bed\u8a00\u5b8c\u6210":54,"\u6570\u636e\u957f\u5ea6\u53ca\u6821\u9a8c\u503c\u7ec4\u6210":44,"\u6570\u636e\u96c6\u9700\u8981\u9884\u5148\u88ab\u8f6c\u6362\u6210paddlepaddle\u5206\u5e03\u5f0f\u8bad\u7ec3\u4f7f\u7528\u7684\u5b58\u50a8\u683c":35,"\u6570\u636e\u9884\u5904\u7406\u4efb\u52a1":35,"\u6587\u4ef6":54,"\u6587\u4ef6\u4f20\u8f93\u7684\u7684\u5173\u952e\u5728\u4e8e\u9700\u8981pfsclient\u7aef\u5bf9\u6bd4source\u548cdestination\u7684\u6587\u4ef6chunks\u7684checksum\u662f\u5426\u4fdd\u6301\u4e00\u81f4":44,"\u6587\u4ef6\u5185\u5bb9\u4e3a":54,"\u6587\u4ef6\u540d\u4e3a\u6b64uuid":34,"\u6587\u4ef6\u5bf9\u5e94\u7684data":35,"\u6587\u4ef6\u7684\u4e0a\u4f20\u548c\u4e0b\u8f7d\u90fd\u662f\u901a\u8fc7\u5bf9chunk\u7684\u64cd\u4f5c\u6765\u5b9e\u73b0\u7684":44,"\u65b0\u624b\u5165\u95e8\u7ae0\u8282":71,"\u65b9\u4fbf\u6d4b\u8bd5\u4eba\u5458\u6d4b\u8bd5paddlepaddle\u7684\u884c\u4e3a":71,"\u65b9\u4fbf\u7528\u6237\u4e0a\u4f20\u81ea\u5df1\u7684\u8bad\u7ec3\u6570\u636e\u4ee5\u8fdb\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3":44,"\u65e0\u6cd5\u505a\u5230\u5bf9\u4e8e\u5404\u79cd\u8bed\u8a00\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u7684\u9002\u914d":54,"\u65e0\u8bba\u5728\u672c\u5730\u8fd8\u662f\u5728\u4e91\u7aef":35,"\u65e0\u8bba\u662f\u4ece":35,"\u65e0\u8bba\u662f\u5728\u672c\u5730\u6216\u662f\u4e91\u7aef\u8f6c\u6362":35,"\u65f6":34,"\u662f":44,"\u662f\u4e00\u4e2a\u591a\u8bed\u8a00\u63a5\u53e3\u7684\u4ee3\u7801\u751f\u6210\u5668":54,"\u662f\u4e00\u4e2a\u7c7b\u578b\u7684\u6807\u5fd7":55,"\u662f\u4e0d\u5e38\u89c1\u7684\u505a\u6cd5":54,"\u662f\u5404\u4e2a\u5b9e\u73b0\u4e2d\u5171\u4eab\u7684\u5934\u6587\u4ef6":55,"\u662f\u56e0\u4e3ac99\u652f\u6301":54,"\u662f\u5bf9\u7528\u6237\u6587\u4ef6\u5b58\u50a8\u7a7a\u95f4\u7684\u62bd\u8c61":44,"\u662f\u6307":55,"\u662f\u7528\u6237\u4f7f\u7528c":55,"\u662fc":55,"\u6682\u65f6\u4e0d\u8003\u8651\u591a\u4e2aparamet":34,"\u66b4\u9732\u8fd9\u4e2a\u6982\u5ff5\u5fc5\u8981\u51fd\u6570":55,"\u6700\u540e\u5220\u9664":71,"\u6700\u5e38\u89c1\u7684\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u662fexcept":54,"\u6709\u6807\u51c6\u7684":54,"\u6709\u7684\u65f6\u5019":54,"\u672c\u5217\u8868\u8bf4\u660epaddlepaddle\u53d1\u7248\u4e4b\u524d\u9700\u8981\u6d4b\u8bd5\u7684\u529f\u80fd\u70b9":71,"\u672c\u6587\u6863\u63cf\u8ff0paddl":55,"\u673a\u5668\u7ffb\u8bd1":71,"\u6765\u4fdd\u8bc1\u8bad\u7ec3\u8fc7\u7a0b\u53ef\u4ee5\u4ece\u4e2d\u95f4\u72b6\u6001\u91cd\u65b0\u542f\u52a8":34,"\u6765\u786e\u4fdd\u628a":54,"\u6765\u8868\u793apaddle\u5185\u90e8\u7c7b":54,"\u6765\u8bbf\u95ee\u7528\u6237\u81ea\u5df1\u7684\u6570\u636e":35,"\u6765\u8fdb\u884c\u8ba8\u8bba":55,"\u6807\u51c6\u8868\u793apaddlepaddle\u7248\u672c\u53f7":71,"\u68c0\u67e5\u70b9\u4fdd\u5b58\u7a0b\u5e8f\u6d41\u7a0b":34,"\u6a21\u578b\u53c2\u6570\u68c0\u67e5\u70b9\u901a\u8fc7\u5b9a\u671f\u5411\u78c1\u76d8\u4e0a\u4fdd\u5b58\u4e00\u4efd\u5b58\u50a8\u5728paramet":34,"\u6a21\u578b\u6570\u636e\u68c0\u67e5\u70b9\u7684\u5b9e\u73b0":34,"\u6a21\u578b\u914d\u7f6e\u89e3\u6790":54,"\u6b64\u65f6master\u5c06\u8d1f\u8d23\u542f\u52a8\u4e00\u4e2a\u65b0\u7684train":34,"\u6bcf\u4e00\u4e2a":71,"\u6bcf\u4e00\u4e2a\u6587\u4ef6\u662f\u6570\u636e\u96c6\u7684\u4e00\u4e2ashard":35,"\u6bcf\u4e2a\u503c\u7684\u7c7b\u578b\u53ef\u4ee5\u662f\u6574\u5f62":35,"\u6bcf\u4e2adata":35,"\u6bcf\u4e2aparamet":34,"\u6bcf\u4e2ashard\u5206\u522b\u5b58\u50a8\u5728\u5176\u4e2d\u4e00\u53f0paramet":34,"\u6bcf\u6b21\u8f93\u51fa\u4e00\u4e2adata":35,"\u6bcf\u969410\u5206\u949f":34,"\u6bd4\u5982":35,"\u6bd4\u5982\u5c06":71,"\u6bd4\u5982\u6bcf\u969410\u5206\u949f\u6700\u65b0\u7684\u5feb\u7167":34,"\u6bd4\u5982\u6d41\u5f0f\u6570\u636e\u5904\u7406":35,"\u6bd4\u5982imagenet\u8fd9\u4e2a\u6570\u636e\u96c6\u53ef\u80fd\u88ab\u5206\u62101000\u4e2ashard":35,"\u6ce8":34,"\u6d4b\u8bd5docker\u955c\u50cf":71,"\u6d6e\u70b9\u578b\u6570\u636e":35,"\u7136\u540e\u5728etcd\u7684":34,"\u7136\u540e\u5c31\u53ef\u4ee5\u5e76\u53d1\u5199\u5165\u591a\u4e2achunk":44,"\u7136\u540e\u624d\u80fd\u4f7f\u7528pfsclient":44,"\u7248\u672c\u5206\u652f":71,"\u7248\u672c\u53f7":71,"\u7248\u672c\u53f7rc":71,"\u7248\u672cfork\u51fa\u81ea\u5df1\u7684\u529f\u80fd\u5206\u652f":71,"\u73b0\u9636\u6bb5paddle\u6709\u4e00\u4e2a\u95ee\u9898\u662f":54,"\u751f\u4ea7\u73af\u5883\u4e2d\u7684\u8bad\u7ec3\u6570\u636e\u96c6\u901a\u5e38\u4f53\u79ef\u5f88\u5927":35,"\u751f\u4ea7\u73af\u5883\u7684\u65e5\u5fd7\u6570\u636e\u4f1a\u901a\u8fc7\u5b9e\u65f6\u6d41\u7684\u65b9\u5f0f":35,"\u751f\u6210\u5404\u79cd\u8bed\u8a00\u7684\u7ed1\u5b9a\u4ee3\u7801":54,"\u751f\u6210\u6587\u6863":54,"\u751f\u6210\u7684":35,"\u751f\u6210\u7ed9\u5b9a":35,"\u751f\u6210api\u6587\u6863":54,"\u751f\u6210pfsclient\u548cpfsserver\u7684\u6846\u67b6\u90e8\u5206":44,"\u7528":44,"\u7528\u6237\u4e0a\u4f20\u6570\u636e\u540e":35,"\u7528\u6237\u4e5f\u53ef\u4ee5\u4e0a\u4f20label":35,"\u7528\u6237\u53ef\u4ee5\u5b89\u5168\u7684\u91ca\u653e\u67d0\u4e2ac":55,"\u7528\u6237\u53ef\u4ee5\u628a\u81ea\u5df1\u7684\u6570\u636e\u5206\u4eab\u7ed9\u522b\u4eba":35,"\u7528\u6237\u53ef\u4ee5\u76f4\u63a5\u4f7f\u7528\u8fd9\u4e2a\u52a8\u6001\u5e93\u6765\u5f15\u5165paddl":55,"\u7528\u6237\u5728\u672c\u5730\u8f6c\u6362\u597d\u518d\u4e0a\u4f20":35,"\u7528\u6237\u6587\u4ef6\u53ef\u80fd\u662f\u6bd4\u8f83\u5927\u7684":44,"\u7528\u6237\u901a\u8fc7c":55,"\u7531\u4e8e\u5bf9parameters\u7684\u66f4\u65b0\u9700\u8981\u83b7\u53d6parameters\u5185\u5b58\u7684":34,"\u7531\u4e8e\u96c6\u7fa4\u4e2d\u540c\u65f6\u5b58\u5728\u4e24\u53f0\u673a\u5668\u6545\u969c\u7684\u6982\u7387\u6781\u4f4e":34,"\u7531\u4e8ec":54,"\u7531\u4e8echunk\u6bd4\u8f83\u5c0f":44,"\u7531\u4e8epypi":71,"\u7533\u8bf7\u7528\u6237\u7a7a\u95f4":44,"\u7684\u547d\u540d\u98ce\u683c\u5e76\u4e0d\u80fd\u9002\u5e94\u5176\u4ed6\u7b2c\u4e09\u65b9\u8bed\u8a00":54,"\u7684\u5934\u6587\u4ef6":54,"\u7684\u63a5\u53e3\u6837\u5f0f":54,"\u7684\u6570\u636e\u6d41\u56fe":35,"\u7684\u6e90\u7801\u91cc\u4f7f\u7528\u4e86":54,"\u7684\u7f29\u5199":44,"\u7684\u89c4\u8303":54,"\u7684\u89d2\u5ea6":35,"\u7684\u914d\u7f6e\u5199\u5230\u914d\u7f6e\u6587\u4ef6\u4e2d":35,"\u76ee\u524d\u53ea\u8003\u8651\u52a8\u6001\u6269\u5bb9trainer\u6570\u91cf":34,"\u76ee\u524d\u5d4c\u5165python\u89e3\u91ca\u5668":54,"\u76ee\u524d\u6211\u4eec\u7528cephfs\u6765\u642d\u5efa":44,"\u76ee\u524dpaddle\u7684\u8fdb\u7a0b\u6a21\u578b\u662fc":54,"\u76ee\u5f55\u4e0b":55,"\u76f4\u63a5\u4f7f\u7528c\u8bed\u8a00\u7684":54,"\u76f4\u63a5\u5220\u9664\u8fd9\u4e2a\u53c2\u6570\u5373\u53ef":55,"\u76f4\u63a5\u5bfc\u51fa\u5230c\u7684\u63a5\u53e3\u6bd4\u8f83\u56f0\u96be":54,"\u793e\u533a\u53c2\u4e0e\u56f0\u96be":54,"\u793e\u533a\u8d21\u732e\u4ee3\u7801\u5b66\u4e60\u6210\u672c\u9ad8":54,"\u795e\u7ecf\u7f51\u7edc\u4e2d\u7684\u53c2\u6570":34,"\u79bb\u7ebf\u6279\u5904\u7406":35,"\u7b2c\u4e00\u4e2atag\u4e3a":71,"\u7b2c\u4e09\u6b65\u5b8c\u6210\u540e":71,"\u7b2c\u4e8c\u4e2a\u4e3a":71,"\u7b49":55,"\u7b49\u5168\u90e8\u9759\u6001\u5e93\u4e2d\u7684\u76ee\u6807\u6587\u4ef6\u5168\u90e8\u6253\u5305\u540e\u4ea7\u751f\u7684\u6587\u4ef6":55,"\u7b49\u6587\u4ef6":55,"\u7c7b\u4f3c":55,"\u7c7b\u540d\u548cc":54,"\u7c7b\u578b":54,"\u7ea2\u697c\u68a6":108,"\u7ed3\u8bba":54,"\u7edf\u4e00\u7528":35,"\u7f16\u8bd1\u5668\u6ca1\u6709":54,"\u7f16\u8bd1\u578b\u8bed\u8a00":54,"\u7f16\u8bd1\u8fd9\u4e2a\u7248\u672c\u7684docker\u53d1\u884c\u955c\u50cf":71,"\u7f16\u8bd1\u8fd9\u4e2a\u7248\u672c\u7684python":71,"\u7f16\u8bd1\u8fd9\u4e2a\u7248\u672c\u7684ubuntu":71,"\u7f16\u8bd1c":55,"\u7f16\u8bd1master\u5206\u652f\u7684docker\u53d1\u884c\u955c\u50cf":71,"\u7f16\u8bd1ubuntu\u7684deb\u5305":71,"\u800c\u4e0d\u5fc5\u5728\u610fpaddl":55,"\u800c\u4e0d\u652f\u6301pypy\u89e3\u91ca\u5668":54,"\u800c\u4e0d\u66b4\u9732\u6982\u5ff5\u7684\u5b9e\u73b0":55,"\u800c\u4e14\u5728\u4f20\u8f93\u7684\u8fc7\u7a0b\u4e2d\u4e5f\u53ef\u80fd\u51fa\u73b0\u7f51\u7edc\u4e0d\u7a33\u5b9a\u7684\u60c5\u51b5":44,"\u800c\u51fa\u73b0\u9636\u6bb5\u6027\u7684\u8fd0\u884c\u505c\u6ede":34,"\u800c\u5728cpp\u91cc\u9762\u5b9e\u73b0\u8fd9\u4e2ac\u7684\u63a5\u53e3":54,"\u800c\u591a\u8bed\u8a00\u63a5\u53e3\u9700\u8981\u76f4\u63a5\u8bfb\u53d6\u751f\u6210\u7684\u4e8c\u8fdb\u5236":54,"\u800c\u5bf9\u4e8egolang":54,"\u800c\u5bf9\u4e8egolang\u9519\u8bef\u5904\u7406\u5e94\u8be5\u4f7f\u7528\u8fd4\u56de\u503c":54,"\u800c\u662f\u76f4\u63a5\u4fee\u6539paddl":55,"\u800c\u662f\u76f4\u63a5\u7528api\u7684\u63a5\u53e3\u8fdc\u7a0b\u8bbf\u95ee":35,"\u800cswig\u53ea\u80fd\u7b80\u5355\u7684\u66b4\u9732c":54,"\u81ea\u52a8\u6302\u8f7d\u5206\u5e03\u5f0f\u5b58\u50a8\u76ee\u5f55":34,"\u81f3\u4e8e\u4e3a\u4ec0\u4e48\u9700\u8981c":55,"\u826f\u597d\u7684\u6587\u6863":54,"\u83b7\u53d6\u6700\u65b0\u7684\u68c0\u67e5\u70b9\u7684\u6587\u4ef6uuid":34,"\u867d\u7136\u4e0d\u9f13\u52b1\u8fd9\u6837":55,"\u89e3\u91ca\u578b\u8bed\u8a00\u53ea\u80fd\u8c03\u7528\u52a8\u6001\u5e93":54,"\u89e3\u91ca\u6027\u8bed\u8a00\u5b9e\u9645\u8fd0\u884c\u7684\u4e8c\u8fdb\u5236\u662f\u89e3\u91ca\u5668\u672c\u8eab":54,"\u8ba1\u7b97\u8fd9\u4e2a\u6587\u4ef6\u7684md5":34,"\u8ba9paddle\u6838\u5fc3\u4e2d":55,"\u8bad\u7ec3\u4efb\u52a1\u7684\u8fd0\u884c\u53ef\u80fd\u4f1a\u5360\u6ee1trainer\u548cparamet":34,"\u8bad\u7ec3\u548c\u7eaf\u4f7f\u7528":71,"\u8bad\u7ec3\u6a21\u578b\u6b63\u786e\u6027":71,"\u8bb0\u5f55\u4e0b\u6240\u6709\u5931\u8d25\u7684\u4f8b\u5b50":71,"\u8bbe\u7f6e":55,"\u8bc6\u522b\u6570\u5b57":71,"\u8bcd\u5411\u91cf":71,"\u8be6\u7ec6\u8bbe\u8ba1":44,"\u8bed\u610f\u89d2\u8272\u6807\u6ce8":71,"\u8bf4\u660e":34,"\u8bf7\u53c2\u8003":55,"\u8f6c\u6362\u751f\u6210\u7684\u6587\u4ef6\u540d\u4f1a\u662f\u4ee5\u4e0b\u683c\u5f0f":35,"\u8fbe\u5230\u5bb9\u707e\u7684\u76ee\u7684":34,"\u8fd4\u56de\u7b2c\u4e8c\u6b65":71,"\u8fd8\u662f\u4ece":35,"\u8fd9\u4e00\u5c42\u8fdb\u884c\u5c01\u88c5":55,"\u8fd9\u4e00\u6982\u5ff5\u4e0d\u518d\u7410\u788e":55,"\u8fd9\u4e09\u4e2a\u5206\u652f":71,"\u8fd9\u4e2a\u52a8\u6001\u5e93\u7684\u8fde\u63a5\u53c2\u6570\u4e0epaddle\u7684\u5176\u4ed6\u4e8c\u8fdb\u5236":55,"\u8fd9\u4e2a\u53c2\u6570\u4e5f\u4e0d\u4f1a\u4e00\u5e76\u5220\u9664":55,"\u8fd9\u4e2a\u5934\u6587\u4ef6\u4e0d\u5047\u8bbe\u5176\u4ed6\u6587\u4ef6\u7684\u5f15\u7528\u987a\u5e8f":55,"\u8fd9\u4e2a\u63a5\u53e3\u9700\u8981\u505a\u5230":54,"\u8fd9\u4e2a\u6587\u4ef6\u5177\u6709\u72ec\u7279\u7684\u8bed\u6cd5":54,"\u8fd9\u4e2a\u76ee\u5f55\u4e2d\u9664\u4e86":55,"\u8fd9\u4e2a\u7ed3\u6784\u4f53\u4e2d\u7684\u53e6\u4e00\u4e2a\u9879\u76ee\u662f":55,"\u8fd9\u4e2a\u7ed3\u6784\u4f53\u5305\u542b\u4e24\u4e2a\u9879\u76ee":55,"\u8fd9\u4e2a\u9759\u6001\u5e93\u5305\u542b\u4e86paddle\u7684\u5168\u90e8\u7b26\u53f7":55,"\u8fd9\u4e2ainstance\u53ef\u4ee5\u662f\u5355\u4e2a\u503c":35,"\u8fd9\u4e9b\u5206\u5e03\u5f0f\u5b58\u50a8\u670d\u52a1\u901a\u5e38\u4f1a\u628a\u6570\u636e\u5207\u5272\u6210\u591a\u4e2a\u5206\u7247\u5206\u5e03\u5f0f\u7684\u5b58\u50a8\u5728\u591a\u4e2a\u8282\u70b9\u4e4b\u4e0a":35,"\u8fd9\u5bf9\u4e8e\u901a\u5e38\u7684java\u7684\u5f00\u53d1\u8005\u6765\u8bf4":54,"\u8fd9\u662f\u56e0\u4e3a":54,"\u8fd9\u6837":55,"\u8fd9\u6837\u4fdd\u8bc1":71,"\u8fd9\u6837\u5c31\u53ef\u4ee5\u5728\u4e91\u7aef\u6267\u884c\u591a\u79cd\u6570\u636e\u7c7b\u8ba1\u7b97\u4efb\u52a1":35,"\u8fd9\u6837\u5df2\u7ecf\u4f20\u8f93\u6210\u529f\u7684\u90e8\u5206\u5c31\u4e0d\u7528\u91cd\u65b0\u4f20\u8f93\u4e86":44,"\u8fd9\u90fd\u9700\u8981\u8fd9\u4e2a\u63a5\u53e3\u6309\u7167\u7ea6\u5b9a\u4fd7\u6210\u7684\u89c4\u5219\u6765\u6ce8\u91ca\u5b8c\u5907":54,"\u8fd9\u91cc\u9700\u8981\u7528\u6237\u989d\u5916\u6ce8\u610f":34,"\u8fdb\u800c\u8fdb\u884c\u4ee3\u7801\u8bc4\u5ba1":71,"\u900f\u4f20\u7528\u6237\u8eab\u4efd\u7684\u529e\u6cd5":44,"\u901a\u5e38":55,"\u901a\u5e38\u6307\u5c06\u4e00\u4e2a\u6574\u4f53\u62c6\u5206\u6210\u591a\u4efd\u7684\u5176\u4e2d\u7684\u4e00\u4efd":34,"\u901a\u8fc7\u6a21\u578b\u63a8\u65adapi\u7684\u5b9e\u73b0\u4f5c\u4e3a\u4e00\u4e2a\u6837\u4f8b":55,"\u903b\u8f91\u5212\u4e0a\u6587\u4ef6\u5206\u5757\u7684\u5355\u4f4d":44,"\u9075\u5faa\u4ee5\u4e0b\u6d41\u7a0b":71,"\u90a3\u4e48":55,"\u90fd\u662f\u4e94\u4f4d\u7684\u6570\u5b57":35,"\u90fd\u662fabi\u8c03\u7528\u6807\u51c6\u7684":54,"\u914d\u7f6e\u7684\u65b9\u6cd5\u53c2\u8003":44,"\u91ca\u653e\u5bf9paramters\u5185\u5b58\u7684\u9501\u5b9a":34,"\u91cc\u6240\u6709\u7684\u7b26\u53f7\u90fd\u5199\u5165\u81ea\u5df1\u7684\u7a0b\u5e8f\u7684\u4e8c\u8fdb\u5236\u6587\u4ef6\u91cc":54,"\u91cd\u547d\u540d\u6210":54,"\u94fe\u63a5\u5230\u81ea\u5df1\u7684\u7a0b\u5e8f\u91cc":54,"\u9519\u8bef\u5904\u7406":54,"\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u662f\u8fd4\u56de\u503c":54,"\u9519\u8bef\u5904\u7406\u7684\u65b9\u5f0f\u4e5f\u4e0d\u5c3d\u76f8\u540c":54,"\u9664\u6784\u9020\u67d0\u79cd\u7c7b\u578b\u7684\u51fd\u6570":55,"\u9700\u8981":35,"\u9700\u8981\u4fee\u6539build":71,"\u9700\u8981\u53ef\u4ee5\u8de8\u5e73\u53f0\u6267\u884c":44,"\u9700\u8981\u5728cmake\u7684\u65f6\u5019":55,"\u9700\u8981\u5c06bugfix\u7684\u5206\u652f\u540c\u65f6merge\u5230":71,"\u9700\u8981\u5f15\u7528":55,"\u9700\u8981\u6709\u7a33\u5b9a\u7684\u5bfc\u51fa\u7b26\u53f7":54,"\u9700\u8981\u6ce8\u610f\u7684\u662f":71,"\u9700\u8981\u88ab\u66b4\u9732\u5230\u5176\u4ed6\u8bed\u8a00":55,"\u9700\u8981\u91cd\u547d\u540dwheel\u5305\u4e2dplatform\u76f8\u5173\u7684\u540e\u7f00":71,"\u9ed8\u8ba4256k":44,"abstract":[59,70,72,86,95],"api\u4e2d\u4f7f\u7528":54,"api\u5bfc\u51fa\u7684\u52a8\u6001\u5e93":55,"api\u5bfc\u51fa\u7684\u9759\u6001\u5e93":55,"api\u63a5\u53d7\u7684\u7c7b\u578b\u5168\u662f":55,"api\u63a5\u53e3":44,"api\u63a5\u53e3\u7684\u53c2\u6570\u8f6c\u53d1\u7ed9":55,"api\u65f6":55,"api\u65f6\u6240\u552f\u4e00\u9700\u8981\u5f15\u5165\u7684\u5934\u6587\u4ef6":55,"api\u662f\u591a\u8bed\u8a00api\u7684\u57fa\u7840\u90e8\u5206":55,"api\u66b4\u9732\u7684\u7c7b\u578b":55,"api\u751f\u6210\u7684\u4e8c\u8fdb\u5236\u6587\u4ef6\u4f1a\u88ab\u5b89\u88c5\u5230":55,"api\u7684\u5b9e\u4f8b":55,"api\u7684\u5b9e\u73b0\u7ec6\u8282":55,"api\u7684\u63a5\u53e3":55,"api\u7684\u65f6\u5019\u63a8\u8350paddle\u4e0d\u5d4c\u5165python\u89e3\u91ca\u5668":55,"api\u7684\u7f16\u8bd1\u9009\u9879\u9ed8\u8ba4\u5173\u95ed":55,"api\u76ee\u5f55\u7ed3\u6784\u5982\u4e0a\u56fe\u8868\u6240\u793a":55,"api\u83b7\u5f97\u4e86\u795e\u7ecf\u7f51\u7edc\u7684\u53c2\u6570\u5b9e\u4f8b":55,"block\u6784\u6210\u4e00\u4e2amodel":34,"book\u4e2d\u6240\u6709\u7ae0\u8282\u529f\u80fd\u7684\u6b63\u786e\u6027":71,"boolean":[21,45,51,54],"break":[13,32,75,111],"bugfix\u5206\u652f\u4e5f\u662f\u5728\u5f00\u53d1\u8005\u81ea\u5df1\u7684fork\u7248\u672c\u5e93\u7ef4\u62a4":71,"bugfix\u5206\u652f\u9700\u8981\u5206\u522b\u7ed9\u4e3b\u7248\u672c\u5e93\u7684":71,"byte":[15,44,53],"c99\u662f\u76ee\u524dc\u6700\u5e7f\u6cdb\u7684\u4f7f\u7528\u6807\u51c6":54,"c\u6709\u6807\u51c6\u7684abi":54,"c\u8bed\u8a00\u662f\u6709\u5bfc\u51fa\u7b26\u53f7\u7684\u6807\u51c6\u7684":54,"case":[8,20,21,36,55,59,62,64,65,68,83,86,87,92,96,98,106,109,111],"char":38,"class":[4,5,6,7,8,9,10,11,13,14,18,20,23,26,28,29,31,42,46,47,49,50,52,54,57,58,61,65,68,69,70,72,73,74,75,88,94],"const":[31,36,38,46,52,60,62,69,72,74,75,86,87,88],"core\u4e2d\u7684\u6a21\u578b\u8fd8\u5728\u4f7f\u7528\u8fd9\u4e2a\u53c2\u6570":55,"core\u4e2d\u8fd9\u4e00\u7c7b\u578b\u63a5\u53e3\u7684\u667a\u80fd\u6307\u9488":55,"core\u662f\u5426\u8fd8\u5728\u4f7f\u7528\u8fd9\u4e2a\u5b9e\u4f8b":55,"core\u6982\u5ff5":55,"data\u5230\u5206\u5e03\u5f0f\u5b58\u50a8\u8865\u5145\u8bad\u7ec3\u6570\u636e":35,"deb\u5305":71,"deb\u5305\u7f16\u8bd1\u95ee\u9898":71,"default":[2,6,7,8,9,10,11,13,14,15,18,21,25,28,29,31,32,48,52,53,56,62,63,72,73,74,76,77,78,80,84,85,87,91,93,95,97,98,99,103,104,106,111],"enum":[36,38,61,73,74,76],"export":[50,78,89,93],"final":[8,9,30,50,56,57,75,86,87],"float":[2,6,7,8,10,13,21,46,52,74,86,87,88,92,97,108,110],"function":[2,4,8,9,13,21,26,29,31,33,37,38,39,41,42,46,49,52,56,57,59,60,61,62,64,65,66,68,69,70,72,74,75,83,86,87,88,91,92,93,95,106,109,111],"golang\u53ef\u4ee5\u4f7f\u7528":54,"golang\u7684":54,"h\u5e76\u4e0d\u56f0\u96be":54,"images\u6570\u636e\u96c6\u4e0a\u4f20\u5230\u4e91\u7aef\u7684":35,"import":[2,4,7,8,29,31,32,48,50,51,56,57,61,66,72,80,81,83,87,92,93,98,104,108,109,110,111],"ingress\u9700\u8981\u628apfsclient\u7684\u8eab\u4efd\u4fe1\u606f\u4f20\u7ed9pfsserv":44,"instance\u4e0e\u751f\u6210\u6570\u636e\u96c6\u65f6":35,"instance\u5305\u6db5\u4e24\u4e2a\u503c":35,"instance\u662f\u4e00\u6a21\u4e00\u6837\u7684":35,"int":[2,6,7,8,9,13,14,15,21,31,36,37,38,41,51,52,54,55,61,63,64,73,74,75,76,86,88,93,97,111],"interface\u6587\u4ef6\u7684\u5199\u6cd5\u975e\u5e38":54,"list\u4f5c\u4e3a\u68c0\u67e5\u5217\u8868":71,"long":[1,8,9,13,21,92],"model\u505a\u5206\u652f\u7ba1\u7406":71,"ndarray\u7c7b\u578b\u7684\u503c\u548c\u6574\u578b\u7684\u503c":35,"new":[2,8,13,30,31,32,33,36,37,38,39,40,46,57,59,63,64,65,67,68,70,74,75,81,84,85,90,98,99,106,109,111],"note\u7684\u4e66\u5199":71,"null":[50,86,95],"org\u76ee\u524d\u9075\u5faa":71,"paddle\u4e00\u4e2a\u52a8\u6001\u5e93\u53ef\u4ee5\u5728\u4efb\u4f55linux\u7cfb\u7edf\u4e0a\u8fd0\u884c":54,"paddle\u5185\u5d4c\u7684python\u89e3\u91ca\u5668\u548c\u5916\u90e8\u4f7f\u7528\u7684python\u5982\u679c\u7248\u672c\u4e0d\u540c":54,"paddle\u5185\u90e8\u7684\u7c7b\u4e3ac":54,"paddle\u7684\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0\u5305\u62ec\u4e00\u4e0b\u51e0\u4e2a\u65b9\u9762":54,"paddle\u7684\u7c7b\u578b\u5168\u90e8\u9000\u5316\u6210":55,"paddle\u7684\u94fe\u63a5\u65b9\u5f0f\u6bd4\u8f83\u590d\u6742":54,"paddle\u7684c":55,"paddle\u8bad\u7ec3\u4efb\u52a1":35,"paddle\u8def\u5f84\u4e0b":55,"paddle\u9700\u8981\u4e00\u4e2a\u591a\u8bed\u8a00\u63a5\u53e3":54,"paddle\u9700\u8981\u66b4\u9732\u7684api\u5f88\u591a":55,"paddle\u9759\u6001\u5e93\u94fe\u63a5\u590d\u6742":54,"paddle_\u7c7b\u578b\u540d":55,"paddle_\u7c7b\u578b\u540d_\u51fd\u6570\u540d":55,"paddlepaddle\u4f7f\u7528git":71,"paddlepaddle\u5f00\u53d1\u8fc7\u7a0b\u4f7f\u7528":71,"paddlepaddle\u63d0\u4f9b\u4e13\u7528\u7684":35,"paddlepaddle\u6bcf\u6b21\u53d1\u65b0\u7684\u7248\u672c":71,"paddlepaddle\u6bcf\u6b21\u53d1\u7248\u672c\u9996\u5148\u8981\u4fdd\u8bc1paddlepaddl":71,"paddlepaddle\u7684\u4e3b\u7248\u672c\u5e93\u9075\u5faa":71,"patch\u53f7":71,"patch\u53f7\u52a0\u4e00":71,"pfsclient\u9700\u8981\u548cingress\u4e4b\u95f4\u505a\u53cc\u5411\u9a8c\u8bc1":44,"pfsclient\u9700\u8981\u5728\u4f20\u8f93\u5b8c\u6bd5\u6700\u540e\u4e00\u4e2achunk\u7684\u65f6\u5019\u68c0\u67e5destination\u6587\u4ef6\u7684md5\u503c\u662f\u5426\u548csource\u6587\u4ef6\u4e00\u81f4":44,"pfsserver\u63d0\u4f9brest":44,"public":[14,18,31,46,47,52,69,72,74,75,86,87,88,93,98,99],"py\u4e2d":71,"pypi\u4e0a\u7684package\u540d\u79f0\u4e3apaddlepaddle\u548cpaddlepaddl":71,"reader\u7684\u4f7f\u7528\u65b9\u5f0f\u90fd\u662f\u4e00\u81f4\u7684":35,"reader\u8f93\u51fa\u7684data":35,"release\u9875\u9762":71,"return":[2,6,7,8,9,11,13,14,15,18,21,23,28,29,30,31,35,36,38,41,42,46,47,48,50,52,56,57,58,61,62,63,65,67,69,72,74,75,83,86,87,88,98,109,110,111],"s3\u4e4b\u7c7b\u7684\u5206\u5e03\u5f0f\u5b58\u50a8\u4e4b\u4e0a":35,"server\u4e4b\u4e0a":34,"server\u4e4b\u95f4\u7684\u7f51\u7edc\u5e26\u5bbd":34,"server\u4f1a\u6682\u505c\u53c2\u6570\u66f4\u65b0\u5e76\u7b49\u5f85":34,"server\u4f1a\u83b7\u53d6parameters\u5185\u5b58\u7684":34,"server\u5185\u5b58\u4e2d\u7684\u6a21\u578b\u6570\u636e\u7684\u5b8c\u6574\u955c\u50cf":34,"server\u540c\u6b65\u7684\u4fdd\u5b58\u4e00\u4e2a\u7279\u5b9a\u65f6\u95f4\u70b9\u7684\u5168\u5c40\u68c0\u67e5\u70b9":34,"server\u5728\u96c6\u7fa4\u4e2d\u542f\u52a8\u540e":34,"server\u6545\u969c\u540e\u88abkubernetes\u91cd\u65b0\u542f\u52a8":34,"server\u6b64\u65f6\u8fd8\u9700\u8981\u901a\u8fc7\u7f51\u7edc\u8bbf\u95ee\u5206\u5e03\u5f0f\u5b58\u50a8\u4ee5\u4fdd\u5b58\u5feb\u7167":34,"server\u751f\u6210\u4e00\u4e2auuid":34,"server\u7684\u5355\u70b9\u6216\u591a\u70b9\u540c\u65f6\u6545\u969c":34,"server\u7684\u6570\u636e\u5feb\u7167":34,"server\u7684\u68c0\u67e5\u70b9\u5404\u81ea\u72ec\u7acb\u4fdd\u5b58":34,"server\u7b2c\u4e00\u6b21\u542f\u52a8\u6216\u4efb\u610f\u65f6\u95f4paramet":34,"short":[8,9,21,46,48,63,72,75,87],"static":[28,38,55,72,74,98,106],"super":[63,86],"swig\u652f\u6301\u7684\u8bed\u8a00\u6216\u8005\u89e3\u91ca\u5668\u6709\u5c40\u9650":54,"swig\u66b4\u9732\u7684\u63a5\u53e3\u4fdd\u7559\u4e86c":54,"swig\u751f\u6210\u7684\u4ee3\u7801\u4e0d\u80fd\u4fdd\u8bc1\u591a\u8bed\u8a00\u4ee3\u7801\u98ce\u683c\u7684\u4e00\u81f4\u6027":54,"swig\u76f4\u63a5\u8bfb\u53d6c":54,"swig\u9700\u8981\u5199\u4e00\u4e2ainterface\u6587\u4ef6":54,"switch":[31,55,67,98],"tag\u4e3a":71,"throw":98,"true":[2,6,7,8,9,10,11,13,15,20,21,28,29,31,36,51,58,61,62,64,71,74,75,83,86,93,95,97,98,110],"try":[32,33,36,37,38,50,64,72,78,80,84,92,109],"type\u5b57\u6bb5\u5747\u4e0d\u5c3d\u76f8\u540c":55,"ubuntu\u5b89\u88c5\u5305\u7684\u529f\u80fd\u6b63\u786e\u6027":71,"var":[31,47,49,51,58,61,62,63,68,72,75,89],"void":[31,36,38,46,47,52,53,54,55,61,62,73,74,86,87,88],"wheel\u5305":71,"while":[1,2,6,13,21,31,40,50,57,59,60,64,70,72,83,87,88,95,109,111],AGE:[98,99],AWS:[13,35,90,100,101],And:[2,7,8,10,13,14,15,28,30,36,40,41,43,48,50,64,72,83,97,98,99,108,110],But:[2,8,9,13,21,30,47,72,80,85,93,106],EOS:8,For:[1,2,7,8,9,10,13,21,28,29,31,37,38,39,41,42,47,49,52,53,56,57,59,60,61,62,63,64,65,67,68,69,70,73,74,76,77,78,81,83,85,86,87,88,91,92,94,95,97,103,104,106,108,110,111],Has:2,IDE:84,IDs:[14,21,40,57,111],IPs:93,IRs:65,Ids:111,Into:98,Its:[2,7,73,83,87,98],K8s:106,NMS:8,NOT:[63,87],Not:[29,33,80,106],ONE:2,OPs:[65,66],One:[7,9,28,30,40,53,67,72,83,86,95,109,111],Ops:[68,70,74,87],PFS:44,PRs:89,QoS:99,Such:[52,63,75],THE:2,TLS:[29,44,98],That:[8,13,78,95,97],The:[1,2,4,5,6,7,8,9,10,13,14,15,18,21,25,28,29,30,32,33,37,39,40,41,42,43,45,46,50,53,55,56,57,59,61,62,63,64,65,66,67,70,72,73,74,75,76,78,80,83,84,85,86,87,88,89,91,92,93,95,97,98,99,103,107,108,109,110,111],Their:[2,8,33],Then:[4,8,9,47,52,62,77,78,80,83,86,91,92,93,98,99,108],There:[7,8,14,21,28,29,31,32,33,38,40,41,45,46,50,56,57,59,60,63,65,70,72,73,87,92,98,103,109,110,111],These:[7,15,31,46,49,58,70,73,74,77,93,97],Use:[2,7,13,29,45,64,68,77,86,91,92,95,96,98,104],Used:[9,18,22,68],Useful:2,Using:[33,59,64,68,70,72,79,84,88,99],VMs:84,VPS:98,WITH:104,Will:[13,28],With:[2,8,9,61,75,109],YES:41,Yes:[78,84],___fc_layer_0__:98,__align__:46,__cuda_align__:46,__device__:46,__doc__:74,__file__:41,__forceinline__:46,__fp16:46,__global__:46,__gradient_machines__:28,__hadd:46,__half:46,__half_raw:46,__impl__:74,__init__:[42,48,58,63,75,86,91],__main__:[48,110],__metaclass__:87,__name__:[48,110],__param_conf__:28,__rnn_step__:83,__tmp_params__:28,__va_args__:69,__x:46,_binari:32,_create_global_var:63,_dtype:50,_error:109,_librari:32,_link:9,_loss:48,_op:[50,87],_proj:8,_res2_1_branch1_bn:110,_source_language_embed:[83,108],_target_language_embed:[83,108],_test:32,_update_op:42,_value_index:50,a75:46,a_op:87,aaaaa:35,aaaaaaaaaaaaa:98,abbrevi:15,abc:8,abi:103,abil:48,abl:[8,29,52,58,63,65,67,76,106,109],about:[4,8,9,15,31,32,41,45,56,64,65,72,74,78,80,81,87,91,92,94,95,98,103,104],abov:[2,4,7,8,21,29,31,32,33,37,46,47,49,56,57,58,59,61,63,65,67,74,75,78,79,81,84,85,87,88,91,92,98,99,103,106,109,110,111],abs:[8,9,20,30,48,109],abs_numerical_grad:30,absolut:[1,78,103,104],absp:8,acceler:[8,34,59,78,97],accept:[2,4,6,8,13,29,68,103,111],access:[1,8,9,21,29,32,37,40,41,63,65,80,83,93],accessmod:98,accessor:63,accord:[1,2,7,8,15,21,30,38,49,57,65,66,68,75,87,93,94,95,97],accordingli:[4,7,8,86],account:[68,85,106],accrodingli:36,accumul:[33,38,42,59],accur:[30,40],accuraci:[7,18,42,86,111],achiev:[59,92],ack:95,across:[8,13,56],act1:50,act2:50,act:[8,9,21,22,31,50,57,63,65,67,75,81,83,111],act_output:74,act_typ:[50,111],action:[65,98],activ:[4,9,21,27,32,50,57,63,65,70,74,81,83,86,88,89,95,111],activi:9,actual:[2,8,36,48,50,59,74,88],adadelta:111,adagrad:[23,59,73,111],adagradoptim:58,adam:[23,29,38,48,111],adamax:[23,111],adamoptim:[108,111],adapt:[7,10,21,28],add:[2,8,9,13,20,21,23,26,28,30,31,32,36,40,42,46,47,51,58,59,62,63,65,66,68,70,72,77,84,85,86,87,88,92,97,104,111],add_activ:63,add_bia:63,add_depend:32,add_execut:32,add_input:[56,86],add_memori:56,add_output:56,add_scalar:[31,57,61],add_sum:63,add_test:[32,86],add_two:[31,56],add_unittest_without_exec:86,addattr:[74,87],addbia:86,addcom:[74,87],added:[2,7,8,18,26,28,31,46,59,66,70,85,86,87],adding:[70,85,110],addinput:[74,87],addit:[8,9,68,70,75,87,88,111],addition:56,addop:[47,88],addoutput:87,addr:33,address:[33,38,65,67,78,92,93,95,106],addrow:86,addtolay:8,addtyp:74,admin:106,administr:[40,84,106],adopt:[46,48],advanc:[30,83,92,95],advantag:[30,46,59,64,93],adversari:[48,64],advic:92,affect:[8,31],affili:57,afford:37,afi:2,aforement:32,after:[8,9,14,15,31,32,37,38,40,45,46,60,63,65,66,77,78,83,85,86,87,88,91,93,95,97,98,99,103,109,110,111],again:[29,33,59,92],against:98,age:14,agg_level:8,aggreg:[42,98],ago:32,aid:92,alexnet_pass1:97,alexnet_pass2:97,algorithm:[8,10,37,57,59,70,83,108],alia:[5,6,8,21],align:[8,9,13],all:[2,6,7,8,18,20,21,28,29,31,32,33,36,38,40,41,43,45,48,49,50,53,55,56,57,58,59,61,63,65,67,68,74,78,83,84,86,88,92,93,94,95,97,98,99,103,104,106,108,109,110,111],alloc:[6,38,41,86,88,97],allow:[29,38,59,65,70,85,86,92,95,98,111],allow_only_one_model_on_one_gpu:[94,95,97],almost:[84,93,108],along:[15,21],alpha:[32,70],alreadi:[32,33,63,65,66,72,78,92,93,95,98,99,104],alreali:94,also:[1,2,8,9,14,21,29,31,32,36,39,46,47,48,49,50,56,57,59,60,61,62,63,64,70,72,74,75,77,78,80,83,84,86,87,89,92,93,99,103,106,109,110,111],altern:[87,91],altogeth:106,alwai:[4,8,9,28,32,53,67,73,95,98],amazon:[98,99,111],amazonaw:98,amazonec2fullaccess:98,amazonelasticfilesystemfullaccess:98,amazonroute53domainsfullaccess:98,amazonroute53fullaccess:98,amazons3fullaccess:98,amazonvpcfullaccess:98,ambigu:64,amd64:98,amend:85,amnih:8,among:98,amount:[21,92],analysi:[91,92],analyz:111,ancestor:[61,63],andd:98,android:[104,105],android_abi:103,android_api:103,android_arm_neon:103,android_native_api_level:103,android_standalone_toolchain:103,android_toolchain:103,angl:8,ani:[1,2,8,9,13,21,23,29,32,33,38,40,41,46,52,53,57,59,63,64,65,66,67,69,70,77,84,87,88,92,93,98,104,111],annoi:93,announc:46,anoth:[2,8,13,28,29,31,41,57,63,72,74,84,95,98],anroid_arm_mod:103,ans:98,answer:[85,98],anyth:[13,57,64,98],anytim:48,anywai:[91,103],api:[14,18,28,29,32,38,39,41,42,44,47,48,50,56,60,68,71,75,76,77,80,86,87,91,92,93,98,102,103,104,106,107,109,111],api_shar:32,api_test:32,api_trainer_config_helpers_lay:83,apiserv:98,apivers:[98,99],appear:49,append:[2,13,21,28,42,57,63,64,83,85,86,93],append_backward:91,append_backward_op:[23,58],append_batch_s:21,append_gradient_machin:28,append_op:63,append_oper:63,appleyard:92,appli:[8,21,48,49,72,83,86,111],applic:[25,46,49,63,68,85,87,91,92,93,98,99,106],applyl1:36,appoint:87,appreci:85,approach:[8,59,60,65,66,70,103,104,106],appropri:65,approxim:[20,59],apt:[78,91],arang:21,arbitrari:[8,53,65,88],arch:103,architectur:[46,77,93,103,108],archiv:[14,54,55],area:48,arg:[2,7,9,25,50,58,74,87,94,109,110,111],arg_nam:8,argu:62,argument:[2,4,8,13,15,25,31,36,37,58,60,62,63,67,77,83,85,86,95,96,108,109,110],argv:110,arithmet:46,arm64:103,arm64_standalone_toolchain:103,arm:[46,103,104],arm_standalone_toolchain:103,armeabi:103,armv7:46,armv8:46,arn:98,around:[2,8,40,63,98,106],arrai:[4,6,8,13,15,21,28,38,49,57,61,63,64,68,75,87,110],arrang:75,arrow:48,articl:[49,84,93,99,104],artifact:[80,98],artifici:[20,109],arxiv:[8,9,20,48,109],as_row_vector:8,as_step_input:31,asgd:59,ask:[33,40,84],assign:[7,8,37,46,88,93,95,98,106],associ:[60,69,88],assum:[7,8,31,65,78,83,97,108],assumpt:65,assur:1,astyp:[64,87,109],asyc:33,async:[33,94],async_count:95,async_lagged_grad_discard_ratio:95,async_lagged_ratio_default:[94,95],async_lagged_ratio_min:[94,95],asynchron:[33,93,95],att_seq:9,attach:9,attend:9,attended_sequ:9,attenion:9,attent:[8,9],attr1:8,attr2:8,attr:[6,8,9,21,31,50,61,62,63,74,83,87],attr_map:74,attrdesc:61,attribut:[2,8,9,21,27,31,61,66,68,72,74,75,86,87,108],attributemap:87,attrproto:74,attrtyp:[61,74,87],attrvalu:74,auc:[42,94],aucvalidationlay:95,authent:98,author:[44,98,110],auto:[31,36,54,62,68,72,75,77,85,86,87,88,92],autom:[93,98],automat:[8,29,38,47,58,65,66,68,74,77,83,85,86,87,91,93,94,95,98],avail:[33,38,46,66,67,80,98,106],averag:[7,8,11,28,37,95,110,111],average_test_period:[94,95],avg:[92,111],avgcost:111,avgpool:[8,111],avoid:[30,31,33,59,60,65,77,92],avx2:77,avx:[77,78],await:99,awar:[29,42,56,63,84,91,98,103],awk:93,aws:44,aws_account_id:98,awsaccountid:98,awskeymanagementservicepowerus:98,axes:21,axi:[8,21],axis:8,aync:66,b2t:108,b363:99,b8561f5c79193550d64fa47418a9e67ebdd71546186e840f88de5026b8097465:99,ba5f:98,back:[2,8,21,28,33,46,48,59,65,78,87],background:[7,8,70,93],background_id:[7,8],backpropag:30,backward:[5,8,9,26,30,31,36,38,48,58,59,60,62,69,70,83,86,95,97],backward_first:83,backward_op:30,backwardactiv:86,bag:111,baidu:[99,108],baik:108,bake:65,balanc:[66,95,98,109],bandwidth:46,bare:[99,106],barrier:[93,95],barrierstatset:92,basci:50,base:[7,8,11,13,14,18,20,21,23,26,29,37,42,46,52,58,59,65,68,69,70,75,83,84,86,87,91,92,95,98,103,108,109,111],baseactiv:9,baseev:28,basematrix:86,basenam:7,basepoolingtyp:[8,9],basestr:[6,7,8,9,11,28],bash:[77,78,84,93,98,99],basic:[2,8,28,50,61,65,68,69,75,86,111],batch:[2,8,9,13,15,18,21,28,29,31,33,35,36,42,48,51,56,57,59,75,85,86,95,98,99,109,110,111],batch_0:110,batch_id:[28,48],batch_im:48,batch_images_from_tar:15,batch_label:48,batch_norm:48,batch_norm_lay:9,batch_norm_typ:8,batch_read:[35,64],batch_siz:[2,13,48,57,108,109,111],batch_szi:48,batch_z:48,batchnorm:[21,48],batchsiz:[8,86],bazel:32,bbbbb:35,bbox:[7,8],bcd:8,bcebo:14,bcm2708:104,bdist_wheel:71,beacus:50,beam:[8,83,95],beam_gen:[8,83],beam_search:[28,57,83],beam_siz:[8,57,83,94,95,97],becaus:[4,7,8,14,29,31,32,33,38,46,57,60,63,64,70,72,73,75,76,83,84,86,88,91,97,98,103,111],becom:[66,72,92],been:[2,8,9,32,37,84,85,111],befor:[4,8,9,33,40,45,49,59,60,64,70,77,78,85,87,91,98,103,106],begin:[4,7,8,18,36,38,42,45,49,57,86,93],beginiter:[28,29],beginn:83,beginpass:[28,29],begintrain:29,behavior:92,behind:75,being:[21,40,62,64,88,91,109],belong:[7,8,65,72],below:[2,8,31,33,38,46,53,60,64,65,66,70,75,76,77,80,83,86,92,93,98,103,109,111],benchmark:53,benefit:[9,40,41,57],bengio:20,besid:[1,8,14,65,80],best:[32,78,84,95,111],besteffort:99,beta1:[10,23],beta2:[10,23],beta:[48,110],better:[9,32,57,98,106,109],between:[7,8,15,21,28,32,33,38,46,55,60,65,66,67,69,72,78,87,88,98,109,111],bgr:[15,110],bi_gru:9,bi_lstm:9,bia:[8,9,21,57,63,83,86,110],bias:[8,86],bias_attr:[8,9,21,63,83],bias_initi:21,bias_param_attr:9,biases_:86,biasparameter_:86,biassiz:86,bidi:99,bidirect:[8,9,83],big:[66,92,106],bigger:33,bilinear:8,bilinear_interpol:8,bilinearfwdbwd:92,bin:[78,93,98,99],binari:[2,7,8,13,32,41,46,48,53,65,77,79,80,84,91,92,98,108,111],bind:[46,47,72],bit:[46,111],bla:78,black:48,blank:[8,98],block:[8,34,36,38,42,43,52,56,58,65,66,86,87,88,92,95,110],block_i:8,block_id:43,block_x:8,blockdesc:[31,49,63,68],blockdescbind:52,blueprint:57,bn_bias_attr:9,bn_layer_attr:9,bn_param_attr:9,book:[14,68,83,89,107],bool:[2,6,7,8,9,10,11,13,15,28,31,46,51,62,63,73,74,75,76,86,95,97,111],boot:[8,83,106],boot_bia:8,boot_bias_active_typ:8,boot_lay:83,boot_stat:75,boot_with_const_id:8,bootstrapp:106,borrow:[48,75],bos_id:[8,83],both:[5,6,8,9,15,21,29,31,32,33,40,46,48,52,57,62,65,66,73,83,86,87,88,89,92,93,98,109,110,111],bottleneck:[92,110],bottom:28,bound:8,boundari:65,boundri:7,bow:111,box:[8,48,92],brace:[31,49],brain:40,branch:[8,29,31,32,51,61,65,71,80,85,87,89],breadth:95,break_if:75,brief:[32,38,46,88],briefli:92,bring:88,broadcast:[21,33,68,106],broken:85,browser:[78,89,91,98],bsd:84,bucket_nam:98,buddy_alloc:85,buf:36,buf_siz:13,buffer:[2,13,36,53,59,64,72,95],buffer_s:13,buffered_read:64,bufsiz:13,bug:[85,98],build:[8,14,32,41,49,50,59,65,70,71,74,78,80,81,85,87,91,93,95,98,100,101,105,108,110,111],build_android:103,build_dict:14,build_model:48,builder:85,built:[32,46,65,74,75,77,80,84,91,103,104,106,109],bunch:[53,92,93,111],button:[85,89,98],c11:54,c703c041:85,c99:55,c99e:98,c_pre_init:21,cach:[46,65,77,111],cache_pass_in_mem:[2,111],cachetyp:[2,111],cacul:[9,42,93],caff:31,caffe2:31,calc_batch_s:2,calcul:[2,7,8,9,18,21,30,33,38,42,46,78,83,86,92,93,95,97,109],call:[2,7,8,9,13,21,28,29,30,31,36,37,38,39,41,48,49,56,57,58,63,65,68,69,72,74,75,81,83,84,86,87,88,91,92,93,95,98,110,111],callabl:[2,6,8,13,14],callback:86,caller:[30,91,98],can:[1,2,4,6,7,8,9,13,14,15,21,25,28,29,30,31,32,33,36,37,40,41,46,47,48,49,50,52,56,57,58,59,61,62,63,64,65,66,67,68,69,70,74,75,77,78,79,80,81,83,84,85,86,87,88,89,91,92,93,94,95,97,98,99,103,104,106,108,109,110,111],can_over_batch_s:2,cancel:40,candid:[8,57],candidate_activ:21,cannot:[67,68,72,75,86,87],cantain:50,capabl:[46,60,68],capac:[70,98],capi:[54,77],capi_prvi:55,caption:57,captur:[8,93],card:93,care:[9,41,64,78,94,95,106],carefulli:[95,110],caret:28,carpedm20:48,carri:21,cast:46,cast_to_op_attr:74,cat:[13,15,78,93,110],categori:[8,14,33,111],categorig:14,categoryfil:99,caus:[33,45,80,87],caution:[98,99],cburg:8,cc_:32,cc_binari:32,cc_test:32,cclient:39,cde:8,cdn:14,ceil:8,ceil_mod:8,cell:[8,9,21],cell_activ:21,center:[2,15],center_crop:15,cento:[77,80,81,106],central:70,ceph:[13,35,99],cephf:[35,41,44],certain:[1,58,72,94],certif:[29,44,98],cffi:54,cfg:99,cgo:54,chain:[13,49,86],challeng:[8,33,51],chanc:[29,46,86,111],chang:[8,14,32,37,41,60,61,64,69,72,77,83,85,86,87,88,92,93,95,98,103,111],channel:[8,9,15,21,92,110],channel_shar:8,channl:110,chapter:[56,57],chapter_data:56,chapter_out:56,charact:111,characterist:97,check:[2,13,31,32,62,68,77,80,85,87,89,95,97,98],check_align:13,check_attr:74,check_eq:86,check_fail_continu:2,check_grad:[30,87],check_l:86,check_sparse_distribution_batch:[94,95],check_sparse_distribution_in_pserv:[94,95],check_sparse_distribution_ratio:[94,95],check_sparse_distribution_unbalance_degre:[94,95],check_styl:85,checker:68,checkgrad:95,checkgrad_ep:95,checkmark:106,checkout:85,checkpoint:[62,66],checksum:44,child:31,china:78,chines:89,chmod:98,choic:[32,78],choos:[43,77,78,79,84,95,103,111],chosen:[1,48],chunk:[37,44,109],chunk_schem:7,chunktyp:7,chw:15,cifar:109,circl:49,claim:98,claimnam:98,clang:[46,54,85,103],clarifi:7,clariti:57,classfic:110,classic:8,classif:[2,4,8,20,49,97,110,111],classifi:[8,48,109,110,111],classification_cost:[65,111],classification_error_evalu:[7,109,111],classification_evalu:7,claster:98,clean:[4,31,32,60,68,77,84,85],clear:[7,32,57,60,72,104],clearer:[60,63],clearli:72,cli:98,click:[80,85,89,91,92,98],client:[36,39,68],clip:[6,9,95,111],clock:8,clone:[8,77,84,85,89,91,103,104],close:[2,64,67,85],cloud:[32,33,41,44,45,67,68,106],cloud_read:13,cls:111,cludform:98,cluster:[13,28,29,31,33,38,65,67,94,95,99,111],cluster_test_fil:93,cluster_train:93,cluster_train_fil:93,cluster_train_v2:93,cm469:98,cmake:[55,77,84,85,86,87,89,91,92,103],cmake_build_typ:[91,103,104],cmake_install_prefix:103,cmake_system_nam:[103,104],cmakelist:[32,86],cmatrix:[54,55],cmd:[13,99],cname:98,cnn:[8,99,110,111],coars:47,code:[2,4,8,13,29,32,40,43,46,47,48,49,53,58,59,60,62,64,65,66,68,69,70,74,75,77,78,80,81,83,84,86,87,88,89,90,92,93,98,99,109,111],codebas:[68,85],coeff:8,collabor:33,collect:[8,14,28],collectbia:86,color:[15,110],colour:14,column:[7,8,21,49,64,67,86,91,108],column_evalu:7,com:[8,9,14,32,48,77,84,85,89,91,98,99,103,104,106,110],combin:[7,8,9,13,23,28,58,68,72,109],come:[21,42,61,65,75],comma:[25,28,38,95,108],command:[1,4,13,25,32,36,41,45,77,78,80,81,84,85,86,87,89,90,91,92,98,99,100,101,103,104,108,109,110],commandlin:92,commenc:111,comment:[32,50,74,85,87,111],commit:[32,85],common:[15,20,23,26,35,70,83,86,94],commonli:[45,70,83,91,92,97,104],commun:[33,38,39,65,66,85,86,93,98],compar:[30,32,68,84,86,87,109,111],comparison:32,compat:[2,46,47],competit:109,compil:[8,32,50,52,65,69,73,74,76,84,86,89,93],complaint:32,complet:[4,8,9,14,23,28,31,33,37,38,44,49,53,68,79,86,87,88,91,98,99,106,111],complex:[1,2,9,40,57,68,83,92,111],complic:[8,47,64,65,75],compon:[50,65,75,86],compos:[13,29,47,50,56,63,68,109],composenotalign:13,composit:47,compress:37,compromis:84,comput:[8,9,21,25,29,30,33,46,50,53,58,59,65,66,67,69,72,76,78,83,84,85,86,87,91,92,93,97,98,103,104,111],computation:[8,83],computationgraph:50,concat:[48,83],concaten:[8,9,48,56,75],concentr:68,concept:[2,7,29,47,48,50,56,57,59,60,61,67,72,75,76,83],conceptu:[48,50],concern:[29,42],concis:[48,75],conckerneltrac:25,conclud:87,concret:[68,87],concurr:[33,40,66,93],concurrentremoteparameterupdat:95,cond:[21,31,51,61],condit:[8,37,51,65,83,99],condtion:48,conduct:92,conf:[4,8,93,108,109,110],conf_paddle_gradient_num:98,conf_paddle_n:98,conf_paddle_port:98,conf_paddle_ports_num:98,conf_paddle_ports_num_spars:98,confer:20,confid:8,confidence_threshold:8,config:[2,6,8,25,35,45,57,86,94,95,98,99,106,108,109,110,111],config_:[36,95],config_arg:[94,95,97,110,111],config_bas:[7,8,28],config_lay:86,config_len:38,config_pars:[4,86],config_proto:38,configur:[0,1,2,4,8,21,28,36,38,40,41,50,63,66,81,82,84,85,86,87,88,92,93,95,103,104,106,108,110],confirm:45,conflict:[72,85],confus:[15,48],congest:95,conll:14,connect:[1,9,21,41,65,66,86,93,98,99,106,109,110,111],connectionist:8,consequ:[8,9],consid:[7,8,20,62,84,92,97,106],consider:[2,8,9],consist:[7,8,14,15,37,53,61,64,68,69,74,87,110,111],consol:[92,98],consolid:[31,89],const_cast:87,constant:[8,20,21,50,52,67,86,87],constantiniti:21,constraint:[65,72],construct:[2,4,7,29,50,56,63,68,72,74,76,83],constructbackwardgraph:49,constructoptimizationgraph:49,constructor:[21,46,63,68,72,74,86,87],consum:[33,91],contact:40,contain:[2,7,8,9,11,13,14,15,21,28,29,31,37,43,48,50,53,60,63,67,68,69,72,73,74,75,76,79,80,83,84,87,93,98,110,111],container:93,containerport:98,content:[38,45,53,57,89,99],content_dir:89,content_len:38,context:[8,9,14,72,73,83,87,88,108,111],context_attr:9,context_len:[8,9,111],context_proj_layer_nam:9,context_proj_param_attr:9,context_project:9,context_start:[8,9,111],contin:98,continu:[2,7,33,53,93,95,103],contrast:8,contrib:70,contribut:[70,84,90],contributor:68,control:[6,31,67,95,98,99,106],conv2d:48,conv:[9,21,48],conv_act:[9,22],conv_batchnorm_drop_r:[9,22],conv_bias_attr:9,conv_filter_s:[9,22],conv_layer_attr:9,conv_num_filt:[9,22],conv_op:8,conv_pad:[9,22],conv_param_attr:9,conv_strid:9,conv_with_batchnorm:[9,22],conveni:[29,50,58,74],convent:[38,85,87],converg:[93,109],convers:[46,65],convert:[2,4,14,21,35,46,64,69,108,110,111],convlay:8,convolut:[8,9,13,21,22,48,63,109,110],convoper:8,convproject:8,convtranslay:8,convtransproject:8,cool:[2,85],coordin:[33,38],copi:[28,29,37,40,45,49,56,57,59,75,77,85,93,98,109],copy_shared_paramet:109,copytonumpymat:109,core:[2,6,18,50,55,59,60,75,84,95],coreo:[98,106],corner:68,corpu:14,correct:[2,8,21,30,46,86,87,98],correctli:[7,13,46,48,86,109],corresond:46,correspoind:29,correspond:[2,4,8,21,26,29,31,32,46,50,51,56,57,63,67,68,69,70,74,86,87,88,91],corss_entropi:29,cortex:46,cos:[8,74],cosin:[8,21,74],cosineop:74,cosineopproto:74,cosineopprotomak:74,cost:[4,21,28,29,49,58,61,62,65,67,95,109,111],cost_id:8,cost_np:62,cost_val:65,could:[2,4,8,13,28,29,30,37,46,56,58,59,60,61,63,64,65,66,69,84,91,92,93,98,103,111],count:[7,33,41,42,62,64,92,93,95,97,99,108],counter:[25,33,37,49],cours:[7,41,84],covari:8,cover:88,cp27:80,cp27m:80,cp27mu:80,cpickl:110,cpp:[30,36,47,54,55,60,66,68,76,86,92,111],cprofil:91,cprofilev:91,cpu:[1,2,6,8,30,41,46,59,60,65,67,68,70,71,77,78,84,87,88,91,92,95,99,109],cpu_avx_mkl:80,cpu_avx_openbla:80,cpu_per_p:67,cpu_per_train:67,cpuinfo:78,cpuplac:[87,88],cpusparsematrix:55,crash:[33,92,93,95],creat:[4,6,8,13,18,21,28,29,30,31,33,38,42,44,45,46,47,48,49,56,58,59,60,63,69,70,78,81,84,85,86,87,89,93,95,103,106,108,109],create_backward_pass:58,create_bias_paramet:86,create_block:63,create_cloud_job:67,create_doc_str:74,create_input_paramet:86,create_op:87,create_oper:47,create_optimization_pass:[23,58],create_paramet:63,create_python_ops_creatation_funct:74,create_rnn:31,create_rnn_op:56,create_st:18,create_tmp_var:63,create_tmp_vari:63,create_var:63,create_whileloop:75,createargu:109,createfromconfigproto:[4,109],creategradientoper:69,createop:74,createoper:31,createstack:98,createvari:31,creation:[47,98],creationd:98,creator:[13,14,35,68,69],creator_:69,credenti:45,credit:109,critic:[48,91],crlf:85,crop:[15,110],crop_siz:[15,110],crope:15,cross:[8,63,87,111],cross_compil:104,cross_entropi:[8,29,48,67,109],cross_entropy_with_selfnorm:8,crt:44,csc:86,csr:86,csv:25,ctc:7,ctc_evalu:7,ctest:[77,84,87],ctor:63,ctrl:[84,93],ctx:[87,88],cuda7:80,cuda8:[77,80],cuda:[25,32,68,78,80,84,92,93,95],cuda_dir:[94,95],cuda_fp16:46,cuda_profil:25,cuda_so:78,cudaconfigurecal:92,cudadevicegetattribut:92,cudaeventcr:92,cudaeventcreatewithflag:92,cudafre:92,cudagetdevic:92,cudagetdevicecount:92,cudagetdeviceproperti:92,cudagetlasterror:92,cudahostalloc:92,cudalaunch:92,cudamalloc:92,cudamemcpi:92,cudaprofilerstart:92,cudaprofilerstop:92,cudaruntimegetvers:92,cudasetdevic:92,cudasetupargu:92,cudastreamcr:92,cudastreamcreatewithflag:92,cudastreamsynchron:92,cudeviceget:92,cudevicegetattribut:92,cudevicegetcount:92,cudevicegetnam:92,cudevicetotalmem:92,cudnn:[8,11,32,95],cudnn_batch_norm:8,cudnn_conv:8,cudnn_conv_workspace_limit_in_mb:[94,95],cudnn_convt:8,cudnn_dir:[94,95],cudnnavginclpadpool:8,cudnnavgpool:8,cudnnv5:77,cudrivergetvers:92,cuinit:92,cumtim:91,cumul:8,cur_mem:57,curl:[13,98],curli:[31,49],current:[2,8,31,32,33,36,38,42,56,57,59,60,63,66,67,72,75,78,80,83,84,86,89,93,95,98,107,111],current_block:[61,63],current_oper:61,current_word:83,currentcost:111,currentev:111,curv:29,custom:[1,2,23,29,41,46,48,57,59,68,86,98],custom_batch_read:64,cut:[66,75],cut_lin:13,cutoff:14,cv2:15,cxx_compil:[103,104],cxxabi_1:80,cycl:33,cyclic:8,cython:54,d3e0:98,d_b0:48,d_b1:48,d_b2:48,d_block:48,d_f:48,d_g:48,d_h0:48,d_h0_bn:48,d_h0_relu:48,d_h1:48,d_h1_bn:48,d_h1_relu:48,d_h2:48,d_loss:48,d_loss_fak:48,d_loss_real:48,d_optim:48,d_step:48,d_t:48,d_w0:48,d_w1:48,d_w2:48,daili:85,dalla:2,dandroid_abi:103,dandroid_arm_mod:103,dandroid_arm_neon:103,dandroid_standalone_toolchain:103,danger:2,dangl:84,darwin:98,dash:48,dat:35,data:[0,1,2,4,7,14,15,18,28,29,30,31,35,36,37,42,44,46,48,49,50,52,53,56,57,58,59,60,61,63,66,67,68,70,72,73,74,75,76,81,83,86,87,88,92,93,94,95,97,100,110],data_batch_gen:109,data_dir:108,data_fil:15,data_i:48,data_initialz:111,data_lay:[2,36,109,111],data_layout:21,data_read:[13,64],data_reader_creator_random_imag:64,data_shar:75,data_sourc:109,data_typ:[13,14,53,73,76,81,83],data_x:48,databas:14,datacent:[35,45],datacenter1:35,datacenter2:35,datacenter_1:35,datacenter_2:35,datacenter_nam:35,datadim:8,datafeed:16,dataflow:50,dataprovid:1,dataprovider_bow:111,dataprovider_emb:111,dataproviderconvert:4,datasci:8,dataset:[0,2,35,41,59,64,81,83,91,95,108,110,111],dataset_nam:15,datatyp:[14,18,73,76],date:93,dcgan:[48,109],dcmake_install_prefix:[103,104],dcmake_system_nam:[103,104],dcuda_arch_nam:77,dcudnn_root:77,ddim:88,dead:33,deal:[106,109],debug:[2,30,45,65,78,85,91],debug_str:50,decai:[10,23,26],decar:13,decayr:36,decent:37,decid:[29,40,48,53,59,69,70,73,107],declar:[8,21,31,48,56],declear:21,decod:[8,9,83],decoder_boot:83,decoder_dim:57,decoder_group_nam:83,decoder_input:[57,83],decoder_mem:[57,83],decoder_prev:9,decoder_s:83,decoder_st:[9,83],deconv:[8,48],deconvolut:[8,21],decor:[2,13,86],decrypt:98,deduc:68,deep:[8,20,40,48,49,68,70,78,92,109,110,111],deeper:[78,110],def:[2,8,13,29,30,35,41,42,47,48,50,56,57,58,63,64,74,75,83,86,87,109,110,111],def_block:48,defalut:[95,97],default_block:48,default_devic:97,default_main_program:18,default_param_attr:63,default_st:75,default_startup_program:18,default_valu:97,defaultinfervartyp:52,defect:60,defer:40,defferenct:2,defin:[1,2,8,9,13,20,23,26,28,29,31,32,33,40,46,47,48,50,56,61,63,64,66,68,72,74,75,83,86,88,91,93,95,108,109],define_py_data_sources2:[2,110,111],definit:[2,31,33,37,43,61,65,69,74,75,78,87,91,108,111],definiton:47,degre:8,delai:[59,88,95],delar:111,delet:[41,44,85,107],deletestack:98,delimit:7,deliv:106,delta:[8,30],delv:[8,20],demand:33,demo:[8,14,68,93,99,100,108,109,110,111],demolish:99,demonstr:[83,88,109],denot:[87,97,111],dens:[2,8,13,38,39,73,86,98,111],dense_arrai:13,dense_vector:[2,4,13,81],dense_vector_sequ:13,dep:32,depend:[31,32,33,41,43,50,62,65,66,73,78,84,87,93,97,103,104,106],dependent_var:62,deploi:[8,93,97,106],deploy:[50,53,68,93,98,106],deprec:8,depth:31,dequeu:66,deriv:[5,29,51,58,103],desc:[31,53,63,74,75],desc_:31,descend:75,descent:[8,33,59,93],descproto:53,describ:[29,31,32,37,53,56,57,60,61,63,68,73,74,76,86,87,88,98,99,109,111],describestack:98,describestackev:98,describestackresourc:98,descript:[4,7,31,32,52,53,69,73,76,77,80,85,87,93,96,98],deseri:[28,53,60],deserializ:68,desgin:49,design:[2,8,13,20,36,54,59,70,87,106],desir:[13,33,59,98,99,108],destin:[38,45],destroi:31,destruct:72,destructor:86,det_output:7,detail:[2,4,6,7,8,9,10,21,30,37,41,45,48,50,53,56,63,65,67,70,72,75,76,77,81,83,84,86,87,88,89,91,92,93,96,97,98,99,104,106,108,109,110,111],detect:[52,77,85,103],detection_evalu:7,detection_output:7,determin:[2,8,13,31,68,86,109],dev:[68,78,84,91,103,106],dev_ctx:31,develop:[32,52,60,63,69,71,78,80,85,88,89,91,93,94,95,104],deverlop:95,deviat:[6,20],devic:[6,42,43,46,50,60,65,67,68,78,87,88,95],device_context:87,device_context_:87,devicecontext:[31,87],deviceid:97,devid:[8,95],devtools2:77,dhcp:106,diagnos:93,diagram:[56,93,110],diamond:48,dic:15,dict:[2,7,14,28,63,67,93,111],dict_fil:[7,111],dict_siz:[14,36,57],dictionai:111,dictionari:[2,7,8,14,28,29,30,63,67,97,110,111],did:[2,60,78],diff_mat:30,differ:[2,7,8,28,31,32,33,38,40,42,43,46,48,50,51,57,59,62,65,66,67,69,72,75,78,83,86,87,88,91,93,95,98,99,108,110,111],differenti:[47,87],difficult:[7,30,84],difficulti:20,dig:[78,92,98],digit:[2,8,93],digraph:50,dilat:8,dilation_i:8,dim0:87,dim1:87,dim:[8,13,36,53,56,68,73,76,86,87,88,108,110,111],dim_:88,dimens:[5,8,9,11,13,21,48,68,73,75,86,87,88,97,108,111],dimension:[2,8,21,83,86,88,109,111],dimenst:108,dimes:8,dir:[103,110,111],direct:[8,9,15,59,91,110],directli:[1,2,9,20,23,26,32,39,41,46,60,65,74,75,79,93,99],director:87,directori:[1,8,32,35,40,44,45,77,78,84,88,89,92,93,95,99,103,104,107,110,111],diretcoti:110,dis_conf:109,dis_train:109,dis_training_machin:109,disabl:2,disadvantag:[59,63],discard:[13,33,37,57,95],discount:8,discov:33,discoveri:98,discrep:92,discret:8,discrim:48,discrimin:109,discriminator_train:109,discuss:[29,31,37,38,39,65],disk:[53,84,99],dispatch:[60,65,93,95],displai:[41,45,85],dist:[71,77],dist_train:[29,41],distanc:[7,8],distibut:108,distinguish:[32,93,109],distribut:[8,20,31,37,38,39,40,42,66,67,68,76,80,90,99,100,101,106,109,111],distribute_test:[94,95],distributedli:86,disucss:29,div:21,divid:[8,10,42,43,74,76,91,94],diy_beam_search_prob_so:[94,95],django:89,dnn:77,dns:98,do_forward_backward:64,doc:[4,13,50,56,75,87,88,89,93],doc_cn:89,dockefil:84,docker:[71,77,79,85,89,93,98,100,101,106],docker_build:29,docker_clust:93,docker_push:29,dockerfil:[84,103,104],dockerhub:78,document:[2,4,8,9,30,44,49,56,57,65,68,77,84,85,87,88,90,93,97,111],documentari:2,doe:[2,4,9,33,37,38,40,41,46,50,56,60,63,65,66,68,69,70,80,84,86,87,88,92,111],doesn:[6,8,13,29,31,64,67,78,84,85,91,92,99,103],dog:110,doing:[36,40,49,65,92],domain:98,don:[9,29,32,47,49,64,77,78,84,85,87,89,98],done:[7,8,9,32,33,37,38,52,53,59,65,69,70,85,91,92,98,109],dot:[8,9,87,95,110],dot_period:[95,97,109],dotmuloper:8,dotmulproject:8,doubl:[2,46,49,65,77,87,95],down:[92,111],download:[14,32,33,36,40,44,77,78,80,93,106,109,111],dozen:32,draw:57,drive:72,driver:[78,93],drop:[2,8,9,21,57],drop_rat:6,dropout:[6,9,86,111],dropout_prob:21,dropout_r:8,drpi_arm_neon:104,drpi_toolchain:104,drwxr:99,dst:38,dtoh:92,dtype:[4,18,21,50,63,110],due:[37,40,48,57,63,91],dummi:[28,37],dump:53,duplic:[21,66],durat:[37,92],dure:[1,2,8,9,21,26,31,33,37,40,41,42,59,68,86,87,94,95,98,106,111],durn:2,duse_eigen_for_bla:103,dwith_c_api:[55,103,104],dwith_gpu:[77,104],dwith_profil:92,dwith_python:[55,104],dwith_swig_pi:[55,103,104],dwith_test:[77,87],dwith_tim:92,dynam:[1,2,38,55,56,64,77,92,95],dynamic_cast:86,dynamic_recurrent_op:75,e2e:106,each:[1,2,4,7,8,9,11,13,14,18,21,28,30,32,33,36,37,38,40,41,42,43,49,52,56,57,60,62,63,64,65,66,68,69,72,73,74,75,83,86,91,93,95,97,98,106,108,110,111],each_feature_vector:5,each_pixel_str:2,each_time_step_output:5,each_word:2,earli:[46,85,87],eas:[13,52,87,110],easi:[30,57,59,64,68,70,85,86,93,111],easier:[29,46,64,66,75,84,85,86],easili:[29,48,64,69,72],echo:78,edg:[15,67],edit:[7,78,84,98],editor:[63,84],edu:[8,14,98,99],eeoi3ezpr86c:98,effect:[2,8,28,77,95,98],effici:[1,2,8,53,64,65,83,84,86],effort:65,efg:8,efs:98,efs_dns_nam:98,efsvol:98,eigen:[46,59,68,70,87,103],eigen_test:88,eigen_use_gpu:87,eigenmatrix:88,eigenscalar:88,eigentensor:88,eigenvector:88,either:[8,9,13,28,29,48,51,52,56,59,65,70,79,92,111],elb:98,elbapis:98,elec:111,electron:[99,111],elem_dim:8,element:[2,4,7,8,9,13,15,21,28,30,37,50,57,66,67,68,87,88,111],element_typ:38,elementari:68,elementwis:21,elif:[29,74],els:[29,36,41,48,51,52,65,66,72,74,77,78,84,86,87,110,111],emac:84,email:85,emailweixu:32,emb1:36,emb2:36,emb:[99,111],embed:[29,31,36,52,57,66,73,75,83,93],embedding_lay:[36,111],embedding_nam:[8,83],embedding_s:[8,83],emphas:92,empir:8,emplace_back:86,emploi:[74,83],empti:[7,13,33,57,87],emul:46,enabl:[2,6,8,31,32,37,50,66,84,85,92,93,95,98],enable_grad_shar:[94,95],enable_parallel_vector:95,enableonstart:25,enc_proj:[9,83],enc_seq:9,enc_vec:83,encapsul:[38,67],encod:[9,37,57,83],encoded_proj:[9,83],encoded_sequ:[9,83],encoded_vector:83,encoder_ctx:57,encoder_ctx_expand:57,encoder_ctx_proj:57,encoder_dim:57,encoder_last:8,encoder_out_seq:57,encoder_s:83,encount:36,encourag:65,encrypt:98,encrypt_decrypt:98,end2end:106,end:[2,7,8,28,31,50,57,60,64,72,80,83,84,85,95,108],end_pass:29,end_po:8,endforwardbackward:28,endian:53,enditer:[28,29],endpass:[28,29],endpoint:[13,35,98],endtrain:29,engin:[41,92],english:[2,8,89],enjoi:78,enlarg:87,enough:[31,84],enqueu:66,ensembl:9,ensur:[2,33,72,78,80,84,86],enter:31,enterpris:68,entir:[8,9,38,40,87],entiti:[7,31,72],entranc:43,entri:[13,37,41,52,84,85,86,98,103],entropi:[8,63,111],entry_point:41,enueu:66,enumer:[5,111],env:[89,91,98],environ:[29,67,77,80,84,85,91,92,93,94,95,98,99,109],environmenterror:93,eos_id:[8,83],epoch:48,epsilon:[8,10,21,23],equal:[8,9,21,33,75,87,95],equat:[7,8,9,10,21,87],equilibrium:109,equip:83,equival:[29,31,51,74,106],error:[6,7,8,9,21,29,30,37,45,46,72,86,87,93,95,98,110,111],error_clipping_threshold:6,especi:[2,8,9,84],essenc:29,essenti:[8,29,43,46,88],estim:[8,29,59,66],eta:99,etc:[7,13,21,31,42,59,64,65,72,77,93,94,97,98,106],etcd:[13,28,33,37,38,40],etcd_addr:38,etcd_endpoint:13,eth0:98,euclidean:8,eval:[7,18,31,42,48,67,68,111],eval_program:[18,42],eval_result:42,evalu:[1,8,16,27,28,40,43,50,62,65,67,92,111],evaluate_difficult:7,even:[29,46,63,64,84,85,92,95],evenli:[38,98],event:99,event_handl:[28,29],eventu:[65,75],everi:[1,2,7,8,9,13,18,29,33,37,38,40,42,49,50,52,56,63,66,72,74,83,85,86,87,88,93,95,111],everyon:85,everyth:[48,65,66,103],everywher:84,evid:60,exactli:[2,8,9,98],exampl:[1,2,7,8,9,13,14,15,21,28,31,41,42,45,47,48,49,50,52,56,57,60,61,63,64,66,68,69,70,73,75,83,84,85,86,87,88,91,92,93,94,95,97,98,99,104,110,111],exceed:8,except:[2,8,14,40,49,75,97,108],exchang:60,exclud:8,exclude_mod:8,excluded_chunk_typ:7,exconv:8,exconvt:8,excut:13,exdb:14,exec:95,execut:[32,33,37,41,42,43,48,50,67,69,84,86,91,92,98],executioncontext:[87,88],executor:[16,18,42,46,48,58,61,91],exist:[29,31,33,45,57,63,64,67,69,74,75,80,84,86,88,95,98],exit:[38,45,95,99],expand:[57,78,86],expand_a:8,expand_level:8,expandconvlay:8,expans:8,expect:[8,92],experi:[53,97],experienc:85,expert:32,expir:33,explain:[2,7,33,47,49,85,91,93,109],explan:[8,30,41,65,72,111],explicit:[75,86],explicitli:[2,29,65,67,87,88],explor:[8,57,70],expon:8,exponenti:5,expos:[39,53,75,98],express:[29,42,50,66,87,98],ext:8,extend:[7,59,66,75],extens:[40,57,66,87,103],extent:55,extern:[2,32,54,55,68],external_librari:32,extra:[6,8,9,65,70,106],extra_lay:28,extraattr:[6,97],extraattribut:8,extraattributenon:8,extract:[7,8,60,65,87,98],extract_fea_c:110,extract_fea_pi:110,extract_para:108,extralayerattribut:[6,9],extralayeroutput:9,extrem:[8,92],extremli:1,f120da72:99,f7e3:98,fa0wx:99,face:[32,70],fact:[61,63,110],factor:[6,10,21],factor_s:8,factori:54,fail:[2,33,37,57,67,87,95,97,99],failur:[33,38,87],fake:[48,109],fake_imag:64,faked_imag:48,fall:[46,62],falloc:44,fals:[2,6,7,8,9,10,13,21,22,23,30,31,51,56,61,62,64,73,76,81,83,86,87,93,95,97,99,108,111],false_block:[31,51,61],false_label:64,false_neg:42,false_posit:42,false_read:64,familiar:2,fan_in:20,fan_out:20,fanscin:2,fantast:111,far:75,fascinatingli:1,fashion:65,fast:[8,37,92],faster:[8,9,26,33,78,83,92],fastli:85,fault:[28,37,68,77],favorit:84,fbd1f2bb71f4:99,fc1:[50,67,86,97],fc1_bia:50,fc1_weight:50,fc2:[50,67,97],fc3:[50,97],fc4:97,fc8a365:98,fc8a:98,fc_act:9,fc_attr:9,fc_bias_attr:9,fc_layer:[63,74,86,97,111],fc_layer_nam:9,fc_mat:28,fc_op:74,fc_out:31,fc_output:74,fc_param_attr:9,fc_without_b:31,fclayer:86,fcop:47,fea:110,fea_output:110,feasibl:59,featur:[2,5,8,13,14,46,50,65,85,95,111],feed:[9,28,29,49,56,65,67,70],feed_dict:[48,67],feeder:13,feedforward:20,feel:85,festiv:2,fetch:[14,33,36,62,65,83,86],fetch_op:62,few:[2,32,33,59,64,65,73,84],fewer:[8,63],fg0:8,field1:28,field2:28,field:[8,28,31,50,52,53,62,69,73,74,92,98],fifth:49,figur:[29,32,48,56,63,65,66,83,86,92,108,109,110],file:[1,2,4,7,8,13,14,15,25,28,29,32,33,35,37,38,40,41,44,45,50,53,55,64,65,68,76,78,80,81,83,84,85,86,87,88,93,95,103,104,106,108,110],file_list:2,file_nam:[2,110,111],file_typ:13,filenam:[2,15,35,63,91],fileoffset:44,filesystem:[40,41,44,65,98],fill:[8,21,33,37,63,98,111],fill_zero_grad:68,filter:[8,9,21,110],filter_s:[8,9,21,22],filter_size_h:21,filter_size_i:8,filter_size_w:21,filter_strid:21,find:[8,21,31,33,40,46,50,57,72,77,80,92,93,103],find_var:30,findop:31,findvar:[31,72],fine:[6,37,47],fingerprint:98,finish:[2,33,37,40,41,74,77,93,98,99],finit:86,finnal:78,first:[2,8,21,28,29,31,33,37,40,41,45,48,49,50,56,57,61,62,63,65,68,73,74,75,77,83,84,85,86,87,88,92,95,97,98,106,108,109,110,111],first_seq:83,firstli:[7,8],firstn:13,firstseen:99,fit:[1,14,46,53,57,68],five:[61,92,111],fix:[2,6,8,54,65,85,91],flag:[8,14,21,85,87,89,95,109],flatten0:50,flatten:[50,61,63,88],flexibl:[1,8,9,29,38,49,56,57,59,64,65,75,83],flip:15,flist:93,float16_t:46,float32:[4,13,21,46,47,48,63,64,87,109,110],float_16:21,float_to_half_rn:46,floor:8,flow:[21,31,56,71],fluid:[0,18,20,21,22,23,25,26,91],fly:111,fnt03:98,focu:[2,50,91,92],focus:87,folder:[32,35,41,45,98],follow:[1,2,7,8,9,10,13,15,21,28,29,30,31,32,33,37,41,46,47,48,49,50,51,52,56,57,59,61,62,63,64,65,66,68,69,70,72,73,74,75,77,78,80,81,83,84,85,86,87,88,89,91,92,93,97,98,99,100,101,103,104,106,108,109,110,111],fool:109,forbid:29,forc:63,force_load:54,forest:31,forget:[10,29],forget_bia:21,fork:[8,85],form:[1,2,8,9,42,80,92],format:[1,2,7,13,14,15,21,25,28,30,37,46,57,75,81,85,86,87,88,93,95,98,108],former:[29,32,59],formula:[8,9,10,30],formular:8,forth:48,forward:[5,8,9,30,31,36,38,48,53,58,60,61,64,68,69,70,73,83,86,97,109],forward_op:30,forwardactiv:86,forwardbackward:28,forwardtest:4,found:[2,4,8,46,61,70,72,83,93,104,109,111],four:[2,7,42,49,108,110,111],foward:62,fp16:[46,68,76],fp32:[68,76],fp64:76,fpga:67,frac:21,frame:[7,68,75],framework:[29,31,42,46,50,59,61,68,70,72,74,85,86,87,91,93,110,111],free:[14,85,106],frequenc:[14,92,108,111],frequent:[37,64,68,70,93,103],fresh:[40,67],friend:72,friendli:48,fritz:8,from:[2,4,7,8,9,13,14,15,20,21,28,30,31,32,33,35,36,37,38,42,45,46,47,48,49,50,51,56,57,58,60,61,63,64,65,66,67,68,69,72,75,78,80,83,84,85,86,87,88,91,92,93,95,97,98,99,103,106,108,109,110,111],from_no_sequ:8,from_sequ:8,from_tar:28,fromfil:[64,110],front:50,fuction:25,fulfil:92,full:[8,33,40,56,59,83,86,106],full_matrix_project:[9,83],fulli:[21,65,66,86,92,106,109,110,111],fullmatrixproject:8,fullsiz:36,fully_matrix_project:9,fullyconnect:[50,63,108],fullyconnectedlay:86,func:[13,37,69],funciton:[9,21],functor:[47,50],fundament:[46,66,68],further:[8,74,106],furthermor:67,futur:[8,40,46,56,65,68,103,107],fvs:74,fwd_op:69,g_b0:48,g_b1:48,g_b2:48,g_block:48,g_h0:48,g_h0_bn:48,g_h0_relu:48,g_h1:48,g_h1_bn:48,g_h1_relu:48,g_h2:48,g_im:48,g_loss:48,g_optim:48,g_program:63,g_step:48,g_w0:48,g_w1:48,g_w2:48,gain:8,game:109,gamma:110,gan:29,gan_train:109,gangliao:32,gap:95,gate:[8,9],gate_act:[8,9],gate_activ:21,gate_attr:8,gate_bias_attr:8,gate_param_attr:8,gate_recurr:8,gather:[8,60,86,87],gauss:6,gaussian:[20,109],gaussian_normal_random:48,gcc:[46,54,68,77,84,91,103,104],gcc_3:80,gcreators_:74,gen:8,gen_conf:109,gen_train:109,gen_training_machin:109,gender:14,gendrated_id:57,gener:[1,2,4,7,8,9,13,18,28,29,30,31,32,33,35,37,38,40,47,52,59,61,62,63,64,65,66,68,69,73,74,75,85,87,92,93,95,97,98,103,104,108,110,111],generated_id:57,generated_scor:57,generated_word_embed:8,generatedinput:[8,83],generator_conf:109,generator_machin:109,generator_train:109,genert:2,geq:8,gereat:7,get:[2,7,8,13,14,28,30,31,32,33,37,38,40,41,44,48,50,56,57,63,67,68,69,72,74,75,77,78,80,83,85,86,87,91,92,93,98,102,110,111],get_all_op_proto:74,get_block:63,get_cloud_job:67,get_config_arg:[97,111],get_data:[99,111],get_dict:14,get_dim:30,get_embed:14,get_float_el:30,get_grad:28,get_input_lay:86,get_mnist_data:109,get_model:110,get_movie_title_dict:14,get_nois:109,get_numeric_gradi:30,get_numerical_gradi:30,get_output:30,get_shap:28,get_support:80,get_symbol:50,get_tensor:30,get_training_loss:109,get_vari:31,get_word_dict:14,getbatchs:86,geteigendevic:88,getenv:[29,41,93],getinfervartyp:52,getinput:86,getinputgrad:86,getinputvalu:86,getkerneltyp:46,getlayeroutput:28,getmat:36,getoptconfig:36,getoutputgrad:86,getoutputvalu:86,getparam:36,getparameterconfig:36,getparameterptr:86,getparameterspars:36,getparametersremot:36,getplac:[87,88],getsiz:86,getslotvalu:109,gettask:37,gettempl:98,gettranspos:86,getw:86,getweight:86,getwgrad:86,gist:9,git:[71,77,84,85,89,103,104],github:[9,32,48,77,84,85,89,91,103,104,110],give:[2,33,56,63,68,84,85,86,92,98,111],given:[2,8,13,21,28,38,40,47,48,57,64,66,67,70,75,86,95,109,111],glibc:[80,103,104],glibc_2:80,glibcxx_3:80,glide:32,global:[2,6,21,23,29,31,32,33,50,60,67,68,72,74,84,92,95,98],global_block:63,global_learning_r:6,global_pool:21,global_step:23,globalstat:92,globalstatinfo:92,globe:2,glog:85,glog_v:85,glog_vmodul:85,glorot10a:20,glorot:20,gnueabihf:104,go_librari:32,go_test:32,goal:[46,49,68,92],gob:37,godep:32,godoc:54,goe:[8,9,33,51,72],going:[47,59,91,93,106,111],golang:32,good:[48,59,64,70,91,92,106],goodfellow13:8,googl:[29,68,85,91,93,103],googleapi:98,got:72,gpg2:98,gpg:98,gprotos_:74,gpu:[1,2,6,8,11,30,41,42,46,59,60,65,67,68,70,71,77,80,81,84,87,88,90,93,106,109,110],gpu_id:[95,97,109],gpu_per_train:[65,67],gpugpu_id:94,gpukernel:68,gpuplac:87,gpustarttimestamp:25,grab:33,grad:[30,38,73,95],grad_op_class:68,grad_op_maker_:69,grad_op_typ:[68,69],grad_op_type_:69,grad_share_block_num:[94,95],grad_var_nam:30,gradient:[6,7,8,10,20,21,23,26,28,33,37,49,52,58,59,60,68,73,87,91,93,95,111],gradient_check:87,gradient_clipping_threshold:[6,111],gradient_evalu:7,gradient_flat:30,gradient_machin:[28,55],gradientcheck:87,gradientmachin:[4,28,55,60,109],gradientmachine_:36,gradopdescmak:[52,69],gradopdescmakerbas:69,gradopmak:69,gradual:92,grai:15,grain:47,gram:108,grandient:28,grant:98,graph:[8,21,28,31,32,33,42,43,48,56,59,61,65,67,76,88,108],graphviz:110,grayscal:2,great:[66,106],greater:[8,59,93],greaterthan:74,green:48,grep:[78,93],gridsize3d:25,groudtruth:83,ground:[7,8,111],group:[9,21,22,37,50,106],group_input1:83,group_input2:83,group_input:83,grouplen:14,grow:85,grpc:106,gru:[8,57,83,111],gru_bias_attr:9,gru_decod:83,gru_decoder_with_attent:83,gru_encoder_decod:108,gru_layer_attr:9,gru_memori:9,gru_out:57,gru_siz:111,gru_step:[57,83],gru_step_lay:9,grumemori:[9,83],gserver:[8,86],gsizex:92,guarante:[63,86],guard:36,guest:[80,84],gui:[91,92],guid:[25,44,68,83,85,86,92,98,99,108],gzip:[13,37,99],h0_bn:48,h_prev:31,had:84,hadoop:[13,29],half:[46,98],half_to_float:46,hand:[68,88,93],handi:32,handl:[13,29,41,50,60,64,65,67,72,75],handler:[28,31],handwrit:2,happen:[37,74],hard:[57,65,66,75,84,98,111],hardwar:[84,92],has:[2,4,7,8,9,14,21,25,29,30,31,32,33,37,38,40,42,46,48,50,53,57,61,65,66,67,68,73,74,83,84,85,86,87,92,98,99,106,108,111],has_kei:28,has_selected_colum:8,hasdependentvar:62,have:[1,2,4,8,9,13,29,30,31,32,33,37,38,40,41,46,47,48,49,53,56,57,59,60,61,63,64,65,66,67,68,69,72,73,76,77,78,83,84,86,87,92,93,95,97,98,104,106,108,111],haven:84,hdf:[1,13,35],head:[85,87,93,108],header:[38,53,55,68,86,103,104,108,110],headip:93,heard:84,heavi:93,height:[8,13,15,31,54,64,86,87],height_:73,held:33,hello:29,help:[2,4,8,21,31,45,50,57,64,68,75,84,85,91,93],helper:[8,21,65,69,75,86],henc:[59,63,65,69,70,72],here:[2,4,6,7,8,9,13,29,32,33,39,45,49,50,56,64,70,74,77,78,80,83,85,87,89,93,94,97,98,99,104,106,108,110,111],heterogen:[65,66],heurist:[8,57,66,95],hidden:[8,9,58,65,83,98,111],hidden_dim:21,hidden_out:31,hidden_s:9,hierarch:[8,61,63,68,83],hierarchi:68,high:[6,20,46,86,93,106,109],higher:[1,47,56,75,85],highest:[13,31],highli:[1,2,14,75,83,97],him:29,hint:[66,91],his:67,histor:47,histori:10,hl_get_sync_flag:86,hold:[29,33,37,39,46,48,50,52,67,72,74,88,98],holder_:88,home:[35,45,65,67,78,91,93,98,99],honor:37,hook:[2,6],hookattr:6,hookattribut:6,horizont:[8,15,110],host:[32,41,98,99,103,104],host_c:[103,104],hostfil:93,hostnam:98,hostpath:99,hostport:98,hour:84,hourli:85,hous:[2,14,81,108],how:[1,2,6,8,21,29,31,33,37,45,47,50,56,57,60,65,70,74,83,84,91,93,95,98,99,102,104,110,111],howev:[2,8,9,30,40,59,60,63,64,65,69,70,73,74,83,94,95,98],howto:93,hpp:[46,54],html:[14,20],htod:92,http:[8,9,13,14,20,32,41,48,77,78,84,85,89,91,98,99,103,104,106,109,110],huber:8,huge:59,human:[8,20],hwc:15,hyper:[8,48,86],hyperparamet:[8,70],hyperplan:13,i1117:92,iOS:104,iamfullaccess:98,iamusersshkei:98,icml2006_gravesfgs06:8,icml_rank:8,ics:14,id_input:7,id_rsa:93,idea:[32,59,64,70,91],ideal:65,ident:[8,69,87,98],identifi:[8,51,86],identityoffsetproject:8,identityproject:8,ids:[7,8,21,57,86,111],idx:[37,48,86],ies:45,ifels:[31,61],ifelseop:61,ight:8,ignor:[2,8,21,95,108],illustr:[2,7,33,38,47,56,65,67,83,86,92,111],ilsvrc:110,im_siz:48,imag:[2,11,12,13,14,21,22,29,48,49,57,58,61,64,67,77,84,85,97,98,100,101,106,109,110],image_a:64,image_b:64,image_fil:64,image_h:21,image_lay:64,image_list_provid:110,image_nam:29,image_path:64,image_reader_cr:64,image_s:110,image_w:21,imagenet:[8,20,35],imagepullpolici:98,imageri:8,images_reader_cr:64,imagin:49,img2label:15,img:[2,8,9,65],img_conv_lay:9,img_featur:2,img_pool_lay:9,imgsiz:92,imgsizei:92,imgsizex:92,imikolov:93,immedi:[59,70,77,98],immutable_paramet:29,imperfect:68,implement:[2,8,9,13,20,21,23,26,31,37,38,39,40,41,47,50,51,52,54,55,57,60,62,65,66,72,74,75,83,111],implemet:36,impli:32,implicit:67,imposs:[57,106],improv:[8,66,68,91,92,98],in_fals:21,in_plac:21,in_tru:21,inarg:36,inbound:98,includ:[1,2,7,8,9,14,15,23,29,31,32,38,41,46,48,50,54,55,56,57,61,63,68,74,77,80,83,84,86,87,91,92,93,95,98,99,103,104,108,111],inclus:57,incorpor:8,incorrect:8,increas:[33,37,46,93,95],increment:[42,49,95],incupd:86,inde:13,independ:[8,30,38,67,72,106,111],index:[2,7,8,11,13,14,21,28,30,31,33,37,61,63,75,98],indexslot:8,indic:[2,7,8,21,31,38,48,56,61,69,73,75,93,98,103],indice_map:75,indices_map:75,individu:[33,98],industri:[33,53,106],ineffici:60,infer:[0,15,29,31,33,42,51,52,54,62,63,67,68,73,81,104],infer_shap:63,infer_var_type_:52,inferfac:52,inferior:40,infernec:104,infershap:[31,63,68,87,88],infershapecontext:[87,88],infervartypefn:52,info:[7,8,14,46,56,86,93,106],inform:[4,8,14,21,28,31,41,45,50,53,56,63,65,70,72,73,85,86,87,88,91,92,95,98,103],infrastructur:[98,109],ingor:95,inherit:[31,58,68,87],ininst:29,init:[6,20,28,31,48,56,57,65,81,86,93,97,98,109,111],init_attr:63,init_from_tar:28,init_hook:111,init_model_path:[94,95,97,108,111],initi:[2,4,6,8,9,14,16,21,28,32,37,42,49,56,59,63,65,66,70,74,75,81,83,86,87,95,108,109,111],initial_max:6,initial_mean:[6,8],initial_min:6,initial_std:[6,8],initialize_op_attr:63,initpaddl:[4,109],initrd:106,inlcud:9,inlin:[88,98],inner:[8,86],inner_param_attr:9,inproj_attr:8,inproj_bias_attr:8,inproj_param_attr:8,input0:88,input1:[8,9,88],input2:8,input:[2,4,5,7,8,9,11,13,15,21,22,28,30,31,36,40,42,46,47,48,49,50,52,56,57,59,60,62,63,64,65,66,67,68,69,72,74,75,81,83,85,86,87,88,93,97,108,109,110,111],input_conf:8,input_data:86,input_data_target:86,input_dim_idx:21,input_dtyp:21,input_featur:5,input_hassub_sequence_data:86,input_id:8,input_imag:9,input_index:86,input_label:86,input_lay:86,input_loc:8,input_nam:29,input_proj_bias_attr:9,input_proj_layer_attr:9,input_seg:75,input_seq:8,input_sequence_data:86,input_sequence_label:86,input_sparse_float_value_data:86,input_sparse_non_value_data:86,input_t:86,input_to_check:30,input_typ:111,input_valu:30,input_var:[30,63],inputbuff:36,inputdef:86,inputgradi:69,inputlayers_:86,inputs_to_check:30,inputtyp:[2,13],insert:[62,68,69,85],insid:[7,9,21,33,42,60,64,65,66,68,69,78,98],inspir:108,instal:[8,41,71,77,78,84,85,89,91,93,99,110],install_android:103,instanc:[8,30,31,33,35,39,43,51,56,57,59,63,65,68,69,83,86,88,92,95],instance_ip:98,instanti:[33,43],instead:[8,9,11,13,32,36,41,46,49,50,65,84,85,111],instrins:46,instruct:[31,49,78,84,92,103,111],int16:76,int32:[61,75,76,95],int64:[44,73,76],integ:[2,7,8,13,21,37,41,46,54,57,86,111],integer_valu:[2,13,111],integer_value_sequ:[2,13,57,83,111],integr:[7,77,106],intellig:[20,43],intend:77,inter:[8,65],interact:[8,78,98],intercept:8,interchang:[49,68],interest:[46,92],interfac:[0,4,6,8,9,20,23,25,26,28,31,37,41,45,50,60,68,69,77,87,88,98,106],interg:111,intergr:8,intermedi:[45,48,58,65,84,103,104],intern:[8,9,20,23,28,46,91,93,98],internet:[32,33,106],interpret:[2,7,77,92],interv:8,inth:88,intrins:[46,104],introduc:[2,8,15,31,33,48,53,70,72,74,87,91,93,99],introduct:[3,109],introductori:84,intuit:[40,68],invalid:[64,72],invoc:[32,47,68],invok:[2,8,18,28,60,63,65,66,68,69,74,84,85,92,98],involv:[57,87,109],iob:7,ioe:7,ips:98,ipt:[8,63,74,83],ipx:106,ipython:29,is_color:15,is_discriminator_train:109,is_gener:[108,109],is_generator_train:109,is_loc:28,is_predict:111,is_revers:21,is_seq:[8,83],is_spars:21,is_stat:6,is_target:62,is_tensor:74,is_test:[21,110],is_traget:62,is_train:[2,15],isn:92,isspars:86,issu:[32,48,67,78,80,84,85,92],istag:71,item:[8,13,28,40,46,64,81,106],iter:[8,9,10,13,28,29,33,59,64,65,75],iter_multiple_input_and_param:63,its:[2,8,9,20,26,28,29,30,31,33,37,42,48,49,50,52,53,56,57,59,60,62,63,68,69,72,73,74,80,86,87,88,92,93,95,98,108,109,111],itself:[33,40,59,72],ivs:74,java:[31,54,61,68],jeremi:92,jian:20,jmlr:8,job:[4,14,40,65,67,68,78,94,95,97,110,111],job_dispatch_packag:93,job_id:14,job_mod:108,job_nam:[41,98],job_namespac:98,job_path:98,job_workspac:93,jobpath:98,jobport0:98,jobport1:98,jobport2:98,jobport3:98,jobserv:41,join:33,joint:108,jointli:9,journei:78,jpg:[15,110],json:[50,98,99],jth:9,judg:8,juditski:59,jupyt:[41,78],just:[2,5,7,8,9,14,21,32,37,38,48,52,59,60,63,64,65,68,69,70,72,73,77,80,84,85,93,97,98,103,108],jx4xr:98,jypyt:29,k8s:106,k8s_data:98,k8s_job:29,k8s_token:29,k8s_train:98,k8s_user:29,kafka:35,kaim:20,kaimingh:110,kebilinearinterpbw:92,kebilinearinterpfw:92,keep:[2,8,13,15,20,33,49,57,59,63,72,74,77,85,106],keep_top_k:8,kei:[2,14,15,25,28,30,31,33,35,37,44,46,68,69,74,75,84,85,87,92],kept:[8,63],kera:70,kernel:[8,30,46,59,70,73,87,88,92,111],key1:95,key2:95,key_pair_nam:98,keyid:98,keymetadata:98,keypair:98,keyserv:98,keystat:98,keyusag:98,keyword:[2,63],kill:[33,98],kind:[1,2,29,30,33,39,49,58,76,78,98,99,109,111],kms:98,know:[2,29,37,53,85,86,91,92,93,98,103],known:[21,31,47,56,109],kriz:14,kselectedrow:73,ksimonyan:9,kube_cluster_tl:29,kube_ctrl_start_job:29,kube_list_containers_in_job_and_return_current_containers_rank:29,kubeconfig:98,kubectl:[93,99],kuberent:[33,98],kubernet:[29,33,68,90,100,101,106],kubernetes_service_host:29,kvp:25,kwarg:[2,9,10,13,18,21,42,50,63,74,111],kwd:25,l1_rate:6,l1_regularization_op:70,l2_rate:6,l2_regularization_op:70,l2_sim:8,l2regular:111,l93:36,label:[2,4,7,8,13,14,15,21,28,42,48,49,50,58,61,64,65,66,67,99,109,110,111],label_dim:[8,111],label_fil:64,label_lay:64,label_path:64,lag:95,lake:2,lambdacost:8,lambdarank:8,lan:93,languag:[8,14,49,68,72,97,108],larg:[11,14,53,59,65,66,85],larger:[2,6,7,8],larger_than:[31,51,61],last:[7,8,9,21,56,61,83,95,111],last_seq:57,last_time_step_output:8,lastseen:99,latenc:[8,46,65,93,98],latent:8,later:[32,68,70,77,80,87,88,98,107,111],latest:[8,31,32,33,40,77,78,80,89,99],latter:[59,75,91],launch:[95,98],launcher:29,layer1:[8,9],layer2:8,layer3:8,layer:[4,6,7,9,11,13,16,20,27,28,31,36,48,49,51,58,59,61,64,65,66,68,70,74,75,81,83,90,94,95,108,109,110,111],layer_0:86,layer_attr:[8,83,97],layer_num:[97,110],layer_s:8,layer_typ:8,layerbas:86,layerconfig:86,layergradutil:86,layerhelp:[21,63],layermap:86,layeroutout:8,layeroutput:9,layout:15,lazi:[59,70],lbl:7,lead:92,leaki:48,learing_r:58,learn:[6,7,8,9,10,14,29,38,40,48,49,57,59,64,65,66,67,68,70,78,83,84,86,87,89,92,107,110,111],learnabl:28,learning_method:[108,111],learning_r:[6,23,38,65,108,111],leas:33,least:[7,33,80,103],leav:[2,31,98],lecun:14,left:[8,31,88,110],left_cmd:13,left_right_flip:15,legaci:78,legal:74,len:[2,8,38,44,63,81,86,111],length:[8,9,13,14,15,21,38,46,53,56,57,68,75,83,95,99],leq:8,less:[8,29,106],less_than:29,let02:99,let:[4,7,8,29,31,40,47,49,56,57,58,69,87,91,98],level:[6,8,20,21,46,47,50,53,56,57,75,76,85,93,95,103,109],lgtest:32,lgtest_main:32,lib64:[78,95],lib:[55,77,78,91,93,103,104],libapi:32,libari:55,libc:80,libcuda:78,libgcc_:80,libgoogl:91,libnvidia:78,libpaddl:[54,55,68,91],libpaddle_capi:55,libpaddle_gserv:55,libpaddle_math:55,libpython2:77,librari:[8,32,39,55,65,77,80,87,93,95,104],libstdc:80,life:33,lifecycl:106,lifetim:[72,80],lightweight:47,like:[2,7,8,13,14,21,31,32,33,36,41,47,48,49,50,52,59,63,64,68,69,70,72,73,75,77,80,83,84,85,91,92,93,94,97,98,103,104,106,108,110,111],limit:[8,13,21,53,57,68,70,92,95],linaro:104,line:[1,2,4,7,13,25,32,36,41,45,49,59,61,63,68,70,84,85,90,91,92,97,98,108,110],line_break:13,linear:[8,21,57,81],lineno:91,link1:46,link2:46,link:[8,9,32,44,45,72,80,87,98,106,111],linux:[13,44,78,80,84,85,98,104],linux_x86_64:[71,80],lipeng:108,list:[1,2,7,8,9,13,15,18,21,23,28,29,31,32,37,41,43,45,48,58,60,63,67,69,72,75,81,83,84,86,87,91,93,95,97,98,104,110,111],listdir:93,listen:[33,93,95],littl:[1,2,38,53,95,111],live:87,load:[1,2,4,15,29,33,48,63,65,77,87,95,98,110],load_and_transform:15,load_featur:110,load_feature_c:110,load_feature_pi:110,load_imag:15,load_image_byt:15,load_missing_parameter_strategi:[94,95,97,108],load_mnist:48,load_uniform_data:109,loadparamet:4,loadsave_parameters_in_pserv:[36,94,95],loc:[7,20],local:[6,21,28,30,31,33,39,40,49,56,61,63,68,77,78,84,85,91,93,94,95,99],local_scop:30,localhost:[78,89],localpath:45,locat:[8,28,32,75,83,86,93,104,111],lock:[32,33,37,38,66],lod:[21,53,56,73,75,76],lod_desc:[73,76],lod_expand:57,lod_level:[21,63,73,76],lod_tensor:[21,56,73,76],lod_tensor_arrai:21,lodtensor:[52,53,68,76],lodtensordesc:[53,73],log:[2,37,45,48,65,80,86,93,95,98,99,104],log_barrier_abstract:95,log_barrier_lowest_nod:[94,95],log_barrier_show_log:[94,95],log_clip:[94,95],log_error_clip:[94,95],log_period:[95,97,99,109,111],log_period_serv:[94,95],logarithm:5,logger:2,logic:[2,40,48,52,58,60,65,66,72,75,87],login:[80,93],logit:48,longer:[33,65],look:[2,7,31,41,49,59,63,69,70,93,94,98,99,109,111],lookahead:8,lookup:[21,52,57,111],lookup_t:21,loop:[30,31,64,72],loop_var:75,loss:[8,23,48,50,58,59,70,86,109,111],lot:[57,59,63,65,70,93,94,106],low:[8,20,58,75],low_rnn:56,lower:[8,46,56,57,85,93],lower_level_rnn:56,lowest:95,lpaddle_capi_shar:55,lpaddle_capi_whol:55,lrelu:48,lstm:[8,83,99,111],lstm_bias_attr:9,lstm_cell_attr:9,lstm_group:9,lstm_layer_attr:9,lstm_size:111,lstm_step:9,lstmemori:[9,83],lstmemory_group:8,ltr:8,mac:[55,84,85,103],machin:[9,14,28,48,59,65,66,70,77,80,84,86,93,94,95,97,98,99,104,106,111],machine_transl:83,machinelearn:8,maco:[80,81,84],macro:[47,69,87],made:[2,33,38,83],mai:[2,8,9,30,31,42,46,64,65,66,67,68,72,77,78,88,89,92,93,98,104],main:[2,4,21,50,61,68,80,91,93,98],main_program:[18,21,22,42],mainli:[39,77,87,95],mainlin:80,maintain:[8,31,37,59,63,68,98,107],majel:32,major:[46,65,103,109,110],make:[2,7,8,29,31,32,33,37,38,40,46,49,56,57,59,60,63,64,65,68,70,75,77,84,85,86,87,91,92,93,98,103,104,106,111],make_ddim:88,make_function_oper:47,make_vari:74,maker:[68,69],malloc:86,man:44,manag:[23,28,33,38,39,45,65,72,80,89],mandarin:8,mani:[9,15,32,37,48,57,60,63,68,69,72,73,74,75,84,95,111],manili:50,manipul:[63,69,93],manner:[8,59,70],manual:[58,59,65,69,93,103,106],manylinux1:80,manylinux1_x86_64:[71,80],map:[2,7,8,13,28,29,31,37,63,69,72,74,75,84,95,106,110],map_fn:75,map_read:13,mapper:13,mapreduc:[29,93],mark:[2,48,49,56,57,66,72,83,91,106],market:46,mask:[6,8,21],master:[29,40,68,71,95,104],mastermind:32,mat:[54,55],mat_cache_row:36,mat_norm:36,mat_normal_shar:36,mat_param_attr:9,mat_sparse_row:36,mat_sparse_row_auto_grow:36,mat_sparse_row_id:36,mat_sparse_row_prefetch:36,mat_sparse_row_prefetch_full_s:36,mat_value_shar:36,match:[8,21,32,46,80,92],matchbox:106,math:[9,54,68,85,86,87,92],mathemat:70,matirx:8,matmul:[31,50,56,75,87],matric:[4,83,86],matrix:[7,8,9,13,21,28,36,54,55,86,87,94,97,110],matrixptr:86,matrixtyp:55,matter:2,mattyp:36,matur:93,max:[2,6,8,13,14,21,22,30,63,92,95,97,111],max_diff:30,max_id:[8,28,111],max_job_id:14,max_length:[8,57,83],max_movie_id:14,max_relative_error:[30,87],max_sort_s:8,max_user_id:14,maxframe_evalu:7,maxid:[7,111],maxid_evalu:7,maxid_lay:111,maxim:8,maximum:[7,8,14,31,38,83,87,92,95,111],maxinum:11,maxpool:8,mayb:31,md5:[14,34],mean:[2,6,7,8,9,10,11,13,15,20,28,32,43,50,57,62,64,65,72,78,83,84,87,91,92,95,97,98,106,108,109,110,111],mean_meta:110,mean_meta_224:110,mean_valu:110,mean_var_nam:8,meant:75,measur:[42,92],mechan:[8,9,39,42,63,69,83,98],mem:[8,31,41,57],mem_per_p:67,mem_per_train:67,member:[8,14,29,49,50,60,63,72,87],memcpi:[60,92],memor:8,memori:[1,2,9,21,31,36,37,41,46,53,57,59,68,83,84,85,86,88,92,95,97,99,111],memory_boot:9,memory_nam:8,memory_test:84,memory_threshold_on_load_data:95,mention:[21,32,37,56,59,65,66,84],mere:9,merg:[8,18,38,40,42,56,60,85,89,95,108],mergedict:108,messag:[31,49,53,61,62,63,68,69,73,76,85,95,99],meta:[110,111],metaclass:87,metadata:[44,98,99],metal:106,metaphor:49,metaplotlib:29,method:[2,8,10,20,23,28,30,31,40,46,48,49,50,58,63,64,66,68,72,73,75,78,80,86,87,88,89,91,92,95,97,111],methodolog:59,metric:[18,42],microarchitectur:46,microsoft:8,might:[8,31,32,61,66,84,85,86,91,98,103],mileag:92,million:[14,97],min:[6,8,63,92,97,98],min_block:31,min_count:66,min_desc:31,min_pool_s:2,min_word_freq:14,mind:91,mini:[2,8,13,18,28,31,33,42,51,56],mini_batch:64,minibatch:[8,21,31,42,49,51,61],minim:[2,23,31,48,58,66,68,95,103,104],minimum:[8,103],minimun:95,minsizerel:[103,104],minst:2,minu:69,minus_grad:69,minusgradop:69,minusop:69,minusopgradmak:69,minusopprotoandcheckermak:69,minut:[33,40,78,84,98],mirror:[32,78],mislead:38,miss:[48,95,108],mit:98,mix:[9,75,83],mixed_lay:9,mixed_layer_attr:9,mixedlayertyp:8,mixtur:91,mkdir:[45,77,89,93,98],mkl:[68,77,78],mkldnn:8,mkldnn_batch_norm:8,mlp:50,mlpaper:8,mlr:20,mnist:[2,4,35,48,49,61,64,65,67,68,91],mnist_provid:2,mnist_random_image_batch_read:64,mnist_train:[2,64],mnist_train_batch_read:64,mobil:[46,68,89,102],mod:93,mode:[8,25,28,46,85,95,109,110],model:[0,1,4,8,9,14,28,31,33,34,42,49,58,59,65,66,67,68,70,75,81,85,86,89,90,95,98,107],model_config:[4,109],model_list:[95,97],model_path:97,model_zoo:[108,110],modif:79,modifi:[4,8,46,50,65,70,83,86,87,93,98],modul:[1,2,4,9,14,28,47,48,57,65,75,87,91,110,111],modular:57,modulo:8,moment:[23,91],momentum:[6,21,23,72,111],momentumop:91,mon:99,monitor:111,mono:8,month:[32,111],more:[1,2,4,7,8,9,13,21,29,30,32,33,37,40,41,45,46,47,49,56,57,58,63,64,65,66,68,70,75,77,78,81,83,84,86,87,88,89,91,92,93,97,99,104,106,111],most:[2,4,8,13,21,28,29,32,40,49,50,57,59,64,65,70,80,83,86,91,92,94,106],mostli:[46,106],motiv:68,mount:[41,78,93,98,99],mountpath:[98,99],move:[8,33,37,45,59,78,92,98,106],movement:92,movi:[2,14],movie_categori:14,movie_info:14,movie_review:14,movieinfo:14,moving_average_fract:8,mpi:93,mpirun:93,mse:[49,58,61],msra:20,much:[8,33,58,64,70,75,92],mul:[47,63,67,86,87],mul_grad:87,mul_op:[21,87],mul_ratio:8,mul_result:63,mulgradkernel:87,mulkernel:87,mulop:[47,87],mulopgrad:87,mulopmak:87,multi:[8,42,60,86,91,93,94,95,106,110],multi_binary_label_cross_entropi:8,multi_crop:110,multigradientmachin:60,multinomi:8,multipl:[7,8,9,13,18,21,28,29,30,37,38,40,42,47,65,66,67,68,76,83,86,87,91,93,95,97,98,109,111],multiple_input:63,multiple_param_attr:63,multipli:[7,8,86],multiprocess:13,multithread:2,must:[2,5,7,8,9,13,15,21,38,53,62,63,64,68,74,83,86,87,88,93,95,97,98,103,104],mutabl:88,mutable_data:[87,88],mutuable_data:88,mxnet:31,my_cluster_nam:98,my_external_dns_nam:98,my_lib:93,mypaddl:99,name:[2,6,7,8,9,11,15,18,21,25,28,29,30,31,33,35,36,38,41,42,46,47,50,53,55,57,61,63,67,68,71,73,74,75,76,78,80,81,83,84,86,87,92,93,95,97,99,100,101,106,108,109,110,111],name_prefix:35,namespac:[31,51,54,63,86,87,99],nativ:[8,46,85],natur:[37,40,57,66,75,97],navig:89,ncall:91,ncelm:8,nchw:[8,21],ndarrai:[15,28,35],ndcg:8,ndcg_num:8,ndk:103,nearest:[46,111],nearli:30,necess:75,necessari:[2,8,31,38,40,42,53,57,60,63,74,75,86,93,111],necessarili:86,need:[2,7,8,9,13,20,23,26,29,30,32,36,37,38,40,41,42,45,47,48,57,58,59,60,62,63,65,66,67,68,69,70,72,73,74,75,77,78,79,80,81,83,86,87,88,89,93,94,95,97,98,99,103,104,106,109,110,111],neg:[2,7,8,111],neg_distribut:8,neg_overlap:8,neg_pos_ratio:8,neighbor:111,neon:[46,103,104],ner:7,nerual:21,nest:[2,8,13,31,61],net:[8,9,16,31,48,56,72],net_diagram:110,netop:[31,68],network:[1,2,4,6,7,8,13,20,21,26,27,28,29,30,31,33,36,42,48,50,56,58,59,64,65,66,70,72,74,76,81,86,87,88,92,93,95,106,108],network_config:97,networkadministr:98,neural:[2,4,8,9,13,20,28,29,31,33,50,56,59,65,70,72,76,81,88,92,93,95,108,109,110],neuralnetwork:60,neuron:[4,21,86,111],never:[13,64,72,98,99],new_block_idx:63,new_stat:56,newblock:63,newer:103,newest:38,newli:[46,106],newop:31,newopdesc:63,newprogram:63,newremot:65,newvardesc:63,next:[8,14,33,39,57,75,83,86,87,91,92,95,98,99],nfs4:98,nfs:98,nfsver:98,ngram:14,nic:[94,95],nil:37,nine:14,nlp:[2,8],nltk:14,nms_threshold:8,nms_top_k:8,nnz:86,no_cach:2,no_grad_set:[23,30,87],no_sequ:[2,8],node1ip:93,node2ip:93,node3ip:93,node:[8,32,40,50,57,65,66,67,68,76,84,86,93,95,98,99,106],node_0:98,node_1:98,node_2:98,node_id:93,nodeattr:50,nodeentri:50,nodefil:93,nodesep:50,nohup:93,nois:[8,33,48,93,109],noise_dim:109,noisi:[8,48],non:[8,21,33,46,73,86,87,95,98],none:[1,2,4,6,7,8,9,10,11,15,18,20,21,22,23,28,29,30,31,42,48,50,51,56,57,58,61,63,67,74,75,83,110,111],nonlinear:[20,86],nor:84,norm:[9,48,109],norm_by_tim:8,normal:[2,4,8,9,14,20,59,83,86,93,95,99,108,109,110],normzal:110,notat:8,note:[2,4,6,8,9,11,15,18,28,29,31,36,37,41,53,64,68,77,78,87,88,89,92,93,95,97,98,108],notebook:[41,78],noth:[5,28,63,72,84,95],notic:[69,83,85,86],notif:85,notingradi:87,notion:75,notori:30,now:[2,13,32,33,48,53,59,66,68,69,70,72,95,98,109],np_arrai:13,nproc:84,nullptr:[69,72,86],num:[8,9,93,95,111],num_channel:[8,9],num_chunk_typ:7,num_class:[8,9,50],num_col_dim:21,num_filt:[8,9,21,22],num_flatten_dim:21,num_gradient_serv:[93,94,95],num_group:8,num_hidden:50,num_input:85,num_neg_sampl:8,num_p:[65,67],num_parameter_serv:29,num_pass:[28,94,95,97,99,111],num_per_batch:15,num_repeat:8,num_result:7,num_results_per_sampl:8,num_row:73,num_shard:35,num_step:75,num_train:[65,67],number:[2,7,8,9,13,14,15,21,31,33,35,42,59,64,66,68,74,75,84,86,91,93,95,98,108,110,111],numchunktyp:7,numdevices_:97,numer:[8,87],numeric_grad:30,numerical_grad:30,numlogicaldevices_:97,numofallsampl:7,numofwrongpredict:7,numpi:[6,13,15,21,28,35,46,48,63,64,77,87,109,110],numreal:36,numsampl:92,numtagtyp:7,numtimeout:37,nv_:32,nv_gpu:84,nv_librari:32,nv_test:32,nvcc:[32,46],nvidia:[46,78,84,92,95],obei:7,obj:[2,110,111],object:[2,4,6,8,9,13,28,29,36,42,48,50,54,58,63,67,68,70,72,88,92,109,110,111],observ:[8,86,92],obtain:[40,59,111],obvious:[32,91],occupi:46,occur:[14,28],occurr:31,oct:99,odd:8,off:[55,77,78,84,93,103,104,106],offer:[4,31,67,68,74],offici:[8,32,85,89,98,103],offlin:[33,35,106],offset:[8,36],often:[8,36,50,85,91,93,111],ograd:86,old:[30,38,40,57,68,95],older:103,omega:70,omit:111,omp_num_thread:91,ompi_comm_world_rank:93,on_init:2,onc:[2,8,33,37,42,49,59,65,66,85,86,89,98,111],one:[2,5,7,8,9,11,13,20,23,26,28,29,30,31,33,36,37,38,40,41,42,43,46,47,48,50,52,53,56,57,58,59,60,61,62,63,64,65,67,68,69,72,73,75,78,84,85,86,87,93,95,97,98,99,106,108,109,110,111],onehotcrossentropyopkernel:87,ones:[47,48,68,85,87],onli:[1,2,4,7,8,9,11,15,21,26,28,29,30,32,36,37,38,39,40,41,42,45,46,48,49,56,57,58,60,63,65,66,67,68,73,74,75,77,79,80,83,84,86,87,88,89,92,94,95,97,98,99,106,108,110,111],onlin:[8,10,33,35,64],only_cpu:30,onto:[21,65,66,93,98],op_:87,op_check:87,op_class:[68,74],op_desc:[52,62],op_maker_class:[68,74],op_proto:74,op_test_util:87,op_typ:68,opattrcheck:87,opcreat:74,opdesc:[31,49,61,62,63,68,69,74,76],opdescbind:[52,69],opdescbuild:31,open:[2,8,15,29,35,48,64,85,91,93,98,110,111],openbla:[77,78,103],opencv:15,openmp:91,oper:[8,9,13,15,20,21,23,26,30,31,42,43,46,48,49,50,52,56,57,58,62,65,67,70,72,76,83,85,86,88,92,95,98,103,108],operand:46,operator_grad:30,operatorbas:[31,47,68,69,74,87],operatorwithkernel:87,opinfo:[52,68,69],opinfomak:52,opinfomap:69,opkernel:88,opkernelkei:68,opmak:74,opproto:87,opprotoandcheckermak:[69,87],opprotomak:[74,87],opregist:74,opregistri:74,ops:[23,30,31,32,49,50,59,61,62,63,68,87,106],ops_:31,ops_test:32,opt:[29,58,62,67,74,77],opt_op_list:58,optestmeta:87,optim:[2,6,16,27,28,30,43,48,59,60,61,65,66,67,68,70,73,86,91,92,93,103,104],optimis:58,optimize_op_attr:63,option:[2,7,8,18,21,25,29,32,48,53,61,62,63,68,73,74,76,84,86,91,93,97,103,106],optmization_op_list:58,opts_np:62,optyp:[52,74],opwithkernel:73,order:[2,8,9,13,15,28,43,49,53,64,70,75,77,86,91,93,95,98,99,106,109,110,111],ordereddict:28,oregon:98,org:[7,8,9,14,20,35,44,48,78,109],organ:[7,8],orient:74,origin:[1,2,8,9,13,14,30,46,48,72,75,85,88,109],other:[2,7,8,9,13,21,23,31,33,38,45,46,52,56,59,62,70,72,74,77,83,84,85,91,93,97,98,99,103,104,106,108,109,110,111],otherchunktyp:7,otherwis:[1,8,13,14,15,21,28,29,33,38,40,48,52,64,83,85,93,97],our:[29,32,48,52,59,65,66,72,75,77,80,83,84,85,86,91,98,99,103,108,111],out:[8,21,28,29,31,32,37,40,50,56,57,63,65,81,83,87,88,91,92,93,95,98,99],out_dir:98,out_left:8,out_mem:83,out_memori:9,out_right:8,out_size_i:8,out_size_x:8,outer:8,outlier:8,outlin:96,outout_lay:28,outout_layer1:28,outout_layer2:28,output:[4,5,6,7,9,11,13,21,25,28,29,30,31,35,40,45,47,48,49,50,51,52,53,56,57,59,61,62,63,64,65,66,68,69,72,73,74,75,77,83,84,85,86,87,88,91,92,95,97,99,103,108,109,110,111],output_:[8,86],output_all_step:56,output_dim_idx:21,output_dir:110,output_dtyp:21,output_fil:25,output_id:8,output_lay:[28,81,110],output_max_index:11,output_mem:[8,83],output_mod:25,output_nam:30,output_num:56,output_path:35,output_s:21,output_seg:75,outputbuff:36,outputgradi:69,outputh:8,outputw:8,outsid:[2,8,9,65,72],outter_kwarg:2,outupt:75,outv:86,over:[1,8,9,28,29,59,75,85,86,92,111],overal:[48,59,85,106],overfit:[21,70],overhead:92,overlap:[7,8,86],overlap_threshold:[7,8],overload:46,overrid:[31,33,45,67,86,87,88],overview:[37,38,39],overwhelm:85,overwrit:[45,93],own:[8,38,40,50,52,58,59,65,67,70,74,87,93,98,103],owner:[84,85],paam:15,pack:[75,103],packag:[2,13,14,37,41,47,71,77,78,85,87,91,98],pad:[9,21,111],pad_c:8,pad_h:8,pad_w:8,paddepaddl:1,padding_attr:8,padding_h:21,padding_i:8,padding_w:21,padding_x:8,paddl:[2,4,5,6,7,8,9,10,11,13,14,15,18,20,21,22,23,25,26,28,29,31,32,33,35,41,45,47,48,51,53,54,55,56,57,60,61,65,67,68,70,74,75,77,78,80,81,83,84,85,86,87,89,90,91,92,93,95,97,98,103,106,109,111],paddle_begin_init_param:38,paddle_dir:87,paddle_element_typ:38,paddle_element_type_float32:38,paddle_element_type_float64:38,paddle_element_type_int32:38,paddle_element_type_int64:38,paddle_element_type_uint32:38,paddle_element_type_uint64:38,paddle_enforc:31,paddle_enforce_eq:[87,88],paddle_error:[54,55],paddle_exampl:41,paddle_finish_init_param:38,paddle_get_param:38,paddle_gradi:38,paddle_init_num_gradient_serv:93,paddle_init_param:38,paddle_init_port:93,paddle_init_ports_num:93,paddle_init_ports_num_for_spars:93,paddle_init_pserv:93,paddle_init_trainer_count:93,paddle_init_trainer_id:93,paddle_init_use_gpu:93,paddle_job:41,paddle_manylinux_devel:77,paddle_matrix:[54,55],paddle_matrix_cr:55,paddle_matrix_get_shap:54,paddle_matrix_shap:54,paddle_new_etcd_pserver_cli:38,paddle_new_pserver_cli:38,paddle_on_cloud:41,paddle_output:99,paddle_paramet:38,paddle_pserver2:93,paddle_pserver_cli:38,paddle_pserver_client_releas:38,paddle_root:108,paddle_save_model:38,paddle_send_grad:38,paddle_source_root:108,paddle_train:[55,71,93],paddledev:[78,98,99],paddlepaddl:[1,2,4,8,9,13,14,15,28,32,33,35,38,39,40,41,44,45,47,48,49,51,53,56,57,58,60,63,64,67,68,72,75,76,79,81,83,84,85,86,87,90,91,92,100,101,105,106,107,110,111],paddlepaddlebook:78,paddlepadl:2,paddpepaddl:2,page:[85,98],pain:67,pair:[7,8,23,25,31,49,58,65,68],pairwis:8,pakcag:32,palceholder_just_ignore_the_embed:108,paper:[8,20,48,108,109,110],paper_fil:8,para:36,paraconvert:108,paradigm:68,paragraph:56,paragraph_data:56,paragraph_out:56,parallel:[65,66,68,84,92,93,95,97,98,99],parallel_nn:[6,94,95],param:[6,8,9,13,23,30,31,38,60,63,88,93],param_attr:[8,9,21,36,63,83],param_config_proto:38,param_initi:21,paramattr:[6,8,16,83],paramet:[1,2,4,7,9,10,11,13,14,15,18,21,23,25,27,30,31,32,34,36,40,43,45,48,49,50,52,53,56,58,61,64,65,72,74,75,77,81,85,86,88,90,97,109,111],parameter_block_s:[94,95],parameter_block_size_for_spars:[94,95],parameter_learning_r:6,parameter_list:[23,58],parameter_nam:[28,29],parameter_serv:29,parameter_valu:36,parameterattribut:[6,8,9,36],parameterclient_:36,parametermap:86,parametermutex_:36,parameters_:86,parameters_and_grad:[23,58],parameterserver2:36,parameterset:29,parameterupdat:60,parameterupdater_:36,parametr:8,params_grad:58,paramt:[98,108],paraphrase_data:108,paraphrase_model:108,paraspars:86,parent:[31,61,63,68,86],parent_:[31,72],parent_idx:63,parenthes:68,pars:[4,13,14,32,43,50,84,97,98,109],parse_config:[4,109],parser:13,part:[2,7,8,31,40,53,61,63,65,83,86,91,92,93,106,109,111],parti:[84,92,103,104],partial:[8,28,109],partial_sum:8,particip:87,participl:108,particular:[49,53,68,92],particularli:20,partit:[33,35,43,65,66,68,93,98],pass:[2,8,13,18,21,26,28,31,33,42,48,53,58,59,60,62,63,64,68,70,72,75,85,86,92,93,95,98,99,109,111],pass_gener:8,pass_id:28,pass_idx:64,pass_test:109,passtyp:86,password:93,past:[29,78,81,98],patch:44,path:[1,2,7,13,14,15,28,33,37,38,41,57,64,77,78,93,95,97,98,99,103,104,108,110,111],path_to_paddlepaddle_working_directori:89,pattern:[14,33,54,59,70,98],paus:[33,40],pdf:[8,9],pem:[29,35,98],pend:[33,37],peopl:[8,84],pep425tag:80,pep8:85,per:[7,8,14,15,33,38,59,64,70,87,95,111],percal:91,perf_test:91,perfom:[95,97],perform:[1,8,9,20,21,30,38,42,46,48,60,64,65,68,70,83,84,86,87,90,93,94,103,104,109,111],perftool:91,period:[1,33,40,67,95,111],permiss:98,permut:21,peroid:[8,15],persist:[43,73,76,98],persistentvolum:98,persistentvolumeclaim:98,person:[7,29],perspect:[68,92],perturb:[30,86],peter:67,pex:106,pfs:[35,45,67],pfsclient:35,pfspath:45,pgp:98,phase:[21,57,59,64,69,106],philosophi:[59,70],photo:48,physic:106,pick:[2,98],pickl:93,pictur:111,piec:[8,9,43,88],pil:[15,93],pillow:41,ping:85,pip:[71,77,79,81,85,89,91],pipe:13,pipe_read:13,pipelin:42,pixel:[2,8,13,14],pixels_float:2,pixels_str:2,place:[1,2,21,33,40,65,66,68,86,87,88,92,110],placehold:[48,88,108],placement:66,plain:[1,7,8,13,41,53,55],plan:[33,68,86,103],platform:[31,80,85,87,88,98,103,104],pleas:[2,4,6,8,9,10,15,29,33,37,38,39,50,56,63,64,67,68,76,77,78,80,83,84,85,86,87,88,89,91,93,98,103,104,107,108,111],plot:29,plu:[8,30],plug:59,pne:87,png:110,pnpairvalidationlay:95,pnpairvalidationpredict_fil:94,pod:[35,41,98,99],pod_nam:98,point:[31,33,41,46,67,84,85,87,88,91,92,103,106],pointer:[31,38,50,63,68,72,88],polar:14,polici:98,pollut:40,polyak:59,ponit:50,pool3:86,pool:[2,9,21,27,111],pool_attr:9,pool_bias_attr:9,pool_layer_attr:9,pool_pad:[9,21],pool_siz:[2,8,9,21,22],pool_size_i:8,pool_strid:[9,21,22],pool_typ:[8,9,21,22],pooled_height:8,pooled_width:8,pooling_lay:[9,111],pooling_typ:[8,111],poolingtyp:11,pop:31,popul:38,popular:[32,48,50,110],port:[32,91,93,94,95,98,99],port_num:94,portabl:50,portal:89,ports_num:[93,95],ports_num_for_spars:[36,93,94,95,97],pose:33,posit:[2,7,8,9,111],positive_label:7,possibl:[29,31,37,63,66,70,92,109],post:[41,44],postpon:70,potenti:[46,92],power:[46,88,106,111],ppo_workspac:89,pprof:91,practic:[83,86],pre:[2,8,9,14,29,38,77,85,98,99,103,104,108],pre_activ:63,pre_bia:63,pre_dictandmodel:108,pre_stat:[56,75],preambl:63,precis:[7,42,46,59,77],precision_evalu:7,pred:[50,111],predetermin:[8,95],predic:14,predict:[2,3,7,8,28,65,70,81,83,95,108,111],predict_fil:95,predict_lay:28,predict_output_dir:[94,95,111],predict_sampl:4,predicted_label_id:111,prediction1:28,prediction2:28,prefetch:[36,86],prefix:[7,9,33,35,57,87,98],pregrad:86,premodel:108,prepand:63,prepar:[4,30,41,60,83,100,111],prepend:63,prepend_oper:63,preprocess:[14,15,75,99],present:[29,31,75,110],preserv:45,press:20,prev_batch_st:[94,95],prevent:[1,10,21,29,33,37,40,70,91],preview:[68,89],previou:[8,9,28,33,45,56,57,66,86,91,95,98],previous:[8,99,110],previous_memori:31,price:[14,68,81],primari:49,primarili:[59,70],primer:85,primit:[46,75],principl:[29,32],print:[6,28,29,50,65,80,81,91,93,95,108,111],print_graphviz:50,print_s3_bucket:13,printallstatu:92,printer:7,printstatu:92,priorbox:8,prioriti:68,prite:7,privat:[31,55,63,72,73,74,75,85,88],privileg:[84,98],prob:[7,28,81,109],probabilist:[8,108],probability_of_label_0:111,probability_of_label_1:111,probabl:[7,8,21,28,57,78,83,85,110,111],problem:[4,8,29,30,32,40,48,49,59,68,70,80,84,111],proc:78,proc_from_raw_data:111,proce:[13,33,64,78,98],procedur:[31,53,88,108],proceed:[8,20],process:[1,2,4,6,8,9,13,29,31,35,36,37,40,42,50,53,65,70,74,83,85,91,93,95,97,98,99,108,110,111],process_num:13,process_pr:111,processdata:110,processor:[46,92],produc:[8,9,13,33,50,64,110,111],product:[8,9,21,41,86,98,111],productgraph:99,prof:91,profil:[16,45],proflier:92,program:[1,13,18,20,21,25,29,35,38,40,49,51,58,64,65,67,68,72,91,92,95],programdesc:[43,53,62,63,67,69],programm:[63,65],progress:[33,37,95],proivid:2,proj:8,project:[8,9,41,55,83,86,87],promis:[8,9,57],prompt:45,prone:29,propag:[8,10,59,87,95,97],proper:93,properli:[84,111],properti:[2,50,70,95],propos:[31,57,58,59,66,75],protect:[46,74,86,87],proto:[11,53,61,68,74,76,87],proto_:74,protobuf:[28,31,41,43,49,50,53,61,63,68,69,74],protoc:[103,104],protocol:[7,95,106],prove:111,provi:93,provid:[8,14,29,31,38,41,42,46,48,50,52,59,63,67,70,74,75,78,81,88,91,92,93,98,103,106,108,109,110],providermemory_threshold_on_load_data:94,provis:[98,106],provod:2,prune:[8,31],ps_desir:33,pserver:[28,36,38,39,41,68,93,94,95,98],pserver_addr:38,pserver_cpu:41,pserver_id:34,pserver_mem:41,pserver_num_thread:[36,94,95],pserver_spec:28,pserverstart_pserv:94,pseudo:[29,41,69,75],pseudocod:75,psize:86,ptr:55,pub:93,pull:[32,68,71,85,108],purchas:111,purpos:[8,33,65,66,92],push:[31,85],push_back:86,put:[32,33,36,63,66,86,99,103,111],pvc:98,pwd:[77,78,84,89,103],pxe:106,py_paddl:[4,109],pybind:[31,46],pydataprovid:[1,2,111],pydataprovider2:[3,4,111],pypi:80,pyramid:8,pyramid_height:8,python2:91,python3:80,python:[1,2,3,13,21,28,29,31,39,42,47,48,49,50,54,57,60,68,71,75,77,78,80,81,83,84,85,89,93,108,109],qualcomm:46,qualiti:111,queri:[7,8,98],query_id:7,question:[8,29,66,74,98],queue:66,quick:[50,95,99],quick_start:[41,98,99,100,111],quick_start_data:99,quickli:[57,63,68],quickstart:99,quit:[57,92],r14b:103,r_t:8,rac:8,rais:[13,50,93],rajathkmp:48,ran:[66,92],rand:[48,92,95,97,109],random:[2,6,8,13,20,21,35,48,60,63,64,87,93,95,109],random_crop:15,random_imag:35,randomli:[15,21,40],randomnumberse:94,rang:[2,8,13,20,35,46,48,64,65,67,74,85,95,97],rank:[8,21,29,75,88,98,110,111],rank_tabl:21,rankdir:50,ranktabl:21,rapid:69,rare:2,raspberri:105,raspberry_pi:104,raspberrypi:104,raspbian:104,rate:[6,7,8,9,10,14,38,86,111],rather:[4,41,48,75,98],ratio:[8,95],raw:[8,13,53,111],rdma:95,rdma_tcp:[94,95],reach:[33,92],read:[1,2,13,15,21,28,29,33,35,64,65,66,68,75,78,83,84,89,93,98,103,106,110,111],read_from_realistic_imag:29,read_from_rng:29,read_lock:34,read_mnist_imag:29,read_ranking_model_data:29,readabl:[68,91],reader:[0,14,28,35,46,48,49,61,65,67,91,93],reader_cr:35,reader_creator_bool:64,reader_creator_random_imag:[13,64],reader_creator_random_image_and_label:[13,64],readi:[33,98,99,106],readlockguard:36,readm:55,readonesamplefromfil:2,readwritebuffer_:36,readwritemani:98,real:[2,8,36,48,64,93,109],realist:29,realiz:[31,56],realli:70,reason:[9,29,30,33,85,99],recal:7,receiv:[13,33,41,56,66],recent:59,reciev:95,recognit:[2,8,110],recommand:2,recommend:[1,9,29,77,78,79,83,85,86,89,93,95,103],recompil:92,record:[13,37,74,98],recordio:[13,14,29,35,37,65,67],recov:[33,75,109],recover:68,recoveri:37,rectifi:[8,20],recurr:[56,72],recurrent_group:[9,83],recurrent_lay:9,recurrent_op:75,recurrentgradientmachin:[55,57,75],recurrentgroup:7,recurrentlay:95,recurs:[31,32,45,68],recv:[65,66,98],recvparametertyp:36,red:[48,91],redirect:13,reduc:[8,21,46,66,68,78,85,91,93,95,97],reduce_by_kei:68,reduce_mean:48,refactor:[49,57,59,60,63,65,66,70,75],refer:[1,4,6,8,9,10,15,20,21,25,30,31,33,37,38,39,46,50,56,61,63,68,70,72,75,76,77,78,83,84,86,87,88,99,103,108,111],referenc:[8,37],reflect:37,reformat:85,refrain:87,reg:74,regard:106,region:[8,72,92],regist:[69,86,92],register_gpu_profil:92,register_lay:86,register_op:[47,68,69,74,87],register_op_cpu_kernel:87,register_op_gpu_kernel:87,register_op_without_gradi:[68,87],register_oper:[52,69],register_tim:36,register_timer_info:92,registerop:74,registr:87,registri:[41,52,99,106],regress:8,regular:[6,16,21,86,98,111],regularization_coeff:26,reinforc:67,rel:[1,9,30,40,70,87,103],relat:[2,33,40,41,46,72,85,91,99,104,106],relationship:[69,109],releas:[67,71,98,103,104],relev:87,reli:[30,57,58,59,70,87,91],reliabl:[33,70],relu1:50,relu2:50,relu:[8,48,50,86],relwithdebinfo:91,remain:[75,111],rememb:[8,85],remot:[6,32,36,68,85,86,95,97,98],remote_ess:67,remote_sess:67,remoteparameterupdat:[36,39,95],remov:[13,45,57,85,95,103],ren:20,renam:[45,46,80],reorgan:8,repeat:[31,49,61,62,73,74,76,91],repeatedli:49,replac:[32,37,52,59,67,69],repli:85,replic:65,replicaset:41,repo:[32,85,89,104],report:[37,46,65,92],reportdataset:37,repositori:[8,89,103],repres:[2,4,8,9,31,37,50,53,57,59,63,66,67,68,70,73,75,76,83,86,98,111],represent:[8,38,48,49,57,65,73,111],reproduc:84,request:[32,33,36,40,65,68,71,85,98,99,106,108],requir:[1,7,8,23,29,33,38,40,41,45,46,50,56,59,61,62,65,66,68,70,73,74,76,80,84,85,86,87,89,93,98,99,103,104,106,109,111],res5_3_branch2c_bn:110,res5_3_branch2c_conv:110,research:[8,14,65],reserv:[2,45],reserveoutput:86,reset:[8,18,33,42],reset_program:[18,42],reshap:[30,64,88],reshape_s:8,resid:84,residu:110,resiz:[15,36,87,88],resize_s:15,resize_short:15,resnet_101:110,resnet_152:110,resnet_50:110,resolv:[32,85,99],resourc:[67,98],respect:[2,30,46,48,56,83,86,95,110],respons:[8,36,42,48,59,60,70,98,99],rest:[2,8,31,41,44,106],restart:[33,38,98,99,106],restartpolici:[98,99],restor:[30,59],restrict:[70,72,91,95],result:[4,5,7,8,21,25,28,30,37,42,48,49,50,53,57,58,60,65,87,88,91,92,95,98,110,111],result_fil:7,resum:40,ret:13,retain:88,retran:98,retriev:[31,57,72,84,86,91,99],retriv:93,return_op_list:23,return_seq:9,reuqest:71,reus:[31,40,57,64,68,86,87],rev:84,revamp:65,reveal:[29,91],revers:[8,9,83],review:[14,99,111],reviews_electronics_5:99,revis:111,rewrit:[32,87],rgb:[8,15],rho:10,right:[2,8,30,31,32,41,42,68,70,85,110],rkt:[41,84],rmsprop:[59,111],rmspropoptim:59,rnn:[8,9,21,31,48,57,63,68,72,90,94,111],rnn_bias_attr:83,rnn_layer_attr:83,rnn_out:83,rnn_output:75,rnn_step:8,rnn_use_batch:[94,95],rnnalgorithm:57,rnnlm:14,rnnstep:75,roadmap:75,robust:[8,20],roi:8,role:[14,29,37,38,65,98],rollback:63,root:[10,11,98,99],rot:8,roughli:[2,109],round:46,routin:46,row:[4,7,8,13,21,36,86,110],row_id:8,rows_:73,rpc:37,rpcserver:37,rpi:104,rpi_arm_neon:104,rpi_toolchain:104,rsize:98,rtk:106,rtype:[8,13],rule:[7,49,65,86,98],run:[29,30,31,32,33,41,42,43,46,47,48,49,50,56,58,59,61,62,65,66,67,68,72,73,79,80,81,84,85,86,88,89,91,92,93,95,98,100,101,103,104,106,108,110,111],run_test:77,runinitfunct:92,runnabl:66,running_on_cloud:41,runserv:89,runtim:[1,2,25,31,43,52,56,67,68,76,78,93,103],runtime_table_:31,s_param:109,s_recurrent_group:83,sacrif:1,safe:41,sai:[8,49,51,64,65,84,95,97],sake:86,same:[2,4,7,8,9,20,21,28,29,30,37,38,40,47,48,50,56,57,63,65,67,68,69,72,75,77,83,87,88,93,97,98,108,111],samping_id:8,sampl:[2,4,7,13,14,42,48,74,78,93,95,97,108,109,110,111],sample_dim:109,sample_fil:13,sample_id:7,sample_num:7,sample_pars:13,sampler:48,satifi:7,satisfi:[32,73,80,98,111],save:[2,8,13,28,33,35,37,38,41,49,50,53,59,65,73,76,84,93,95,97,98,99,110,111],save_dir:[95,97,99,109,111],save_only_on:[94,95],save_parameter_to_tar:28,saving_period:[94,95],saving_period_by_batch:[94,95,97,111],saw:2,scalabl:68,scalar:[2,8,21,31,51,75],scale:[5,20,59,65,66,69,74,87,93,110],scaleop:87,scaleopmak:[68,87],scalingproject:8,scan:[37,68],scatter:8,scenario:[57,94],scene:94,schdule:98,schedul:[37,41,66,98,109],scheduler_factor:6,schema:108,scheme:[7,10,36,70,87],scope:[30,43,67],score:[7,8,21,57],scp:93,scrip:111,script:[4,14,77,84,87,93,98,103,110,111],search:[8,33,72,77,83,95],second:[2,8,21,29,45,48,50,56,57,61,62,64,72,74,87,93,108,110,111],secret:98,section:[2,63,66,83,85,86,91,98,111],see:[2,4,8,9,29,33,46,63,67,85,87,88,91,92,98,108,109,110,111],seed:[20,21,92,95],seem:[32,46,80],seen:[70,87],segment:[7,56,75,88],segmentor:108,sel_fc:8,selcet:8,select:[8,57,98],selected_generation_scor:57,selected_id:[8,57],selected_indic:8,selected_row:[73,76],selected_rows_desc:[73,76],selected_scor:57,selectedrow:[52,76],selectiv:8,selector:99,self:[30,42,48,50,53,58,63,75,86,87],selfnorm:8,semant:[14,29,57,71],semat:29,send:[33,38,65,66,68,74,76,85,93,95,98],send_back_parameter_typ:36,sendbackparameterspars:36,sendbackparametertyp:36,sendparameterrequest:36,sendparameterrespons:36,sens:[59,70,85,91],sensit:8,sent:[29,38,65,68,74,99],sentenc:[2,8,14,56,57,75,83,111],sentence_input:75,sentiment:[2,111],sentimental_provid:2,separ:[2,7,25,38,47,59,65,69,70,93,95,108,111],seper:75,seq:[8,14],seq_len:75,seq_pool:8,seq_silc:8,seq_text_print:7,seq_to_seq_data:108,seq_typ:[2,13],seqenc:21,seqtext_evalu:7,seqtoseq:[8,108],seqtoseq_net:[8,108],sequel:2,sequenc:[2,5,7,8,9,11,13,14,21,31,43,49,58,61,75,85,86,108,111],sequence_conv_pool:111,sequence_group:8,sequence_nest_group:8,sequencestartposit:8,sequencetextprint:7,sequencetyp:[2,8],sequenti:[8,31,83,111],seri:[9,80],serial:[2,28,31,37,53,60,68,76],serializ:68,serv:[46,65,68,75,78,92,93,98,109],server:[29,32,36,39,40,65,68,77,86,94,106],serverless:33,servic:[91,93,106],sess:[48,50,58,67],session:[50,58,62,92],set:[1,2,4,6,7,8,9,13,14,15,21,25,28,29,33,41,48,52,56,57,62,63,68,69,72,75,77,83,84,85,86,87,88,89,90,91,92,93,94,95,97,98,99,104,108,110,111],set_active_typ:86,set_default_parameter_nam:6,set_drop_r:86,set_float_el:30,set_input:8,set_siz:86,set_typ:86,setdatatyp:73,setdefault:87,setp:98,setq:84,settup:86,setup:[2,59,65,71,86,87,106,111],sever:[2,7,8,30,36,43,48,56,57,60,63,73,75,77,93,97,98,111],sexstant:106,sgd:[23,28,29,33,41,59,60,65,66,67,73,93,109],sgdasync_count:94,shall:32,shaoq:20,shape:[7,8,13,18,21,28,30,31,48,51,56,61,63,67,68,73,87,88,110],shard:[33,34,35,36,37,38,40,65,66,93,98],share:[8,21,32,48,55,60,63,68,70,75,84,87,92,95,99],shared_bia:9,shared_bias:8,shared_librari:32,shared_ptr:[54,55,72,88],shell:[78,98,110],shift:[8,110],shorten:8,shorter:[15,110],should:[2,4,6,7,8,13,15,18,20,21,23,25,26,28,29,30,31,38,41,42,43,46,47,48,52,56,57,58,59,60,61,64,65,68,69,70,73,74,75,76,81,83,87,89,93,98,103,111],should_be_fals:29,should_be_tru:29,should_shuffl:2,show:[4,7,10,31,33,45,51,53,56,59,61,75,80,84,88,93,95,98,99,108,110,111],show_check_sparse_distribution_log:[94,95],show_layer_stat:[94,95],show_parameter_stats_period:[94,95,97,99,111],shown:[2,8,29,42,65,83,86,88,92,98,109,110,111],shrink:86,shrink_rnn_memori:21,shuffl:[2,13],sid:98,side:[8,28,42,60,88,93,110],sig:98,sigint:93,sigmod:74,sigmod_op:74,sigmod_output:74,sigmoid:[8,22,31,67,74,75,86],sigmoidactiv:9,sign:[44,53,98],signal:93,signatur:98,signific:92,similar:[8,21,31,57,59,64,65,66,68,70,75,87,91,98,106,111],similarli:[8,13,87],simpl:[1,2,5,7,8,9,13,14,23,28,46,49,50,56,59,61,66,70,72,74,75,92,95,111],simple_attent:83,simple_gru:[83,111],simple_lstm:[8,111],simple_rnn:[8,83],simple_transform:15,simpler:60,simplest:98,simpli:[1,8,15,29,38,78,81,83,92,108,110],simplifi:[29,57,63,74,86,99],simultan:98,sinc:[8,9,33,37,39,40,52,59,64,65,67,69,70,75,88,92,98,106,109,111],sincer:85,singl:[2,7,9,13,33,42,46,65,66,67,68,72,77,81,86,91,93,99,110,111],sinlg:28,sit:65,site:[32,91,98],situat:62,six:108,size:[2,7,8,9,13,14,15,21,28,33,35,36,38,46,48,53,57,59,63,64,65,67,73,74,75,78,81,83,86,87,88,95,103,104,109,110,111],size_a:8,size_b:8,size_t:[36,75,86],sizeof:[31,108],skip:[64,85,93,98,110],sliceproject:8,slide:[8,10,14,33],slightli:48,slope:8,slopeinterceptlay:8,slow:[2,92],slowli:[84,91],small:[2,8,14,30,43,48,57,85,86,95],small_messag:[94,95],smaller:[8,21,30,33,46,57,85],smart:72,smooth:8,snap:99,snapdragon:46,snapshot:[34,40,98],snippet:[47,58,83,86,92,98,111],sock:41,sock_recv_buf_s:[94,95],sock_send_buf_s:[94,95],socket:95,softmax:[8,9,29,31,50,51,57,61,65,66,67,83,86,108,111],softmax_param_attr:9,softmax_selfnorm_alpha:8,softmaxactiv:111,softmaxoutput:50,softwar:[46,92,106],solid:48,solut:106,solv:[29,68],some:[2,6,8,13,15,21,28,29,31,32,36,37,38,40,41,46,47,48,49,56,57,58,61,62,63,64,66,68,69,72,75,85,86,87,88,92,93,94,95,97,98,103,104,106,109,111],some_c_api_funct:55,some_inst:55,some_op:[52,56,75],some_python_class:54,somecppclass:54,somedata:28,somegotyp:54,someth:[2,36,63,84,85,91],sometim:[8,64,84,92],someurl:13,somewhat:38,somewher:72,soon:33,sophist:86,sort:[8,14,75,91,95,98],sort_by_length:75,sourc:[8,14,30,32,45,48,53,55,57,64,68,83,84,91,93,98,99,108,111],source_dict_dim:[57,83],source_dict_s:57,source_language_word:[57,83],space:[7,8,46,63,66,70,83,84,92],space_seperated_tokens_from_dictionary_according_to_seq:7,space_seperated_tokens_from_dictionary_according_to_sub_seq:7,spars:[2,6,8,10,13,21,36,86,88,93,95,98,111],sparse_binary_vector:[2,13,111],sparse_binary_vector_sequ:13,sparse_float_vector:[2,13],sparse_float_vector_sequ:13,sparse_non_value_slot:13,sparse_remot:36,sparse_upd:[6,36],sparse_value_slot:13,sparseparam:86,sparseprefetchrowcpumatrix:86,spatial:8,spatial_scal:8,speak:83,spec:[98,99],specfii:95,special:[8,38,46,52,57,58,65,87,108,111],specialvartypeinfer:52,specif:[1,28,32,33,45,57,68,72,84,87,97,103,111],specifi:[1,2,7,8,18,21,29,30,36,37,38,41,42,43,45,48,53,63,65,67,72,74,75,78,83,84,85,86,88,89,91,95,98,103,109,110,111],speech:8,speed:[8,9,46,53,59,77,106],spefici:110,sphinx:[54,89],split:[2,8,13,40,51,57,68,75,93,97,98,108,110,111],split_count:[93,98],sql:1,sqrt:[8,20],squar:[8,10,11,21,50],squarerootnpool:8,srand:95,src:[32,93],src_backward:83,src_embed:[57,83],src_forward:83,src_root:4,src_word_id:[57,83],src_word_vec:57,srl:14,ssd:8,ssh:[93,98,99,104],ssh_server:93,sstabl:29,stabil:[8,30,87],stabl:[71,98],stack:[68,75,98,111],stackexchang:8,stage:[32,39,48,76,93,103],stale:[33,67],stamp:92,standalon:103,standard:[6,13,20,68,70,80,84,91,108],stanford:[14,30,99],star:32,start:[8,9,21,28,32,33,36,37,38,40,41,57,60,65,77,78,80,83,84,91,92,95,102,108],start_mpi_train:93,start_pass:[94,95],start_po:8,start_pserv:95,startup:[21,33,41,98],startup_program:[18,21,22,23],stat:[92,95],state:[8,9,18,23,31,33,42,56,57,72,75,83,95,99,109],state_act:[8,9],statement:[49,86,98],static_cast:88,staticinput:[8,83],statist:[8,18,20,42,95,111],statset:92,statu:[41,57,92,98,99],status:99,std:[28,32,36,50,52,54,55,62,68,69,72,74,86,87,88,95],stdbuf:93,stderr:93,stdout:[13,93],step:[4,8,9,11,23,30,31,33,38,42,48,49,57,59,60,63,65,66,68,74,75,77,78,83,85,86,91,92,93,98,99,103,104,106,111],step_id:75,step_input:75,step_net:31,step_output:75,step_scop:68,stepnet:[31,56,68,72],still:[37,40,65,69,80,88,110],stirng:63,stmt1482205552000:98,stmt1482205746000:98,stochast:[10,33,37,40,59,93],stop:[8,84,93,95,99],stop_gradi:21,storag:[44,46,93,98,99],store:[7,8,14,28,30,31,32,36,50,52,53,57,60,61,63,65,67,68,69,70,72,75,86,87,88,89,93,95,98,99,104,108,110,111],str:[15,18,28,41,75,97],straight:[61,64,73],strategi:[2,11,33,63,66,95],stream:13,streamid:25,street:8,strength:109,strict:[64,93],stride:[8,9,21],stride_h:21,stride_i:8,stride_w:21,stride_x:8,string:[1,2,7,8,13,15,25,28,31,37,45,50,53,61,62,63,67,68,69,72,73,74,76,86,87,95,98],strip:[91,111],strongli:93,struct:[37,38,44,46,52,55,69,74],structur:[31,37,48,53,57,61,63,68,73,93,98,108,111],sts:98,stuff:85,stun:2,style:[2,8,68,74,77],sub:[7,8,13,29,40,48,56,60,63,65,83,86,103,111],sub_nest_seq:8,sub_sequ:[2,8],subclass:[23,63],subcommand:45,subgradi:10,subgraph:[48,66],submiss:65,submit:[68,89,93,94,95,98],subnet0:98,subnet:[29,98],subobjectpath:99,subsequ:8,subsequenceinput:8,subset:[21,86],substanti:110,succeed:[37,99],success:[8,38,98,99,110],successfulcr:99,successfulli:[87,110],successor:95,sudo:[84,98],suffer:30,suffici:95,suffix:[18,41,80,93],suggest:[8,32,85,92],suit:106,suitabl:[73,95],sum:[8,10,31,34,52,63,83,86],sum_:8,summar:[48,111],sumopgradmak:69,sumpool:8,sun:20,suppli:73,support:[6,7,8,10,11,13,15,21,30,31,33,40,41,47,48,53,57,59,60,62,64,65,66,68,69,70,73,77,78,80,81,83,84,86,87,88,89,92,93,95,98,103,104,106],suppos:[9,32,47,73,86,111],suppress:[8,45],sure:[77,84,86,91,98],surpass:[8,20],svs:74,swagger:44,swap_channel:110,swig:[4,39,54,55,77,103],swig_paddl:[4,109],switchop:31,symbol:[8,31,50,55,80],symbols_ready_:31,symbolt:[31,68],symlink:85,sync:[33,59,70,95,109],sync_with_cpp:91,syncflag:86,synchron:[33,37,93,95,98],syntax:[57,64],synthes:109,sys:110,sysroot:103,system:[31,32,33,38,40,44,47,48,65,66,77,78,80,87,89,91,93,99,103,111],t2b:108,tab:[80,111],tabl:[2,7,8,21,31,52,53,73,76,110,111],tablelookup:73,tablelookupgrad:73,tablelookupop:73,tableproject:8,tag:[7,14,78,83,93],tagtyp:7,tail:57,take:[2,4,7,8,9,13,21,28,29,31,32,33,40,43,46,48,49,51,52,59,61,62,63,64,68,69,75,77,83,84,85,86,87,91,92,93,98,99,109],taken:[2,50,65,75],talk:[38,104],tangl:91,tanh:[8,9,21,48,57,65,83,86],tanhactiv:9,tar:[13,15,28,98],tarbal:98,target:[8,14,21,23,28,31,32,48,50,58,62,65,67,68,77,83,87,103,104,108,111],target_dict_dim:83,target_dict_s:57,target_dictionary_dim:8,target_language_embed:8,target_language_word:83,target_link_librari:32,target_word:57,targetinlink:8,task:[2,7,8,53,57,65,74,83,97,108,110],task_queu:37,taskentri:37,taskqueu:37,tbd:39,tcp:[95,98],teach:111,tear:92,technic:33,techniqu:[21,83,86,91],technolog:84,tee:99,tell:[33,37,38,57,74,78,92,103],templat:[47,74,87,88,99,106],tempor:[8,111],temporari:[18,41,59,63,107],ten:84,tensor:[21,30,32,46,48,50,52,53,56,57,66,67,73,75,76,87],tensor_array_read:75,tensor_array_s:75,tensor_array_stack:75,tensor_array_unstack:75,tensor_array_writ:75,tensor_data:53,tensor_s:30,tensor_test:32,tensor_to_check:30,tensorarraydesc:75,tensordesc:[53,73],tensorflow:[31,48,51,65,66,70,75,88],term:[8,9,21,33,69,70],termin:99,tese:1,tessorarrai:75,test100:14,test10:14,test1:35,test:[1,2,8,13,14,15,21,28,29,30,32,50,55,59,64,67,71,81,84,88,92,93,94,108,110,111],test_:87,test_all_data_in_one_period:99,test_check_grad_ingore_i:87,test_check_grad_ingore_x:87,test_check_grad_norm:87,test_data_dir:93,test_fcgrad:86,test_gpuprofil:92,test_layergrad:86,test_list:[2,111],test_mul_op:[77,87],test_norm:87,test_pass:[94,95,97],test_period:[94,95,97],test_recurrent_op:85,test_wait:[94,95],testa:29,testb:29,testbilinearfwdbwd:92,testcas:87,testconfig:86,testfcgrad:86,testfclay:86,testlayergrad:86,testmodel_list:94,testmulgradop:87,testmulop:87,testq:29,testresult:28,testsave_dir:94,testutil:86,text1:45,text:[1,2,7,9,13,29,53,56,98,108,111],text_conv:111,text_fil:13,tflop:92,tftp:106,tgz:[14,80],than:[2,4,6,7,8,9,21,33,41,47,48,63,68,70,75,77,83,84,86,93,98,103,106,110],thank:108,thei:[2,8,18,20,26,29,32,33,38,40,43,45,48,49,57,58,63,66,67,68,74,75,76,83,84,85,86,87,88,92,93,94,98,110],them:[1,2,7,8,9,15,21,29,30,32,33,36,41,43,47,52,57,64,66,68,69,72,73,74,75,76,84,85,87,89,92,94,95,98,110,111],themselv:32,theori:92,therefor:59,therein:[8,31],therun:110,theta:48,theta_d:48,theta_g:48,thi:[1,2,6,7,8,9,10,13,14,15,18,20,21,23,25,26,28,29,30,31,32,33,36,37,38,39,40,41,42,46,47,48,49,50,56,57,58,59,60,61,63,64,65,66,67,68,69,70,73,74,75,78,80,81,83,84,85,86,87,88,89,91,92,93,95,97,98,99,103,104,106,108,109,110,111],thin:52,thing:[2,48,65,68,92],think:[29,32],third:[8,33,50,87,91,92,103,104,110],third_parti:[8,103,104],thirt:84,those:[8,31,32,33,47,49,50,51,61,103,110],though:[75,106],thought:[32,92],thread:[86,91,92,95,97],thread_local_rand_use_global_se:[94,95],threadblocks:25,threadid:97,threadloc:92,three:[2,7,8,30,33,42,46,49,57,58,60,61,64,95,103,109,110],threshold:[6,7,8,33,37,85,95],through:[4,8,21,32,33,37,39,42,58,59,67,83,86,87,89,92,93,108,109],throughout:[43,111],throughput:[92,93],thrust:68,thu:[2,8,40,42,50,86,98,107],tier:99,time:[2,8,9,11,13,28,29,30,32,33,37,40,47,52,56,57,63,64,65,66,68,69,73,74,75,76,77,83,84,88,91,92,95,97,99,106,111],timelin:[8,68,92],timeo:98,timeout:[33,37],timeout_sec:13,timestamp:[8,34],timestep:[2,8,72],tip:103,titl:14,tls:44,tmp:63,to_chw:15,to_no_sequ:8,to_sequ:8,to_tar:28,todo:[7,13,14,31,33,37,40,57,74],toend:8,togeth:[2,8,9,13,28,75,83],token:[7,8,29,83,108],toler:[28,30,77,87],too:[14,30,75],took:106,tool:[77,80,83,84,91,98,103,104],toolchain:[91,103],top:[7,21,28,56,57,87,110],top_k:[7,21,57],top_level_rnn:56,topk_generated_scor:57,topk_id:57,topk_scor:57,toplevel:84,topolog:[29,33,50,53,60,65,67],topoloi:50,topolopi:28,torch:31,toronto:[8,14],total:[21,28,33,42,64,66,91,92,93,99,106,108],total_pass:64,tottim:91,trace:[31,48],track:[33,37,50,63],tractabl:8,tradit:[8,31,46],traffic:65,trail:13,train100:14,train10:14,train:[0,1,2,4,6,7,8,13,14,15,20,21,31,35,37,38,40,42,48,49,53,59,60,61,62,67,68,70,73,76,83,86,90,92,94,100,101,110],train_conf:108,train_config_dir:98,train_data:93,train_data_dir:93,train_id:98,train_list:[2,93,110,111],trainabl:[8,53,63],traindot_period:94,trainer:[2,4,29,34,35,36,37,39,59,60,65,66,68,86,95,97,109,111],trainer_config:[1,2,98,99,111],trainer_config_help:[2,86,111],trainer_count:[81,93,94,95,97,98,99],trainer_cpu:41,trainer_cr:41,trainer_gpu:41,trainer_id:[93,95,98],trainer_intern:36,trainer_mem:41,trainer_packag:41,trainerid:40,trainerintern:111,training_machin:109,trainingtest_period:94,trainonebatch:36,trainonedatabatch:109,tran:[86,95],trane:2,transact:[33,37],transfer:[1,2],transform:[8,9,15,21,68,83,86,88,109,111],transform_param_attr:9,transformed_st:9,translat:[8,9,108],translation_id:57,translation_scor:57,transpar:[57,93],transport:95,transpos:[8,15,86,109],transposedfullmatrixproject:8,travel:2,travers:49,travi:85,treat:[8,31,38],treatment:[38,46],tree:[8,31,63,95,104],trg_dic_siz:57,trg_embed:[57,83],triain:1,trick:57,tricki:54,trigger:[40,60],trim:8,trivial:[2,57,75],trn:111,true_block:[31,51,61],true_imag:64,true_label:64,true_neg:42,true_posit:42,true_read:64,truth:[7,8,111],tst:111,tune:[6,90,91,111],tuninglog_barrier_abstract:94,tupl:[2,8,9,13,14,15,18,21,28,63,64],ture:8,turn:[8,63,64,78,109],tutori:[77,78,83,86,87,91,92,93,98,99,100,101,107,110,111],twice:[48,66],twine:71,two:[1,2,7,8,9,21,29,38,39,40,41,42,45,46,48,49,52,53,57,59,61,64,65,67,68,69,70,72,73,74,75,76,83,87,88,92,93,97,98,103,104,108,109,110,111],txt:[2,32,41,45,86,89,93,98,111],type:[2,7,8,9,11,13,14,15,18,21,23,28,29,31,33,36,37,40,41,44,45,46,52,53,54,55,56,57,61,62,63,64,68,69,70,73,74,76,81,83,84,86,88,93,95,97,98,99,110,111],type_nam:74,typedef:[38,46,54,55],typeid:74,typenam:[47,74,87,88],typic:[4,7,65,92],ubuntu:[71,80,81,91],ubyt:64,uci:14,uci_h:81,ufldl:8,uid:99,uint16_t:46,uint32:[44,53],uint64:[53,54],uint64_t:54,unawar:38,unbalanc:95,unbound:83,unclear:40,under:[32,37,66,77,78,88,89,93,98],underli:57,understand:[20,63,91,92,106,108],understand_senti:83,undeterminist:92,unidirect:8,unifi:[50,73,85],uniform:[6,8,13,20,35,48,63,64,95,109],uniform_random:63,uniqu:[29,31,33,40,41,63,72,87,93,95,98],unique_nam:63,unique_name_gener:63,unique_ptr:[69,72,86],unit:[8,9,21,32,59,70,77,83,84,88],unittest:[55,85,87],unittestcheckgrad_ep:94,unk:[73,76,108],unk_idx:111,unknown:8,unlik:[8,57,87],unnecessari:85,unordered_map:72,unpack:75,unrol:56,unseen:70,unseg:8,unsign:[38,46],unstack:75,unstack_from:75,unsupervis:48,unsupport:87,until:[33,38,66,72,98],unzip:103,updat:[6,8,10,23,33,37,38,44,46,48,56,57,58,59,60,65,72,75,80,86,91,93,95,97],update_equ:28,update_hook:6,update_memori:31,update_op:58,updatecallback:86,updatestack:98,upgrad:[80,107],upload:[33,41,44,71,93],upon:33,upper:8,upstream:85,uri:98,url:[13,14,85],usag:[1,2,7,8,9,15,28,46,51,60,87,92,93,108,109],use:[1,2,4,6,7,8,9,11,13,14,15,20,21,23,26,28,29,30,31,32,33,39,43,46,48,50,52,57,58,60,63,65,66,67,72,73,74,75,76,77,78,80,81,83,84,85,86,87,89,91,92,93,95,97,98,99,103,104,108,109,110,111],use_eigen_bla:103,use_eigen_for_bla:103,use_etcd:28,use_global_stat:8,use_gpu:[81,93,94,95,97,99,109,110,111],use_mkldnn:8,use_nesterov:23,use_old_updat:[36,94,95],use_peephol:21,use_sparse_remote_updat:36,used:[1,2,4,7,8,9,10,11,13,14,15,20,21,25,28,29,30,31,32,33,39,40,46,48,50,56,57,59,60,63,64,65,67,68,70,72,74,75,77,80,83,84,86,87,88,91,92,94,95,97,98,103,104,108,110,111],useful:[1,2,8,9,30,46,72,83,86,97,103,111],usegpu:[86,109],user:[1,2,6,8,9,13,14,15,18,20,23,25,26,28,29,30,31,32,35,37,40,41,42,45,47,48,49,50,52,57,58,59,63,64,65,66,67,68,69,70,72,74,75,78,85,89,91,93,94,95,98,106,110,111],user_info:14,user_nam:35,usercert:35,userinfo:14,userkei:35,usernam:35,uses:[2,8,33,40,46,56,57,60,65,77,80,83,84,85,86,88,89,93,95,98,103,110,111],using:[1,2,4,6,8,9,13,21,28,29,31,32,33,37,38,40,41,45,46,47,48,50,52,56,58,59,61,64,65,69,70,72,74,77,78,79,80,81,83,85,86,87,88,89,92,93,95,97,98,99,103,104,107,108,109,110,111],usr:[77,78,93,95,98],usrdict:108,usrmodel:108,usual:[8,28,41,61,70,85,87,91,92,95,97,98],utf:108,util:[4,83,86,87,92,106],uuid:[34,40],v28:8,v7a:103,v8a:103,valid:[8,15,64,68,72,87,98,110],valu:[2,4,6,7,8,11,13,14,15,20,21,25,28,30,31,33,42,50,51,53,56,57,58,59,60,61,63,65,67,68,72,73,74,75,76,83,86,87,95,97,98,103,109,110],value1:95,value2:95,value_:73,value_evalu:7,value_rang:13,valueerror:50,values_:75,vanilla:83,varabl:66,vardesc:[31,49,61,63,68,73],vardescbuild:31,vari:[92,98],variabl:[2,10,13,14,18,20,21,23,29,30,31,42,43,48,49,50,51,52,56,57,58,59,61,62,65,66,67,69,70,73,74,75,86,87,91,93,98,99,103],variablenamemap:87,varialbl:48,varianc:[8,110],variant:[8,52,75],varibl:50,varienc:75,varient:75,variou:[31,46,70,103],varproto:74,vars_:[31,72],vartyp:[21,73,76],vartypeinfer:52,vec1:8,vec2:8,vector:[2,8,9,13,14,21,29,31,36,38,50,51,56,57,63,68,69,73,75,83,86,88,108,111],vectorenable_parallel_vector:94,veloc:23,vendor:32,verb:14,verbos:[45,85],veri:[2,8,11,32,37,47,48,57,60,64,66,70,72,75,83,91,92,93,111],verifi:[31,86],version:[8,9,32,41,45,48,50,51,53,57,65,71,77,78,81,84,86,91,92,93,94,95,98,99,103,104,108],versu:29,vertic:[8,110],vgg:[9,22],via:[33,79,85,92,98,106,111],view:[8,53],vim:78,viriabl:93,virtual:[52,69,84],virtualenv:84,visibl:[40,72],visit:28,visual:[8,57,92],vlog:[36,85],voc_dim:111,voila:81,volum:[89,99],volumemount:[98,99],volumn:98,vutbr:14,wai:[2,7,9,29,38,40,57,59,63,64,67,70,75,83,84,85,86,97],wait:[33,38,93,95],walk:[4,109],wangkuiyi:32,want:[2,8,29,41,42,48,59,62,64,67,70,72,75,77,78,84,85,86,89,91,93,95,97,103,104,108,110,111],warn:[28,45],warp:[8,92],warpctc:8,watch:33,wbia:[98,110],web:[89,91],websit:[89,111],weight:[7,8,9,10,20,21,26,53,70,83,86,95,97,110],weight_act:9,weightlist:86,weights_:86,weights_t:86,welcom:32,well:[41,47,48,65,67,70,73,86,95,98,111],were:[7,32],west:98,wget:103,what:[6,8,32,48,57,63,65,66,74,87,91,106,111],whatev:[84,93],wheel:80,when:[1,2,6,7,8,10,13,18,21,28,30,31,32,33,36,37,38,41,42,45,46,50,57,59,60,61,63,65,66,68,75,77,79,83,84,85,86,87,88,89,91,92,93,95,97,98,99,103,106,108,109],whenev:[63,85],where:[2,8,9,10,20,21,29,31,33,40,49,56,57,59,61,65,68,70,75,83,86,87,91,92,95,97,108,110],wherea:[31,37,47,51],whether:[7,8,13,15,21,28,30,31,64,73,75,77,78,86,87,95,109],which:[1,2,4,6,7,8,9,13,14,15,21,28,29,30,31,32,33,35,37,38,40,41,43,46,47,48,50,52,53,56,57,58,60,61,62,63,64,65,66,67,68,69,72,73,74,75,77,80,83,84,85,86,87,88,91,92,93,95,97,98,103,106,109,110,111],whichev:109,while_loop:[57,75],whileloop:75,whileop:31,whl:77,who:[47,49,63,85,108,110],whoever:38,whole:[2,7,13,48,51,54,55,56,62,74,85,93,98,99,106,111],whose:[2,8,13,30,33,40,56,68,69,74,75,83],why:[9,30,55,84],wide:[32,48,80,93],width:[7,8,13,15,36,54,64,86,87],wiki:[8,32],wikipedia:[8,14],wilder:2,window:[8,11,14,59,78,84,103],wirt:50,wise:[8,15,66,68,88],wish:[77,80,89,93],with_avx:[77,78,93,103],with_bia:74,with_c_api:[77,103,104],with_doc:77,with_doubl:[77,86,93],with_dso:77,with_golang:77,with_gpu:[77,84,93,103],with_mkl:77,with_profil:92,with_python:[77,93,103],with_rdma:[93,103],with_style_check:[77,85],with_swig_pi:[77,103],with_test:[77,87],with_tim:[92,93],within:[8,37],without:[7,8,23,33,38,63,64,66,68,87,91,93,103],wloop:75,wmt14:83,wmt_shrinked_data:14,won:[92,93,110],wonder:2,word2vec:[41,93],word:[2,7,8,14,49,52,56,57,66,68,74,75,83,93,97],word_dict:[93,111],word_dim:111,word_id:2,word_idx:14,word_vector:111,word_vector_dim:[8,57,83,108],words_freq_sort:14,work:[2,4,8,13,21,29,31,32,33,46,58,59,63,65,78,83,84,85,86,89,91,92,93,95,98,99,106,111],worker:[66,76,98],workercount:98,workflow:[68,98],workspac:[85,93,95],world:93,wors:109,would:[28,31,32,33,40,47,48,49,58,59,60,63,64,65,66,73,75,78,84,85,91,98,103,106,109,111],wouldn:49,wrap:[47,48,106],wrapper:[9,32,47,59,65,69,75,92],write:[2,13,21,29,33,40,46,47,50,52,58,59,63,64,65,66,68,69,75,84,90,93,98],write_lock:34,writer:[29,63],written:[25,31,48,53,59,68,69,73,77,78,87,88,91,93],wrong:[2,64],wrote:[50,66],wsize:98,wustl:8,www:[8,14],x64:[103,104],x86_64:103,x_i:8,x_j:8,x_neg:30,x_num_col_dim:21,x_po:30,xarg:[7,78,86,93],xavier:20,xavieriniti:21,xgbe0:95,xgbe1:95,xiangyu:20,xmap_read:13,xrang:[30,48,64,81,86],xxx:[29,75,110],xxxx:34,xxxxxxxxx:98,xxxxxxxxxx:98,xxxxxxxxxxxxx:98,xxxxxxxxxxxxxxxxxxx:98,y_dim:48,y_neg:30,y_num_col_dim:21,y_po:30,y_predict:81,yaml:[32,93,98,106],yancey1989:41,yann:14,yapf:85,yeild:28,yep:91,yet:106,yield:[2,13,29,35,64,111],yin:8,yoshua:20,you:[1,2,4,6,8,9,13,28,30,41,46,72,77,78,79,80,81,83,84,85,86,89,91,92,93,95,97,98,103,104,106,108,109,110,111],your:[2,8,13,28,29,32,36,41,45,68,77,79,80,84,85,86,89,92,93,97,98,103,104,106,111],your_access_key_id:98,your_secrete_access_kei:98,your_source_root:55,yourself:77,yuyang18:[13,14],yuyang:91,z_dim:48,z_size:48,zero:[2,6,8,9,10,13,14,18,30,33,48,57,60,63,73,86,95,98,111],zhang:20,zhidao:108,zip:[14,63,103],zone:98,zxvf:98},titles:["API","Introduction","PyDataProvider2","API","Python Prediction","Activation","Parameter Attribute","Evaluators","Layers","Networks","Optimizer","Pooling","Data Reader Interface and DataSets","Data Reader Interface","Dataset","Image Interface","Fluid","DataFeeder","Evaluator","Executor","Initializer","Layers","Nets","Optimizer","ParamAttr","Profiler","Regularizer","Model Configuration","Training and Inference","PaddlePaddle Design Doc","Auto Gradient Checker Design","Design Doc: Block and Scope","Required CMake Function","Design Doc: Distributed Training","\u6a21\u578b\u53c2\u6570\u68c0\u67e5\u70b9\uff08Checkpointing\uff09","\u8bad\u7ec3\u6570\u636e\u7684\u5b58\u50a8\u548c\u5206\u53d1","Alalysis of large model distributed training in Paddle","Design Doc: Master Server","Design Doc: The Client Library of Parameter Server","Design Doc: Remote Parameter Updater for Cluster Train","Design Doc: Save Model","Submit a Distributed Training Job","Evaluator Design","Executor Design Doc","FileManager\u8bbe\u8ba1\u6587\u6863","PFSClient","Design Doc: float16","Design Doc: Functions, Operators, and Layers","Design for GAN","Design Doc: Computations as a Graph","Survey on Graph","The IfElse Operator","Design Doc: InferVarType","Design Doc: Model Format","Paddle\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0","C-API \u6a21\u578b\u63a8\u65ad\u5b9e\u73b0\u6587\u6863","RNNOp design","Design: Sequence Decoder Generating LoDTensors","Optimizer Design","Averaging Parameter in PaddlePaddle","Design Doc: The C++ Class Parameters","Design Doc: PaddlePaddle Programs","Prune","Design Doc: Python API","Python Data Reader Design Doc","Design Doc: Distributed Training Architecture","Design Doc: Operation Graph Based Parameter Server","Design Doc: Session","Design Doc: Refactorization Overview","Design Doc: Gradient Operators Registration","Regularization in PaddlePaddle","PaddlePaddle\u53d1\u884c\u89c4\u8303","Design of Scope in Paddle","Design Doc: Selected Rows","Interaction between C++ and Python","Design for TensorArray","Background","Build from Sources","Run in Docker Containers","Install and Build","Install Using pip","GET STARTED","RNN Models","RNN Configuration","Build using Docker","Contribute Code","Write New Layers","How to write a new operator","How to use Eigen in Paddle","Contribute Documentation","HOW TO","Profiling the Python Code","Tune GPU Performance","PaddlePaddle Distributed Training","Argument Outline","Detail Description","Set Command-line Parameters","Use Case","Distributed PaddlePaddle Training on AWS with Kubernetes","Paddle On Kubernetes","<no title>","<no title>","PaddlePaddle Documentation","Build PaddlePaddle for Android","Build PaddlePaddle for Raspberry Pi","MOBILE","Cluster bootstrapping tool survey","<no title>","Chinese Word Embedding Model Tutorial","Generative Adversarial Networks (GAN)","Model Zoo - ImageNet","Quick Start"],titleterms:{"\u4e0a\u4f20\u8bad\u7ec3\u6587\u4ef6":35,"\u4e0d\u4f7f\u7528":54,"\u4e0d\u4f7f\u7528swig\u8fd9\u79cd\u4ee3\u7801\u751f\u6210\u5668":54,"\u4e0d\u5bfc\u51fapaddle\u5185\u90e8\u7684\u7ed3\u6784\u4f53":54,"\u4e0d\u5f15\u7528\u5176\u4ed6\u52a8\u6001\u5e93":54,"\u4ec5\u4ec5\u4f7f\u7528void":54,"\u4ece\u5feb\u7167\u6062\u590d":34,"\u4f7f\u7528\u52a8\u6001\u5e93\u6765\u5206\u53d1paddl":54,"\u4f7f\u7528\u8f6c\u6362\u5e93":35,"\u5177\u4f53\u67d0\u79cd\u7c7b\u578b\u7684\u5934\u6587\u4ef6":55,"\u5177\u4f53\u67d0\u79cd\u7c7b\u578b\u7684\u5b9e\u73b0\u6587\u4ef6":55,"\u5206\u5757\u6587\u4ef6\u4f20\u8f93":44,"\u5206\u652f\u89c4\u8303":71,"\u52a0\u901f\u6267\u884c":34,"\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165\u4efb\u4f55\u5176\u4ed6\u8bed\u8a00\u7684\u89e3\u91ca\u5668":54,"\u52a8\u6001\u6269\u5bb9":34,"\u539f\u56e0":54,"\u539f\u56e0\u5217\u8868":54,"\u53c2\u8003\u6587\u6863":44,"\u540d\u8bcd\u89e3\u91ca":44,"\u57fa\u672c\u8981\u6c42":54,"\u5b9e\u73b0":54,"\u5b9e\u73b0\u65b9\u5f0f":55,"\u5bfc\u51fac":54,"\u5feb\u7167\u4fdd\u5b58\u7684\u8bbe\u8ba1\u5982\u4e0b":34,"\u6307\u9488\u4f5c\u4e3a\u7c7b\u578b\u7684\u53e5\u67c4":54,"\u63a8\u6d4b\u6267\u884c":34,"\u652f\u6301\u7528\u6237\u81ea\u5b9a\u4e49\u7684\u6570\u636e\u9884\u5904\u7406job":35,"\u6587\u4ef6\u4f20\u8f93\u4f18\u5316":44,"\u6587\u4ef6\u8bbf\u95ee\u65b9\u5f0f":35,"\u6587\u4ef6\u8bbf\u95ee\u7684\u6743\u9650":35,"\u6587\u4ef6\u9884\u5904\u7406":35,"\u66b4\u9732\u63a5\u53e3\u539f\u5219":55,"\u672f\u8bed":34,"\u67b6\u6784\u56fe":44,"\u6846\u67b6\u751f\u6210":44,"\u6982\u5ff5\u89e3\u91ca":35,"\u6a21\u5757":44,"\u6a21\u578b\u53c2\u6570\u68c0\u67e5\u70b9":34,"\u6a21\u578b\u63a8\u65ad\u5b9e\u73b0\u6587\u6863":55,"\u6d41\u7a0b\u4ecb\u7ecd":35,"\u751f\u6210sparse\u6587\u4ef6":44,"\u7528\u6237\u4f7f\u7528\u6d41\u7a0b":44,"\u76ee\u5f55\u7ed3\u6784":55,"\u76ee\u6807":44,"\u793a\u4f8b\u7a0b\u5e8f":35,"\u7b26\u53f7":54,"\u7c7b":54,"\u7f16\u8bd1\u9009\u9879":55,"\u7f29\u5bb9":34,"\u800c\u662f\u624b\u5199\u591a\u8bed\u8a00\u7ed1\u5b9a":54,"\u80cc\u666f":54,"\u8986\u76d6\u4e0d\u4e00\u81f4\u7684\u90e8\u5206":44,"\u8bad\u7ec3\u6570\u636e\u5b58\u50a8":35,"\u8bad\u7ec3\u6570\u636e\u7684\u5b58\u50a8\u548c\u5206\u53d1":35,"\u8f6c\u6362\u5e93":35,"\u8fd9\u4e2a\u52a8\u6001\u5e93\u4f7f\u7528c99\u6807\u51c6\u7684\u5934\u6587\u4ef6\u5bfc\u51fa\u4e00\u4e9b\u51fd\u6570":54,"\u8fdb\u884c\u8bad\u7ec3":35,"abstract":[65,66,67,106],"book\u4e2d\u6240\u6709\u7ae0\u8282":71,"case":97,"class":[48,60,63,86,87],"filemanager\u8bbe\u8ba1\u6587\u6863":44,"function":[32,47,48,63,108],"new":[86,87],"paddle\u52a8\u6001\u5e93\u4e2d":54,"paddle\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0":54,"paddlepaddle\u53d1\u884c\u89c4\u8303":71,"paddlepaddle\u56de\u5f52\u6d4b\u8bd5\u5217\u8868":71,"return":64,"tensor\u5230eigentensor\u7684\u8f6c\u6362":88,AWS:98,Abs:5,DNS:98,EFS:98,For:[32,99],KMS:98,NOT:43,Not:84,The:[31,38,48,49,51,52,58,60,68,69,104],Use:[31,61,89,93,97,99],Using:[32,38,80,93,103],With:41,about:48,absolut:57,access:98,account:98,accuraci:21,activ:[5,8],adadelta:10,adagrad:10,adagradoptim:23,adam:10,adamax:10,adamaxoptim:23,adamoptim:23,add:98,address:98,addto:8,adversari:109,aggreg:8,aggregatelevel:8,alalysi:36,algorithm:[30,33,56,62,65,111],all:[72,75],analysi:65,android:103,api:[0,3,55,58,59,63,70,74],appendix:[106,111],applic:3,approach:92,architectur:[65,83,111],argument:[45,64,93,94,97,103,104,111],arrai:30,array_length:21,array_read:21,array_to_lod_tensor:21,array_writ:21,asset:98,assign:21,associ:[72,98],assumpt:106,async:95,attent:83,attribut:[6,70],auc:7,auto:30,averag:59,avg:11,aws:98,backgraound:30,background:[66,67,75,76,87],backward:[49,68,87],base:[41,57,66],basepool:11,basic:106,batch:64,batch_norm:[8,21],batch_siz:64,beam:57,beam_search:8,beam_search_decod:21,benefit:[66,68],between:[29,63,68,74],bidirectional_gru:9,bidirectional_lstm:9,bilinear_interp:8,binari:31,bind:87,bla:77,block:[31,48,49,61,63,68],block_expand:8,blockdesc:61,book:78,bool:77,bootstrap:106,bottleneck:91,brelu:5,bring:106,bucket:98,build:[48,68,77,79,84,89,99,103,104],built:92,cach:2,can:72,capi:55,capi_priv:55,cast:21,challeng:[62,66],chang:57,check:[8,30,86,93],checker:30,checkpoint:[33,34,40],chines:108,choos:[32,98],chunk:7,cifar:14,classif:7,classification_error:7,classification_error_print:7,client:38,clip:8,close:30,cloudform:98,cluster:[39,93,97,98,106],cmake:[32,104],code:[41,63,85,91],column_sum:7,command:[93,96,97,111],commit:99,common:95,commun:95,compar:106,comparis:63,compil:[31,46,61,68,77,87,103,104],compos:64,comput:[31,49,68,70,88],con:106,concat:[8,21],concept:[63,68,98],conclus:[40,50,106],condit:48,config:[3,97],configur:[27,83,90,98,111],conll05:14,connect:8,constantiniti:20,construct:49,contain:[78,99],content:[55,92,98],context_project:8,contribut:[85,89],control:68,conv2d:21,conv2d_transpos:21,conv:8,conv_oper:8,conv_project:8,conv_shift:8,convert:[40,65,66],convolut:111,core:[30,63,98],cos_sim:[8,21],cost:8,cpu:97,creat:[64,67,68,72,98,99],create_arrai:21,creation:[37,59,70],creator:64,credenti:98,crf:8,crf_decod:8,cross:[103,104],cross_channel_norm:8,cross_entropi:21,cross_entropy_cost:8,cross_entropy_with_selfnorm_cost:8,ctc:8,ctc_error:7,cuda:[46,77],cudnn:77,cudnnavg:11,cudnnmax:11,current:[46,69],custom:64,data:[8,12,13,21,33,64,65,98,99,108,109,111],datafeed:[13,17],dataprovid:[2,3,95],dataset:[12,14,33,37,93],datatyp:13,decayedadagrad:10,decayedadagradoptim:23,decod:57,decor:64,deep:31,defin:[87,98,111],definit:76,delet:98,demo:[48,98],dens:40,dep:80,depend:[48,77,80],deploi:41,deriv:86,describ:58,descript:[45,68,95,109],design:[29,30,31,33,37,38,39,40,42,43,46,47,48,49,52,53,56,57,58,60,61,63,64,65,66,67,68,69,72,73,75],destroi:[72,98],detail:[36,95],detect:[7,8],detection_map:7,detection_output:8,develop:[68,84,90],devic:97,dictionari:[64,108],differ:[68,97],directori:98,discrimin:48,discuss:[48,66],dispatch:[33,37],distribut:[29,33,36,41,65,93,95,98],doc:[29,31,33,37,38,39,40,43,46,47,49,52,53,60,61,63,64,65,66,67,68,69,73],docker:[41,78,84,99,103],document:[89,102],doe:[43,64],dot_prod:8,dot_product_attent:9,dotmul_oper:8,dotmul_project:8,down:98,download:[98,99,108,110],dropout:[8,21],dure:[57,64],dylib:55,dynam:[33,75],dynamic_lstm:21,dynet:50,each:80,ec2:98,eigen:88,elast:98,elect:40,elementwise_add:21,elementwise_div:21,els:31,embed:[8,21,108,111],engin:48,enough:30,entri:64,environ:[41,103],eos:8,equat:86,eval:65,evalu:[7,18,42],event:[28,29],examin:91,exampl:[29,32,51,55,67,108,109],execut:[31,61,68],executor:[19,43],exp:5,expand:8,expandlevel:8,explain:30,extern:98,extract:[108,110],fabric:93,factor:8,factorization_machin:8,faq:[79,80],fault:33,featur:110,file:[31,91,98,99,111],fill_const:21,fill_constant_batch_size_lik:21,find:98,first_seq:8,float16:46,fluid:16,format:[31,33,53,111],forward:[49,87],frame:31,framework:[30,88],from:[29,40,74,77,79],full_matrix_project:8,fulli:8,gan:[48,109],gate:83,gated_unit:8,gener:[48,57,83,84,91,106,109],get:[81,99],get_output:8,give:64,global:[61,63],gotcha:84,gpu:[78,92,95,97],gradient:[30,38,69,86],gradient_print:7,graident:30,graph:[49,50,66,68,70],group:[8,98],gru:[9,95],gru_group:9,gru_step:8,gru_unit:9,grumemori:8,hand:92,handler:[29,54],happen:40,hardwar:46,helper:63,hierarchi:31,high:[58,59,70,74],how:[30,36,59,64,68,77,87,88,89,90,92],hsigmoid:8,huber_classification_cost:8,huber_regression_cost:8,iam:98,ident:5,identifi:91,identity_project:8,ifels:51,ifelseop:31,imag:[8,9,15,41,78,99,103],imagenet:110,imdb:14,img_cmrnorm:8,img_conv:8,img_conv_bn_pool:9,img_conv_group:[9,22],img_pool:8,imikolov:14,implement:[30,32,36,42,43,46,53,56,59,63,64,68,69,70,86,87,88,109],increment:21,infer:[28,103,111],infershap:[61,73],infervartyp:52,info:110,ingredi:29,ingress:44,init_hook:2,initi:[20,38,48,97,98],input_typ:2,insid:72,inspect:98,instal:[79,80,81,98,103,104,106,111],instanc:98,instead:64,integr:98,interact:74,interfac:[12,13,15,30,33,38,39,58,64,67,72,110],intermedi:68,interpol:8,introduc:[57,75],introduct:[1,70,93,108,110],isn:64,issu:46,job:[33,41,93,98,99],join:8,kei:98,kernel:68,kill:93,kmax_sequence_scor:8,kube:98,kubectl:98,kubernet:[41,93,98,99],l1decayregular:26,l2_distanc:8,l2decayregular:26,lambda_cost:8,languag:31,larg:36,last_seq:8,launch:[78,93],layer:[8,21,29,47,63,86,97],learn:31,learnabl:8,less_than:21,leval:74,level:[58,59,70,74],libpaddle_capi_shar:55,libpaddle_capi_whol:55,librari:[38,46,68,103],limit:65,line:[93,96,111],linear:5,linear_chain_crf:21,linear_comb:8,linux:[93,103],list:[34,64],local:[65,67,72,97,98],lod:57,lod_rank_t:21,lod_tensor_to_arrai:21,lodtensor:[56,57,75],lodtensordesc:76,log:[5,85,111],logic:37,logist:111,look:91,low:[59,70,74],lstm:[9,21,95],lstm_step:8,lstmemori:8,lstmemory_group:9,lstmemory_unit:9,machin:[8,57],macro:68,main:48,manag:[32,93],map:[64,68],master:[33,37,41],math:8,mathemat:30,matrix:95,max:11,max_sequence_len:21,maxframe_print:7,maxid:8,maxid_print:7,maxout:8,mean:21,member:48,memori:[8,56],merge_lod_tensor:21,messag:74,method:57,might:48,migrat:68,mileston:68,mini:64,minibatch:13,misc:8,mix:[8,97],mnist:[14,109],mobil:105,model:[2,3,27,29,36,38,40,48,53,57,82,83,93,97,108,109,110,111],modifi:99,modul:[68,88],momentum:10,momentumoptim:23,more:48,motiv:[43,53,62],movielen:14,msrainiti:20,mul:21,multi_binary_label_cross_entropy_cost:8,multibox_loss:8,multipl:64,multiplex:8,mxnet:50,name:[72,98],nce:8,necess:63,necessari:68,need:[64,84,92],nest:56,net:22,network:[9,68,83,97,109,110,111],neural:[83,111],nlp:[9,95],non:2,norm:[8,70],normaliniti:20,note:30,numer:30,numpi:30,nvprof:92,nvvp:92,object:33,observ:[108,110],offset:57,ones:21,onli:[64,72],op_mak:68,openmpi:93,oper:[47,51,59,61,63,66,68,69,73,75,87],opinfomap:68,opkernel:[68,87],opproto:74,ops:70,optim:[10,23,33,38,49,58,63,90,111],option:[45,77,108],opwithkernel:68,order:45,org:89,origin:68,orthogon:72,out_prod:8,outlin:94,output:[8,93,98],overview:[40,43,68,72,111],pack:57,packag:[32,80],pad:8,paddl:[36,64,72,88,99],paddlejob:41,paddlepaddl:[29,31,59,61,65,70,71,77,78,80,89,93,98,102,103,104,108],pair:98,parallel_nn:97,paramattr:24,paramet:[6,8,28,29,33,38,39,41,59,60,63,66,70,93,95,96,98,108,110],parameteraverageoptim:59,paraphras:108,parent:72,part:49,partit:38,pass:[77,97],path:[40,45],penalti:70,perform:[59,91,92,95],persist:37,pfsclient:[44,45],pfsserver:44,pip:80,placement:65,platform:93,pnpair:7,point:98,pool2d:21,pool:[8,11],pose:[52,69],power:8,precision_recal:7,predict:[4,110],prefetch:64,prelu:8,prepar:[93,98,108,109],preprocess:[108,111],pretrain:108,print:7,privat:98,pro:106,problem:[42,52,58,69],procedur:106,process:[33,38,41,58,68,84],profil:[25,91,92],program:[31,61,63,78,93],programdesc:61,project:32,propos:[52,69,70],protobuf:73,protomak:87,provid:[2,64,111],prune:62,pserver:40,pull:78,pydataprovider2:2,python:[4,30,41,56,58,59,63,64,65,70,74,76,86,87,91,110,111],qualiti:68,queue:[33,37],quick:[81,111],randomnumb:95,rank:7,rank_cost:8,raspberri:104,reader:[12,13,29,64],realiz:68,recoveri:33,recurr:[8,9,83,111],recurrent_group:8,ref:30,refactor:68,refer:[2,65,66,92],region:98,regist:[52,68,74,87],registr:[68,69],registri:68,regress:111,regular:[26,38,70],rel:57,relat:[68,75],relu:5,remark:87,remot:[39,67],render:98,repeat:8,represent:[31,68],requir:[32,48],reshap:[8,21],resiz:8,resnet:110,result:[93,99],retri:37,reus:63,review:85,revis:108,rmsprop:10,rnn:[56,75,82,83,95],rnnop:[31,56,68],roi_pool:8,rotat:8,route53:98,row:73,row_conv:8,row_l2_norm:8,run:[77,78,87,99],runtim:[41,65,80],sampl:8,sampling_id:8,save:40,scale:[8,21,33],scale_shift:8,scaling_project:8,scope:[31,56,68,72],script:99,search:57,secur:98,select:[38,73],selectedrow:73,selective_fc:8,sentiment:14,separ:68,seq_concat:8,seq_reshap:8,seq_slic:8,seqtext_print:7,sequenc:[57,83],sequence_conv:21,sequence_conv_pool:[9,22],sequence_pool:21,sequencesoftmax:5,sequenti:2,server:[33,37,38,41,66,93,95,98],servic:98,session:[65,67],set:96,setup:[98,103],sextant:106,sgd:95,sgdoptim:23,shape:57,share:[29,72],should:72,shrink_memori:21,shuffl:64,sigmoid:[5,21],sigmoid_cross_entropy_with_logit:21,simpl:[57,83],simple_attent:9,simple_gru2:9,simple_gru:9,simple_img_conv_pool:[9,22],simple_lstm:9,singl:64,slice:8,slice_project:8,slope_intercept:8,small_vgg:9,smooth_l1_cost:8,softmax:5,softrelu:5,softsign:5,solut:[52,62,69],some:84,sourc:[77,79],spars:[38,39,40,73,97],specifi:[97,108],split_lod_tensor:21,spp:8,squar:5,square_error_cost:[8,21],squarerootn:11,stack:31,standard:[85,111],stanh:5,start:[29,81,93,98,99,111],startup:99,statement:42,step:[56,79],storag:70,store:33,structur:109,style:85,sub_nested_seq:8,subcommond:45,submit:41,suffici:64,suitabl:32,sum:[7,11,21],sum_cost:8,sum_to_one_norm:8,summar:29,summari:[53,111],support:[46,75],survei:[46,50,70,106],synopsi:45,system:98,tabl:55,table_project:8,tanh:5,task:[33,37],tear:98,tecton:106,templat:98,tensor:[8,68,88],tensorarrai:[57,75],tensordesc:76,tensorflow:50,test:[77,85,86,87,95,97],text_conv_pool:9,theori:30,thi:72,think:48,three:75,timelin:40,timer:92,tip:92,todo:[34,35],togeth:72,toi:109,toler:33,tool:[32,89,92,93,106],toolchain:104,topk:21,train:[28,29,33,36,39,41,58,64,65,78,93,95,97,98,99,108,109,111],trainer:[28,33,38,40,41,93,98],tran:8,trans_full_matrix_project:8,transfer:111,translat:57,transpos:21,tune:[92,95],tutori:108,two:30,type:[77,87],uci_h:14,uniform:75,uniforminiti:20,unit:[85,86,87,95],unpack:57,updat:[29,39,40,89,98],usag:[56,57,64,88,90],use:[36,64,88],user:[33,108],using:84,util:7,value_print:7,vardesc:76,variabl:[63,68,72,76],vector:95,verifi:98,version:[46,80],vgg_16_network:9,visual:110,volum:98,vpc:98,warp_ctc:8,weightdecayregular:26,what:[36,40,43,84,92],when:[40,72],whl:80,why:[46,59,64,68,75,92],wmt14:14,word:[108,111],workflow:85,wrapper:86,write:[85,86,87,89,111],www:89,xavieriniti:20,yaml:99,your:78,zero:21,zoo:110}})
\ No newline at end of file
diff --git a/doc/survey/cluster_bootstrapping_tools.html b/doc/survey/cluster_bootstrapping_tools.html
new file mode 100644
index 0000000000000000000000000000000000000000..62af92897f2333337d1a2d6aee77efb76c2fa85b
--- /dev/null
+++ b/doc/survey/cluster_bootstrapping_tools.html
@@ -0,0 +1,368 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Cluster bootstrapping tool survey — PaddlePaddle  documentation
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Cluster bootstrapping tool survey
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Cluster bootstrapping tool survey
    
+
    
+
    Abstract
    
+
    In order to bring up a cluster from bare metal machine to a fully functional kubernetes cluster for Paddlepaddle to run, we need to utilize some tools. Here we are going to compare Sextant and Tectonic installer
    
+
    
+
    
+
    Basic assumptions
    
+
    Here are some basic assumptions before we move on to  details
    
+
      
+
    1. You are an administrator of a bare metal machine cluster, which means:
      2. 
+
    
+
      
+
    • you have full control to each of the machines.
      • 
+
    • you have full control to the network which machines are connected to.
      • 
+
    
+
      
+
    1. Machines can be booted from network with PEX or iPXE
      2. 
+
    2. You understand the general procedure to bring up a cluster
      3. 
+
    
+
    if your cluster is able to mark above items with checkmarks, then keep reading.
    
+
    
+
    
+
    Comparing Sextant and Tectonic installer
    
+
    
+
    Sextant
    
+
    Sextant is an end2end solution to bring up a bare metal cluster to a fully functional k8s cluster, it integrates DHCP, name service, PEX, cloud-config-service, docker registry services altogether.
    
+
    
+
    Pros
    
+
      
+
    1. End2End: basically all admin need to do is to config the cluster.yaml and power on the cluster.
      2. 
+
    2. Offline cluster configuration: Sextant has 2 phases during working with it, config time and deploy time. when admin is configuring, it requires admin’s machine has internet connectivity, which will download some images, etc. But in deploy time, it’s completely OK to go offline since all dependencies are ready during config time.
      3. 
+
    3. docker registry integrated.
      4. 
+
    4. GPU machine took care of.
      5. 
+
    
+
    
+
    
+
    
+
    Cons
    
+
      
+
    1. k8s API server is not deployed with high availability in considering by default.
      2. 
+
    2. No grouping support.
      3. 
+
    3. No API interface, a one-off service.
      4. 
+
    
+
    
+
    
+
    Tectonic installer
    
+
    First of all, Tectonic is not free, it requires coreos.com account as a step of installation, and free user can only create less than 10 nodes.
    
+
    Tectonic is a suite of software which wraps around k8s and providing more utility regarding dev ops, ie,
+Tectonic installer as it’s named, it installs Tectonic to a bare metal cluster which means it’s not totally an equivalent of Sextant. At the “booting a cluster” part, it mostly utilizes Matchbox, which is a general cluster bootstrapper.
    
+
    Matchbox’s Approach is similar to Sexstant.
    
+
    
+
    
+
    Pros
    
+
      
+
    1. supports grouping machines.
      2. 
+
    2. supports running provisioning service in rtk. (not a big deal though).
      3. 
+
    3. supports http/gRPC API interface.
      4. 
+
    4. supports multi-template.
      5. 
+
    
+
    
+
    
+
    Cons
    
+
      
+
    1. Not an e2e solution to bring up a cluster, need a lot of extra work and other software.
      2. 
+
    2. Not fully supporting centOS deployment yet.
      3. 
+
    
+
    
+
    
+
    
+
    Conclusion
    
+
    Sextant is a better solution overall for paddle cloud deploying to a bare metal cluster. It would be great if Sextant can also 1) deploy k8s api server with high availability by default; 2) not designed as a one-off service.
    
+
    
+
    
+
    Appendix: General procedure to bring up a cluster
    
+
    It’s physically impossible for a cluster admin to manually install OS and applications into cluster nodes one by one, here is what an admin would do in cloud industry:
    
+
      
+
    1. setup a bootstrap machine with static IP in the cluster, which has following services:
      2. 
+
    
+
      
+
    • DHCP: assigns ip address for rest of the nodes.
      • 
+
    • name service: to map node name to a IP
      • 
+
    • PXE related services: the booting related info will be delivered to newly booted machines as their IP is assigned via DHCP service, PXE service will provide further booting and installing info and image with TFTP and http protocol.
      • 
+
    • cluster config service: this is for providing cluster node with OS config via http
      • 
+
    • optional docker registry: a built-in docker registry makes the whole cluster independent from connecting internet, and speeds up software distribution.
      • 
+
    
+
      
+
    1. New node powers on, it will
      2. 
+
    
+
      
+
    • broadcast the request for an IP address
      • 
+
    • DHCP server assigns the IP address, and deliver the PXE booting related info to the node.
      • 
+
    • cluster node will request config files with booting info delivered with DHCP via the TFTP service, and in most of the cases, the config file will point to a http service for the booting image.
      • 
+
    • Since PXE is configured with initrd, it will utilize the cloud config service and do further installations like coreOS or K8s installations.
      • 
+
    • then restart the node.
      • 
+
    
+
    For further understanding, following 2 links from Matchbox are some good readings:
    
+
    
+
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc/tutorials/image_classification/index_en.html b/doc/tutorials/image_classification/index_en.html
deleted file mode 100644
index 854d1b9f4deca83a1d5d9c64388cb40a0074719d..0000000000000000000000000000000000000000
--- a/doc/tutorials/image_classification/index_en.html
+++ /dev/null
@@ -1,432 +0,0 @@
-
-
-
-
-  
-
-  
-  
-  
-  
-  Image Classification Tutorial — PaddlePaddle  documentation
-  
-
-  
-  
-
-  
-
-  
-  
-    
-
-  
-
-  
-  
-    
-  
-
-  
-  
-        
-        
-     
-
-  
-  
-  
-
-  
-
-  
-  
-
-
-
-
-
-  
-  
    
-    
-    
    
-      
    
-        Folk me on Github
-        
    
-          
-            English
-            
-            
-          
-          
-        
    
-        
-      
    
-      
    
-        
-        
-
-        
-
    
-  
    
-    
-    
-    
-  
    
-
            
-      
    
-    
    
-  
    
-  
-  
    
-
-    
-    
-    
-    
    
-
-      
-
- 
-
-
-
-
-
-
-
-
    
-  
      
-      
-    
    • Image Classification Tutorial
      • 
-  
    
-
    
-      
-      
    
-        
    
-          
    
-           
    
-            
-  
    
-
    Image Classification Tutorial
    
-
    This tutorial will guide you through training a convolutional neural network to classify objects using the CIFAR-10 image classification dataset.
-As shown in the following figure, the convolutional neural network can recognize the main object in images, and output the classification result.
    
-
    Image Classification
    
-
    
-
    Data Preparation
    
-
    First, download CIFAR-10 dataset. CIFAR-10 dataset can be downloaded from its official website.
    
-
    https://www.cs.toronto.edu/~kriz/cifar.html
    
-
    We have prepared a script to download and process CIFAR-10 dataset. The script will download CIFAR-10 dataset from the official dataset.
-It will convert it to jpeg images and organize them into a directory with the required structure for the tutorial. Make sure that you have installed pillow and its dependents.
-Consider the following commands:
    
-
      
-
    1. install pillow dependents
      2. 
-
    
-
    sudo apt-get install libjpeg-dev
-pip install pillow
-
    
-
    
-
      
-
    1. download data and preparation
      2. 
-
    
-
    cd demo/image_classification/data/
-sh download_cifar.sh
-
    
-
    
-
    The CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.
    
-
    Here are the classes in the dataset, as well as 10 random images from each:
-
    Image Classification
    
-
    After downloading and converting, we should find a directory (cifar-out) containing the dataset in the following format:
    
-
    train
----airplane
----automobile
----bird
----cat
----deer
----dog
----frog
----horse
----ship
----truck
-test
----airplane
----automobile
----bird
----cat
----deer
----dog
----frog
----horse
----ship
----truck
-
    
-
    
-
    It has two directories:train and test. These two directories contain training data and testing data of CIFAR-10, respectively. Each of these two folders contains 10 sub-folders, ranging from airplane to truck. Each sub-folder contains images with the corresponding label. After the images are organized into this structure, we are ready to train an image classification model.
    
-
    
-
    
-
    Preprocess
    
-
    After the data has been downloaded, it needs to be pre-processed into the Paddle format. We can run the following command for preprocessing.
    
-
    cd demo/image_classification/
-sh preprocess.sh
-
    
-
    
-
    preprocess.sh calls ./demo/image_classification/preprocess.py to preprocess image data.
    
-
    export PYTHONPATH=$PYTHONPATH:../../
-data_dir=./data/cifar-out
-python preprocess.py -i $data_dir -s 32 -c 1
-
    
-
    
-
    ./demo/image_classification/preprocess.py has the following arguments
    
-
      
-
    • -i or --input specifes  the input data directory.
      • 
-
    • -s or --size specifies the processed size of images.
      • 
-
    • -c or --color specifes whether images are color images or gray images.
      • 
-
    
-
    
-
    
-
    Model Training
    
-
    We need to create a model config file before training the model. An example of the config file (vgg_16_cifar.py) is listed below. Note, it is slightly different from the vgg_16_cifar.py which also applies to the prediction.
    
-
    from paddle.trainer_config_helpers import *
-data_dir='data/cifar-out/batches/'
-meta_path=data_dir+'batches.meta'
-args = {'meta':meta_path, 'mean_img_size': 32,
-        'img_size': 32, 'num_classes': 10,
-        'use_jpeg': 1, 'color': "color"}
-define_py_data_sources2(train_list=data_dir+"train.list",
-                        test_list=data_dir+'test.list',
-                        module='image_provider',
-                        obj='processData',
-                        args=args)
-settings(
-    batch_size = 128,
-    learning_rate = 0.1 / 128.0,
-    learning_method = MomentumOptimizer(0.9),
-    regularization = L2Regularization(0.0005 * 128))
-
-img = data_layer(name='image', size=3*32*32)
-lbl = data_layer(name="label", size=10)
-# small_vgg is predined in trainer_config_helpers.network
-predict = small_vgg(input_image=img, num_channels=3)
-outputs(classification_cost(input=predict, label=lbl))
-
    
-
    
-
    The first line imports python functions for defining networks.
    
-
    from paddle.trainer_config_helpers import *
-
    
-
    
-
    Then define an define_py_data_sources2 which use python data provider
-interface. The arguments in args are used in image_provider.py which
-yeilds image data and transform them to Paddle.
    
-
      
-
    • meta: the mean value of training set.
      • 
-
    • mean_img_size: the size of mean feature map.
      • 
-
    • img_size: the height and width of input image.
      • 
-
    • num_classes: the number of classes.
      • 
-
    • use_jpeg: the data storage type when preprocessing.
      • 
-
    • color: specify color image.
      • 
-
    
-
    settings specifies the training algorithm. In the following example,
-it specifies learning rate as 0.1, but divided by batch size, and the weight decay
-is 0.0005 and multiplied by batch size.
    
-
    settings(
-    batch_size = 128,
-    learning_rate = 0.1 / 128.0,
-    learning_method = MomentumOptimizer(0.9),
-    regularization = L2Regularization(0.0005 * 128)
-)
-
    
-
    
-
    The small_vgg specifies the network. We use a small version of VGG convolutional network as our network
-for classification. A description of VGG network can be found here http://www.robots.ox.ac.uk/~vgg/research/very_deep/.
    
-
    # small_vgg is predined in trainer_config_helpers.network
-predict = small_vgg(input_image=img, num_channels=3)
-
    
-
    
-
    After writing the config, we can train the model by running the script train.sh.
    
-
    config=vgg_16_cifar.py
-output=./cifar_vgg_model
-log=train.log
-
-paddle train \
---config=$config \
---dot_period=10 \
---log_period=100 \
---test_all_data_in_one_period=1 \
---use_gpu=1 \
---save_dir=$output \
-2>&1 | tee $log
-
-python -m paddle.utils.plotcurve -i $log > plot.png
-
    
-
    
-
      
-
    • Here we use GPU mode to train. If you have no gpu environment, just set use_gpu=0.
      • 
-
    • ./demo/image_classification/vgg_16_cifar.py is the network and data configuration file. The meaning of the other flags can be found in the documentation of the command line flags.
      • 
-
    • The script plotcurve.py requires the python module of matplotlib, so if it fails, maybe you need to install matplotlib.
      • 
-
    
-
    After training finishes, the training and testing error curves will be saved to plot.png using plotcurve.py script. An example of the plot is shown below:
    
-
    Training and testing curves.
    
-
    
-
    
-
    Prediction
    
-
    After we train the model, the model file as well as the model parameters are stored in path ./cifar_vgg_model/pass-%05d. For example, the model of the 300-th pass is stored at ./cifar_vgg_model/pass-00299.
    
-
    To make a prediction for an image, one can run predict.sh as follows. The script will output the label of the classfiication.
    
-
    sh predict.sh
-
    
-
    
-
    predict.sh:
    
-
    model=cifar_vgg_model/pass-00299/
-image=data/cifar-out/test/airplane/seaplane_s_000978.png
-use_gpu=1
-python prediction.py $model $image $use_gpu
-
    
-
    
-
    
-
    
-
    Exercise
    
-
    Train a image classification of birds using VGG model and CUB-200 dataset. The birds dataset can be downloaded here. It contains an image dataset with photos of 200 bird species (mostly North American).
    
-
    http://www.vision.caltech.edu/visipedia/CUB-200.html
    
-
    
-
    
-
    Delve into Details
    
-
    
-
    Convolutional Neural Network
    
-
    A Convolutional Neural Network is a feedforward neural network that uses convolution layers. It is very suitable for building neural networks that process and understand images. A standard convolutional neural network is shown below:
    
-
    Convolutional Neural Network
    
-
    Convolutional Neural Network contains the following layers:
    
-
      
-
    • Convolutional layer: It uses convolution operation to extract features from an image or a feature map.
      • 
-
    • Pooling layer: It uses max-pooling to downsample feature maps.
      • 
-
    • Fully Connected layer: It uses fully connected connections to transform features.
      • 
-
    
-
    Convolutional Neural Network achieves amazing performance for image classification because it exploits two important characteristics of images: local correlation and spatial invariance. By iteratively applying convolution and max-pooing operations, convolutional neural network can well represent these two characteristics of images.
    
-
    For more details of how to define layers and their connections, please refer to the documentation of layers.
    
-
    
-
    
-
    
-
-
-           
    
-          
    
-          
    
-  
-
-  
    
-
-  
    
-    
    
-        © Copyright 2016, PaddlePaddle developers.
-
-    
    
-  
    
-  Built with Sphinx using a theme provided by Read the Docs. 
-
-
    
-
-        
    
-      
    
-
-    
    
-
-  
    
-  
-
-
-  
-
-    
-      
-      
-      
-      
-       
-  
-
-  
-  
-    
-  
-  
-  
-  
-   
-
-
-
\ No newline at end of file
diff --git a/doc/tutorials/rec/ml_regression_en.html b/doc/tutorials/rec/ml_regression_en.html
deleted file mode 100644
index 42f55c2406bb0cca2307ad456df4f613099ba016..0000000000000000000000000000000000000000
--- a/doc/tutorials/rec/ml_regression_en.html
+++ /dev/null
@@ -1,845 +0,0 @@
-
-
-
-
-  
-
-  
-  
-  
-  
-  Regression MovieLens Ratting — PaddlePaddle  documentation
-  
-
-  
-  
-
-  
-
-  
-  
-    
-
-  
-
-  
-  
-    
-  
-
-  
-  
-        
-        
-     
-
-  
-  
-  
-
-  
-
-  
-  
-
-
-
-
-
-  
-  
    
-    
-    
    
-      
    
-        Folk me on Github
-        
    
-          
-            English
-            
-            
-          
-          
-        
    
-        
-      
    
-      
    
-        
-        
-
-        
-
    
-  
    
-    
-    
-    
-  
    
-
            
-      
    
-    
    
-  
    
-  
-  
    
-
-    
-    
-    
-    
    
-
-      
-
- 
-
-
-
-
-
-
-
-
    
-  
      
-      
-    
    • Regression MovieLens Ratting
      • 
-  
    
-
    
-      
-      
    
-        
    
-          
    
-           
    
-            
-  
    
-
    Regression MovieLens Ratting
    
-
    Here we demonstrate a Cosine Similarity Regression job in movie lens dataset.
-This demo will show how paddle does (word) embedding job,
-handles the similarity regression,
-the character-level convolutional networks for text, and how does paddle handle
-multiple types of inputs.
-Note that the model structure is not fine-tuned and just a demo to show how paddle works.
    
-
    YOU ARE WELCOME TO BUILD A BETTER DEMO
-BY USING PADDLEPADDLE, AND LET US KNOW TO MAKE THIS DEMO BETTER.
    
-
    
-
    Data Preparation
    
-
    
-
    Download and extract dataset
    
-
    We use MovieLens Dataset here.
-To download and unzip the dataset, simply run the following commands.
    
-
    cd demo/recommendation/data
-./ml_data.sh
-
    
-
    
-
    And the directory structure of demo/recommendation/data/ml-1m is:
    
-
    +--ml-1m
-     +--- movies.dat    # movie features
-     +--- ratings.dat   # ratings
-     +--- users.dat     # user features
-     +--- README        # dataset description
-
    
-
    
-
    
-
    
-
    Field config file
    
-
    Field config file is used to specify the fields of the dataset and the file format,
-i.e, specific WHAT type it is in each feature file.
    
-
    The field config file of ml-1m shows in demo/recommendation/data/config.json.
-It specifics the field types and file names: 1) there are four types of field for user file: id, gender, age and occupation;
-2) the filename is “users.dat”, and the delimiter of file is ”::”.
    
-
    {
-  "user": {
-    "file": {
-      "name": "users.dat",
-      "delimiter": "::"
-    },
-    "fields": ["id", "gender", "age", "occupation"]
-  },
-  "movie": {
-    "file": {
-      "name": "movies.dat",
-      "delimiter": "::"
-    },
-    "fields": ["id", "title", "genres"]
-  }
-}
-
    
-
    
-
    
-
    
-
    
-
    Preprocess Data
    
-
    You need to install python 3rd party libraries.
-IT IS HIGHLY RECOMMEND TO USE VIRTUALENV MAKE A CLEAN PYTHON ENVIRONMENT.
    
-
    pip install -r requirements.txt
-
    
-
    
-
    The general command for preprocessing the dataset is:
    
-
    cd demo/recommendation
-./preprocess.sh
-
    
-
    
-
    And the detail steps are introduced as follows.
    
-
    
-
    Extract Movie/User features to python object
    
-
    There are many features in movie or user in movielens 1m dataset.
-Each line of rating file just provides a Movie/User id to refer each movie or user.
-We process the movie/user feature file first, and pickle the feature (Meta) object as a file.
    
-
    
-
    Meta config file
    
-
    Meta config file is used to specific HOW to parse each field in dataset.
-It could be translated from field config file, or written by hand.
-Its file format could be either json or yaml syntax file. Parser will automatically choose the file format by extension name.
    
-
    To convert Field config file to meta config file, just run:
    
-
    cd demo/recommendation/data
-python config_generator.py config.json > meta_config.json
-
    
-
    
-
    The meta config file shows below:
    
-
    {
-  "meta": {
-    "movie": {
-      "fields": [
-        {
-          "type": "id", 
-          "pos": 0
-        }, 
-        {
-          "regex": {
-            "pattern": "^(.*)\\((\\d+)\\)$", 
-            "group_id": 1, 
-            "strip": true
-          }, 
-          "type": {
-            "seq_type": "sequence", 
-            "name": "embedding"
-          }, 
-          "dict": {
-            "type": "char_based"
-          }, 
-          "name": "title", 
-          "pos": 1
-        }, 
-        {
-          "type": "one_hot_dense", 
-          "dict": {
-            "delimiter": "|", 
-            "type": "split"
-          }, 
-          "name": "genres", 
-          "pos": 2
-        }
-      ], 
-      "file": {
-        "delimiter": "::", 
-        "type": "split", 
-        "name": "movies.dat"
-      }
-    }, 
-    "user": {
-      "fields": [
-        {
-          "type": "id", 
-          "pos": 0
-        }, 
-        {
-          "type": "embedding", 
-          "dict": {
-            "type": "char_based"
-          }, 
-          "name": "gender", 
-          "pos": 1
-        }, 
-        {
-          "type": "embedding", 
-          "dict": {
-            "sort": true, 
-            "type": "whole_content"
-          }, 
-          "name": "age", 
-          "pos": 2
-        }, 
-        {
-          "type": "embedding", 
-          "dict": {
-            "sort": "true", 
-            "type": "whole_content"
-          }, 
-          "name": "occupation", 
-          "pos": 3
-        }
-      ], 
-      "file": {
-        "delimiter": "::", 
-        "type": "split", 
-        "name": "users.dat"
-      }
-    }
-  }
-}
-
    
-
    
-
    There are two kinds of features in meta: movie and user.
    
-
      
-
    • 
-
      in movie file, whose name is movies.dat
      
-
        
-
      • we just split each line by ”::”
        • 
-
      • pos 0 is id.
        • 
-
      • 
-
        pos 1 feature:
        
-
          
-
        • name is title.
          • 
-
        • it uses regex to parse this feature.
          • 
-
        • it is a char based word embedding feature.
          • 
-
        • it is a sequence.
          • 
-
        
-
        
-
        
-
        • 
-
      • 
-
        pos 2 feature:
        
-
          
-
        • name is genres.
          • 
-
        • type is one hot dense vector.
          • 
-
        • dictionary is auto generated by parsing, each key is split by ‘|’
          • 
-
        
-
        
-
        
-
        • 
-
      
-
      
-
      
-
      • 
-
    • 
-
      in user file, whose name is users.dat
      
-
        
-
      • we just split each line by ”::”
        • 
-
      • pos 0 is id.
        • 
-
      • 
-
        pos 1 feature:
        
-
          
-
        • name is gender
          • 
-
        • just simple char based embedding.
          • 
-
        
-
        
-
        
-
        • 
-
      • 
-
        pos 2 feature:
        
-
          
-
        • name is age
          • 
-
        • just whole word embedding.
          • 
-
        • embedding id will be sort by word.
          • 
-
        
-
        
-
        
-
        • 
-
      • 
-
        pos 3 feature:
        
-
          
-
        • name is occupation.
          • 
-
        • just simple whole word embedding.
          • 
-
        
-
        
-
        
-
        • 
-
      
-
      
-
      
-
      • 
-
    
-
    
-
    
-
    
-
    Meta file
    
-
    After having meta config file, we can generate Meta file, a python pickle object which stores movie/user information.
-The following commands could be run to generate it.
    
-
    python meta_generator.py ml-1m meta.bin --config=meta_config.json
-
    
-
    
-
    And the structure of the meta file meta.bin is:
    
-
    +--+ movie
-|      +--+ __meta__
-|      |       +--+ raw_meta  # each feature meta config. list
-|      |       |       +
-|      |       |       |     # ID Field, we use id as key
-|      |       |       +--+ {'count': 3883, 'max': 3952, 'is_key': True, 'type': 'id', 'min': 1}
-|      |       |       |
-|      |       |       |     # Titile field, the dictionary list of embedding.
-|      |       |       +--+ {'dict': [ ... ], 'type': 'embedding', 'name': 'title', 'seq': 'sequence'}
-|      |       |       |
-|      |       |       |     # Genres field, the genres dictionary
-|      |       |       +--+ {'dict': [ ... ], 'type': 'one_hot_dense', 'name': 'genres'}
-|      |       |
-|      |       +--+ feature_map [1, 2] # a list for raw_meta index for feature field.
-|      |                               # it means there are 2 features for each key.
-|      |                               #    * 0 offset of feature is raw_meta[1], Title.
-|      |                               #    * 1 offset of feature is raw_meta[2], Genres.
-|      |
-|      +--+ 1 # movie 1 features
-|      |    +
-|      |    +---+ [[...], [...]] # title ids, genres dense vector
-|      |
-|      +--+ 2
-|      |
-|      +--+ ...
-|
-+--- user
-       +--+ __meta__
-       |       +
-       |       +--+ raw_meta
-       |       |       +
-       |       |       +--+ id field as user
-       |       |       |
-       |       |       +--+ {'dict': ['F', 'M'], 'type': 'embedding', 'name': 'gender', 'seq': 'no_sequence'}
-       |       |       |
-       |       |       +--+ {'dict': ['1', '18', '25', '35', '45', '50', '56'], 'type': 'embedding', 'name': 'age', 'seq': 'no_sequence'}
-       |       |       |
-       |       |       +--+ {'dict': [...], 'type': 'embedding', 'name': 'occupation', 'seq': 'no_sequence'}
-       |       |
-       |       +--+ feature_map [1, 2, 3]
-       |
-       +--+ 1 # user 1 features
-       |
-       +--+ 2
-       +--+ ...
-
    
-
    
-
    
-
    
-
    Split Training/Testing files
    
-
    We split ml-1m/ratings.dat into a training and testing file. The way to split file is for each user, we split the
-rating by two parts. So each user in testing file will have some rating information in training file.
    
-
    Use separate.py to separate the training and testing file.
    
-
    python split.py ml-1m/ratings.dat --delimiter="::" --test_ratio=0.1
-
    
-
    
-
    Then two files will be generated: ml-1m/ratings.dat.train and ml-1m/rating.data.test.
-Move them to workspace data, shuffle the train file, and prepare the file list for paddle train.
    
-
    shuf ml-1m/ratings.dat.train > ratings.dat.train
-cp ml-1m/ratings.dat.test .
-echo "./data/ratings.dat.train" > train.list
-echo "./data/ratings.dat.test" > test.list
-
    
-
    
-
    
-
    
-
    
-
    Neural Network Configuration
    
-
    
-
    Trainer Config File
    
-
    The network structure shows below.
    
-rec_regression_network
-
    The demo’s neural network config file trainer_config.py show as below.
    
-
    from paddle.trainer_config_helpers import *
-
-try:
-    import cPickle as pickle
-except ImportError:
-    import pickle
-
-is_predict = get_config_arg('is_predict', bool, False)
-
-META_FILE = 'data/meta.bin'
-
-with open(META_FILE, 'rb') as f:
-    # load meta file
-    meta = pickle.load(f)
-
-settings(
-    batch_size=1600, learning_rate=1e-3, learning_method=RMSPropOptimizer())
-
-
-def construct_feature(name):
-    """
-    Construct movie/user features.
-
-    This method read from meta data. Then convert feature to neural network due
-    to feature type. The map relation as follow.
-
-    * id: embedding => fc
-    * embedding:
-        is_sequence:  embedding => context_projection => fc => pool
-        not sequence: embedding => fc
-    * one_hot_dense:  fc => fc
-
-    Then gather all features vector, and use a fc layer to combined them as
-    return.
-
-    :param name: 'movie' or 'user'
-    :type name: basestring
-    :return: combined feature output
-    :rtype: LayerOutput
-    """
-    __meta__ = meta[name]['__meta__']['raw_meta']
-    fusion = []
-    for each_meta in __meta__:
-        type_name = each_meta['type']
-        slot_name = each_meta.get('name', '%s_id' % name)
-        if type_name == 'id':
-            slot_dim = each_meta['max']
-            embedding = embedding_layer(
-                input=data_layer(
-                    slot_name, size=slot_dim), size=256)
-            fusion.append(fc_layer(input=embedding, size=256))
-        elif type_name == 'embedding':
-            is_seq = each_meta['seq'] == 'sequence'
-            slot_dim = len(each_meta['dict'])
-            din = data_layer(slot_name, slot_dim)
-            embedding = embedding_layer(input=din, size=256)
-            if is_seq:
-                fusion.append(
-                    text_conv_pool(
-                        input=embedding, context_len=5, hidden_size=256))
-            else:
-                fusion.append(fc_layer(input=embedding, size=256))
-        elif type_name == 'one_hot_dense':
-            slot_dim = len(each_meta['dict'])
-            hidden = fc_layer(input=data_layer(slot_name, slot_dim), size=256)
-            fusion.append(fc_layer(input=hidden, size=256))
-
-    return fc_layer(name="%s_fusion" % name, input=fusion, size=256)
-
-
-movie_feature = construct_feature("movie")
-user_feature = construct_feature("user")
-similarity = cos_sim(a=movie_feature, b=user_feature)
-if not is_predict:
-    outputs(mse_cost(input=similarity, label=data_layer('rating', size=1)))
-
-    define_py_data_sources2(
-        'data/train.list',
-        'data/test.list',
-        module='dataprovider',
-        obj='process',
-        args={'meta': meta})
-else:
-    outputs(similarity)
-
    
-
    
-
    In this trainer_config.py, we just map each feature type to
-a feature vector, following shows how to map each feature to a vector shows below.
    
-
      
-
    • id: Just simple embedding, and then add to fully connected layer.
      • 
-
    • 
-
      embedding:
      
-
        
-
      • if is_sequence, get the embedding and do a text convolutional operation,
-get the average pooling result.
        • 
-
      • if not sequence, get the embedding and add to fully connected layer.
        • 
-
      
-
      
-
      
-
      • 
-
    • 
-
      one_host_dense:
      
-
        
-
      • just two fully connected layer.
        • 
-
      
-
      
-
      
-
      • 
-
    
-
    Then we combine each features of movie into one movie feature by a
-fc_layer with multiple inputs, and do the same thing to user features,
-get one user feature. Then we calculate the cosine similarity of these two
-features.
    
-
    In these networks, we use several APIs in Model Config API . There are
    
-
-
    
-
    
-
    Data Provider
    
-
    from paddle.trainer.PyDataProvider2 import *
-import common_utils  # parse
-
-
-def __list_to_map__(lst):
-    ret_val = dict()
-    for each in lst:
-        k, v = each
-        ret_val[k] = v
-    return ret_val
-
-
-def hook(settings, meta, **kwargs):
-    """
-    Init hook is invoked before process data. It will set obj.slots and store
-    data meta.
-
-    :param obj: global object. It will passed to process routine.
-    :type obj: object
-    :param meta: the meta file object, which passed from trainer_config. Meta
-                 file record movie/user features.
-    :param kwargs: unused other arguments.
-    """
-    del kwargs  # unused kwargs
-
-    # Header define slots that used for paddle.
-    #    first part is movie features.
-    #    second part is user features.
-    #    final part is rating score.
-    # header is a list of [USE_SEQ_OR_NOT?, SlotType]
-    movie_headers = list(common_utils.meta_to_header(meta, 'movie'))
-    settings.movie_names = [h[0] for h in movie_headers]
-    headers = movie_headers
-    user_headers = list(common_utils.meta_to_header(meta, 'user'))
-    settings.user_names = [h[0] for h in user_headers]
-    headers.extend(user_headers)
-    headers.append(("rating", dense_vector(1)))  # Score
-
-    # slot types.
-    settings.input_types = __list_to_map__(headers)
-    settings.meta = meta
-
-
-@provider(init_hook=hook, cache=CacheType.CACHE_PASS_IN_MEM)
-def process(settings, filename):
-    with open(filename, 'r') as f:
-        for line in f:
-            # Get a rating from file.
-            user_id, movie_id, score = map(int, line.split('::')[:-1])
-
-            # Scale score to [-5, +5]
-            score = float(score) * 2 - 5.0
-
-            # Get movie/user features by movie_id, user_id
-            movie_meta = settings.meta['movie'][movie_id]
-            user_meta = settings.meta['user'][user_id]
-
-            outputs = [('movie_id', movie_id - 1)]
-
-            # Then add movie features
-            for i, each_meta in enumerate(movie_meta):
-                outputs.append((settings.movie_names[i + 1], each_meta))
-
-            # Then add user id.
-            outputs.append(('user_id', user_id - 1))
-
-            # Then add user features.
-            for i, each_meta in enumerate(user_meta):
-                outputs.append((settings.user_names[i + 1], each_meta))
-
-            # Finally, add score
-            outputs.append(('rating', [score]))
-            # Return data to paddle
-            yield __list_to_map__(outputs)
-
    
-
    
-
    The data provider just read the meta.bin and rating file, yield each sample for training.
-In this dataprovider.py, we should set:
    
-
      
-
    • obj.slots: The feature types and dimension.
      • 
-
    • use_seq: Whether this dataprovider.py in sequence mode or not.
      • 
-
    • process: Return each sample of data to paddle.
      • 
-
    
-
    The data provider details document see PyDataProvider2.
    
-
    
-
    
-
    
-
    Train
    
-
    After prepare data, config network, writting data provider, now we can run paddle training.
    
-
    The run.sh is shown as follow:
    
-
    paddle train \
-    --config=trainer_config.py \
-    --save_dir=./output \
-    --use_gpu=false \
-    --trainer_count=4\
-    --test_all_data_in_one_period=true \
-    --log_period=100 \
-    --dot_period=1 \
-    --num_passes=50  2>&1 | tee 'log.txt'
-paddle usage -l log.txt -e $? -n "recommendation" >/dev/null 2>&1
-
    
-
    
-
    It just start a paddle training process, write the log to log.txt,
-then print it on screen.
    
-
    Each command line argument in run.sh, please refer to the Set Command-line Parameters page. The short description of these arguments is shown as follow.
    
-
      
-
    • config: Tell paddle which file is neural network configuration.
      • 
-
    • save_dir: Tell paddle save model into ./output.
      • 
-
    • use_gpu: Use gpu or not. Default is false.
      • 
-
    • trainer_count: The compute thread in one machine.
      • 
-
    • test_all_data_in_one_period: Test All Data during one test period. Otherwise,
-will test a batch_size data in one test period.
      • 
-
    • log_period: Print log after train log_period batches.
      • 
-
    • dot_period: Print a . after train dot_period batches.
      • 
-
    • num_passes: Train at most num_passes.
      • 
-
    
-
    If training process starts successfully, the output likes follow:
    
-
    I0601 08:07:22.832059 10549 TrainerInternal.cpp:157]  Batch=100 samples=160000 AvgCost=4.13494 CurrentCost=4.13494 Eval:  CurrentEval:
-
-I0601 08:07:50.672627 10549 TrainerInternal.cpp:157]  Batch=200 samples=320000 AvgCost=3.80957 CurrentCost=3.48421 Eval:  CurrentEval:
-
-I0601 08:08:18.877369 10549 TrainerInternal.cpp:157]  Batch=300 samples=480000 AvgCost=3.68145 CurrentCost=3.42519 Eval:  CurrentEval:
-
-I0601 08:08:46.863963 10549 TrainerInternal.cpp:157]  Batch=400 samples=640000 AvgCost=3.6007 CurrentCost=3.35847 Eval:  CurrentEval:
-
-I0601 08:09:15.413025 10549 TrainerInternal.cpp:157]  Batch=500 samples=800000 AvgCost=3.54811 CurrentCost=3.33773 Eval:  CurrentEval:
-I0601 08:09:36.058670 10549 TrainerInternal.cpp:181]  Pass=0 Batch=565 samples=902826 AvgCost=3.52368 Eval:
-I0601 08:09:46.215489 10549 Tester.cpp:101]  Test samples=97383 cost=3.32155 Eval:
-I0601 08:09:46.215966 10549 GradientMachine.cpp:132] Saving parameters to ./output/model/pass-00000
-I0601 08:09:46.233397 10549 ParamUtil.cpp:99] save dir ./output/model/pass-00000
-I0601 08:09:46.233438 10549 Util.cpp:209] copy trainer_config.py to ./output/model/pass-00000
-I0601 08:09:46.233541 10549 ParamUtil.cpp:147] fileName trainer_config.py
-
    
-
    
-
    The model is saved in output/ directory. You can use Ctrl-C to stop training whenever you want.
    
-
    
-
    
-
    Evaluate and Predict
    
-
    After training several passes, you can evaluate them and get the best pass. Just run
    
-
    ./evaluate.sh
-
    
-
    
-
    You will see messages like this:
    
-
    Best pass is 00009,  error is 3.06949, which means predict get error as 0.875998002281
-evaluating from pass output/pass-00009
-
    
-
    
-
    Then, you can predict what any user will rate a movie. Just run
    
-
    python prediction.py 'output/pass-00009/'
-
    
-
    
-
    Predictor will read user input, and predict scores. It has a command-line user interface as follows:
    
-
    Input movie_id: 9
-Input user_id: 4
-Prediction Score is 2.56
-Input movie_id: 8
-Input user_id: 2
-Prediction Score is 3.13
-
    
-
    
-
    
-
    
-
-
-           
    
-          
    
-          
    
-  
-
-  
    
-
-  
    
-    
    
-        © Copyright 2016, PaddlePaddle developers.
-
-    
    
-  
    
-  Built with Sphinx using a theme provided by Read the Docs. 
-
-
    
-
-        
    
-      
    
-
-    
    
-
-  
    
-  
-
-
-  
-
-    
-      
-      
-      
-      
-       
-  
-
-  
-  
-    
-  
-  
-  
-  
-   
-
-
-
\ No newline at end of file
diff --git a/doc/tutorials/semantic_role_labeling/index_en.html b/doc/tutorials/semantic_role_labeling/index_en.html
deleted file mode 100644
index 8a1a9d5b66c67dbab52ddf8d058e0519ccc65496..0000000000000000000000000000000000000000
--- a/doc/tutorials/semantic_role_labeling/index_en.html
+++ /dev/null
@@ -1,432 +0,0 @@
-
-
-
-
-  
-
-  
-  
-  
-  
-  Semantic Role labeling Tutorial — PaddlePaddle  documentation
-  
-
-  
-  
-
-  
-
-  
-  
-    
-
-  
-
-  
-  
-    
-  
-
-  
-  
-        
-        
-     
-
-  
-  
-  
-
-  
-
-  
-  
-
-
-
-
-
-  
-  
    
-    
-    
    
-      
    
-        Folk me on Github
-        
    
-          
-            English
-            
-            
-          
-          
-        
    
-        
-      
    
-      
    
-        
-        
-
-        
-
    
-  
    
-    
-    
-    
-  
    
-
            
-      
    
-    
    
-  
    
-  
-  
    
-
-    
-    
-    
-    
    
-
-      
-
- 
-
-
-
-
-
-
-
-
    
-  
      
-      
-    
    • Semantic Role labeling Tutorial
      • 
-  
    
-
    
-      
-      
    
-        
    
-          
    
-           
    
-            
-  
    
-
    Semantic Role labeling Tutorial
    
-
    Semantic role labeling (SRL) is a form of shallow semantic parsing whose goal is to discover the predicate-argument structure of each predicate in a given input sentence. SRL is useful as an intermediate step in a wide range of natural language processing tasks, such as information extraction. automatic document categorization and question answering.  An instance is as following [1]:
    
-
    [ A0 He ] [ AM-MOD would ][ AM-NEG n’t ] [ V accept] [ A1 anything of value ] from [A2 those he was writing about ].
    
-
      
-
    • V: verb
      • 
-
    • A0: acceptor
      • 
-
    • A1: thing accepted
      • 
-
    • A2: accepted-from
      • 
-
    • A3: Attribute
      • 
-
    • AM-MOD: modal
      • 
-
    • AM-NEG: negation
      • 
-
    
-
    Given the verb “accept”, the chunks in sentence would play certain semantic roles. Here, the label scheme is from Penn Proposition Bank.
    
-
    To this date, most of the successful SRL systems are built on top of some form of parsing results where pre-defined feature templates over the syntactic structure are used. This tutorial will present an end-to-end system using deep bidirectional long short-term memory (DB-LSTM)[2] for solving the SRL task, which largely outperforms the previous state-of-the-art systems. The system regards SRL task as the sequence labelling problem.
    
-
    
-
    Data Description
    
-
    The relevant paper[2] takes the data set in CoNLL-2005&2012 Shared Task for training and testing. Accordingto data license,  the demo adopts the test data set of CoNLL-2005, which can be reached on website.
    
-
    To download and process the original data, user just need to execute the following command:
    
-
    cd data
-./get_data.sh
-
    
-
    
-
    Several new files appear in the datadirectory as follows.
    
-
    conll05st-release：the test data set of CoNll-2005 shared task 
-test.wsj.words：the Wall Street Journal data sentences
-test.wsj.props:  the propositional arguments
-feature: the extracted features from data set
-
    
-
    
-
    
-
    
-
    Training
    
-
    
-
    DB-LSTM
    
-
    Please refer to the Sentiment Analysis demo to learn more about the long short-term memory unit.
    
-
    Unlike Bidirectional-LSTM that used in Sentiment Analysis demo,  the DB-LSTM adopts another way to stack LSTM layer. First a standard LSTM processes the sequence in forward direction. The input and output of this LSTM layer are taken by the next LSTM layer as input, processed in reversed direction. These two standard LSTM layers compose a pair of LSTM. Then we stack LSTM layers pair after pair to obtain the deep LSTM model.
    
-
    The following figure shows a temporal expanded 2-layer DB-LSTM network.
-
    
-pic
-
    
-
    
-
    
-
    Features
    
-
    Two input features play an essential role in this pipeline: predicate (pred) and argument (argu). Two other features: predicate context (ctx-p) and region mark (mr) are also adopted. Because a single predicate word can not exactly describe the predicate information, especially when the same words appear more than one times in a sentence. With the predicate context, the ambiguity can be largely eliminated. Similarly, we use region mark mr = 1 to denote the argument position if it locates in the predicate context region, or mr = 0 if does not. These four simple features are all we need for our SRL system. Features of one sample with context size set to 1 is showed as following[2]:
-
    
-pic
-
    
-
    In this sample, the coresponding labelled sentence is:
    
-
    [ A1 A record date ] has [ AM-NEG n’t ] been [ V set ] .
    
-
    In the demo, we adopt the feature template as above, consists of :  argument, predicate, ctx-p (p=-1,0,1), mark and use B/I/O scheme to label each argument. These features and labels are stored in feature file, and separated by \t.
    
-
    
-
    
-
    Data Provider
    
-
    dataprovider.py is the python file to wrap data. hook() function is to define the data slots for network. The  Six features and label are all IndexSlots.
    
-
    def hook(settings, word_dict, label_dict, **kwargs):
-    settings.word_dict = word_dict
-    settings.label_dict = label_dict
-    #all inputs are integral and sequential type
-    settings.slots = [
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(predicate_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(2),
-        integer_value_sequence(len(label_dict))]
-
    
-
    
-
    The corresponding data iterator is as following:
    
-
    @provider(init_hook=hook, should_shuffle=True, calc_batch_size=get_batch_size,
-          can_over_batch_size=False, cache=CacheType.CACHE_PASS_IN_MEM)
-def process(settings, file_name):
-    with open(file_name, 'r') as fdata:
-        for line in fdata:
-            sentence, predicate, ctx_n2, ctx_n1, ctx_0, ctx_p1, ctx_p2,  mark, label = \
-                line.strip().split('\t')
-
-            words = sentence.split()
-            sen_len = len(words)
-            word_slot = [settings.word_dict.get(w, UNK_IDX) for w in words]
-
-            predicate_slot = [settings.predicate_dict.get(predicate)] * sen_len
-            ctx_n2_slot = [settings.word_dict.get(ctx_n2, UNK_IDX)] * sen_len
-            ctx_n1_slot = [settings.word_dict.get(ctx_n1, UNK_IDX)] * sen_len
-            ctx_0_slot = [settings.word_dict.get(ctx_0, UNK_IDX)] * sen_len
-            ctx_p1_slot = [settings.word_dict.get(ctx_p1, UNK_IDX)] * sen_len
-            ctx_p2_slot = [settings.word_dict.get(ctx_p2, UNK_IDX)] * sen_len
-
-            marks = mark.split()
-            mark_slot = [int(w) for w in marks]
-
-            label_list = label.split()
-            label_slot = [settings.label_dict.get(w) for w in label_list]
-            yield word_slot, predicate_slot, ctx_n2_slot, ctx_n1_slot, \
-                  ctx_0_slot, ctx_p1_slot, ctx_p2_slot, mark_slot, label_slot
-
    
-
    
-
    The processfunction yield 9 lists which are 8 features and label.
    
-
    
-
    
-
    Neural Network Config
    
-
    db_lstm.py is the neural network config file to load the dictionaries and define the  data provider module and network architecture during the training procedure.
    
-
    Nine data_layer load instances from data provider. Eight features are transformed into embedddings respectively, and mixed by mixed_layer .  Deep bidirectional LSTM layers extract features for the softmax layer. The objective function is cross entropy of labels.
    
-
    
-
    
-
    Run Training
    
-
    The script for training is train.sh, user just need to execute:
    
-
      ./train.sh
-
    
-
    
-
    The content in train.sh:
    
-
    paddle train \
-  --config=./db_lstm.py \
-  --use_gpu=0 \
-  --log_period=5000 \
-  --trainer_count=1 \
-  --show_parameter_stats_period=5000 \
-  --save_dir=./output \
-  --num_passes=10000 \
-  --average_test_period=10000000 \
-  --init_model_path=./data \
-  --load_missing_parameter_strategy=rand \
-  --test_all_data_in_one_period=1 \
-2>&1 | tee 'train.log'
-
    
-
    
-
      
-
    • --config=./db_lstm.py : network config file.
      • 
-
    • --use_gpu=false: use CPU to train, set true, if you install GPU version of PaddlePaddle and want to use GPU to train, until now crf_layer do not support GPU
      • 
-
    • --log_period=500: print log every 20 batches.
      • 
-
    • --trainer_count=1: set thread number (or GPU count).
      • 
-
    • --show_parameter_stats_period=5000: show parameter statistic every 100 batches.
      • 
-
    • --save_dir=./output: output path to save models.
      • 
-
    • --num_passes=10000: set pass number, one pass in PaddlePaddle means training all samples in dataset one time.
      • 
-
    • --average_test_period=10000000:  do test on average parameter every average_test_period batches
      • 
-
    • --init_model_path=./data: parameter initialization path
      • 
-
    • --load_missing_parameter_strategy=rand: random initialization unexisted parameters
      • 
-
    • --test_all_data_in_one_period=1: test all data in one period
      • 
-
    
-
    After training, the models  will be saved in directory output. Our training curve is as following:
-
    
-pic
-
    
-
    
-
    
-
    Run testing
    
-
    The script for testing is test.sh, user just need to execute:
    
-
      ./test.sh
-
    
-
    
-
    The main part in tesh.sh
    
-
    paddle train \
-  --config=./db_lstm.py \
-  --model_list=$model_list \
-  --job=test \
-  --config_args=is_test=1 \
-
    
-
    
-
      
-
    • --config=./db_lstm.py: network config file
      • 
-
    • --model_list=$model_list.list: model list file
      • 
-
    • --job=test: indicate the test job
      • 
-
    • --config_args=is_test=1: flag to indicate test
      • 
-
    • --test_all_data_in_one_period=1: test all data in 1 period
      • 
-
    
-
    
-
    
-
    Run prediction
    
-
    The script for prediction is predict.sh, user just need to execute:
    
-
      ./predict.sh
-  
-
    
-
    
-
    In predict.sh, user should offer the network config file, model path, label file, word dictionary file, feature file
    
-
    python predict.py 
-     -c $config_file \
-     -w $best_model_path \
-     -l $label_file \
-     -p $predicate_dict_file  \
-     -d $dict_file \
-     -i $input_file \
-     -o $output_file
-
    
-
    
-
    predict.py is the main executable python script, which includes functions: load model, load data, data prediction. The network model will output the probability distribution of labels. In the demo, we take the label with maximum probability as result. User can also implement the beam search or viterbi decoding upon the probability distribution matrix.
    
-
    After prediction,  the result is saved in predict.res.
    
-
    
-
    
-
    
-
    Reference
    
-
    [1] Martha Palmer, Dan Gildea, and Paul Kingsbury. The Proposition Bank: An Annotated Corpus of Semantic Roles , Computational Linguistics, 31(1), 2005.
    
-
    [2] Zhou, Jie, and Wei Xu. “End-to-end learning of semantic role labeling using recurrent neural networks.” Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.
    
-
    
-
    
-
-
-           
    
-          
    
-          
    
-  
-
-  
    
-
-  
    
-    
    
-        © Copyright 2016, PaddlePaddle developers.
-
-    
    
-  
    
-  Built with Sphinx using a theme provided by Read the Docs. 
-
-
    
-
-        
    
-      
    
-
-    
    
-
-  
    
-  
-
-
-  
-
-    
-      
-      
-      
-      
-       
-  
-
-  
-  
-    
-  
-  
-  
-  
-   
-
-
-
\ No newline at end of file
diff --git a/doc/tutorials/sentiment_analysis/index_en.html b/doc/tutorials/sentiment_analysis/index_en.html
deleted file mode 100644
index 86f11ded6f39daa34175d63987fb95ca75069618..0000000000000000000000000000000000000000
--- a/doc/tutorials/sentiment_analysis/index_en.html
+++ /dev/null
@@ -1,544 +0,0 @@
-
-
-
-
-  
-
-  
-  
-  
-  
-  Sentiment Analysis Tutorial — PaddlePaddle  documentation
-  
-
-  
-  
-
-  
-
-  
-  
-    
-
-  
-
-  
-  
-    
-  
-
-  
-  
-        
-        
-     
-
-  
-  
-  
-
-  
-
-  
-  
-
-
-
-
-
-  
-  
    
-    
-    
    
-      
    
-        Folk me on Github
-        
    
-          
-            English
-            
-            
-          
-          
-        
    
-        
-      
    
-      
    
-        
-        
-
-        
-
    
-  
    
-    
-    
-    
-  
    
-
            
-      
    
-    
    
-  
    
-  
-  
    
-
-    
-    
-    
-    
    
-
-      
-
- 
-
-
-
-
-
-
-
-
    
-  
      
-      
-    
    • Sentiment Analysis Tutorial
      • 
-  
    
-
    
-      
-      
    
-        
    
-          
    
-           
    
-            
-  
    
-
    Sentiment Analysis Tutorial
    
-
    Sentiment analysis has many applications. A basic task in sentiment analysis is classifying the polarity of a given text at the document, sentence or feature/aspect level. One simple example is to classify the customer reviews in a shopping website, a tourism website, and group buying websites like Amazon, TaoBao, Tmall etc.
    
-
    Sentiment analysis is also used to monitor social media based on large amount of reviews or blogs. For example, the researchers analyzed several surveys on consumer confidence and political opinion, found they correlate to sentiment word frequencies in contemporaneous Twitter messages [1]. Another example is to forecast stock movements through analyzing the text content of a daily Twitter blog [2].
    
-
    On the other hand, grabbing the user comments of products and analyzing their sentiment are useful to understand user preferences for companies, products, even competing products.
    
-
    This tutorial will guide you through the process of training a Long Short Term Memory (LSTM) Network to classify the sentiment of sentences from Large Movie Review Dataset, sometimes known as the Internet Movie Database (IMDB). This dataset contains movie reviews along with their associated binary sentiment polarity labels, namely positive and negative. So randomly guessing yields 50% accuracy.
    
-
    
-
    Data Preparation
    
-
    
-
    IMDB Data Introduction
    
-
    Before training models, we need to preprocess the data and build a dictionary. First, you can use following script to download IMDB dataset and Moses tool, which is a statistical machine translation system. We provide a data preprocessing script, which is capable of handling not only IMDB data, but also other user-defined data. In order to use the pre-written script, it needs to move labeled train and test samples to another path, which has been done in get_imdb.sh.
    
-
    cd demo/sentiment/data
-./get_imdb.sh
-
    
-
    
-
    If the data is obtained successfuly, you will see the following files at ./demo/sentiment/data:
    
-
    aclImdb  get_imdb.sh  imdb  mosesdecoder-master
-
    
-
    
-
      
-
    • aclImdb: raw dataset downloaded from website.
      • 
-
    • imdb: only contains train and test data.
      • 
-
    • mosesdecoder-master: Moses tool.
      • 
-
    
-
    IMDB dataset contains 25,000 highly polar movie reviews for training, and 25,000 for testing. A negative review has a score ≤ 4 out of 10, and a positive review has a score ≥ 7 out of 10. After running ./get_imdb.sh, we can find the dataset has the following structure in aclImdb.
    
-
    imdbEr.txt  imdb.vocab  README  test  train
-
    
-
    
-
      
-
    • train: train sets.
      • 
-
    • test : test sets.
      • 
-
    • imdb.vocab: dictionary.
      • 
-
    • imdbEr.txt: expected rating for each token in imdb.vocab.
      • 
-
    • README: data documentation.
      • 
-
    
-
    The file in train set directory is as follows. The test set also contains them except unsup and urls_unsup.txt.
    
-
    labeledBow.feat  neg  pos  unsup  unsupBow.feat  urls_neg.txt  urls_pos.txt  urls_unsup.txt
-
    
-
    
-
      
-
    • pos: positive samples, contains 12,500 txt files, each file is one movie review.
      • 
-
    • neg: negative samples, contains 12,500 txt files, each file is one movie review.
      • 
-
    • unsup: unlabeled samples, contains 50,000 txt files.
      • 
-
    • urls_xx.txt: urls of each reviews.
      • 
-
    • xxBow.feat: already-tokenized bag of words (BoW) features.
      • 
-
    
-
    
-
    
-
    IMDB Data Preparation
    
-
    In this demo, we only use labled train and test set and not use imdb.vocab as dictionary. By default, dictionary is builded on train set. Train set is shuffled and test set is not. tokenizer.perl in Moses tool is used to tokenize the words and punctuation. Simply execute the following command to preprcess data.
    
-
    cd demo/sentiment/
-./preprocess.sh
-
    
-
    
-
    preprocess.sh:
    
-
    data_dir="./data/imdb"
-python preprocess.py -i data_dir
-
    
-
    
-
      
-
    • data_dir: input data directory.
      • 
-
    • preprocess.py: preprocess script.
      • 
-
    
-
    If running successfully, you will see demo/sentiment/data/pre-imdb directory as follows:
    
-
    dict.txt  labels.list  test.list  test_part_000  train.list  train_part_000
-
    
-
    
-
      
-
    • test_part_000 and train_part_000: all labeled test and train sets. Train sets have be shuffled.
      • 
-
    • train.list and test.list: train and test file lists.
      • 
-
    • dict.txt: dictionary generated on train sets by default.
      • 
-
    • labels.txt: neg  0, pos 1, means label 0 is negative review, label 1 is positive review.
      • 
-
    
-
    
-
    
-
    User-defined Data Preparation
    
-
    If you perform other sentiment classifcation task, you can prepare data as follows. We have provided the scripts to build dictionary and preprocess data. So just organize data as follows.
    
-
    dataset
-|----train
-|    |----class1
-|    |    |----text_files
-|    |----class2
-|    |    |----text_files
-|    |    ...
-|----test
-|    |----class1
-|    |    |----text_files
-|    |----class2
-|    |    |----text_files
-|    |    ...
-
    
-
    
-
      
-
    • dataset: 1st directory.
      • 
-
    • train, test: 2nd directory.
      • 
-
    • class1,class2,...: 3rd directory.
      • 
-
    • text_files: samples with text file format.
      • 
-
    
-
    All samples with text files format under the same folder are same category. Each text file contains one or more samples and each line is one sample. In order to shuffle fully, the preprocessing is a little different for data with multiple lines in one text file, which needs to set -m True in preprocess.sh. And tokenizer.perl is used by default. If you don’t need it, only set -t False in `preprocess.sh’.
    
-
    
-
    
-
    
-
    Training
    
-
    In this task, we use Recurrent Neural Network (RNN) of LSTM architecure to train sentiment analysis model. LSTM model was introduced primarily in order to overcome the problem of vanishing gradients. LSTM network resembles a standard recurrent neural network with a hidden layer, but each ordinary node in the hidden layer is replaced by a memory cell. Each memory cell contains four main elements: an input gate, a neuron with a self-recurrent connection, a forget gate and an output gate. More details can be found in the literature [4]. The biggest advantage of the LSTM architecture is that it learns to memorize information over long time intervals without the loss of short time memory. At each time step with a new coming word, historical information stored in the memory block is updated to iteratively learn the sequence representation.
    
-
    LSTM
    
-
    Figure 1. LSTM [3]
    
-
    Sentiment analysis is among the most typical problems in natural language understanding. It aims at predicting the attitude expressed in a sequence. Usually, only some key words, like adjectives and adverbs words, play a major role in predicting the sentiment of sequences or paragraphs. However, some review or comment contexts are very long, such as IMDB dataset. We use LSTM to perform this task for its improved design with the gate mechanism. First, it is able to summarize the representation from word level to context level with variable context length which is adapted by the gate values. Second, it can utilize the expanded context at the sentence level, while most methods are good at utilizing n-gram level knowledge. Third, it learns the paragraph representation directly rather than combining the context level information. This results in this end-to-end framework.
    
-
    In this demo we provide two network, namely bidirectional-LSTM and three layers of stacked-LSTM.
    
-
    
-
    Bidirectional-LSTM
    
-
    One is a bidirectional LSTM network, connected by fully connected layer and softmax, as shown in Figure 2.
    
-
    BiLSTM
    
-
    Figure 2. Bidirectional-LSTM 
    
-
    
-
    
-
    Stacked-LSTM
    
-
    Another is three-layer LSTM structure in Figure 3. The bottom of the figure is word embedding. Next, three LSTM-Hidden layers are connected and the second LSTM is reversed. Then extract the maximum hidden vectors of all time step of hidden and LSTM layer as the representation for the entire sequence. Finally, a fully connected feed forward layer with softmax activation is used to perform the classification task. This network is refered to paper [5].
    
-
    StackedLSTM
    
-
    Figure 3. Stacked-LSTM for sentiment analysis 
    
-
    Config
    
-
    Switch into demo/sentiment directory, trainer_config.py file is an example of the config, containing algorithm and newtork configure. The first line imports predefined networks from sentiment_net.py.
    
-
    trainer_config.py:
    
-
    from sentiment_net import *
-
-data_dir  = "./data/pre-imdb"
-# whether this config is used for test
-is_test = get_config_arg('is_test', bool, False)
-# whether this config is used for prediction
-is_predict = get_config_arg('is_predict', bool, False)
-dict_dim, class_dim = sentiment_data(data_dir, is_test, is_predict)
-
-################## Algorithm Config #####################
-
-settings(
-  batch_size=128,
-  learning_rate=2e-3,
-  learning_method=AdamOptimizer(),
-  average_window=0.5,
-  regularization=L2Regularization(8e-4),
-  gradient_clipping_threshold=25
-)
-
-#################### Network Config ######################
-stacked_lstm_net(dict_dim, class_dim=class_dim,
-                 stacked_num=3, is_predict=is_predict)
-#bidirectional_lstm_net(dict_dim, class_dim=class_dim, is_predict=is_predict)
-
    
-
    
-
      
-
    • Data Definition:
        
-
      • get_config_arg(): get arguments setted by --config_args=xx in commandline argument.
        • 
-
      • Define data provider, here using Python interface to load data. For details, you can refer to the document of PyDataProvider2.
        • 
-
      
-
      • 
-
    • Algorithm Configuration:
        
-
      • set batch size of 128.
        • 
-
      • set global learning rate.
        • 
-
      • use adam optimization.
        • 
-
      • set average sgd window.
        • 
-
      • set L2 regularization.
        • 
-
      • set gradient clipping threshold.
        • 
-
      
-
      • 
-
    • Network Configuration:
        
-
      • dict_dim: dictionary dimension.
        • 
-
      • class_dim: category number, IMDB has two label, namely positive and negative label.
        • 
-
      • stacked_lstm_net: predefined network as shown in Figure 3, use this network by default.
        • 
-
      • bidirectional_lstm_net: predefined network as shown in Figure 2.
        • 
-
      
-
      • 
-
    
-
    Training
    
-
    Install PaddlePaddle first if necessary. Then you can use script train.sh as follows to launch local training.
    
-
    cd demo/sentiment/
-./train.sh
-
    
-
    
-
    train.sh:
    
-
    config=trainer_config.py
-output=./model_output
-paddle train --config=$config \
-             --save_dir=$output \
-             --job=train \
-             --use_gpu=false \
-             --trainer_count=4 \
-             --num_passes=10 \
-             --log_period=20 \
-             --dot_period=20 \
-             --show_parameter_stats_period=100 \
-             --test_all_data_in_one_period=1 \
-             2>&1 | tee 'train.log'
-
    
-
    
-
      
-
    • --config=$config: set network config.
      • 
-
    • --save_dir=$output: set output path to save models.
      • 
-
    • --job=train: set job mode to train.
      • 
-
    • --use_gpu=false: use CPU to train, set true, if you install GPU version of PaddlePaddle and want to use GPU to train.
      • 
-
    • --trainer_count=4: set thread number (or GPU count).
      • 
-
    • --num_passes=15: set pass number, one pass in PaddlePaddle means training all samples in dataset one time.
      • 
-
    • --log_period=20: print log every 20 batches.
      • 
-
    • --show_parameter_stats_period=100: show parameter statistic every 100 batches.
      • 
-
    • --test_all_data_in_one_period=1: test all data every testing.
      • 
-
    
-
    If the run succeeds, the output log is saved in path of demo/sentiment/train.log and model is saved in path of demo/sentiment/model_output/. The output log is explained as follows.
    
-
    Batch=20 samples=2560 AvgCost=0.681644 CurrentCost=0.681644 Eval: classification_error_evaluator=0.36875  CurrentEval: classification_error_evaluator=0.36875
-...
-Pass=0 Batch=196 samples=25000 AvgCost=0.418964 Eval: classification_error_evaluator=0.1922
-Test samples=24999 cost=0.39297 Eval: classification_error_evaluator=0.149406
-
    
-
    
-
      
-
    • Batch=xx: means passing xx batches.
      • 
-
    • samples=xx: means passing xx samples.
      • 
-
    • AvgCost=xx: averaged cost from 0-th batch to current batch.
      • 
-
    • CurrentCost=xx: current cost of latest log_period batches.
      • 
-
    • Eval: classification_error_evaluator=xx: means classfication error from 0-th batch ro current batch.
      • 
-
    • CurrentEval: classification_error_evaluator: current classfication error of the lates log_period batches.
      • 
-
    • Pass=0: Going through all training set one time is called one pass. 0 means going through training set first time.
      • 
-
    
-
    By default, we use the stacked_lstm_net network, which converges at a faster rate than bidirectional_lstm_net when passing same sample number. If you want to use bidirectional LSTM, just remove comment in the last line and comment stacked_lstm_net.
    
-
    
-
    
-
    
-
    Testing
    
-
    Testing means evaluating the labeled validation set using trained model.
    
-
    cd demo/sentiment
-./test.sh
-
    
-
    
-
    test.sh:
    
-
    function get_best_pass() {
-  cat $1  | grep -Pzo 'Test .*\n.*pass-.*' | \
-  sed  -r 'N;s/Test.* error=([0-9]+\.[0-9]+).*\n.*pass-([0-9]+)/\1 \2/g' | \
-  sort | head -n 1
-}
-
-log=train.log
-LOG=`get_best_pass $log`
-LOG=(${LOG})
-evaluate_pass="model_output/pass-${LOG[1]}"
-
-echo 'evaluating from pass '$evaluate_pass
-
-model_list=./model.list
-touch $model_list | echo $evaluate_pass > $model_list
-net_conf=trainer_config.py
-paddle train --config=$net_conf \
-             --model_list=$model_list \
-             --job=test \
-             --use_gpu=false \
-             --trainer_count=4 \
-             --config_args=is_test=1 \
-             2>&1 | tee 'test.log'
-
    
-
    
-
    The function get_best_pass gets the best model by classification error rate for testing. In this example, We use test dataset of IMDB as validation by default. Unlike training, it needs to specify --job=test and model path, namely --model_list=$model_list here. If running successfully, the log is saved in path of demo/sentiment/test.log. For example, in our test, the best model is model_output/pass-00002, the classification error is 0.115645 as follows.
    
-
    Pass=0 samples=24999 AvgCost=0.280471 Eval: classification_error_evaluator=0.115645
-
    
-
    
-
    
-
    
-
    Prediction
    
-
    predict.py provides a predicting interface. You should install python api of PaddlePaddle before using it. One example to predict unlabeled review of IMDB is as follows. Simply running:
    
-
    cd demo/sentiment
-./predict.sh
-
    
-
    
-
    predict.sh:
    
-
    #Note the default model is pass-00002, you shold make sure the model path
-#exists or change the mode path.
-model=model_output/pass-00002/
-config=trainer_config.py
-label=data/pre-imdb/labels.list
-cat ./data/aclImdb/test/pos/10007_10.txt | python predict.py \
-     --tconf=$config\
-     --model=$model \
-     --label=$label \
-     --dict=./data/pre-imdb/dict.txt \
-     --batch_size=1
-
    
-
    
-
      
-
    • cat ./data/aclImdb/test/pos/10007_10.txt : the input sample.
      • 
-
    • predict.py : predicting interface.
      • 
-
    • --tconf=$config : set network configure.
      • 
-
    • --model=$model : set model path.
      • 
-
    • --label=$label : set dictionary about corresponding relation between integer label and string label.
      • 
-
    • --dict=data/pre-imdb/dict.txt : set dictionary.
      • 
-
    • --batch_size=1 : set batch size.
      • 
-
    
-
    Note you should make sure the default model path model_output/pass-00002
-exists or change the model path.
    
-
    Predicting result of this example:
    
-
    Loading parameters from model_output/pass-00002/
-./data/aclImdb/test/pos/10014_7.txt: predicting label is pos
-
    
-
    
-
    We sincerely appreciate your interest and welcome your contributions.
    
-
    
-
    
-
    Reference
    
-
    [1] Brendan O’Connor, Ramnath Balasubramanyan, Bryan R. Routledge, and Noah A. Smith. 2010. From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series. In ICWSM-2010. 
    
-[2] Johan Bollen, Huina Mao, Xiaojun Zeng. 2011. Twitter mood predicts the stock market, Journal of Computational Science.
    
-[3] Alex Graves, Marcus Liwicki, Santiago Fernan- dez, Roman Bertolami, Horst Bunke, and Ju ̈rgen Schmidhuber. 2009. A novel connectionist system for unconstrained handwriting recognition. IEEE Transactions on Pattern Analysis and Machine In- telligence, 31(5):855–868.
    
-[4] Zachary C. Lipton, A Critical Review of Recurrent Neural Networks for Sequence Learning, arXiv:1506.00019. 
    
-[5] Jie Zhou and Wei Xu; End-to-end Learning of Semantic Role Labeling Using Recurrent Neural Networks; ACL-IJCNLP 2015. 
    
-
    
-
    
-
-
-           
    
-          
    
-          
    
-  
-
-  
    
-
-  
    
-    
    
-        © Copyright 2016, PaddlePaddle developers.
-
-    
    
-  
    
-  Built with Sphinx using a theme provided by Read the Docs. 
-
-
    
-
-        
    
-      
    
-
-    
    
-
-  
    
-  
-
-
-  
-
-    
-      
-      
-      
-      
-       
-  
-
-  
-  
-    
-  
-  
-  
-  
-   
-
-
-
\ No newline at end of file
diff --git a/doc/tutorials/text_generation/index_en.html b/doc/tutorials/text_generation/index_en.html
deleted file mode 100644
index 6f7d9f54ced24a520311505e733697cc97fe3bcb..0000000000000000000000000000000000000000
--- a/doc/tutorials/text_generation/index_en.html
+++ /dev/null
@@ -1,592 +0,0 @@
-
-
-
-
-  
-
-  
-  
-  
-  
-  Text generation Tutorial — PaddlePaddle  documentation
-  
-
-  
-  
-
-  
-
-  
-  
-    
-
-  
-
-  
-  
-    
-  
-
-  
-  
-        
-        
-     
-
-  
-  
-  
-
-  
-
-  
-  
-
-
-
-
-
-  
-  
    
-    
-    
    
-      
    
-        Folk me on Github
-        
    
-          
-            English
-            
-            
-          
-          
-        
    
-        
-      
    
-      
    
-        
-        
-
-        
-
    
-  
    
-    
-    
-    
-  
    
-
            
-      
    
-    
    
-  
    
-  
-  
    
-
-    
-    
-    
-    
    
-
-      
-
- 
-
-
-
-
-
-
-
-
    
-  
      
-      
-    
    • Text generation Tutorial
      • 
-  
    
-
    
-      
-      
    
-        
    
-          
    
-           
    
-            
-  
    
-
    Text generation Tutorial
    
-
    Sequence to sequence has been proven to be a powerful model for language generation. It can be used for machine translation, query rewriting, image captioning, etc.
    
-
    This tutorial guides you through training a sequence to sequence model for neural machine translation (NMT) network that translates French to English.
    
-
    We follow the paper Neural Machine Translation by Jointly Learning to Align and Translate , which details the model architecture and training procedure for good performance on WMT-14 dataset. This tutorial reproduces this result in PaddlePaddle.
    
-
    We thank @caoying for the pull request that defines the model architecture and solver configurations.
    
-
    
-
    Data Preparation
    
-
    
-
    Download and Extract
    
-
    Download the WMT-14 dataset from http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/, extract it, and divide Develop and Test data into separate folder.
    
-
-
    To do this, simply run the following commands in linux, otherwise, you need to download, extract, divide, and rename the file suffix respectively.
    
-
    cd demo/seqToseq/data
-./wmt14_data.sh
-
    
-
    
-
    We should find that the dataset wmt14 has three folders as shown in the following table.
    
-
------
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
    





    folder nameFrench-English parallel corpora filenumber of total filesize
    train_dataccb2_pc30.src, ccb2_pc30.trg, etctwelve3.55G
    test_datantst1213.src, ntst1213.trgtwo1636k
    gen_datantst14.src, ntst14.trgtwo864k
    
-
      
-
    • Each folder has French-English parallel corpora
      • 
-
    • XXX.src are source French files; XXX.trg are target English files.
      • 
-
    • The number of lines of XXX.src and XXX.trg should be the same.
      • 
-
    • Each line is a French/English sentence.
      • 
-
    • There is a one-to-one correspondence between the sentence at the i-th line of XXX.src and XXX.trg.
      • 
-
    
-
    
-
    
-
    User Defined Dataset
    
-
    If you need to do other sequence-to-sequence tasks, such as Paraphrasing, you only need to organize the data as follows, and place them in demo/seqToseq/data:
    
-
    dataset
-  train
-    file1.src file1.trg
-    file2.src file2.trg
-    ......
-  test
-    file1.src file1.trg
-    file2.src file2.trg
-    ......
-  gen
-    file1.src file1.trg
-    file2.src file2.trg
-    ......
-
    
-
    
-
      
-
    • 1st directory: dataset folder name
      • 
-
    • 2nd directory: folder of train, test, and gen. The names of these three folders are fixed.
      • 
-
    • 3rd file: Source-Target parallel corpora files.
        
-
      • XXX.src are source files, XXX.trg are target files.
        • 
-
      • Each line of the file must be a sequence.
        • 
-
      • There should be a one-to-one correspondence between the i-th sequence of XXX.src and XXX.trg.
        • 
-
      
-
      • 
-
    
-
    
-
    
-
    
-
    Data Preprocess
    
-
    
-
    Preprocessing Workflow
    
-
      
-
    • Concat each Source-Target parallel corpora to be one file:
        
-
      • concat each XXX.src and XXX.trg to be XXX.
        • 
-
      • the i-th line of XXX = the i-th line of XXX.src + ‘\t’ + the i-th line of XXX.trg
        • 
-
      
-
      • 
-
    • Build source and target dictionary of train data, each dictionary has DICTSIZE words:
        
-
      • the most frequent (DICTSIZE-3) words
        • 
-
      • 3 special token:
          
-
        • <s>: the start of a sequence
          • 
-
        • <e>: the end of a sequence
          • 
-
        • <unk>: a word not included in dictionary
          • 
-
        
-
        • 
-
      
-
      • 
-
    
-
    
-
    
-
    Preprocessing Command and Result
    
-
    The general command for preprocessing the dataset is:
    
-
    cd demo/seqToseq/
-python preprocess.py -i INPUT [-d DICTSIZE] [-m]
-
    
-
    
-
      
-
    • -i INPUT: the path of input original dataset
      • 
-
    • -d DICTSIZE: the specified word count of dictionary, if not set, dictionary will contain all the words in input dataset
      • 
-
    • -m --mergeDict: merge source and target dictionary, thus, two dictionaries have the same context
      • 
-
    
-
    And you will see messages like this:
    
-
    concat parallel corpora for dataset
-build source dictionary for train data
-build target dictionary for train data
-dictionary size is XXX
-
    
-
    
-
    Here, you can simply run the command:
    
-
    python preprocess.py -i data/wmt14 -d 30000
-
    
-
    
-
    It will take several minutes, and store the preprocessed dataset in demo/seqToseq/data/pre-wmt14, the directory has following structure.
    
-
    train test gen train.list test.list gen.list src.dict trg.dict
-
    
-
    
-
      
-
    • train, test, gen: folder contains French-English parallel corpora of train data, test data and gen data respectively. Each line of file in folder contains two parts, the former is a French sequence, and the latter is a corresponding English sequence.
      • 
-
    • train.list, test.list, gen.list: text contains a file list in train folder, test folder and gen folder respectively
      • 
-
    • src.dict, trg.dict: source (French) / target (English) dictionary, each dictionary has 30000 words: the most frequent 29997 words and 3 special token
      • 
-
    
-
    
-
    
-
    
-
    Model Training
    
-
    
-
    Introduction
    
-
    Neural machine translation (NMT) aims at building a single neural network that can be jointly tuned to maximize translation performance. Recently proposed NMT models often belong to a family of encoder–decoder models. Encoder-Decoder models encode a source sentence into a fixed-length vector from which a decoder generates a target sentence.
    
-
    In this task, we use an extension to the encoder–decoder model which learns to align and translate jointly. Each time the model generates a word in a translation, it searches for a set of positions in the source sentence for the most relevant information.  The decoder predicts a target word based on the context vectors associated with these source positions and all the previous generated target words. For more detailed explanation, readers can refer to paper Neural Machine Translation by Jointly Learning to Align and Translate.
    
-
    The most distinguishing feature of this model is that it doesn’t encode an input sentence into a single fixed-length vector. Instead, it encodes the input sentence into a sequence of vectors, where one vector corresponds to an input element. A subset of these vectors is chosen adaptively while decoding the translated sentence. This frees a NMT model from having to squash all the information of a source sentence, regardless of its length, into a fixed-length vector. The improvement of this model is more apparent for longer sentences, but the improvement can be observed for sentences of any length.
-
    
-
    Figure 1. Encoder-Decoder-Attention-Model
    
-
    
-
    
-
    Training Model in PaddlePaddle
    
-
    We need to create a model config file before training. Here is an example demo/seqToseq/translation/train.conf. The first three lines import python function for defining network, and define the job_mode and attention_mode.
    
-
    from seqToseq_net import *
-is_generating = False
-
-### Data Definiation
-train_conf = seq_to_seq_data(data_dir = "./data/pre-wmt14",
-                             is_generating = is_generating)
-
-### Algorithm Configuration
-settings(
-    learning_method = AdamOptimizer(),
-    batch_size = 50,
-    learning_rate = 5e-4)
-
-### Network Architecture
-gru_encoder_decoder(train_conf, is_generating)
-
    
-
    
-
      
-
    1. Data Definiation: We define a SeqToSeq train and test data in our example. It returns train_conf as the configuration, following is its input arguments:
        
-
      • data_dir: directory of train data and test data
        • 
-
      • is_generating: whether this config is used for generating, here is false
        • 
-
      
-
      2. 
-
    2. Algorithm Configuration: We use the SGD training algorithm (default), ADAM learning method in our example, specify batch_size as 50, and learning rate as 5e-4.
      3. 
-
    3. Network Architecture: We use an attention version of GRU Encoder-Decoder network in our example. It consists a bidirectional GRU as an encoder and a decoder that emulates searching through a source sentence during decoding a translation.
      4. 
-
    
-
    
-
    
-
    Training Command and Result
    
-
    After writing the model config, we can train the model by running the command:
    
-
    cd demo/seqToseq/translation
-./train.sh
-
    
-
    
-
    The train.sh is shown as follows:
    
-
    paddle train \
---config='translation/train.conf' \
---save_dir='translation/model' \
---use_gpu=false \
---num_passes=16 \
---show_parameter_stats_period=100 \
---trainer_count=4 \
---log_period=10 \
---dot_period=5 \
-2>&1 | tee 'translation/train.log'
-
    
-
    
-
      
-
    • config: set config of neural network
      • 
-
    • save_dir: set output path to save models
      • 
-
    • use_gpu: whether to use GPU to train, here use CPU
      • 
-
    • num_passes: set number of passes. One pass in paddle means training all samples in dataset one time
      • 
-
    • show_parameter_stats_period: here show parameter statistic every 100 batches
      • 
-
    • trainer_count: set number of CPU threads or GPU devices
      • 
-
    • log_period: here print log every 10 batches
      • 
-
    • dot_period: here print ‘.’ every 5 batches
      • 
-
    
-
    The training loss function is printed every 10 batch by default, and you will see messages like this:
    
-
    I0719 19:16:45.952062 15563 TrainerInternal.cpp:160]  Batch=10 samples=500 AvgCost=198.475 CurrentCost=198.475 Eval: classification_error_evaluator=0.737155  CurrentEval: classification_error_evaluator=0.737155
-I0719 19:17:56.707319 15563 TrainerInternal.cpp:160]  Batch=20 samples=1000 AvgCost=157.479 CurrentCost=116.483 Eval: classification_error_evaluator=0.698392  CurrentEval: classification_error_evaluator=0.659065
-.....
-
    
-
    
-
      
-
    • AvgCost: Average Cost from 0th batch to current batch
      • 
-
    • CurrentCost: Cost in current batch
      • 
-
    • classification_error_evaluator(Eval): False prediction rate for each word from 0th evaluation to current evaluation
      • 
-
    • classification_error_evaluator(CurrentEval): False prediction rate for each word in current evaluation
      • 
-
    
-
    And when the classification_error_evaluator is less than 0.35, the model is trained sucessfully.
    
-
    
-
    
-
    
-
    Text Generation
    
-
    
-
    Introduction
    
-
    Generally speaking, the NMT model is conditioned on the encodings of the source sentence, and then to predict the next target word by given the current target word. In the training process, the current word is always knowns as the ground truth, by contrast. In the generating process, the current word is the output of the decoder in last time step, which is accessed to from a memory in PaddlePaddle.
    
-
    Besides, we use Beam Search to generate sequences. Beam search uses breadth-first search to build its search tree. At each level of the tree, it generates all successors of the states at the current level, sorting them in increasing order of heuristic cost. However, it only stores a predetermined number of best states at each level (called the beam size).
    
-
    
-
    
-
    Pretrained model
    
-
    We trained the model on a cluster with 50 nodes, each node has two 6-core CPUs. We trained 16 passes in 5 days, where each pass takes 7 hours. The model_dir has 16 sub-folder, each of which contains the whole model parameters with 202MB size. And we find pass-00012 model has the highest BLEU 27.77 (see paper BLEU: a Method for Automatic Evaluation of Machine Translation). To download and extract this model, simply run the following commands in linux.
    
-
    cd demo/seqToseq/data
-./wmt14_model.sh
-
    
-
    
-
    
-
    
-
    Generating Model in PaddlePaddle
    
-
    We need to create a model config file before translating French sequence. Here is an example demo/seqToseq/translation/gen.conf, the first three lines import python function for defining network, and define the job_mode and attention_mode.
    
-
    from seqToseq_net import *
-is_generating = True
-
-################## Data Definiation #####################
-gen_conf = seq_to_seq_data(data_dir = "./data/pre-wmt14",
-                           is_generating = is_generating,
-                           gen_result = "./translation/gen_result")
-
-############## Algorithm Configuration ##################
-settings(
-  learning_method = AdamOptimizer(),
-  batch_size = 1,
-  learning_rate = 0)
-
-################# Network configure #####################
-gru_encoder_decoder(gen_conf, is_generating)
-
    
-
    
-
      
-
    1. Data Definiation: We defines an SeqToSeq gen data in our example. It returns gen_conf as the configuration, following is its input arguments:
        
-
      • data_dir: directory of gen data
-  - is_generating: whether this config is used for generating, here is true
-  - gen_result: file to store the generation result
        • 
-
      
-
      2. 
-
    2. Algorithm Configuration: We use SGD traing algorithm in generation, and specify batch_size as 1 (each time generate one sequence), and learning rate as 0.
      3. 
-
    3. Network Architecture: Essentially the same as the training model.
      4. 
-
    
-
    
-
    
-
    Generating Command and Result
    
-
    After writing the model config, we can do text translation from French to English by running the command:
    
-
    cd demo/seqToseq/translation
-./gen.sh
-
    
-
    
-
    The gen.sh is shown as follows, unlike training, there are some different arguments to specify:
    
-
    paddle train \
---job=test \
---config='translation/gen.conf' \
---save_dir='data/wmt14_model' \
---use_gpu=true \
---num_passes=13 \
---test_pass=12 \
---trainer_count=1 \
-2>&1 | tee 'translation/gen.log'
-
    
-
    
-
      
-
    • job: set job mode to test
      • 
-
    • save_dir: the path of saved models
      • 
-
    • num_passes and test_pass: loading model parameters from test_pass to (num_passes - 1), here only loads data/wmt14_model/pass-00012
      • 
-
    
-
    You will see messages like this:
    
-
    I0706 14:48:31.178915 31441 GradientMachine.cpp:143] Loading parameters from data/wmt14_model/pass-00012
-I0706 14:48:40.012039 31441 Tester.cpp:125]  Batch=100 samples=100 AvgCost=0
-I0706 14:48:48.898632 31441 Tester.cpp:125]  Batch=200 samples=200 AvgCost=0
-...
-
    
-
    
-
    And the generating result in demo/seqToseq/translation/gen_result likes:
    
-
    0
-0       -11.1314         The <unk> <unk> about the width of the seats while large controls are at stake <e>
-1       -11.1519         The <unk> <unk> on the width of the seats while large controls are at stake <e>
-2       -11.5988         The <unk> <unk> about the width of the seats while large controls are at stake . <e>
-
-1
-0       -24.4149         The dispute is between the major aircraft manufacturers about the width of the tourist seats on the <unk> flights , paving the way for a <unk> confrontation during the month of the Dubai <unk> . <e>
-1       -26.9524         The dispute is between the major aircraft manufacturers about the width of the tourist seats on the <unk> flights , paving the way for a <unk> confrontation during the month of Dubai &apos; s <unk> . <e>
-2       -27.9574         The dispute is between the major aircraft manufacturers about the width of the tourist seats on the <unk> flights , paving the way for a <unk> confrontation during the month of Dubai &apos; s Dubai <unk> . <e>
-...
-
    
-
    
-
      
-
    • This is the beam search result, where beam size is 3
      • 
-
    • ‘0’ in 1st-line and ‘1’ in 6th-line mean the sequence-id in gen data
      • 
-
    • Other six lines list the beam search results
        
-
      • The 2nd-column is the score of beam search (from large to small)
        • 
-
      • The 3rd-colunm is the generating English sequence
        • 
-
      
-
      • 
-
    • There is 2 special tokens:
        
-
      • <e>: the end of a sequence
        • 
-
      • <unk>: a word not included in dictionary
        • 
-
      
-
      • 
-
    
-
    
-
    
-
    Bleu Evalutaion
    
-
    Human evaluations of machine translation are extensive but expensive. Paper BLEU: a Method for Automatic Evaluation of Machine Translation presents a method as an automated understudy to skilled human judges which substitutes for them when there is need for quick or frequent evaluations. Moses is a statistical machine translation system, and we use multi-bleu.perl of it to do Bleu Evalution. To download this script, simply run the following command:
    
-
    cd demo/seqToseq/translation
-./moses_bleu.sh
-
    
-
    
-
    Since the standard translation is alrealy downloaded as data/wmt14/gen/ntst14.trg, we can do Bleu Evalution by running the command:
    
-
    cd demo/seqToseq/translation
-./eval_bleu.sh FILE BEAMSIZE
-
    
-
    
-
      
-
    • FILE: the generation result file
      • 
-
    • BEAMSIZE: expand width in beam search
      • 
-
    
-
    
-
    
-
    
-
-
-           
    
-          
    
-          
    
-  
-
-  
    
-
-  
    
-    
    
-        © Copyright 2016, PaddlePaddle developers.
-
-    
    
-  
    
-  Built with Sphinx using a theme provided by Read the Docs. 
-
-
    
-
-        
    
-      
    
-
-    
    
-
-  
    
-  
-
-
-  
-
-    
-      
-      
-      
-      
-       
-  
-
-  
-  
-    
-  
-  
-  
-  
-   
-
-
-
\ No newline at end of file
diff --git a/doc/tutorials/index_en.html b/doc/v1_api_tutorials/README.html
similarity index 68%
rename from doc/tutorials/index_en.html
rename to doc/v1_api_tutorials/README.html
index e44145f43c5a9885e122b1d26aba6d9dda379960..de3f7b30fca09790fd91eabacd96af185fffebbf 100644
--- a/doc/tutorials/index_en.html
+++ b/doc/v1_api_tutorials/README.html
@@ -8,7 +8,7 @@
   
   
   
-  TUTORIALS — PaddlePaddle  documentation
+  <no title> — PaddlePaddle  documentation
   
 
   
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,7 +84,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -166,7 +194,7 @@
 
    
   
      
       
-    
    • TUTORIALS
      • 
+    
    • <no title>
      • 
   
    
 
    
       
@@ -175,30 +203,10 @@
           
    
            
    
             
-  
+  
    The tutorials in v1_api_tutorials are using v1_api currently, and will be upgraded to v2_api later.
+Thus, v1_api_tutorials is a temporary directory. We decide not to maintain it and will delete it in future.
    
+
    Please go to PaddlePaddle/book and
+PaddlePaddle/models to learn PaddlePaddle.
    
 
 
            
    
diff --git a/doc/tutorials/embedding_model/index_en.html b/doc/v1_api_tutorials/embedding_model/index_en.html
similarity index 88%
rename from doc/tutorials/embedding_model/index_en.html
rename to doc/v1_api_tutorials/embedding_model/index_en.html
index 0f063ffcd7067366a99f6b79390466d2348dbc34..2770f5533f28a7e57657de9f646145d1bc22a170 100644
--- a/doc/tutorials/embedding_model/index_en.html
+++ b/doc/v1_api_tutorials/embedding_model/index_en.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,7 +84,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
@@ -283,7 +311,7 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL
 
  • --init_model_path: path of the initialization model, here is data/paraphrase_model
    • 
 
  • --load_missing_parameter_strategy: operations when model file is missing, here use a normal distibution to initialize the other parameters except for the embedding layer
    • 
 
-
    For users who want to understand the dataset format, model architecture and training procedure in detail, please refer to Text generation Tutorial.
    
+
    For users who want to understand the dataset format, model architecture and training procedure in detail, please refer to Text generation Tutorial.
    
 
    
 
    
 
    
diff --git a/doc/tutorials/gan/index_en.html b/doc/v1_api_tutorials/gan/index_en.html
similarity index 89%
rename from doc/tutorials/gan/index_en.html
rename to doc/v1_api_tutorials/gan/index_en.html
index d26e1d6ca64237efb80cb295a6a1ed4d647e8642..db21d7810d43744ab37f807586300581c5f01c5a 100644
--- a/doc/tutorials/gan/index_en.html
+++ b/doc/v1_api_tutorials/gan/index_en.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,7 +84,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
diff --git a/doc/tutorials/imagenet_model/resnet_model_en.html b/doc/v1_api_tutorials/imagenet_model/resnet_model_en.html
similarity index 91%
rename from doc/tutorials/imagenet_model/resnet_model_en.html
rename to doc/v1_api_tutorials/imagenet_model/resnet_model_en.html
index 85a8e7cb3ec069923f876a5d0c7d3bf8063628c8..cfdda88a544e5d442a3aacca431bf90eada89514 100644
--- a/doc/tutorials/imagenet_model/resnet_model_en.html
+++ b/doc/v1_api_tutorials/imagenet_model/resnet_model_en.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,7 +84,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
diff --git a/doc/tutorials/quick_start/index_en.html b/doc/v1_api_tutorials/quick_start/index_en.html
similarity index 94%
rename from doc/tutorials/quick_start/index_en.html
rename to doc/v1_api_tutorials/quick_start/index_en.html
index 7136215114fbe84276b81dc5c079074570bd2997..243eebd6cd2276a232a309a04a0050b417e937dd 100644
--- a/doc/tutorials/quick_start/index_en.html
+++ b/doc/v1_api_tutorials/quick_start/index_en.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -84,7 +84,7 @@
 
  • GET STARTED
    • 
 
  • HOW TO
    • 
 
  • API
    • 
-
  • ABOUT
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +108,10 @@
           
 
-
  • Run Distributed Training
    • 
+
  • PaddlePaddle Distributed Training
    • 
 
  • Paddle On Kubernetes
    • 
 
  • Distributed PaddlePaddle Training on AWS with Kubernetes
    • 
 
  • Write New Layers
    • 
 
  • Contribute Code
    • 
-
  • RNN Models
    • 
+
  • Contribute Documentation
    • 
+
  • RNN Models
+
    • 
 
  • Tune GPU Performance
    • 
 
 
@@ -135,17 +140,40 @@
 
  • Model Configuration
 
    • 
-
  • Data Reader Interface and DataSets
    • 
+
  • Data Reader Interface and DataSets
+
    • 
 
  • Training and Inference
    • 
+
  • Fluid
+
    • 
+
+
+
  • MOBILE
 
    • 
-
  • ABOUT
    • 
 
 
         
diff --git a/doc_cn/_images/bi_lstm1.jpg b/doc_cn/_images/bi_lstm1.jpg
deleted file mode 100644
index adec1606d64d6e35ffe7e62abfa9a09309b05c84..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/bi_lstm1.jpg and /dev/null differ
diff --git a/doc_cn/_images/cifar.png b/doc_cn/_images/cifar.png
deleted file mode 100644
index f54a0c58837cb3385b32dc57d02cec92666ef0f1..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/cifar.png and /dev/null differ
diff --git a/doc_cn/_images/curve.jpg b/doc_cn/_images/curve.jpg
deleted file mode 100644
index baa35ae7f0a0b6c246f3a0d331735477ab8bcd70..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/curve.jpg and /dev/null differ
diff --git a/doc_cn/_images/encoder-decoder-attention-model1.png b/doc_cn/_images/encoder-decoder-attention-model1.png
deleted file mode 100644
index 79f911d4ba12ac0c0d1a936c9df639c302786914..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/encoder-decoder-attention-model1.png and /dev/null differ
diff --git a/doc_cn/_images/feature.jpg b/doc_cn/_images/feature.jpg
deleted file mode 100644
index 0e3310e4ace5613917e7779d3198ccbb3cdc5ada..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/feature.jpg and /dev/null differ
diff --git a/doc_cn/_images/graph_construction_example_all.png b/doc_cn/_images/graph_construction_example_all.png
new file mode 100644
index 0000000000000000000000000000000000000000..261611a5721f9aa97874f7e6d897fe48cf667db2
Binary files /dev/null and b/doc_cn/_images/graph_construction_example_all.png differ
diff --git a/doc_cn/_images/graph_construction_example_forward_backward.png b/doc_cn/_images/graph_construction_example_forward_backward.png
new file mode 100644
index 0000000000000000000000000000000000000000..4c69687f4a6a181138f3df72ce5e8aa48487b5be
Binary files /dev/null and b/doc_cn/_images/graph_construction_example_forward_backward.png differ
diff --git a/doc_cn/_images/graph_construction_example_forward_only.png b/doc_cn/_images/graph_construction_example_forward_only.png
new file mode 100644
index 0000000000000000000000000000000000000000..e668c16e0cac73acb4e5dc2b1827557ae77126b4
Binary files /dev/null and b/doc_cn/_images/graph_construction_example_forward_only.png differ
diff --git a/doc_cn/_images/image_classification.png b/doc_cn/_images/image_classification.png
deleted file mode 100644
index 14f255805081c1b4fab27eaf336fd389fa93ca19..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/image_classification.png and /dev/null differ
diff --git a/doc_cn/_images/lenet.png b/doc_cn/_images/lenet.png
deleted file mode 100644
index 1e6f2b32bad797f3fccb929c72a121fc935b0cbb..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/lenet.png and /dev/null differ
diff --git a/doc_cn/_images/lstm.png b/doc_cn/_images/lstm.png
deleted file mode 100644
index aaf1fc690da2ffb8418cde5ed81848ddb5263030..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/lstm.png and /dev/null differ
diff --git a/doc_cn/_images/network_arch.png b/doc_cn/_images/network_arch.png
deleted file mode 100644
index 4ae7864212f2a0a38102ee7ff600527ea99fec82..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/network_arch.png and /dev/null differ
diff --git a/doc_cn/_images/paddleci.png b/doc_cn/_images/paddleci.png
new file mode 100644
index 0000000000000000000000000000000000000000..16087ce059aa3c07ce8c927d983eb86351915825
Binary files /dev/null and b/doc_cn/_images/paddleci.png differ
diff --git a/doc_cn/_images/parameters.png b/doc_cn/_images/parameters.png
deleted file mode 100644
index 2ec67480951e21f0400bce1c34b3108dcd65c18c..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/parameters.png and /dev/null differ
diff --git a/doc_cn/_images/plot.png b/doc_cn/_images/plot.png
deleted file mode 100644
index a31f99791c670e18bb8c62b7604ec8cb0284ffb4..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/plot.png and /dev/null differ
diff --git a/doc_cn/_images/pprof_1.png b/doc_cn/_images/pprof_1.png
new file mode 100644
index 0000000000000000000000000000000000000000..8e9edbf377672d0ef40f2fc7bd39e746923550cb
Binary files /dev/null and b/doc_cn/_images/pprof_1.png differ
diff --git a/doc_cn/_images/pprof_2.png b/doc_cn/_images/pprof_2.png
new file mode 100644
index 0000000000000000000000000000000000000000..172ba20399ba974d27f4c072425277b69b02520b
Binary files /dev/null and b/doc_cn/_images/pprof_2.png differ
diff --git a/doc_cn/_images/rec_regression_network.png b/doc_cn/_images/rec_regression_network.png
deleted file mode 100644
index 7d2b54d4fcf560cd5b667628f0012c3822efd9b2..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/rec_regression_network.png and /dev/null differ
diff --git a/doc_cn/_images/stacked_lstm.jpg b/doc_cn/_images/stacked_lstm.jpg
deleted file mode 100644
index 4239055050966e0095e188a8c81d860711bce29d..0000000000000000000000000000000000000000
Binary files a/doc_cn/_images/stacked_lstm.jpg and /dev/null differ
diff --git a/doc_cn/_sources/about/index_cn.md.txt b/doc_cn/_sources/about/index_cn.md.txt
deleted file mode 100644
index 3bf030004d4de8c6f3cb773c6e78c09f40878c5f..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/about/index_cn.md.txt
+++ /dev/null
@@ -1,11 +0,0 @@
-关于PaddlePaddle
-================
-
-PaddlePaddle是一个最早由百度科学家和工程师共同研发的并行分布式深度学习平台，兼备易用性、高效性、灵活性和可扩展性，目前已被百度内部多个产品线广泛使用。
-PaddlePaddle目前已经开放源码, 但是远未完善，我们希望能在这个基础上不断的改进、扩展和延伸。
-同时我们希望广大开发者积极提供反馈和贡献源代码，建立一个活跃的开源社区。
-
-致谢
---------
-
-在此，特别感谢PaddlePaddle的[所有贡献者](https://github.com/PaddlePaddle/Paddle/graphs/contributors)。
diff --git a/doc_cn/_sources/api/v1/index_cn.rst.txt b/doc_cn/_sources/api/v1/index_cn.rst.txt
index 3718cd73a2003b8ef6c406a9bd51dc68e76402dc..cf146dc088e3905a751ff55c26fd82ef0ba02c89 100644
--- a/doc_cn/_sources/api/v1/index_cn.rst.txt
+++ b/doc_cn/_sources/api/v1/index_cn.rst.txt
@@ -21,7 +21,7 @@ Model Config API
     trainer_config_helpers/optimizers.rst
     trainer_config_helpers/data_sources.rst
     trainer_config_helpers/layers.rst
-    trainer_config_helpers/activations.rst 
+    trainer_config_helpers/activations.rst
     trainer_config_helpers/poolings.rst
     trainer_config_helpers/networks.rst
     trainer_config_helpers/evaluators.rst
diff --git a/doc_cn/_sources/api/v1/trainer_config_helpers/activations.rst.txt b/doc_cn/_sources/api/v1/trainer_config_helpers/activations.rst.txt
deleted file mode 100644
index 269e6491e7ebe3899c3fb24fca756a393043473b..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/api/v1/trainer_config_helpers/activations.rst.txt
+++ /dev/null
@@ -1,108 +0,0 @@
-===========
-Activations
-===========
-
-BaseActivation
-==============
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: BaseActivation
-    :noindex:
-    
-AbsActivation
-===============
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: AbsActivation
-    :noindex:
-    
-ExpActivation
-===============
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: ExpActivation
-    :noindex:
-    
-IdentityActivation
-==================
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: IdentityActivation
-    :noindex:
-    
-LinearActivation
-==================
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: LinearActivation
-    :noindex:
-
-LogActivation
-==================
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: LogActivation
-    :noindex:
-    
-SquareActivation
-================
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: SquareActivation
-    :noindex:
-    
-SigmoidActivation
-=================
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: SigmoidActivation
-    :noindex:
-    
-SoftmaxActivation
-=================
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: SoftmaxActivation
-    :noindex:
-    
-SequenceSoftmaxActivation
-=========================
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: SequenceSoftmaxActivation
-    :noindex:
-    
-ReluActivation
-==============
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: ReluActivation
-    :noindex:
-    
-BReluActivation
-===============
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: BReluActivation
-    :noindex:
-    
-SoftReluActivation
-==================
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: SoftReluActivation
-    :noindex:
-    
-TanhActivation
-==============
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: TanhActivation
-    :noindex:
-    
-STanhActivation
-===============
-
-..  automodule:: paddle.trainer_config_helpers.activations
-    :members: STanhActivation
-    :noindex:
diff --git a/doc_cn/_sources/api/v1/trainer_config_helpers/attrs.rst.txt b/doc_cn/_sources/api/v1/trainer_config_helpers/attrs.rst.txt
deleted file mode 100644
index ac63127bf7d9db6351365ab7b58f43db12347a8e..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/api/v1/trainer_config_helpers/attrs.rst.txt
+++ /dev/null
@@ -1,5 +0,0 @@
-Parameter Attributes
-=======================
-
-..  automodule:: paddle.trainer_config_helpers.attrs
-    :members:
diff --git a/doc_cn/_sources/api/v1/trainer_config_helpers/data_sources.rst.txt b/doc_cn/_sources/api/v1/trainer_config_helpers/data_sources.rst.txt
deleted file mode 100644
index b9dd4dda01ae59d1260356aff50ddf298d02c87f..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/api/v1/trainer_config_helpers/data_sources.rst.txt
+++ /dev/null
@@ -1,7 +0,0 @@
-..  _api_trainer_config_helpers_data_sources:
-
-DataSources
-===========
-
-..  automodule:: paddle.trainer_config_helpers.data_sources
-    :members:
diff --git a/doc_cn/_sources/api/v1/trainer_config_helpers/evaluators.rst.txt b/doc_cn/_sources/api/v1/trainer_config_helpers/evaluators.rst.txt
deleted file mode 100644
index 11dc735164284d6ed1d661fab1e7690d263b3a7c..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/api/v1/trainer_config_helpers/evaluators.rst.txt
+++ /dev/null
@@ -1,108 +0,0 @@
-..  _api_trainer_config_helpers_evaluators:
-
-==========
-Evaluators
-==========
-
-Base
-====
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members: evaluator_base
-    :noindex:
-
-Classification 
-==============
-
-classification_error_evaluator
-------------------------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members: classification_error_evaluator
-    :noindex:
-
-auc_evaluator
--------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members: auc_evaluator
-    :noindex:
-
-ctc_error_evaluator
--------------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members: ctc_error_evaluator
-    :noindex:
-
-chunk_evaluator
----------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members: chunk_evaluator
-    :noindex:
-
-precision_recall_evaluator
---------------------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members:  precision_recall_evaluator
-    :noindex:
-
-Rank
-====
-
-pnpair_evaluator
-----------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members:  pnpair_evaluator
-    :noindex:
-
-Utils
-=====
-
-sum_evaluator
--------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members: sum_evaluator
-    :noindex:
-
-column_sum_evaluator
---------------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members: column_sum_evaluator
-    :noindex:
-
-Print
-=====
-
-classification_error_printer_evaluator
---------------------------------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members:  classification_error_printer_evaluator
-    :noindex:
-
-gradient_printer_evaluator
---------------------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members:  gradient_printer_evaluator
-    :noindex:
-
-maxid_printer_evaluator
------------------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members:  maxid_printer_evaluator
-    :noindex:
-
-maxframe_printer_evaluator
----------------------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members:  maxframe_printer_evaluator
-    :noindex:
-
-seqtext_printer_evaluator
--------------------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members:  seqtext_printer_evaluator
-    :noindex:
-
-value_printer_evaluator
------------------------
-..  automodule:: paddle.trainer_config_helpers.evaluators
-    :members:  value_printer_evaluator
-    :noindex:
-
diff --git a/doc_cn/_sources/api/v1/trainer_config_helpers/layers.rst.txt b/doc_cn/_sources/api/v1/trainer_config_helpers/layers.rst.txt
deleted file mode 100644
index 24389c2d8574dfda4bec9298776aa6b1aee51535..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/api/v1/trainer_config_helpers/layers.rst.txt
+++ /dev/null
@@ -1,508 +0,0 @@
-..  _api_trainer_config_helpers_layers:
-
-======
-Layers
-======
-
-Base
-======
-
-LayerType
----------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: LayerType
-    :noindex:
-
-LayerOutput
------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: LayerOutput
-    :noindex:
-
-Data layer
-===========
-
-..  _api_trainer_config_helpers_layers_data_layer:
-
-data_layer
-----------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: data_layer
-    :noindex:
-
-Fully Connected Layers
-======================
-
-..  _api_trainer_config_helpers_layers_fc_layer:
-
-fc_layer
---------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: fc_layer
-    :noindex:
-
-selective_fc_layer
-------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: selective_fc_layer
-    :noindex:
-
-Conv Layers
-===========
-
-conv_operator
--------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: conv_operator
-    :noindex:
-
-conv_projection
----------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: conv_projection
-    :noindex:
-
-conv_shift_layer
-------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: conv_shift_layer
-    :noindex:
-
-img_conv_layer
---------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: img_conv_layer
-    :noindex:
-
-..  _api_trainer_config_helpers_layers_context_projection:
-
-context_projection 
-------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: context_projection
-    :noindex:
-
-Image Pooling Layer
-===================
-
-img_pool_layer
---------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: img_pool_layer
-    :noindex:   
-
-spp_layer
---------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: spp_layer
-    :noindex:
-
-maxout_layer
-------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: maxout_layer
-    :noindex:
-
-Norm Layer
-==========
-
-img_cmrnorm_layer
------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: img_cmrnorm_layer
-    :noindex:
-
-batch_norm_layer
----------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: batch_norm_layer
-    :noindex:
-
-sum_to_one_norm_layer
----------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: sum_to_one_norm_layer
-    :noindex:
-    
-Recurrent Layers
-================
-
-recurrent_layer
------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: recurrent_layer
-    :noindex:
-
-lstmemory
----------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: lstmemory
-    :noindex:
-
-grumemory
----------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: grumemory
-    :noindex:
-
-Recurrent Layer Group
-=====================
-
-memory
-------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: memory
-    :noindex:
-
-recurrent_group
----------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: recurrent_group
-    :noindex:
-    
-lstm_step_layer
----------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: lstm_step_layer
-    :noindex:
-
-gru_step_layer
----------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: gru_step_layer
-    :noindex:
-
-beam_search
-------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: beam_search
-    :noindex:
-    
-get_output_layer
------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: get_output_layer
-    :noindex:
-    
-Mixed Layer
-===========
-
-..  _api_trainer_config_helpers_layers_mixed_layer:
-
-mixed_layer
------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: mixed_layer
-    :noindex:
-
-..  _api_trainer_config_helpers_layers_embedding_layer:
-
-embedding_layer
----------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: embedding_layer
-    :noindex:
-
-scaling_projection
-------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: scaling_projection
-    :noindex:
-
-dotmul_projection
------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: dotmul_projection
-    :noindex:
-
-dotmul_operator
----------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: dotmul_operator
-    :noindex:
-
-full_matrix_projection
-----------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: full_matrix_projection
-    :noindex:
-
-identity_projection
--------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: identity_projection
-    :noindex:
-
-
-table_projection
-----------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: table_projection
-    :noindex:
-
-trans_full_matrix_projection
-----------------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: trans_full_matrix_projection
-    :noindex:
-    
-Aggregate Layers
-================
-
-..  _api_trainer_config_helpers_layers_pooling_layer:
-
-pooling_layer
--------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: pooling_layer
-    :noindex:
-
-..  _api_trainer_config_helpers_layers_last_seq:
-
-last_seq
---------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: last_seq
-    :noindex:
-
-..  _api_trainer_config_helpers_layers_first_seq:
-
-first_seq
----------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: first_seq
-    :noindex:
-
-concat_layer
-------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: concat_layer
-    :noindex:
-
-seq_concat_layer
-----------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: seq_concat_layer
-    :noindex:
-
-Reshaping Layers
-================
-
-block_expand_layer
-------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: block_expand_layer
-    :noindex:
-
-..  _api_trainer_config_helpers_layers_expand_layer:
-
-expand_layer
-------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: expand_layer
-    :noindex:
-
-repeat_layer
-------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: repeat_layer
-    :noindex:
-
-rotate_layer
-------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: rotate_layer
-    :noindex:
-
-seq_reshape_layer
------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: seq_reshape_layer
-    :noindex:
-
-Math Layers
-===========
-
-addto_layer
------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: addto_layer
-    :noindex:
-
-linear_comb_layer
------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: linear_comb_layer
-    :noindex:
-
-interpolation_layer
--------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: interpolation_layer
-    :noindex:
-
-bilinear_interp_layer
-----------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: bilinear_interp_layer
-    :noindex:
-
-power_layer
------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: power_layer
-    :noindex:
-
-scaling_layer
--------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: scaling_layer
-    :noindex:
-
-slope_intercept_layer
-----------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: slope_intercept_layer
-    :noindex:
-
-tensor_layer
-------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: tensor_layer
-    :noindex:
-
-..  _api_trainer_config_helpers_layers_cos_sim:
-
-cos_sim
--------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: cos_sim
-    :noindex:
-
-trans_layer
-------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: trans_layer
-    :noindex:
-
-Sampling Layers
-===============
-
-maxid_layer
------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: maxid_layer
-    :noindex:
-
-sampling_id_layer
------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: sampling_id_layer
-    :noindex:
-
-Slicing and Joining Layers
-==========================
-
-pad_layer
------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: pad_layer
-    :noindex:
-
-..  _api_trainer_config_helpers_layers_cost_layers:
-
-Cost Layers
-===========
-
-cross_entropy
--------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: cross_entropy
-    :noindex:
-
-cross_entropy_with_selfnorm
----------------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: cross_entropy_with_selfnorm
-    :noindex:
-
-multi_binary_label_cross_entropy
---------------------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: multi_binary_label_cross_entropy
-    :noindex:
-
-mse_cost
----------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: mse_cost
-    :noindex:
-
-huber_cost
-----------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: huber_cost
-    :noindex:
-
-lambda_cost
------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: lambda_cost
-    :noindex:
-
-rank_cost
----------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: rank_cost
-    :noindex:
-
-sum_cost
----------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: sum_cost
-    :noindex:
-
-crf_layer
------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: crf_layer
-    :noindex:
-
-crf_decoding_layer
--------------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: crf_decoding_layer
-    :noindex:
-
-ctc_layer
------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: ctc_layer
-    :noindex:
-
-warp_ctc_layer
---------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: warp_ctc_layer
-    :noindex:
-
-nce_layer
------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: nce_layer
-    :noindex:
-
-hsigmoid
----------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: hsigmoid
-    :noindex:
-
-Check Layer 
-============
-
-eos_layer
-------------
-..  automodule:: paddle.trainer_config_helpers.layers
-    :members: eos_layer
-    :noindex:
diff --git a/doc_cn/_sources/api/v1/trainer_config_helpers/networks.rst.txt b/doc_cn/_sources/api/v1/trainer_config_helpers/networks.rst.txt
deleted file mode 100644
index edb53acbf0c31532aa34bda044066fed72eaa426..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/api/v1/trainer_config_helpers/networks.rst.txt
+++ /dev/null
@@ -1,123 +0,0 @@
-========
-Networks
-========
-
-The networks module contains pieces of neural network that combine multiple layers.
-
-NLP
-===
-
-sequence_conv_pool
-------------------
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: sequence_conv_pool
-    :noindex:
-
-..  _api_trainer_config_helpers_network_text_conv_pool:
-
-text_conv_pool
---------------
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: text_conv_pool
-    :noindex:
-
-Images
-======
-
-img_conv_bn_pool
-----------------
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: img_conv_bn_pool
-    :noindex:
-
-img_conv_group
---------------
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: img_conv_group
-    :noindex:
-
-..  _api_trainer_config_helpers_network_simple_img_conv_pool:
-
-simple_img_conv_pool
---------------------
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: simple_img_conv_pool
-    :noindex:
-
-vgg_16_network
----------------
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: vgg_16_network
-    :noindex:
-
-Recurrent
-=========
-
-LSTM
-----
-
-lstmemory_unit
-``````````````
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: lstmemory_unit
-    :noindex:
-
-lstmemory_group
-```````````````
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: lstmemory_group
-    :noindex:
-
-simple_lstm
-```````````
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: simple_lstm
-    :noindex:
-
-bidirectional_lstm
-``````````````````
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: bidirectional_lstm
-    :noindex:
-
-GRU
----
-
-gru_unit
-````````
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: gru_unit
-    :noindex:
-
-gru_group
-`````````
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: gru_group
-    :noindex:
-
-simple_gru
-``````````
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: simple_gru
-    :noindex:
-
-simple_attention
-----------------
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: simple_attention
-    :noindex:
-
-Miscs
-=====
-
-dropout_layer
---------------
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: dropout_layer
-    :noindex:
-
-outputs
--------
-..  automodule:: paddle.trainer_config_helpers.networks
-    :members: outputs
-    :noindex:
diff --git a/doc_cn/_sources/api/v1/trainer_config_helpers/optimizers.rst.txt b/doc_cn/_sources/api/v1/trainer_config_helpers/optimizers.rst.txt
deleted file mode 100644
index d2f4958c92b8e3b7426945f1af07112ab4071136..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/api/v1/trainer_config_helpers/optimizers.rst.txt
+++ /dev/null
@@ -1,61 +0,0 @@
-..  _api_trainer_config_helpers_optimizers:
-
-==========
-Optimizers
-==========
-
-BaseSGDOptimizer
-================
-..  automodule:: paddle.trainer_config_helpers.optimizers
-    :members: BaseSGDOptimizer
-    :noindex:
-
-MomentumOptimizer
-=================
-..  automodule:: paddle.trainer_config_helpers.optimizers
-    :members: MomentumOptimizer
-    :noindex:
-
-AdamOptimizer
-=============
-..  automodule:: paddle.trainer_config_helpers.optimizers
-    :members: AdamOptimizer
-    :noindex:
-
-AdamaxOptimizer
-================
-..  automodule:: paddle.trainer_config_helpers.optimizers
-    :members: AdamaxOptimizer
-    :noindex:
-
-AdaGradOptimizer
-================
-..  automodule:: paddle.trainer_config_helpers.optimizers
-    :members: AdaGradOptimizer
-    :noindex:
-
-DecayedAdaGradOptimizer
-=======================
-..  automodule:: paddle.trainer_config_helpers.optimizers
-    :members: DecayedAdaGradOptimizer
-    :noindex:
-
-AdaDeltaOptimizer
-=================
-..  automodule:: paddle.trainer_config_helpers.optimizers
-    :members: AdaDeltaOptimizer
-    :noindex:
-
-RMSPropOptimizer
-================
-..  automodule:: paddle.trainer_config_helpers.optimizers
-    :members: RMSPropOptimizer
-    :noindex:
-
-..  _api_trainer_config_helpers_optimizers_settings:
-
-settings
-========
-..  automodule:: paddle.trainer_config_helpers.optimizers
-    :members: settings
-    :noindex:
diff --git a/doc_cn/_sources/api/v1/trainer_config_helpers/poolings.rst.txt b/doc_cn/_sources/api/v1/trainer_config_helpers/poolings.rst.txt
deleted file mode 100644
index 66566809d26f59263597b5286c5b27e0bbc9415a..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/api/v1/trainer_config_helpers/poolings.rst.txt
+++ /dev/null
@@ -1,33 +0,0 @@
-========
-Poolings
-========
-
-BasePoolingType
-===============
-..  automodule:: paddle.trainer_config_helpers.poolings
-    :members: BasePoolingType
-    :noindex:
-
-AvgPooling
-==========
-..  automodule:: paddle.trainer_config_helpers.poolings
-    :members: AvgPooling
-    :noindex:
-
-MaxPooling
-==========
-..  automodule:: paddle.trainer_config_helpers.poolings
-    :members: MaxPooling
-    :noindex:
-
-SumPooling
-==========
-..  automodule:: paddle.trainer_config_helpers.poolings
-    :members: SumPooling
-    :noindex:
-
-SquareRootNPooling
-==================
-..  automodule:: paddle.trainer_config_helpers.poolings
-    :members: SquareRootNPooling
-    :noindex:
diff --git a/doc_cn/_sources/api/v2/config/activation.rst.txt b/doc_cn/_sources/api/v2/config/activation.rst.txt
index eca3ce03bcdc599edca802d8dfca48d4f28275a2..5317e66b64bbd85c61f19700a9d2c1d239dee573 100644
--- a/doc_cn/_sources/api/v2/config/activation.rst.txt
+++ b/doc_cn/_sources/api/v2/config/activation.rst.txt
@@ -99,3 +99,10 @@ STanh
 ..  automodule:: paddle.v2.activation
     :members: STanh
     :noindex:
+    
+SoftSign
+========
+
+..  automodule:: paddle.v2.activation
+    :members: SoftSign
+    :noindex:
diff --git a/doc_cn/_sources/api/v2/config/evaluators.rst.txt b/doc_cn/_sources/api/v2/config/evaluators.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..9ac972fb193a2fb525edc507f7ba1303d2c8eabe
--- /dev/null
+++ b/doc_cn/_sources/api/v2/config/evaluators.rst.txt
@@ -0,0 +1,110 @@
+..  _api_v2:
+
+==========
+Evaluators
+==========
+
+Classification 
+==============
+
+classification_error
+--------------------
+..  automodule:: paddle.v2.evaluator
+    :members: classification_error
+    :noindex:
+
+auc
+---
+..  automodule:: paddle.v2.evaluator
+    :members: auc
+    :noindex:
+
+ctc_error
+---------
+..  automodule:: paddle.v2.evaluator
+    :members: ctc_error
+    :noindex:
+
+chunk
+-----
+..  automodule:: paddle.v2.evaluator
+    :members: chunk
+    :noindex:
+
+precision_recall
+----------------
+..  automodule:: paddle.v2.evaluator
+    :members:  precision_recall
+    :noindex:
+
+Rank
+====
+
+pnpair
+------
+..  automodule:: paddle.v2.evaluator
+    :members:  pnpair
+    :noindex:
+
+Utils
+=====
+
+sum
+---
+..  automodule:: paddle.v2.evaluator
+    :members: sum
+    :noindex:
+
+column_sum
+----------
+..  automodule:: paddle.v2.evaluator
+    :members: column_sum
+    :noindex:
+
+Print
+=====
+
+classification_error_printer
+----------------------------
+..  automodule:: paddle.v2.evaluator
+    :members:  classification_error_printer
+    :noindex:
+
+gradient_printer
+----------------
+..  automodule:: paddle.v2.evaluator
+    :members:  gradient_printer
+    :noindex:
+
+maxid_printer
+-------------
+..  automodule:: paddle.v2.evaluator
+    :members:  maxid_printer
+    :noindex:
+
+maxframe_printer
+----------------
+..  automodule:: paddle.v2.evaluator
+    :members:  maxframe_printer
+    :noindex:
+
+seqtext_printer
+---------------
+..  automodule:: paddle.v2.evaluator
+    :members:  seqtext_printer
+    :noindex:
+
+value_printer
+-------------
+..  automodule:: paddle.v2.evaluator
+    :members:  value_printer
+    :noindex:
+
+Detection
+=====
+
+detection_map
+-------------
+..  automodule:: paddle.v2.evaluator
+    :members:  detection_map
+    :noindex:
diff --git a/doc_cn/_sources/api/v2/config/layer.rst.txt b/doc_cn/_sources/api/v2/config/layer.rst.txt
index 2a02baf17ba0d1119a8d222024616ef8ae33f8d5..c3f9c18d0663a7a24880b441981875c1e4f015aa 100644
--- a/doc_cn/_sources/api/v2/config/layer.rst.txt
+++ b/doc_cn/_sources/api/v2/config/layer.rst.txt
@@ -54,18 +54,23 @@ img_conv
 
 ..  _api_v2.layer_context_projection:
 
-context_projection 
+context_projection
 ------------------
 ..  autoclass:: paddle.v2.layer.context_projection
     :noindex:
 
+row_conv
+--------
+..  autoclass:: paddle.v2.layer.row_conv
+    :noindex:
+
 Image Pooling Layer
 ===================
 
 img_pool
 --------
 ..  autoclass:: paddle.v2.layer.img_pool
-    :noindex:   
+    :noindex:
 
 spp
 ---
@@ -77,6 +82,11 @@ maxout
 ..  autoclass:: paddle.v2.layer.maxout
     :noindex:
 
+roi_pool
+--------
+..  autoclass:: paddle.v2.layer.roi_pool
+    :noindex:
+
 Norm Layer
 ==========
 
@@ -94,12 +104,17 @@ sum_to_one_norm
 ---------------
 ..  autoclass:: paddle.v2.layer.sum_to_one_norm
     :noindex:
-    
+
 cross_channel_norm
 ------------------
 ..  autoclass:: paddle.v2.layer.cross_channel_norm
     :noindex:
-    
+
+row_l2_norm
+-----------
+..  autoclass:: paddle.v2.layer.row_l2_norm
+    :noindex:
+
 Recurrent Layers
 ================
 
@@ -130,7 +145,7 @@ recurrent_group
 ---------------
 ..  autoclass:: paddle.v2.layer.recurrent_group
     :noindex:
-    
+
 lstm_step
 ---------
 ..  autoclass:: paddle.v2.layer.lstm_step
@@ -145,12 +160,12 @@ beam_search
 ------------
 ..  autoclass:: paddle.v2.layer.beam_search
     :noindex:
-    
+
 get_output
 ----------
 ..  autoclass:: paddle.v2.layer.get_output
     :noindex:
-    
+
 Mixed Layer
 ===========
 
@@ -193,6 +208,10 @@ identity_projection
 ..  autoclass:: paddle.v2.layer.identity_projection
     :noindex:
 
+slice_projection
+-------------------
+..  autoclass:: paddle.v2.layer.slice_projection
+    :noindex:
 
 table_projection
 ----------------
@@ -203,10 +222,15 @@ trans_full_matrix_projection
 ----------------------------
 ..  autoclass:: paddle.v2.layer.trans_full_matrix_projection
     :noindex:
-    
+
 Aggregate Layers
 ================
 
+AggregateLevel
+--------------
+..  autoclass:: paddle.v2.layer.AggregateLevel
+    :noindex:
+
 ..  _api_v2.layer_pooling:
 
 pooling
@@ -238,6 +262,21 @@ seq_concat
 ..  autoclass:: paddle.v2.layer.seq_concat
     :noindex:
 
+seq_slice
+---------
+..  autoclass:: paddle.v2.layer.seq_slice
+    :noindex:
+
+kmax_sequence_score
+-------------------
+..  autoclass:: paddle.v2.layer.kmax_sequence_score
+    :noindex:
+
+sub_nested_seq
+--------------
+..  autoclass:: paddle.v2.layer.sub_nested_seq
+    :noindex:
+
 Reshaping Layers
 ================
 
@@ -248,6 +287,11 @@ block_expand
 
 ..  _api_v2.layer_expand:
 
+ExpandLevel
+-----------
+..  autoclass:: paddle.v2.layer.ExpandLevel
+    :noindex:
+
 expand
 ------
 ..  autoclass:: paddle.v2.layer.expand
@@ -291,6 +335,16 @@ bilinear_interp
 ..  autoclass:: paddle.v2.layer.bilinear_interp
     :noindex:
 
+dot_prod
+---------
+.. autoclass:: paddle.v2.layer.dot_prod
+    :noindex:
+
+out_prod
+--------
+.. autoclass:: paddle.v2.layer.out_prod
+    :noindex:
+
 power
 -----
 ..  autoclass:: paddle.v2.layer.power
@@ -301,6 +355,16 @@ scaling
 ..  autoclass:: paddle.v2.layer.scaling
     :noindex:
 
+clip
+----
+..  autoclass:: paddle.v2.layer.clip
+    :noindex:
+
+resize
+------
+..  autoclass:: paddle.v2.layer.resize
+    :noindex:
+
 slope_intercept
 ---------------
 ..  autoclass:: paddle.v2.layer.slope_intercept
@@ -318,11 +382,21 @@ cos_sim
 ..  autoclass:: paddle.v2.layer.cos_sim
     :noindex:
 
+l2_distance
+-----------
+..  autoclass:: paddle.v2.layer.l2_distance
+    :noindex:
+
 trans
 -----
 ..  autoclass:: paddle.v2.layer.trans
     :noindex:
 
+scale_shift
+-----------
+..  autoclass:: paddle.v2.layer.scale_shift
+    :noindex:
+
 Sampling Layers
 ===============
 
@@ -336,6 +410,19 @@ sampling_id
 ..  autoclass:: paddle.v2.layer.sampling_id
     :noindex:
 
+multiplex
+---------
+..  autoclass:: paddle.v2.layer.multiplex
+    :noindex:
+
+Factorization Machine Layer
+============================
+
+factorization_machine
+---------------------
+..  autoclass:: paddle.v2.layer.factorization_machine
+    :noindex:
+
 Slicing and Joining Layers
 ==========================
 
@@ -364,9 +451,14 @@ multi_binary_label_cross_entropy_cost
 ..  autoclass:: paddle.v2.layer.multi_binary_label_cross_entropy_cost
     :noindex:
 
-huber_cost
-----------
-..  autoclass:: paddle.v2.layer.huber_cost
+huber_regression_cost
+-------------------------
+..  autoclass:: paddle.v2.layer.huber_regression_cost
+    :noindex:
+
+huber_classification_cost
+-------------------------
+..  autoclass:: paddle.v2.layer.huber_classification_cost
     :noindex:
 
 lambda_cost
@@ -374,9 +466,9 @@ lambda_cost
 ..  autoclass:: paddle.v2.layer.lambda_cost
     :noindex:
 
-mse_cost
+square_error_cost
 --------
-..  autoclass:: paddle.v2.layer.mse_cost
+..  autoclass:: paddle.v2.layer.square_error_cost
     :noindex:
 
 rank_cost
@@ -419,10 +511,49 @@ hsigmoid
 ..  autoclass:: paddle.v2.layer.hsigmoid
     :noindex:
 
-Check Layer 
+smooth_l1_cost
+--------------
+..  autoclass:: paddle.v2.layer.smooth_l1_cost
+    :noindex:
+
+multibox_loss
+--------------
+..  autoclass:: paddle.v2.layer.multibox_loss
+    :noindex:
+
+Check Layer
 ============
 
 eos
 ---
 ..  autoclass:: paddle.v2.layer.eos
     :noindex:
+
+Miscs
+=====
+
+dropout
+--------------
+..  autoclass:: paddle.v2.layer.dropout
+    :noindex:
+
+Activation with learnable parameter
+===================================
+
+prelu
+--------
+..  autoclass:: paddle.v2.layer.prelu
+    :noindex:
+
+gated_unit
+-----------
+..  autoclass:: paddle.v2.layer.gated_unit
+    :noindex:
+
+Detection output Layer
+======================
+
+detection_output
+----------------
+..  autoclass:: paddle.v2.layer.detection_output
+    :noindex:
diff --git a/doc_cn/_sources/api/v2/config/networks.rst.txt b/doc_cn/_sources/api/v2/config/networks.rst.txt
index 6f209bc95bec7279051118bb857a96515e0371a9..048379cf01f4aec5e73e2fe3ddfa728f3c17a5d1 100644
--- a/doc_cn/_sources/api/v2/config/networks.rst.txt
+++ b/doc_cn/_sources/api/v2/config/networks.rst.txt
@@ -44,6 +44,12 @@ simple_img_conv_pool
     :members: simple_img_conv_pool
     :noindex:
 
+small_vgg
+---------
+..  automodule:: paddle.v2.networks
+    :members: small_vgg
+    :noindex:
+
 vgg_16_network
 ---------------
 ..  automodule:: paddle.v2.networks
@@ -101,17 +107,26 @@ simple_gru
     :members: simple_gru
     :noindex:
 
+simple_gru2
+```````````
+..  automodule:: paddle.v2.networks
+    :members: simple_gru2
+    :noindex:
+
+bidirectional_gru
+``````````````````
+..  automodule:: paddle.v2.networks
+    :members: bidirectional_gru
+    :noindex:
+
 simple_attention
 ----------------
 ..  automodule:: paddle.v2.networks
     :members: simple_attention
     :noindex:
 
-Miscs
-=====
-
-dropout_layer
---------------
+dot_product_attention
+---------------------
 ..  automodule:: paddle.v2.networks
-    :members: dropout_layer
+    :members: dot_product_attention
     :noindex:
diff --git a/doc_cn/_sources/api/v2/data.rst.txt b/doc_cn/_sources/api/v2/data.rst.txt
index fef87c4fbdb452771ecdb361c6eeae5b32bcee14..b56c7332cc284649c7e04328e51a7faa78593a39 100644
--- a/doc_cn/_sources/api/v2/data.rst.txt
+++ b/doc_cn/_sources/api/v2/data.rst.txt
@@ -2,112 +2,9 @@
 Data Reader Interface and DataSets
 ==================================
 
+..  toctree::
+    :maxdepth: 1
 
-DataTypes
-=========
-
-..  automodule:: paddle.v2.data_type
-    :members:
-    :noindex:
-
-DataFeeder
-==========
-
-..  automodule:: paddle.v2.data_feeder
-    :members:
-    :noindex:
-
-Reader
-======
-
-..  automodule:: paddle.v2.reader
-    :members:
-    :noindex:
-
-..  automodule:: paddle.v2.reader.creator
-    :members:
-    :noindex:
-
-minibatch
-=========
-
-..  automodule:: paddle.v2.minibatch
-    :members:
-    :noindex:
-
-Dataset
-=======
-
-..  automodule:: paddle.v2.dataset
-    :members:
-    :noindex:
-
-mnist
-+++++
-
-..  automodule:: paddle.v2.dataset.mnist
-    :members:
-    :noindex:
-
-cifar
-+++++
-
-..  automodule:: paddle.v2.dataset.cifar
-    :members:
-    :noindex:
-
-conll05
-+++++++
-
-..  automodule:: paddle.v2.dataset.conll05
-    :members: get_dict,get_embedding,test
-    :noindex:
-
-imdb
-++++
-
-..  automodule:: paddle.v2.dataset.imdb
-    :members:
-    :noindex:
-
-imikolov
-++++++++
-
-..  automodule:: paddle.v2.dataset.imikolov
-    :members:
-    :noindex:
-
-movielens
-+++++++++
-
-..  automodule:: paddle.v2.dataset.movielens
-    :members:
-    :noindex:
-
-..  autoclass:: paddle.v2.dataset.movielens.MovieInfo
-    :noindex:
-    
-..  autoclass:: paddle.v2.dataset.movielens.UserInfo
-    :noindex:
-
-sentiment
-+++++++++
-
-..  automodule:: paddle.v2.dataset.sentiment
-    :members:
-    :noindex:
-
-uci_housing
-+++++++++++
-
-..  automodule:: paddle.v2.dataset.uci_housing
-    :members:
-    :noindex:
-
-wmt14
-+++++
-
-..  automodule:: paddle.v2.dataset.wmt14
-    :members:
-    :noindex:
-
+    data/data_reader.rst
+    data/image.rst
+    data/dataset.rst
diff --git a/doc_cn/_sources/api/v2/data/data_reader.rst.txt b/doc_cn/_sources/api/v2/data/data_reader.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..2ccfec9c284877a7576e9751526b169a4ac78d8e
--- /dev/null
+++ b/doc_cn/_sources/api/v2/data/data_reader.rst.txt
@@ -0,0 +1,36 @@
+=====================
+Data Reader Interface
+=====================
+
+
+DataTypes
+=========
+
+..  automodule:: paddle.v2.data_type
+    :members:
+    :noindex:
+
+DataFeeder
+==========
+
+..  automodule:: paddle.v2.data_feeder
+    :members:
+    :noindex:
+
+Reader
+======
+
+..  automodule:: paddle.v2.reader
+    :members:
+    :noindex:
+
+..  automodule:: paddle.v2.reader.creator
+    :members:
+    :noindex:
+
+minibatch
+=========
+
+..  automodule:: paddle.v2.minibatch
+    :members:
+    :noindex:
diff --git a/doc_cn/_sources/api/v2/data/dataset.rst.txt b/doc_cn/_sources/api/v2/data/dataset.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..6a8ecc5bb1d855e0ded3719943ab3adb810de365
--- /dev/null
+++ b/doc_cn/_sources/api/v2/data/dataset.rst.txt
@@ -0,0 +1,75 @@
+Dataset
+=======
+
+..  automodule:: paddle.v2.dataset
+    :members:
+    :noindex:
+
+mnist
++++++
+
+..  automodule:: paddle.v2.dataset.mnist
+    :members:
+    :noindex:
+
+cifar
++++++
+
+..  automodule:: paddle.v2.dataset.cifar
+    :members:
+    :noindex:
+
+conll05
++++++++
+
+..  automodule:: paddle.v2.dataset.conll05
+    :members: get_dict,get_embedding,test
+    :noindex:
+
+imdb
+++++
+
+..  automodule:: paddle.v2.dataset.imdb
+    :members:
+    :noindex:
+
+imikolov
+++++++++
+
+..  automodule:: paddle.v2.dataset.imikolov
+    :members:
+    :noindex:
+
+movielens
++++++++++
+
+..  automodule:: paddle.v2.dataset.movielens
+    :members:
+    :noindex:
+
+..  autoclass:: paddle.v2.dataset.movielens.MovieInfo
+    :noindex:
+    
+..  autoclass:: paddle.v2.dataset.movielens.UserInfo
+    :noindex:
+
+sentiment
++++++++++
+
+..  automodule:: paddle.v2.dataset.sentiment
+    :members:
+    :noindex:
+
+uci_housing
++++++++++++
+
+..  automodule:: paddle.v2.dataset.uci_housing
+    :members:
+    :noindex:
+
+wmt14
++++++
+
+..  automodule:: paddle.v2.dataset.wmt14
+    :members:
+    :noindex:
diff --git a/doc_cn/_sources/api/v2/data/image.rst.txt b/doc_cn/_sources/api/v2/data/image.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..97651ffa6be56cf3ecaca2caca38a353fa5c1f49
--- /dev/null
+++ b/doc_cn/_sources/api/v2/data/image.rst.txt
@@ -0,0 +1,5 @@
+Image Interface
+===============
+
+..  automodule:: paddle.v2.image
+    :members:
diff --git a/doc_cn/_sources/api/v2/fluid.rst.txt b/doc_cn/_sources/api/v2/fluid.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..43fc19dc492bbc119f2356034b81c65e443db2fa
--- /dev/null
+++ b/doc_cn/_sources/api/v2/fluid.rst.txt
@@ -0,0 +1,18 @@
+======================
+Fluid
+======================
+
+..  toctree::
+    :maxdepth: 1
+
+    fluid/layers.rst
+    fluid/data_feeder.rst
+    fluid/executor.rst
+    fluid/initializer.rst
+    fluid/evaluator.rst
+    fluid/nets.rst
+    fluid/optimizer.rst
+    fluid/param_attr.rst
+    fluid/profiler.rst
+    fluid/regularizer.rst
+
diff --git a/doc_cn/_sources/api/v2/fluid/data_feeder.rst.txt b/doc_cn/_sources/api/v2/fluid/data_feeder.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..0fa78f7dfb04c13be7eb83b7fd35cb03f2f4a7fa
--- /dev/null
+++ b/doc_cn/_sources/api/v2/fluid/data_feeder.rst.txt
@@ -0,0 +1,9 @@
+===========
+DataFeeder
+===========
+
+DataFeeder
+-----------
+..  automodule:: paddle.v2.fluid.data_feeder
+    :members: DataFeeder
+    :noindex:
diff --git a/doc_cn/_sources/api/v2/fluid/evaluator.rst.txt b/doc_cn/_sources/api/v2/fluid/evaluator.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..a23f3301d0331e0ea3733f06444515eb4680cd31
--- /dev/null
+++ b/doc_cn/_sources/api/v2/fluid/evaluator.rst.txt
@@ -0,0 +1,9 @@
+===========
+Evaluator
+===========
+
+Evaluator
+-----------
+..  automodule:: paddle.v2.fluid.evaluator
+    :members: Evaluator
+    :noindex:
diff --git a/doc_cn/_sources/api/v2/fluid/executor.rst.txt b/doc_cn/_sources/api/v2/fluid/executor.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..3a283538c120cfa1ef646c390bb71c6251c23675
--- /dev/null
+++ b/doc_cn/_sources/api/v2/fluid/executor.rst.txt
@@ -0,0 +1,9 @@
+===========
+Executor
+===========
+
+Executor
+-----------
+..  automodule:: paddle.v2.fluid.executor
+    :members: Executor
+    :noindex:
diff --git a/doc_cn/_sources/api/v2/fluid/initializer.rst.txt b/doc_cn/_sources/api/v2/fluid/initializer.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..8f587837e9873370722062404f511654a9460587
--- /dev/null
+++ b/doc_cn/_sources/api/v2/fluid/initializer.rst.txt
@@ -0,0 +1,50 @@
+===========
+Initializer
+===========
+
+
+
+Initializer
+-----------
+..  automodule:: paddle.v2.fluid.initializer
+    :members: Initializer
+    :noindex:
+
+
+
+ConstantInitializer
+-------------------
+..  automodule:: paddle.v2.fluid.initializer
+    :members: ConstantInitializer
+    :noindex:
+
+
+
+UniformInitializer
+------------------
+..  automodule:: paddle.v2.fluid.initializer
+    :members: UniformInitializer
+    :noindex:
+
+
+
+NormalInitializer
+-----------------
+..  automodule:: paddle.v2.fluid.initializer
+    :members: NormalInitializer
+    :noindex:
+
+
+XavierInitializer
+-----------------
+..  automodule:: paddle.v2.fluid.initializer
+    :members: XavierInitializer
+    :noindex:
+
+
+MSRAInitializer
+---------------
+..  automodule:: paddle.v2.fluid.initializer
+    :members: MSRAInitializer
+    :noindex:
+
diff --git a/doc_cn/_sources/api/v2/fluid/layers.rst.txt b/doc_cn/_sources/api/v2/fluid/layers.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..89e5fec13bf9062dc7a7187b1334c8f5486a980b
--- /dev/null
+++ b/doc_cn/_sources/api/v2/fluid/layers.rst.txt
@@ -0,0 +1,302 @@
+==========
+Layers
+==========
+
+
+fc
+---
+..  autofunction:: paddle.v2.fluid.layers.fc
+    :noindex:
+
+embedding
+---------
+..  autofunction:: paddle.v2.fluid.layers.embedding
+    :noindex:
+
+dynamic_lstm
+------------
+..  autofunction:: paddle.v2.fluid.layers.dynamic_lstm
+    :noindex:
+
+data
+---------
+..  autofunction:: paddle.v2.fluid.layers.data
+    :noindex:
+
+mean
+---------
+..  autofunction:: paddle.v2.fluid.layers.mean
+    :noindex:
+
+mul
+---------
+..  autofunction:: paddle.v2.fluid.layers.mul
+    :noindex:
+
+elementwise_add
+---------------
+..  autofunction:: paddle.v2.fluid.layers.elementwise_add
+    :noindex:
+
+elementwise_div
+---------------
+..  autofunction:: paddle.v2.fluid.layers.elementwise_div
+    :noindex:
+
+
+dropout
+---------
+..  autofunction:: paddle.v2.fluid.layers.dropout
+    :noindex:
+
+
+reshape
+---------
+..  autofunction:: paddle.v2.fluid.layers.reshape
+    :noindex:
+
+
+sigmoid
+---------
+..  autofunction:: paddle.v2.fluid.layers.sigmoid
+    :noindex:
+
+
+scale
+---------
+..  autofunction:: paddle.v2.fluid.layers.scale
+    :noindex:
+
+
+reshape
+---------
+..  autofunction:: paddle.v2.fluid.layers.reshape
+    :noindex:
+
+
+transpose
+---------
+..  autofunction:: paddle.v2.fluid.layers.transpose
+    :noindex:
+
+
+sigmoid_cross_entropy_with_logits
+---------
+..  autofunction:: paddle.v2.fluid.layers.esigmoid_cross_entropy_with_logits
+    :noindex:
+
+
+cast
+---------
+..  autofunction:: paddle.v2.fluid.layers.cast
+    :noindex:
+
+
+concat
+---------
+..  autofunction:: paddle.v2.fluid.layers.concat
+    :noindex:
+
+
+sums
+---------
+..  autofunction:: paddle.v2.fluid.layers.sums
+    :noindex:
+
+
+linear_chain_crf
+---------
+..  autofunction:: paddle.v2.fluid.layers.linear_chain_crf
+    :noindex:
+
+
+assign
+---------
+..  autofunction:: paddle.v2.fluid.layers.embedding
+    :noindex:
+
+
+split_lod_tensor
+---------
+..  autofunction:: paddle.v2.fluid.layers.split_lod_tensor
+    :noindex:
+
+
+merge_lod_tensor
+---------
+..  autofunction:: paddle.v2.fluid.layers.merge_lod_tensor
+    :noindex:
+
+cos_sim
+---------
+..  autofunction:: paddle.v2.fluid.layers.cos_sim
+    :noindex:
+
+
+cross_entropy
+---------
+..  autofunction:: paddle.v2.fluid.layers.cross_entropy
+    :noindex:
+
+
+
+square_error_cost
+---------
+..  autofunction:: paddle.v2.fluid.layers.square_error_cost
+    :noindex:
+
+
+accuracy
+---------
+..  autofunction:: paddle.v2.fluid.layers.accuracy
+    :noindex:
+
+
+sequence_conv
+---------
+..  autofunction:: paddle.v2.fluid.layers.sequence_conv
+    :noindex:
+
+
+conv2d
+---------
+..  autofunction:: paddle.v2.fluid.layers.conv2d
+    :noindex:
+
+
+sequence_pool
+---------
+..  autofunction:: paddle.v2.fluid.layers.sequence_pool
+    :noindex:
+
+
+pool2d
+---------
+..  autofunction:: paddle.v2.fluid.layers.pool2d
+    :noindex:
+
+
+batch_norm
+---------
+..  autofunction:: paddle.v2.fluid.layers.batch_norm
+    :noindex:
+
+
+beam_search_decode
+---------
+..  autofunction:: paddle.v2.fluid.layers.beam_search_decode
+    :noindex:
+
+
+lstm
+---------
+..  autofunction:: paddle.v2.fluid.layers.lstm
+    :noindex:
+
+
+lod_rank_table
+---------
+..  autofunction:: paddle.v2.fluid.layers.lod_rank_table
+    :noindex:
+
+
+max_sequence_len
+---------
+..  autofunction:: paddle.v2.fluid.layers.max_sequence_len
+    :noindex:
+
+
+topk
+---------
+..  autofunction:: paddle.v2.fluid.layers.topk
+    :noindex:
+
+
+lod_tensor_to_array
+---------
+..  autofunction:: paddle.v2.fluid.layers.lod_tensor_to_array
+    :noindex:
+
+
+
+array_to_lod_tensor
+---------
+..  autofunction:: paddle.v2.fluid.layers.array_to_lod_tensor
+    :noindex:
+
+
+
+
+fill_constant
+---------
+..  autofunction:: paddle.v2.fluid.layers.fill_constant
+    :noindex:
+
+
+
+fill_constant_batch_size_like
+---------
+..  autofunction:: paddle.v2.fluid.layers.fill_constant_batch_size_like
+    :noindex:
+
+
+ones
+---------
+..  autofunction:: paddle.v2.fluid.layers.ones
+    :noindex:
+
+
+zeros
+---------
+..  autofunction:: paddle.v2.fluid.layers.zeros
+    :noindex:
+
+
+increment
+---------
+..  autofunction:: paddle.v2.fluid.layers.increment
+    :noindex:
+
+
+array_write
+---------
+..  autofunction:: paddle.v2.fluid.layers.array_write
+    :noindex:
+
+
+
+create_array
+---------
+..  autofunction:: paddle.v2.fluid.layers.create_array
+    :noindex:
+
+
+less_than
+---------
+..  autofunction:: paddle.v2.fluid.layers.less_than
+    :noindex:
+
+
+array_read
+---------
+..  autofunction:: paddle.v2.fluid.layers.array_read
+    :noindex:
+
+
+shrink_memory
+---------
+..  autofunction:: paddle.v2.fluid.layers.shrink_memory
+    :noindex:
+
+
+array_length
+---------
+..  autofunction:: paddle.v2.fluid.layers.array_length
+    :noindex:
+
+
+conv2d_transpose
+---------
+..  autofunction:: paddle.v2.fluid.layers.conv2d_transpose
+    :noindex:
+
diff --git a/doc_cn/_sources/api/v2/fluid/nets.rst.txt b/doc_cn/_sources/api/v2/fluid/nets.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..2c3d075422de29c96e25458e831133a30270dd39
--- /dev/null
+++ b/doc_cn/_sources/api/v2/fluid/nets.rst.txt
@@ -0,0 +1,22 @@
+===========
+Nets
+===========
+
+simple_img_conv_pool
+-----------
+..  autofunction:: paddle.v2.fluid.nets.simple_img_conv_pool
+    :noindex:
+
+
+img_conv_group
+-----------
+..  autofunction:: paddle.v2.fluid.nets.img_conv_group
+    :noindex:
+
+
+sequence_conv_pool
+-----------
+..  autofunction:: paddle.v2.fluid.nets.sequence_conv_pool
+    :noindex:
+
+
diff --git a/doc_cn/_sources/api/v2/fluid/optimizer.rst.txt b/doc_cn/_sources/api/v2/fluid/optimizer.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..233762fcdfb39e592740adef6721a556fae3feef
--- /dev/null
+++ b/doc_cn/_sources/api/v2/fluid/optimizer.rst.txt
@@ -0,0 +1,54 @@
+===========
+Optimizer
+===========
+
+Optimizer
+-----------
+..  automodule:: paddle.v2.fluid.optimizer
+    :members: Optimizer
+    :noindex:
+
+
+SGDOptimizer
+-----------
+..  automodule:: paddle.v2.fluid.optimizer
+    :members: SGDOptimizer
+    :noindex:
+
+
+
+MomentumOptimizer
+-----------
+..  automodule:: paddle.v2.fluid.optimizer
+    :members: MomentumOptimizer
+    :noindex:
+
+
+
+AdagradOptimizer
+-----------
+..  automodule:: paddle.v2.fluid.optimizer
+    :members: AdagradOptimizer
+    :noindex:
+
+
+AdamOptimizer
+-----------
+..  automodule:: paddle.v2.fluid.optimizer
+    :members: AdamOptimizer
+    :noindex:
+
+
+AdamaxOptimizer
+-----------
+..  automodule:: paddle.v2.fluid.optimizer
+    :members: AdamaxOptimizer
+    :noindex:
+
+
+DecayedAdagradOptimizer
+-----------
+..  automodule:: paddle.v2.fluid.optimizer
+    :members: DecayedAdagradOptimizer
+    :noindex:
+
diff --git a/doc_cn/_sources/api/v2/fluid/param_attr.rst.txt b/doc_cn/_sources/api/v2/fluid/param_attr.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..ca0c8af9e8c4f2271de7a131ad0d27c0e8635f50
--- /dev/null
+++ b/doc_cn/_sources/api/v2/fluid/param_attr.rst.txt
@@ -0,0 +1,11 @@
+===========
+ParamAttr
+===========
+
+
+
+ParamAttr
+-----------
+..  automodule:: paddle.v2.fluid.param_attr
+    :members: ParamAttr
+    :noindex:
diff --git a/doc_cn/_sources/api/v2/fluid/profiler.rst.txt b/doc_cn/_sources/api/v2/fluid/profiler.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..7d4042d1f41c12c4a551ba6576559d612116872a
--- /dev/null
+++ b/doc_cn/_sources/api/v2/fluid/profiler.rst.txt
@@ -0,0 +1,10 @@
+===========
+Profiler
+===========
+
+
+
+Profiler
+-----------
+..  autofunction:: paddle.v2.fluid.profiler.cuda_profiler
+    :noindex:
diff --git a/doc_cn/_sources/api/v2/fluid/regularizer.rst.txt b/doc_cn/_sources/api/v2/fluid/regularizer.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..3af2b07d2ae55d99df705fbf1ad2402eee05c435
--- /dev/null
+++ b/doc_cn/_sources/api/v2/fluid/regularizer.rst.txt
@@ -0,0 +1,25 @@
+===========
+Regularizer
+===========
+
+WeightDecayRegularizer
+-----------
+..  automodule:: paddle.v2.fluid.regularizer
+    :members: WeightDecayRegularizer
+    :noindex:
+
+
+L2DecayRegularizer
+-----------
+..  automodule:: paddle.v2.fluid.regularizer
+    :members: L2DecayRegularizer
+    :noindex:
+
+
+
+L1DecayRegularizer
+-----------
+..  automodule:: paddle.v2.fluid.regularizer
+    :members: L1DecayRegularizer
+
+
diff --git a/doc_cn/_sources/api/v2/model_configs.rst.txt b/doc_cn/_sources/api/v2/model_configs.rst.txt
index a5fae7e29e56d9e236d489353a5c8967d1954641..992b559cbd87244612521d4c96f84f997d6c4196 100644
--- a/doc_cn/_sources/api/v2/model_configs.rst.txt
+++ b/doc_cn/_sources/api/v2/model_configs.rst.txt
@@ -6,6 +6,7 @@ Model Configuration
 
     config/activation.rst
     config/layer.rst
+    config/evaluators.rst
     config/optimizer.rst
     config/pooling.rst
     config/networks.rst
diff --git a/doc_cn/_sources/design/api.md.txt b/doc_cn/_sources/design/api.md.txt
index 8185d2af0ea264a2e7b4e28b9ed05279e4a22014..e6a4638d9100d9b07c3ee6b92b530a17eae1c162 100644
--- a/doc_cn/_sources/design/api.md.txt
+++ b/doc_cn/_sources/design/api.md.txt
@@ -3,7 +3,7 @@
 ## Ingredients
 
 As our design principle is starting from the essence: how could we
-allow users to express and solve their problems at neural networks.
+allow users to express and solve their problems as neural networks.
 Some essential concepts that our API have to provide include:
 
 1. A *topology* is an expression of *layers*.
@@ -233,7 +233,7 @@ paddle.dist_train(model,
                   num_parameter_servers=15)
 ```
 
-The pseudo code if `paddle.dist_train` is as follows:
+The pseudo code of `paddle.dist_train` is as follows:
 
 ```python
 def dist_train(topology, parameters, trainer, reader, ...):
diff --git a/doc_cn/_sources/design/auto_gradient_check.md.txt b/doc_cn/_sources/design/auto_gradient_check.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..f9991541bc51c6e13ffce4e9cec60f73dc800121
--- /dev/null
+++ b/doc_cn/_sources/design/auto_gradient_check.md.txt
@@ -0,0 +1,146 @@
+## Auto Gradient Checker Design
+
+## Backgraound：
+- Generally, it is easy to check whether the forward computation of an Operator is correct or not. However, backpropagation is a notoriously difficult algorithm to debug and get right:
+  1. you should get the right backpropagation formula according to the forward computation.
+  2. you should implement it right in CPP.
+  3. it's difficult to prepare test data.
+
+- Auto gradient checking gets a numerical gradient by forward Operator and use it as a reference of the backward Operator's result. It has several advantages:
+  1. numerical gradient checker only need forward operator.
+  2. user only need to prepare the input data for forward Operator.
+
+## Mathematical Theory
+The following two document from Stanford has a detailed explanation of how to get numerical gradient and why it's useful.
+
+- [Gradient checking and advanced optimization(en)](http://deeplearning.stanford.edu/wiki/index.php/Gradient_checking_and_advanced_optimization)
+- [Gradient checking and advanced optimization(cn)](http://ufldl.stanford.edu/wiki/index.php/%E6%A2%AF%E5%BA%A6%E6%A3%80%E9%AA%8C%E4%B8%8E%E9%AB%98%E7%BA%A7%E4%BC%98%E5%8C%96)
+
+
+## Numeric Gradient Implementation
+### Python Interface
+```python
+def get_numerical_gradient(op,
+                         input_values,
+                         output_name,
+                         input_to_check,
+                         delta=0.005,
+                         local_scope=None):
+    """
+    Get Numeric Gradient for an operator's input.
+
+    :param op: C++ operator instance, could be an network
+    :param input_values: The input variables. Should be an dictionary, whose key is
+    variable name, and value is numpy array.
+    :param output_name: The final output variable name.
+    :param input_to_check: The input variable with respect to which to compute the gradient.
+    :param delta: The perturbation value for numeric gradient method. The
+    smaller delta is, the more accurate result will get. But if that delta is
+     too small, it will suffer from numerical stability problem.
+    :param local_scope: The local scope used for get_numeric_gradient.
+    :return: The gradient array in numpy format.
+    """
+```
+
+### Explaination:
+
+- Why need `output_name`
+  - An Operator may have multiple Output, one can get independent gradient from each Output. So caller should specify the name of the output variable.
+
+- Why need `input_to_check`
+  - One operator may have multiple inputs. Gradient Op can calculate the gradient of these inputs at the same time. But Numeric Gradient needs to calculate them one by one. So `get_numeric_gradient` is designed to calculate the gradient for one input. If you need to compute multiple inputs, you can call `get_numeric_gradient` multiple times.
+
+
+### Core Algorithm Implementation
+
+
+```python
+    # we only compute gradient of one element a time.
+    # we use a for loop to compute the gradient of each element.
+    for i in xrange(tensor_size):
+        # get one input element by its index i.
+        origin = tensor_to_check.get_float_element(i)
+
+        # add delta to it, run op and then get the new value of the result tensor.
+        x_pos = origin + delta
+        tensor_to_check.set_float_element(i, x_pos)
+        y_pos = get_output()
+
+        # plus delta to this element, run op and get the new value of the result tensor.
+        x_neg = origin - delta
+        tensor_to_check.set_float_element(i, x_neg)
+        y_neg = get_output()
+
+        # restore old value
+        tensor_to_check.set_float_element(i, origin)
+
+        # compute the gradient of this element and store it into a numpy array.
+        gradient_flat[i] = (y_pos - y_neg) / delta / 2
+
+    # reshape the gradient result to the shape of the source tensor.
+    return gradient_flat.reshape(tensor_to_check.get_dims())
+```
+
+## Auto Graident Checker Framework
+
+Each Operator Kernel has three kinds of Gradient:
+
+1. Numerical gradient
+2. CPU kernel gradient
+3. GPU kernel gradient (if supported)
+
+The numerical gradient only relies on forward Operator. So we use the numerical gradient as the reference value. And the gradient checking is performed in the following three steps:
+
+1. calculate the numerical gradient
+2. calculate CPU kernel gradient with the backward Operator and compare it with the numerical gradient
+3. calculate GPU kernel gradient with the backward Operator and compare it with the numeric gradient (if supported)
+
+#### Python Interface
+
+```python
+    def check_grad(self,
+                   forward_op,
+                   input_vars,
+                   inputs_to_check,
+                   output_name,
+                   no_grad_set=None,
+                   only_cpu=False,
+                   max_relative_error=0.005):
+        """
+        :param forward_op: used to create backward_op
+        :param input_vars: numpy value of input variable. The following
+            computation will use these variables.
+        :param inputs_to_check: the input variable with respect to which to compute the gradient.
+        :param output_name: The final output variable name.
+        :param max_relative_error: The relative tolerance parameter.
+        :param no_grad_set: used when create backward ops
+        :param only_cpu: only compute and check gradient on cpu kernel.
+        :return:
+        """
+```
+
+### How to check if two numpy array is close enough?
+if `abs_numerical_grad` is nearly zero, then use abs error for numerical_grad
+
+```python
+numerical_grad = ...
+operator_grad = numpy.array(scope.find_var(grad_var_name(name)).get_tensor())
+
+abs_numerical_grad = numpy.abs(numerical_grad)
+# if abs_numerical_grad is nearly zero, then use abs error for numeric_grad, not relative
+# error.
+abs_numerical_grad[abs_numerical_grad < 1e-3] = 1
+
+diff_mat = numpy.abs(abs_numerical_grad - operator_grad) / abs_numerical_grad
+max_diff = numpy.max(diff_mat)
+```
+
+
+#### Notes：
+The Input data for auto gradient checker should be reasonable to avoid numerical  stability problem.
+
+
+#### Refs:
+
+- [Gradient checking and advanced optimization(en)](http://deeplearning.stanford.edu/wiki/index.php/Gradient_checking_and_advanced_optimization)
+- [Gradient checking and advanced optimization(cn)](http://ufldl.stanford.edu/wiki/index.php/%E6%A2%AF%E5%BA%A6%E6%A3%80%E9%AA%8C%E4%B8%8E%E9%AB%98%E7%BA%A7%E4%BC%98%E5%8C%96)
diff --git a/doc_cn/_sources/design/block.md.txt b/doc_cn/_sources/design/block.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..4066122c0e8dfa33776796c3d205ba5aec9e0f52
--- /dev/null
+++ b/doc_cn/_sources/design/block.md.txt
@@ -0,0 +1,336 @@
+# Design Doc: Block and Scope
+
+## The Representation of Computation
+
+Both deep learning systems and programming languages help users describe computation procedures.  These systems use various representations of computation:
+
+- Caffe, Torch, and Paddle: sequences of layers.
+- TensorFlow, Caffe2, Mxnet: graph of operators.
+- PaddlePaddle: nested blocks, like C++ and Java programs.
+
+## Block in Programming Languages and Deep Learning
+
+In programming languages, a block is a pair of curly braces that includes local variables definitions and a sequence of instructions or operators.
+
+Blocks work with control flow structures like `if`, `else`, and `for`, which have equivalents in deep learning:
+
+| programming languages | PaddlePaddle          |
+|-----------------------|-----------------------|
+| for, while loop       | RNN, WhileOp          |
+| if, if-else, switch   | IfElseOp, SwitchOp    |
+| sequential execution  | a sequence of layers  |
+
+A key difference is that a C++ program describes a one pass computation, whereas a deep learning program describes both the forward and backward passes.
+
+## Stack Frames and the Scope Hierarchy
+
+The existence of the backward pass makes the execution of a block of PaddlePaddle different from traditional programs:
+
+| programming languages | PaddlePaddle                    |
+|-----------------------|---------------------------------|
+| stack                 | scope hierarchy                 |
+| stack frame           | scope                           |
+| push at entering block| push at entering block          |
+| pop at leaving block  | destroy when minibatch completes|
+
+1. In traditional programs:
+
+   - When the execution enters the left curly brace of a block, the runtime pushes a frame into the stack, where it realizes local variables.
+   - After the execution leaves the right curly brace, the runtime pops the frame.
+   - The maximum number of frames in the stack is the maximum depth of nested blocks.
+
+1. In PaddlePaddle
+
+   - When the execution enters a block, PaddlePaddle adds a new scope, where it realizes variables.
+   - PaddlePaddle doesn't pop a scope after the execution of the block because variables therein are used by the backward pass.  So it has a stack forest known as a *scope hierarchy*.
+   - The height of the highest tree is the maximum depth of nested blocks.
+   - After the processing of a minibatch, PaddlePaddle destroys the scope hierarchy.
+
+## Use Blocks in C++ and PaddlePaddle Programs
+
+Let us consolidate the discussion by presenting some examples.
+
+### Blocks with `if-else` and `IfElseOp`
+
+The following C++ programs shows how blocks are used with the `if-else` structure:
+
+```c++
+namespace pd = paddle;
+
+int x = 10;
+int y = 1;
+int z = 10;
+bool cond = false;
+int o1, o2;
+if (cond) {
+  int z = x + y;
+  o1 = z;
+  o2 = pd::layer::softmax(z);
+} else {
+  int d = pd::layer::fc(z);
+  o1 = d;
+  o2 = d+1;
+}
+
+```
+
+An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator](./if_else_op.md) is as follows:
+
+```python
+import paddle as pd
+
+x = minibatch([10, 20, 30]) # shape=[None, 1]
+y = var(1) # shape=[1], value=1
+z = minibatch([10, 20, 30]) # shape=[None, 1]
+cond = larger_than(x, 15) # [false, true, true]
+
+ie = pd.ifelse()
+with ie.true_block():
+    d = pd.layer.add_scalar(x, y)
+    ie.output(d, pd.layer.softmax(d))
+with ie.false_block():
+    d = pd.layer.fc(z)
+    ie.output(d, d+1)
+o1, o2 = ie(cond)
+```
+
+In both examples, the left branch computes `x+y` and `softmax(x+y)`, the right branch computes `fc(x)` and `x+1` .
+
+The difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances.
+
+
+### Blocks with `for` and `RNNOp`
+
+The following RNN model in PaddlePaddle from the [RNN design doc](./rnn.md) :
+
+```python
+x = sequence([10, 20, 30]) # shape=[None, 1]
+m = var(0) # shape=[1]
+W = var(0.314, param=true) # shape=[1]
+U = var(0.375, param=true) # shape=[1]
+
+rnn = pd.rnn()
+with rnn.step():
+  h = rnn.memory(init = m)
+  h_prev = rnn.previous_memory(h)
+  a = layer.fc(W, x)
+  b = layer.fc(U, h_prev)  
+  s = pd.add(a, b)
+  act = pd.sigmoid(s)
+  rnn.update_memory(h, act)
+  rnn.output(a, b)
+o1, o2 = rnn()
+```
+has its equivalent C++ program as follows
+
+```c++
+int* x = {10, 20, 30};
+int* m = {0};
+int* W = {0.314};
+int* U = {0.375};
+
+int mem[sizeof(x) / sizeof(x[0]) + 1];
+int o1[sizeof(x) / sizeof(x[0]) + 1];
+int o2[sizeof(x) / sizeof(x[0]) + 1];
+for (int i = 1; i <= sizeof(x)/sizeof(x[0]); ++i) {
+  int x = x[i-1];
+  if (i == 1) mem[0] = m;
+  int a = W * x;
+  int b = Y * mem[i-1];
+  int s = fc_out + hidden_out;
+  int act = sigmoid(sum);
+  mem[i] = act;
+  o1[i] = act;
+  o2[i] = hidden_out;
+}
+```
+
+## Compilation and Execution
+
+Like TensorFlow, a PaddlePaddle program is written in Python. The first part describes a neural network as a protobuf message, and the rest executes the message for training or inference.
+
+The generation of this protobuf message is similar to how a compiler generates a binary executable file. The execution of the message is similar to how the OS executes the binary file.
+
+## The "Binary Executable File Format"
+
+The definition of the protobuf message is as follows:
+
+```protobuf
+message BlockDesc {
+  repeated VarDesc vars = 1;
+  repeated OpDesc ops = 2;
+}
+```
+
+The step net in above RNN example would look like
+
+```
+BlockDesc {
+  vars = {
+    VarDesc {...} // x
+    VarDesc {...} // h
+    VarDesc {...} // fc_out
+    VarDesc {...} // hidden_out
+    VarDesc {...} // sum
+    VarDesc {...} // act
+  }
+  ops = {
+    OpDesc {...} // matmul
+    OpDesc {...} // add_two
+    OpDesc {...} // sigmoid
+  }
+};
+```
+
+Also, the RNN operator in above example is serialized into a protobuf message of type `OpDesc` and would look like:
+
+```
+OpDesc {
+  inputs = {0} // the index of x in vars of BlockDesc above
+  outputs = {5, 3} // indices of act and hidden_out in vars of BlockDesc above
+  attrs {
+    "states" : {1} // the index of h
+    "step_net" : 
+  }
+};
+```
+
+This `OpDesc` value is in the `ops` field of the `BlockDesc` value representing the global block.
+
+
+## The Compilation of Blocks
+
+During the generation of the Protobuf message, the Block should store VarDesc (the Protobuf message which describes Variable) and OpDesc (the Protobuf message which describes Operator).
+
+VarDesc in a block should have its name scope to avoid local variables affect parent block's name scope.
+Child block's name scopes should inherit the parent's so that OpDesc in child block can reference a VarDesc that stored in parent block. For example:
+
+```python
+a = pd.Variable(shape=[20, 20])
+b = pd.fc(a, params=["fc.w", "fc.b"])
+
+rnn = pd.create_rnn()
+with rnn.stepnet():
+    x = a.as_step_input()
+    # reuse fc's parameter
+    fc_without_b = pd.get_variable("fc.w")
+    rnn.output(fc_without_b)
+
+out = rnn()
+```
+The method `pd.get_variable` can help retrieve a Variable by the name. The Variable may be stored in a parent block, but might be retrieved in a child block, so block should have a variable scope that supports inheritance.
+
+In compiler design, the symbol table is a data structure created and maintained by compilers to store information about the occurrence of various entities such as variable names, function names, classes, etc.
+
+To store the definition of variables and operators, we define a C++ class `SymbolTable`, like the one used in compilers.
+
+`SymbolTable` can do the following:
+
+- store the definitions (some names and attributes) of variables and operators,
+- verify if a variable was declared,
+- make it possible to implement type checking (offer Protobuf message pointers to `InferShape` handlers).
+
+
+```c++
+// Information in SymbolTable is enough to trace the dependency graph. So maybe
+// the Eval() interface takes a SymbolTable is enough.
+class SymbolTable {
+ public:
+  SymbolTable(SymbolTable* parent) : parent_(parent) {}
+
+  OpDesc* NewOp(const string& name="");
+
+  // TODO determine whether name is generated by python or C++.
+  // Currently assume that a unique name will be generated by C++ if the
+  // argument name is left default.
+  VarDesc* Var(const string& name="");
+
+  // find a VarDesc by name, if recursive is true, find parent's SymbolTable
+  // recursively.
+  // this interface is introduced to support InferShape, find protobuf messages
+  // of variables and operators, pass pointers into InferShape.
+  //
+  // NOTE maybe some C++ classes such as VarDescBuilder and OpDescBuilder should
+  // be proposed and embedded into pybind to enable python operation on C++ pointers.
+  VarDesc* FindVar(const string& name, bool recursive=true);
+
+  OpDesc* FindOp(const string& name);
+
+  BlockDesc Compile() const;
+
+ private:
+  SymbolTable* parent_;
+
+  map ops_;
+  map vars_;
+};
+```
+
+After all the description of variables and operators is added into SymbolTable,
+the block has enough information to run.
+
+The `Block` class takes a `BlockDesc` as input, and provides `Run` and `InferShape` functions.
+
+
+```c++
+namespace {
+
+class Block : OperatorBase {
+public:
+  Block(const BlockDesc& desc) desc_(desc) {}
+
+  void InferShape(const framework::Scope& scope) const override {
+    if (!symbols_ready_) {
+      CreateVariables(scope);
+      CreateOperators();
+    }
+    // should run InferShape first.
+    for (auto& op : runtime_table_.ops()) {
+      op->InferShape(scope);
+    }
+  }
+
+  void Run(const framework::Scope& scope,
+           const platform::DeviceContext& dev_ctx) const override {
+    PADDLE_ENFORCE(symbols_ready_, "operators and variables should be created first.");
+    for (auto& op : runtime_table_.ops()) {
+      op->Run(scope, dev_ctx);
+    }
+  }
+
+  void CreateVariables(const framework::Scope& scope);
+  void CreateOperators();
+
+  // some other necessary interfaces of NetOp are listed below
+  // ...
+
+private:
+  BlockDesc desc_;
+  bool symbols_ready_{false};
+};
+```
+
+## The Execution of Blocks
+
+Block inherits from OperatorBase, which has a Run method.
+Block's Run method will run its operators sequentially.
+
+There is another important interface called `Eval`, which takes some arguments called targets and generates a minimal graph which treats targets as the end points and creates a new Block. After `Run`, `Eval` will get the latest value and return the targets.
+
+The definition of Eval is as follows:
+
+```c++
+// clean a block description by targets using the corresponding dependency graph.
+// return a new BlockDesc with minimal number of operators.
+// NOTE: The return type is not a Block but the block's description so that this can be distributed
+// to a cluster.
+BlockDesc Prune(const BlockDesc& desc, vector targets);
+
+void Block::Eval(const vector& targets,
+                 const framework::Scope& scope,
+                 const platform::DeviceContext& dev_ctx) {
+  BlockDesc min_desc = Prune(desc_, targets);
+  Block min_block(min_desc);
+  min_block.Run(scope, dev_ctx);
+}
+```
diff --git a/doc_cn/_sources/design/build_system/README.md.txt b/doc_cn/_sources/design/build_system/README.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..bf0e4dddc1b640ecbce489f65820aaf8a4b3b1e7
--- /dev/null
+++ b/doc_cn/_sources/design/build_system/README.md.txt
@@ -0,0 +1,152 @@
+A few months ago when we were trying to replace CMake with Bazel, @emailweixu suggested that we rewrite those handy Bazel functions using CMake. Now it seems that it's the right time to get this done, as we are facing problems from the porting of Majel and the development of new the parameter server using Go and C++.
+
+Here are some initial thoughts. Your comments are welcome!
+
+### Required CMake Function
+
+I think we need only the following few CMake functions to make a project description mean and clean:
+
+| C++ | CUDA C++ | Go |
+|---|---|---|
+| cc_library | nv_library | go_library |
+| cc_binary | nv_binary | go_binary |
+| cc_test | nv_test | go_test |
+
+- The `_library` functions generate  .a files from source code.
+- The `_binary` functions generate executable binary files.
+- The `_test` functions generate executable unit test files. They work like `_binary` but links `-lgtest` and `-lgtest_main`.
+
+The difference between `nv_` functions and `cc_` functions is that the former use `nvcc` instead of the system-default C++ compiler.
+
+Both `nv_` and `cc_` functions enables C++11 (-std=c++11).
+
+Also,
+
+- to describe external dependencies, we need `external_library`.
+- to build shared libraries, we need `shared_library`.
+
+### An Example Project
+
+Suppose that we have aforementioned functions defined in our `/cmake` directory.  The following example `CMakeLists.txt` describes a project including the following source files:
+
+- tensor.h
+- tensor.cc
+- tensor_test.cc
+- ops.h
+- ops.cu
+- ops_test.cu
+- api.go
+- api_test.go
+
+Suppose that ops.cu depends on CUDNN.
+
+```cmake
+# cc_binary parses tensor.cc and figures out that target also depend
+# on tensor.h.
+cc_binary(tensor
+  SRCS
+  tensor.cc)
+
+# The dependency to target tensor implies that if any of
+# tensor{.h,.cc,_test.cc} is changed, tensor_test need to be re-built.
+cc_test(tensor_test
+  SRCS
+  tensor_test.cc
+  DEPS
+  tensor)
+
+# I don't have a clear idea what parameters external_library need to
+# have.  @gangliao as a CMake expert would have better ideas.
+external_library(cudnn
+  ....)
+
+# Suppose that ops.cu depends on external target CUDNN.  Also, ops.cu
+# include global functions that take Tensor as their parameters, so
+# ops depend on tensor.  This implies that if any of tensor.{h.cc},
+# ops.{h,cu} is changed, ops need to be re-built.
+nv_library(ops
+  SRCS
+  ops.cu
+  DEPS
+  tensor
+  cudnn)  # cudnn is defined later.
+
+nv_test(ops_test
+  SRCS
+  ops_test.cu
+  DEPS
+  ops)
+
+# Because api.go defines a GO wrapper to ops and tensor, it depends on
+# both.  This implies that if any of tensor.{h,cc}, ops.{h,cu}, or
+# api.go is changed, api need to be re-built.
+go_library(api
+  SRCS
+  api.go
+  DEPS
+  tensor # Because ops depend on tensor, this line is optional.
+  ops)
+
+go_test(api_test
+  SRCS
+  api_test.go
+  DEPS
+  api)
+
+
+# This builds libapi.so.  shared_library might use CMake target
+# api_shared so to distinguish it from above target api.
+shared_library(api
+  DEPS
+  api)
+
+```
+
+### Implementation
+
+As above example CMakeLists.txt executes, each function invocation adds "nodes" to a dependency graph.  It also use this graph to generate CMake commands including `add_executable`, `add_dependencies`, `target_link_libraries`, and `add_test`.
+
+### Using Package Manager For Go
+
+Building Go binaries and libraries need to satisfy their dependencies, generally
+we can do `go get ./...` to download and compile all external dependencies. The
+problems are:
+
+1. `go get` will always get the latest code from the default branch of the
+    remote repo, so changes of dependents might break the build. This is very
+    different with what we already have in `cmake/external` which download a
+    specific version or commit id of the dependency.
+1. Some locations can not access external dependencies through the internet, as mentioned
+   in https://github.com/PaddlePaddle/Paddle/issues/2605. Using package management
+   tools can package the dependencies as a "vendor" package, which can be mirrored
+   at many cloud file hosting, so users what to compile paddle by themselves can
+   download this "vendor" package from a mirror site.
+
+#### Choose A Suitable Tool
+
+As mentioned by @wangkuiyi, [Here](https://github.com/golang/go/wiki/PackageManagementTools)
+list dozens of Go package managers. We choose the tool using following principles:
+
+- Most "active" projects with more stars, more pull requests or commits
+- Widely used project
+
+After comparing all these projects, we shall choose between the most popular
+tools: Godep and Glide.
+
+Here's a brief comparison between Godep and Glide
+: https://github.com/Masterminds/glide/wiki/Go-Package-Manager-Comparison. There are
+also many complaints about using `Godep`. There's also a new "official" pakcage
+management tool has been started at: https://github.com/golang/dep to resolve
+such problems, but it's currently at Alpha stage. So the best choice now is
+glide obviously.
+
+#### Manage Go Packages
+
+- Dependencies: `go/glide.yaml` will store the dependencies and their versions which
+  is directly imported by paddle. `go/glide.lock` will store all dependencies recursively
+  with their commit id. Builds will "lock" to these packages if we don't `glide up`
+  them
+- Vendor package: `go/vendor` directory will generated when running `cmake` command. `cmake`
+  will download the code corresponding to `go/glide.lock`. If we put a vendor folder
+  under `go/`, cmake will just check the commit id to the packages under the folder,
+  if commit id matches, there will be no download at all.
diff --git a/doc/_sources/design/dist/README.md.txt b/doc_cn/_sources/design/cluster_train/README.md.txt
similarity index 62%
rename from doc/_sources/design/dist/README.md.txt
rename to doc_cn/_sources/design/cluster_train/README.md.txt
index 1788208bcabca30f66cb1c80e80f6b824c0d9579..177a5f5d54bd924fab34795219ce1f7b270c8e25 100644
--- a/doc/_sources/design/dist/README.md.txt
+++ b/doc_cn/_sources/design/cluster_train/README.md.txt
@@ -15,33 +15,37 @@ This poses technical challenges to PaddlePaddle:
 
 A training job will be created once user asks Paddle cloud to train a model. The training job is made up of different processes that collaboratively consume data and produce a trained model. There are three kinds of processes:
 
-1. the *master process*, which dispatches tasks to
+1. the *master server process*, which dispatches tasks to
 1. one or more *trainer processes*, which run distributed training and synchronize gradients/models via
-1. one or more *parameter server processes*, where each holds a shard of the global model.
+1. one or more *parameter server processes*, where each holds a shard of the global model, and receive the uploaded gradients from every *trainer process*, so they can run the optimize functions to update their parameters.
 
 Their relation is illustrated in the following graph:
 
 
 
-### Master Process
+By coordinating these processes, PaddlePaddle supports use both Synchronize Stochastic Gradient Descent (sync SGD) and Asynchronous Stochastic Gradient Descent (async SGD) to train user-defined neural network topologies.
 
-The master process will:
+When training with sync SGD, parameter servers wait for all trainers to finish gradients update and then send the updated parameters to trainers, training can not proceed until the trainer received the updated parameters. This creates a synchronization point between trainers. When training with async SGD, each trainer upload gradient and download new parameters individually, without the synchronization with other trainers. Using asyc SGD will be faster in terms of time per pass, but have more noise in gradient since trainers are likely to have a stale model.
+
+### Master Server Process
+
+The master server process will:
 
 - Partition a dataset into [tasks](#task) and dispatch tasks to trainers.
 - Keep track of training progress on the dataset with [task queue](#task-queue). A training job will iterate on the dataset for a full pass until it goes into next pass.
 
 
-#### Task 
+#### Task
 
 A task is a data shard to be trained. The total number of tasks will be much bigger than the total number of trainers. The number of data instances inside a task will be much bigger than the mini-batch size.
 
 #### Task Queue
 
-The master process has three task queues to track training progress. As illustrated in the graph below, Job A and Job B both have one master process. Each master process has three task queues.
+The master server has three task queues to track training progress. As illustrated in the graph below, Job A and Job B both have one master server. Each master server process has three task queues.
 
 
 
-- The todo queue holds tasks to be dispatched. When a job starts, the master process fills in the todo queue with all tasks.
+- The todo queue holds tasks to be dispatched. When a job starts, the master server fills in the todo queue with all tasks.
 - The pending queue holds tasks that are currently training by trainers.
 - the done queue holds tasks that are already trained.
 
@@ -50,17 +54,18 @@ The life cycle of a single task is illustrated below:
 
 
 1. When a new pass of training starts, all tasks will be placed in the todo queue.
-1. The master process will dispatch few tasks to each trainer at a time, puts them in the pending queue and waits for completion.
-1. The trainer will work on its tasks and tell the master process once a task is completed. The master process will dispatch a new task to that trainer.
-1. If a task timeout. the master process will move it back to the todo queue. The timeout count will increase by one. If the timeout count is above a threshold, the task is likely to cause a trainer to crash, so it will be discarded.
-1. The master process will move completed task to the done queue. When the todo queue is empty, the master process will start a new pass by moving all tasks in the done queue to todo queue and reset the timeout counter of all tasks to zero.
+1. Upon trainer requests for new task, the master server will dispatch a task from todo queue to it, put the task in the pending queue and wait for completion.
+1. The trainer will work on its task and tell the master server once the task is completed and ask for new task. The master server will dispatch a new task to that trainer.
+1. If a task fails for any reason in trainer, or takes longer than a specific period of time,  the master server will move the task back to the todo queue. The timeout count for that task will increase by one. If the timeout count is above a threshold, the task is likely to cause a trainer to crash, then it will be discarded.
+1. The master server will move completed task to the done queue. When the todo queue is empty, the master server will start a new pass by moving all tasks in the done queue to todo queue and reset the timeout counter of all tasks to zero.
 
 ### Trainer Process
 
 The trainer process will:
 
-- Receive tasks from the master.
-- Work on the tasks: calculate and upload gradient to parameter servers, and update local model by downloading new parameters from parameter servers.
+- Request tasks from the master.
+- Work on the tasks
+- Upload gradient to parameter servers, and update local model by downloading new parameters from parameter servers.
 
 ### Parameter Server Process
 
@@ -78,7 +83,7 @@ The communication pattern between the trainers and the parameter servers depends
 - Synchronous Stochastic Gradient Descent (sync-SGD)
 
 	Parameter server will wait for all trainer finish n-th mini-batch calculation and send their gradients before broadcasting new parameters to every trainer. Every trainer will wait for the new parameters before starting n+1-th mini-batch.
-  
+
 - Asynchronous Stochastic Gradient Descent (async-SGD)
 
 	There will no synchronization between different trainers, and parameter server updates its parameter as soon as it receives new gradient:
@@ -89,7 +94,7 @@ The communication pattern between the trainers and the parameter servers depends
 
 ## Fault Tolerant
 
-The training job will pause if the master processes is dead, or any of the parameter server process is dead. They will be started by [Kubernetes](https://kubernetes.io/) and recover in few minutes. Please refer to [fault recovery](#fault-recovery).
+The training job will pause if the master server processes is dead, or any of the parameter server process is dead. They will be started by [Kubernetes](https://kubernetes.io/) and recover in few minutes. Please refer to [fault recovery](#fault-recovery).
 
 The training job will continue to make progress if there is at least one training process running. The strategy depends on the type of optimization algorithm:
 
@@ -109,28 +114,26 @@ Now we will introduce how each process recovers from a failure, the graph below
 
 
 
-### Master Process
+### Master Server Process
 
 When the master is started by the Kubernetes, it executes the following steps at startup:
 
 1. Grabs a unique *master* lock in etcd, which prevents concurrent master instantiations.
 1. Recovers the task queues from etcd if they already exist, otherwise, the master will create them.
-1. Watches the trainer prefix keys `/trainer/` on etcd to find the live trainers.
-1. Starts dispatching the tasks to the trainers, and updates task queue using an etcd transaction to ensure lock is held during the update.
+1. Write its ip address to */master/addr* so that trainers can discover it.
+1. Listens to trainers' request of task, dispatch one upon request, and updates task queue using an etcd transaction to ensure lock is held during the update.
 
-The master process will kill itself if its etcd lease expires.
-
-When the master process is dead for any reason, Kubernetes will restart it. It will be online again with all states recovered from etcd in few minutes.
+When the master server process is dead for any reason, Kubernetes will restart it. It will be online again with all states recovered from etcd in few minutes.
 
 ### Trainer Process
 
 When the trainer is started by the Kubernetes, it executes the following steps at startup:
 
-1. Watches the available parameter server prefix keys `/ps/` on etcd and waits until the count of parameter servers reaches the desired count.
-1. Generates a unique ID, and sets key `/trainer/` with its contact address as value. The key will be deleted when the lease expires, so the master will be aware of the trainer being online and offline.
-1. Waits for tasks from the master to start training.
+1. Watches the available parameter server prefix keys `/ps/` on etcd and waits until the count of parameter servers reaches the desired count */ps_desired*.
+1. Finds and watches */master/addr* to get master's address.
+1. Requests for tasks from the master to start training.
 
-If trainer's etcd lease expires, it will try set key `/trainer/` again so that the master process can discover the trainer again.
+When a trainer fails, Kuberentes would try to restart it. The recovered trainer would fetch tasks from master and go on training.
 
 ### Parameter Server Process
 
@@ -140,11 +143,11 @@ When the parameter server is started by Kubernetes, it executes the following st
 1. Search through etcd keys `/ps/` (`/ps/0`, `/ps/1`, ...) to find the first non-existant key whose index is smaller than the total number of parameter servers. Set the key using a transaction to avoid concurrent writes. The parameter server's index is inferred from the key name.
 
 	The desired number of parameter servers is 3:
-	
+
 	
-	
+
 	The third parameter server joined:
-	
+
 	
 
 1. The parameter server can load parameters if there are already saved parameters in the save path (inferred from its index).
@@ -153,6 +156,13 @@ When the parameter server is started by Kubernetes, it executes the following st
 If the parameter server's etcd lease expires, the parameter server will kill itself.
 
 
+## Parameter Server Checkpointing
+See [here](./checkpointing.md)
+
+## Store and dispatching trainning data
+See [here](./data_dispatch.md)
+
+
 ## Dynamic Scaling
 
 ### Trainer Scaling
diff --git a/doc_cn/_sources/design/cluster_train/checkpointing.md.txt b/doc_cn/_sources/design/cluster_train/checkpointing.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..c87ef2c7d2636208866d05456d5d44316d0bb200
--- /dev/null
+++ b/doc_cn/_sources/design/cluster_train/checkpointing.md.txt
@@ -0,0 +1,44 @@
+## 模型参数检查点（Checkpointing）
+模型数据检查点的实现，可以有效的避免parameter server的单点或多点同时故障。模型参数检查点通过定期向磁盘上保存一份存储在parameter server内存中的模型数据的完整镜像，来保证训练过程可以从中间状态重新启动。在一个不可中断并缺少备份的训练任务中，可以通过阶段性的保存每个parameter server的数据快照（snapshot）到 ***分布式存储服务*** 达到容灾的目的，比如每隔10分钟最新的快照，并删除更早的快照。在出现单点故障时，只需要恢复这台节点，或者将这台节点迁移到另一个节点并启动即可恢复训练任务。
+
+
+
+### 快照保存的设计如下：
+
+说明：
+
+* parameter server在集群中启动后，自动挂载分布式存储目录，并把快照保存到这个目录下。
+* ***注：每个parameter server的检查点各自独立保存，暂时不考虑多个parameter server同步的保存一个特定时间点的全局检查点，因为这样做也没法保证消除随机性。***
+
+检查点保存程序流程：
+
+1. 如果满足条件"每隔10分钟"时，parameter server会获取parameters内存的`read_lock`，启动一个新的线程开始保存检查点。如果已经正在执行保存检查点的线程，则忽略。由于对parameters的更新需要获取parameters内存的`write_lock`，所以在写入快照的过程中，parameter server会暂停参数更新并等待。
+2. parameter server生成一个UUID，向指定的目录中一个新的文件（文件名为此UUID）写入快照数据。在快照写入完成后，计算这个文件的MD5 sum。然后在etcd的`/checkpoints/[pserver_id]`中写入json内容：`{"uuid": [UUID], "md5", "MD5 sum", "timestamp": xxxx}`。
+3. 删除磁盘目录中不是当前uuid的快照文件。
+4. 释放对paramters内存的锁定，停止保存检查点的线程。
+
+这里需要用户额外注意，在您的实际环境中，训练任务的运行可能会占满trainer和parameter server之间的网络带宽，如果parameter server此时还需要通过网络访问分布式存储以保存快照，可能会造成网络拥塞，而出现阶段性的运行停滞。
+
+### 从快照恢复
+
+在parameter server第一次启动或任意时间parameter server故障后被Kubernetes重新启动，则需要回滚到上一个检查点：
+
+  1. 从etcd中读取节点：`/checkpoints/[pserver_id]`获取最新的检查点的文件uuid
+  1. 从磁盘文件中加载uuid文件名的检查点快照文件，并加载其中的参数
+  1. 如果上面两步出现错误，则使用启动参数定义的初始化方法初始化参数
+  1. 开始提供服务
+
+## TODO List
+### 推测执行/加速执行（TODO）
+在异构集群中，如果存在某些trainer执行速度过慢会影响整体集群的速度（如图中Trainer 1），此时master将负责启动一个新的Trainer（Accelerate Trainer 2），使用同样的训练数据block。哪个trainer先完成block的训练，则把另一个慢速的kill掉。
+
+### 动态扩容/缩容
+目前只考虑动态扩容trainer数量，可以减小系统复杂性。
+
+## 术语
+* model: 指深度学习训练之后得到的所有参数，使用这个神经网络可以完成对新数据的预测
+* parameters: 神经网络中的参数，包括权重w和偏置b。一个神经网络的模型由大量的参数组成
+* shard: 分片，通常指将一个整体拆分成多份的其中的一份。
+* model shard: 将一个神经网络参数拆分成多份，每个shard分别存储在其中一台parameter server之上
+* parameter block: 多个parameter block构成一个model shard
+* 单点故障: 任意时刻只可能同时有一台服务器故障。由于集群中同时存在两台机器故障的概率极低（（平均故障率*平均故障修复时间）^2）只对特殊在线系统考虑两台以上同时故障的容灾。
diff --git a/doc_cn/_sources/design/cluster_train/data_dispatch.md.txt b/doc_cn/_sources/design/cluster_train/data_dispatch.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..1f5d22ff5e6abcb576d16cbe7391da1967a1ab8e
--- /dev/null
+++ b/doc_cn/_sources/design/cluster_train/data_dispatch.md.txt
@@ -0,0 +1,160 @@
+## 训练数据的存储和分发
+
+### 概念解释
+
+### 流程介绍
+生产环境中的训练数据集通常体积很大，并被存储在诸如Hadoop HDFS，Ceph，AWS S3之类的分布式存储之上。这些分布式存储服务通常会把数据切割成多个分片分布式的存储在多个节点之上。这样就可以在云端执行多种数据类计算任务，包括：
+
+* 数据预处理任务
+* Paddle训练任务
+* 在线模型预测服务
+
    
+
+
    
+
+在上图中显示了在一个实际生产环境中的应用（人脸识别）的数据流图。生产环境的日志数据会通过实时流的方式（Kafka）和离线数据的方式（HDFS）存储，并在集群中运行多个分布式数据处理任务，比如流式数据处理（online data process），离线批处理（offline data process）完成数据的预处理，提供给paddle作为训练数据。用户也可以上传labeled data到分布式存储补充训练数据。在paddle之上运行的深度学习训练输出的模型会提供给在线人脸识别的应用使用。
+
+### 训练数据存储
+我们选择[CephFS](http://docs.ceph.com/docs/master/cephfs/)作为存储系统。
+
+- 无论是从[PFSClient](../file_manager/README.md)的角度，还是从[Pod](https://kubernetes.io/docs/concepts/workloads/pods/pod/)中运行任务的角度，统一用`/pfs/$DATACENTER/home/$USER`来访问用户自己的数据。  
+- `/pfs/$DATACENTER/common`下存放公共数据集合
+	- 做只读挂载 
+
+
    
+
+
    
+
+### 文件预处理
+
+
+在开始训练之前, 数据集需要预先被转换成PaddlePaddle分布式训练使用的存储格[RecordIO](https://github.com/PaddlePaddle/Paddle/issues/1947)。我们提供两个转换方式：
+
+1. 用户在本地转换好再上传
+1. 用户上传数据后，在机群上运行转换程序
+
+转换生成的文件名会是以下格式：
+
+```text
+name_prefix-aaaaa-of-bbbbb
+```
+
+"aaaaa"和"bbbbb"都是五位的数字，每一个文件是数据集的一个shard，"aaaaa"代表shard的index，"bbbbb"代表这个shard的最大index。
+
+比如ImageNet这个数据集可能被分成1000个shard，它们的文件名是：
+```text
+imagenet-00000-of-00999
+imagenet-00001-of-00999
+...
+imagenet-00999-of-00999
+```
+
+#### 转换库
+
+无论是在本地或是云端转换，我们都提供Python的转换库，接口是：
+```python
+def convert(output_path, reader, num_shards, name_prefix)
+```
+
+- `output_path`: directory in which output files will be saved.
+- `reader`: a [data reader](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/reader/README.md#data-reader-interface), from which the convert program will read data instances.
+- `num_shards`: the number of shards that the dataset will be partitioned into.
+- `name_prefix`: the name prefix of generated files.
+
+`reader`每次输出一个data instance，这个instance可以是单个值，或者用tuple表示的多个值：
+
+```python
+yield 1 # 单个值
+yield numpy.random.uniform(-1, 1, size=28*28) # 单个值
+yield numpy.random.uniform(-1, 1, size=28*28), 0 # 多个值
+```
+
+每个值的类型可以是整形、浮点型数据、字符串，或者由它们组成的list，以及numpy.ndarray。如果是其它类型，会被Pickle序列化成字符串。
+
+### 示例程序
+
+#### 使用转换库
+
+以下`reader_creator`生成的`reader`每次输出一个data instance，每个data instance包涵两个值：numpy.ndarray类型的值和整型的值：
+```python
+def reader_creator():
+	def reader():
+		for i in range(1000):
+			yield numpy.random.uniform(-1, 1, size=28*28), 0 # 多个值
+	return reader
+```
+
+把`reader_creator`生成的`reader`传入`convert`函数即可完成转换：
+```python
+convert("./", reader_creator(), 100, random_images)
+```
+
+以上命令会在当前目录下生成100个文件：
+```text
+random_images-00000-of-00099
+random_images-00001-of-00099
+...
+random_images-00099-of-00099
+```
+
+#### 进行训练
+
+
+PaddlePaddle提供专用的[data reader creator](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/reader/README.md#python-data-reader-design-doc)，生成给定`RecordIO`文件对应的data reader。**无论在本地还是在云端，reader的使用方式都是一致的**：
+
+```python
+# ...
+reader = paddle.reader.creator.RecordIO("/pfs/datacenter_name/home/user_name/random_images-*-of-*")
+batch_reader = paddle.batch(paddle.dataset.mnist.train(), 128)
+trainer.train(batch_reader, ...)
+```
+
+以上代码的reader输出的data instance与生成数据集时，reader输出的data instance是一模一样的。
+
+### 上传训练文件
+
+使用下面命令，可以把本地的数据上传到存储集群中。
+
+```bash  
+paddle pfs cp filename /pfs/$DATACENTER/home/$USER/folder/
+```
+
+比如，把之前示例中转换完毕的random_images数据集上传到云端的`/home/`可以用以下指令：
+
+```bash  
+paddle pfs cp random_images-*-of-* /pfs/$DATACENTER/home/$USER/folder/
+```
+
+需要`$DATACENTER`的配置写到配置文件中，例如
+
+```
+# config file
+[datacenter_1]
+username=user
+usercert=user.pem
+userkey=user-key.pem
+endpoint=datacenter1.paddlepaddle.org
+
+[datacenter_2]
+username=user
+usercert=user.pem
+userkey=user-key.pem
+endpoint=datacenter2.paddlepaddle.org
+```
+## TODO
+### 文件访问的权限
+控制用户权限  
+
+- 用户可以把自己的数据分享给别人
+
+### 文件访问方式
+不用mount的方式来访问数据，而是直接用API的接口远程访问
+
+例如：  
+
+```
+f = open('/pfs/datacenter_name/home/user_name/test1.dat')
+```
+
+
+### 支持用户自定义的数据预处理job
diff --git a/doc_cn/_sources/design/cluster_train/large_model_dist_train.md.txt b/doc_cn/_sources/design/cluster_train/large_model_dist_train.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..0c4b5bc24c854b7062d509249bea9c50d42bd5f1
--- /dev/null
+++ b/doc_cn/_sources/design/cluster_train/large_model_dist_train.md.txt
@@ -0,0 +1,101 @@
+# Alalysis of large model distributed training in Paddle
+
+***NOTE: This is only some note for how we implemeted this scheme in V1, not a new design.***
+
+## What is it
+
+We often encounter cases that the embedding layer parameters(sparse) are so large that we can not store it in the trainer's memory when training. So we need to put them to several servers, and fetch them row by row instead of fetch all of the parameters.
+
+## How to use
+
+Specify command-line argument like  `--loadsave_parameters_in_pserver=true --ports_num_for_sparse=1  --use_old_updater=1` when starting the paddle trainer. And also add something like `--ports_num_for_sparse=1 --pserver_num_threads=5` when starting pserver processes.
+
+Accrodingly, configure your embedding layers like:
+
+```python
+SPARSE_REMOTE=True
+
+w1 = data_layer(name="w1", size=dict_size)
+emb1 = embedding_layer(input=w1, size=32, param_attr=ParameterAttribute(sparse_update=SPARSE_REMOTE))
+w2 = data_layer(name="w2", size=dict_size)
+emb2 = embedding_layer(input=w2, size=32, param_attr=ParameterAttribute(sparse_update=SPARSE_REMOTE))
+...
+```
+
+## Implementation details
+
+```c++
+enum MatType {
+  MAT_NORMAL,
+  MAT_NORMAL_SHARED,
+  MAT_VALUE_SHARED,
+  MAT_SPARSE_ROW_IDS,
+  MAT_SPARSE_ROW_AUTO_GROW,
+  MAT_CACHE_ROW,
+  MAT_SPARSE_ROW,
+  MAT_SPARSE_ROW_PREFETCH,
+  MAT_SPARSE_ROW_PREFETCH_FULL_SIZE,
+};
+```
+
+`MAT_SPARSE_ROW_PREFETCH` is what we use when configured to fetch only row of matrix when training.
+
+In `trainer_internal.cpp:L93 trainOneBatch`:
+
+```c++
+  if (config_->getOptConfig().use_sparse_remote_updater()) {
+    REGISTER_TIMER("prefetch");
+    gradientMachine_->prefetch(inArgs);
+    parameterUpdater_->getParametersRemote();
+  }
+```
+
+When doing actual network forward and backward, at the beginning of each batch, the trainer will try to download one row of data from pserver.
+
+In `trainer/RemoteParameterUpdater.cpp`: `parameterUpdater_->getParametersRemote();`:
+
+```c++
+if (fullSize) {
+    ...
+} else {
+getParams = [&] {
+    parameterClient_->getParameterSparse(
+        /* recvParameterType= */ PARAMETER_VALUE, sendBackParameterType);
+};
+applyL1 = [](Parameter& para, real decayRate) {
+    para.getMat(PARAMETER_VALUE)->applyL1(/*lr=*/1.0f, decayRate);
+};
+}
+```
+
+Calling `parameterClient_->getParameterSparse` will do remote call to pserver's `getParameterSparse`:
+
+```c++
+void ParameterServer2::getParameterSparse(const SendParameterRequest& request,
+                                          std::vector& inputBuffers,
+                                          SendParameterResponse* response,
+                                          std::vector* outputBuffers) {
+  (void)inputBuffers;
+  auto& buffer = *readWriteBuffer_;
+  size_t numReals = 0;
+  for (const auto& block : request.blocks()) {
+    numReals += getParameterConfig(block).dims(1);
+  }
+  buffer.resize(numReals);
+
+  VLOG(3) << "pserver: getParameterSparse, numReals=" << numReals;
+
+  ReadLockGuard guard(parameterMutex_);
+  size_t offset = 0;
+  for (const auto& block : request.blocks()) {
+    size_t width = getParameterConfig(block).dims(1);
+    Buffer buf = {buffer.data() + offset, width};
+    int type = request.send_back_parameter_type();
+    sendBackParameterSparse(block, type, response, &buf, width, outputBuffers);
+    offset += width;
+  }
+}
+```
+
+`getParameterConfig(block).dims(1)` returns the width of the current "parameter block"(a shard of parameter object),
+then `getParameterSparse` remote call returns only one row of data to the client.
diff --git a/doc_cn/_sources/design/cluster_train/master_server.md.txt b/doc_cn/_sources/design/cluster_train/master_server.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..4bf3c506f101361875043f8bfd97972b8c981a22
--- /dev/null
+++ b/doc_cn/_sources/design/cluster_train/master_server.md.txt
@@ -0,0 +1,91 @@
+# Design Doc: Master Server
+
+For an overview of master server's role, please refer to [distributed training design doc](./README.md). In this design doc we will discuss the master server in more details. The master will be implemented in [Go](https://golang.org/).
+
+## Dataset
+
+
+
+A dataset is a list of files in *RecordIO* format. A RecordIO file consists of chunks, whereas each chunk consists some records.
+
+## Task Queue
+
+As mentioned in [distributed training design doc](./README.md), a *task* is a data shard that the master server assigns to the trainer process to train on. A task consists of one or multiple *chunks* from one or multiple files. The master server maintains *task queues* to track the training progress.
+
+### Task Queue Creation
+
+1. Each trainer will make an RPC call (using Go's [rpc](https://golang.org/pkg/net/rpc/) package) to the master server, telling it the RecordIO files representing the dataset specified by the user. Since every trainer will tell the master server the same dataset, only the first RPC call will be honored.
+
+	The RPC interface is:
+	```go
+	func (m *RPCServer) ReportDataset(Paths []string, dummy *int) error {
+	}
+	```
+1. The master server will scan through each RecordIO file to generate the *chunk index* and know how many chunks does each file have. A chunk can be referenced by the file path and the index of the chunk within the file. The chunk index is in memory data structure that enables fast access to each chunk, and the index of the chunk with the file is an integer start from 0, representing the n-th chunk within the file.
+
+	The definition of the chunk is:
+	```go
+	type Chunk struct {
+		Idx   int // index of the chunk within the file
+		Path  string
+		Index recordio.Index // chunk index
+	}
+	```
+1. Chunks are grouped into tasks, and tasks are filled into the todo queue. The pending queue and the done queue are initialized with no element.
+
+	The definition of the task is:
+	```go
+	type Task struct {
+		Index  int
+		Chunks []Chunk
+	}
+	```
+
+	The elements in the tasks queues is of type `TaskEntry`, containing a timeout counter (described in [task retry logic](#task-retry-logic)), and a task:
+	```go
+	type TaskEntry struct {
+		NumTimeout int
+		Task       Task
+	}
+	```
+
+	The definition of task queues is:
+	```go
+	type TaskQueues struct {
+		Todo    []TaskEntry
+		Pending map[int]TaskEntry // map from task index to task entry
+		Done    []TaskEntry
+	}
+	```
+
+### Task Queue Persistence
+
+The task queues need to be persisted on [etcd](https://github.com/coreos/etcd) for fault recovery. Since the task queues only change once a task is completed or timed out, which is not very frequent, we can afford to synchronize with etcd every time the task queues change.
+
+We will serialize the task queues data structure with [gob encoding](https://golang.org/pkg/encoding/gob/), compress with gzip, and save into etcd synchronously under key `/task_queues`.
+
+### Task Dispatch
+
+The trainer will make an RPC call to master to get a new task when:
+
+- the trainer first started, or
+- the trainer finishes a task.
+
+The RPC interface is:
+```go
+func (m *RPCServer) GetTask(finished *Task, result *Task) error {
+}
+```
+Argument `finished` will be `nil` when the trainer is just started.
+
+During the RPC call the master will do the following:
+
+- Make a copy of the task queues, and update the copy reflecting the finished tasks and the new pending tasks.
+- Synchronize the copy of task queues with etcd using a transaction conditioned on holding the master lock.
+- Replace the task queues with the copy and report to the trainer with the new tasks if succeeded, or discard the copy and report the error to the trainer if failed.
+
+### Task Retry Logic
+
+When a task is dispatched to the trainer, the master will schedule a function for execution after the timeout duration (based on the moving average of task completion time). If the task entry in still in the pending queue, its timeout counter will increase by one, and the task will be moved to todo queue. If the timeout counter is above the threshold, the master will log the error and discard the task.
+
+Please note that since a timed out task could be completed after it has been dispatched for retry, so it is possible for a task to be processed multiple times. We do not try to prevent it from happening since it's fine to train on the same task multiple times due to the stochastic nature of the stochastic gradient decent algorithm.
diff --git a/doc_cn/_sources/design/cluster_train/pserver_client.md.txt b/doc_cn/_sources/design/cluster_train/pserver_client.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..474b8c572cd92fc87e9f7f3f2b19d12cccd158de
--- /dev/null
+++ b/doc_cn/_sources/design/cluster_train/pserver_client.md.txt
@@ -0,0 +1,171 @@
+# Design Doc: The Client Library of Parameter Server
+
+For an overview of trainer's role, please refer to [distributed training design doc](README.md). In this design doc, we will discuss the parameter server's client library, which will manage communication with parameter servers. The library will be implemented in [Go](https://golang.org/) and made available as a static or dynamic library with a C header file.
+
+## Parameter Partition
+
+Each parameter will be partitioned into parameter blocks to make the parameters evenly distributed on parameter servers. The partition is done automatically by the client library. The *sparse parameter* require a little different treatment:
+
+### Sparse Parameter
+
+The sparse parameter is a parameter that is updated sparsely. The name is somewhat misleading, it does not have a sparse representation, it has the same representation as a dense vector.
+
+Because a sparse parameter is updated sparsely, the trainer will have to partition the sparse parameter. Because the parameter server will merge all sparse parameter shard into the same file when saving the parameter. It needs special naming convention:
+
+If a sparse parameter is partitioned into n shards, they should be named as:
+
+```text
+name:sparse-0
+name:sparse-1
+...
+name:sparse-n-1
+```
+
+The library is unaware of the partition, and treat each parameter independently. Only when saving parameters, the parameter servers will merge the sparse parameters according to the naming convention.
+
+## Model Optimization Using Gradients
+
+There are two ways to perform model optimization using gradients:
+
+- On Client
+
+  The client does multiple steps of forward and backward update. In each step, the gradients are calculated and a new model is generated. After some steps, the client will calculate the difference between the newest model and the old model at step 0. The difference will be updated to parameter servers. Parameter servers will just update parameters using the difference without any optimization using gradients (such as Adam and L1 regularization).
+
+- On Parameter Server
+
+  The client will send accumulated gradients to parameter servers, the parameter server will do the optimization using gradients.
+
+## L1 and L2 Regularization
+
+PaddlePaddle allows L1 or L2 regularizations to be specified per parameter, so when the trainer initializes the parameter it needs include a parameter configuration when L1 or L2 regularization is necessary.
+
+## Parameter Initialization
+
+The parameters on parameter servers need to be initialized. To provide maximum flexibility, the trainer will initialize the parameters. Only one trainer will do the initialization, the other trainers will wait for the completion of initialization and get the parameters from the parameter servers.
+
+### Trainer Selection
+
+To select the trainer for initialization, every trainer will try to get a distributed lock, whoever owns the lock will do the initialization. As illustrated below:
+
+
+
+### Trainer Selection Process
+
+The trainer select process is encapsulated in the C API function:
+```c
+int paddle_begin_init_params(paddle_pserver_client* client, const char* config_proto);
+```
+The selected trainer's call to `paddle_begin_init_params` will return with 1, and the other trainers' call to `paddle_begin_init_params` will return 0. `paddle_get_params` will be blocked until initialization is completed. As illustrated below:
+
+
+
+## C Interface
+
+```c
+typedef enum {
+  PADDLE_ELEMENT_TYPE_INT32   = 0,
+  PADDLE_ELEMENT_TYPE_UINT32  = 1,
+  PADDLE_ELEMENT_TYPE_INT64   = 2,
+  PADDLE_ELEMENT_TYPE_UINT64  = 3,
+  PADDLE_ELEMENT_TYPE_FLOAT32 = 4,
+  PADDLE_ELEMENT_TYPE_FLOAT64 = 5,
+} paddle_element_type;
+
+typedef struct {
+  char*               name;
+  paddle_element_type element_type;
+  unsigned char*      content;
+  int                 content_len;
+} paddle_parameter, paddle_gradient;
+
+typedef int paddle_pserver_client;
+
+/**
+ * @brief creates a pserver client that talks to etcd for coordination.
+ */
+paddle_pserver_client paddle_new_etcd_pserver_client(char* etcd_addr);
+
+/**
+ * @brief creates a pserver client given pserver addresses.
+ *
+ * @param pserver_addrs comma-separated pserver addresses.
+ * @param selected if current pserver client is selected to initialize all parameter servers.
+ */
+paddle_pserver_client paddle_new_pserver_client(char* pserver_addrs, int selected);
+void paddle_pserver_client_release(paddle_pserver_client c);
+
+/**
+ * @brief paddle_begin_init_params begins to initialize parameters on
+ * parameter servers.
+ *
+ * paddle_begin_init_params will be called from multiple trainers,
+ * only one trainer will be selected to initialize the parameters on
+ * parameter servers. Other trainers need to get the initialized
+ * parameters from parameter servers using @paddle_get_params.
+ *
+ * @return 1 if the trainer is selected to initialize parameter
+ * servers, otherwise 0.
+ */
+int paddle_begin_init_params(paddle_pserver_client client);
+
+/**
+ * @brief paddle_init_param initializes the parameter on parameter
+ * servers.
+ *
+ * @param param the parameter to initialize.
+ * @param param_config_proto the configuration for the parameter.
+ * @param config_len the length of param_config_proto
+ * @return 0 if successful, otherwise -1. On failure, the trainer
+ * needs to restart the entire initialization process (starting from
+ * @paddle_begin_init_param). Or simply exit the program and wait for
+ * the cluster management system to restart the trainer.
+ */
+int paddle_init_param(paddle_pserver_client client, paddle_parameter param, const unsigned char* param_config_proto, int config_len);
+
+/**
+ * @brief paddle_finish_init_params tells parameter servers client has
+ * sent all parameters to parameter servers as initialization.
+ *
+ * @return 0 if successful, otherwise -1. On failure, the trainer
+ * needs to restart the entire initialization process (starting from
+ * @paddle_begin_init_param). Or simply exit the program and wait for
+ * the cluster management system to restart the trainer.
+ */
+int paddle_finish_init_params(paddle_pserver_client client);
+
+/**
+ * @brief paddle_send_grads sends gradients to parameter servers for
+ * updating parameters.
+ *
+ * @param grads the array of gradients to send.
+ * @param len the length of the gradient array.
+ * @param learning_rate the learning rate for the gradients.
+ * @return 0 if successful, otherwise -1.
+ */
+int paddle_send_grads(paddle_pserver_client client, const paddle_gradient* grads, int len);
+
+/**
+ * @brief paddle_get_params gets parameters from parameter servers.
+ *
+ * paddle_get_params will block until parameters are initialized on
+ * the parameter servers.
+ *
+ * @param dst the destination array of parameter pointers to save to.
+ * The parameter pointer must be pre-popullated with required parameter name,
+ * and the content of parameter must be pre-allocated of the size of required
+ * parameter on pserver.
+ * @param len the length of the names array and the paddle_parameter
+ * array.
+ * @return 0 if successful, otherwise -1.
+ */
+int paddle_get_params(paddle_pserver_client client, paddle_parameter** dst, int len);
+
+/**
+ * @brief paddle_save_model indicates parameters to save the parameter
+ * to the given path
+ *
+ * @param path the path to save parameters.
+ * @return 0 if successful, otherwise -1.
+ */
+int paddle_save_model(paddle_pserver_client client, const char* path);
+```
diff --git a/doc_cn/_sources/design/cluster_train/remote_parameter_updater.md.txt b/doc_cn/_sources/design/cluster_train/remote_parameter_updater.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..6e8e5938455b869e0f3367794c41250340b37f77
--- /dev/null
+++ b/doc_cn/_sources/design/cluster_train/remote_parameter_updater.md.txt
@@ -0,0 +1,21 @@
+# Design Doc: Remote Parameter Updater for Cluster Train
+
+For an overview of distribute training, please refer to [distributed training design doc](README.md). In this design doc, we will discuss the parameter updater that will use parameter server cclient [The Client Library of Parameter Server Design Doc](pserver_client.md) to manage and update parameters.
+
+## Parameter Updater
+
+Parameter Updater is used by trainer to manage and update parameter, there are mainly two kind of parameter updater: local and remote, since this design is for cluster train, we will only discuss remote parameter updater here.
+
+### Remote Parameter Updater
+
+Remote Parameter Updater manage parameters through remote parameter server with the client that communicate with pserver([The Client Library of Parameter Server Design Doc](pserver_client.md))
+
+In PaddlePaddle Python V2 API, trainer is implemented in python, and the trainer will hold a instance of parameter updater and call it's functions directly. In this design, we will also expose the api of RemoteParameterUpdater to python with swig.
+
+#### Sparse Remote Parameter Updater
+
+Since we will only implement dense parameter management new, the mechanism for sparse parameter will be discussed in next stage.
+
+### Interface Design
+
+TBD
diff --git a/doc_cn/_sources/design/cluster_train/save_model.md.txt b/doc_cn/_sources/design/cluster_train/save_model.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..b755185c81ad617b9c85c47de0f5f65d2201c658
--- /dev/null
+++ b/doc_cn/_sources/design/cluster_train/save_model.md.txt
@@ -0,0 +1,111 @@
+# Design Doc: Save Model
+
+## Overview
+
+The model is the output of the training process. There are two
+ways from which user can obtain a model:
+
+- Save model triggered by user code: user code asks PaddlePaddle to
+  save a model.
+- Convert model from the checkpoint: model being converted from
+  pservers' periodic checkpoint. In this way, the user can cancel a
+  job at any time, and still have a relatively fresh model (we
+  checkpoint around every 5 minutes).
+
+### Trainer Saving Model vs. Pservers Saving Model
+
+Both trainers and pservers have access to the model. So the model can
+be saved from a trainer or pservers. We need to decide where the model
+is saved from.
+
+#### Dense Update vs. Sparse Update
+
+There are two types of model update methods: dense update and sparse
+update (when the model parameter is configured to be sparse).
+
+- Dense update
+
+  Every trainer has it's own full copy of the model. Every model
+  update will update the entire model.
+
+- Sparse update
+
+  The training input is sparse, and the trainer does not have the
+  entire model. It will only download the sub-model necessary related
+  to the input. When updating the model, only the sub-model related to
+  the training input is updated.
+
+
+#### Pservers Saving Model
+
+The benefit of letting pservers save model is they have the entire
+model all the time. However, since pservers are on different nodes, it
+requires a merging process to merge model shards into the same
+model. Thus requires the pservers to write models to a distributed
+filesystem, making the checkpoint shards visible to the merge program.
+
+#### Trainer Saving Model
+
+The benefit of letting one trainer to save the model is it does not
+require a distributed filesystem. And it's reusing the same save model
+logic when training locally - except when doing sparse update, the
+trainer needs to download the entire model during the saving process.
+
+#### Conclusion
+
+Given trainer saving model does not require a distributed filesystem,
+and is an intuitive extension to trainer saving model when training
+locally, we decide to let the trainer save the model when doing
+distributed training.
+
+
+### Convert Model from Checkpoint
+
+TODO
+
+
+## Timeline
+
+We first implement trainer save the model. Converting the latest
+snapshot to a model will be a TODO for future.
+
+
+## Trainer Save Model
+
+### Trainer Election
+
+One trainer will be elected as the one to save the model. When using
+etcd, trainer ID is a randomly generated UUID, the trainer will
+contact the master server requesting to save the model, and find out
+if itself is elected. When the master server is not used, unique
+trainer IDs will be given by the administrator, the trainer whose ID
+is "0" is elected to save the model.
+
+### Model Save Path
+
+Each trainer will be given the directory to save the model. The
+elected trainer will save the model to
+`given-directory/trainerID`. Since the trainer ID is unique, this
+would prevent concurrent save to the same file when multiple trainers
+are elected to save the model when split-brain problem happens.
+
+### What Happens When Model Is Saving
+
+It takes some time to save model, we need to define what will happen
+when save model is taking place.
+
+When doing dense update, the trainer uses the local model. Pservers
+does not need to pause model update.
+
+When doing sparse update. The trainer needs to download the entire
+model while saving. To get the most accurate model, the model update
+needs to be paused before the download starts and resumed after the
+download finishes. Otherwise, the trainer gets a model that is
+"polluted": some part of the model is old, some part of the model is
+new.
+
+It's unclear that the "polluted" model will be inferior due to the
+stochastic nature of deep learning, and pausing the model update will
+add more complexity to the system. Since supporting sparse update is a
+TODO item. We defer the evaluation of pause the model update or not
+during saving model to the future.
diff --git a/doc_cn/_sources/design/cluster_train/submit-job.md.txt b/doc_cn/_sources/design/cluster_train/submit-job.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..8377d5489dc64bd2fdc5bb4f7bc737e7b489000d
--- /dev/null
+++ b/doc_cn/_sources/design/cluster_train/submit-job.md.txt
@@ -0,0 +1,127 @@
+# Submit a Distributed Training Job
+
+The user can submit a distributed training job with Python code, rather than with a command-line interface.
+
+## Runtime Environment On Kubernetes
+
+For a distributed training job, there is two Docker image called *runtime Docker image* and *base Docker image*. The runtime Docker image is the Docker image that gets scheduled by Kubernetes to run during training. The base Docker image is for building the runtime Docker image.
+
+### Base Docker Image
+
+Usually, the base Docker image is PaddlePaddle product Docker image including paddle binary files and python package. And of course, users can specify any image name hosted on any docker registry which users have the access right.
+
+### Runtime Docker Image
+
+The trainer package which user upload and some Python dependencies are packaged into a runtime Docker image based on base Docker image.
+
+- Handle Python Dependencies
+
+  You need to provide requirements.txt file in your `trainer-package` folder. Example:
+
+  ```txt
+  pillow
+  protobuf==3.1.0
+  ```
+  More [details](https://pip.readthedocs.io/en/1.1/requirements.html) about requirements, an example project looks like:
+  ```bash
+    paddle_example
+      |-quick_start
+        |-trainer.py
+        |-dataset.py
+        |-requirements.txt
+  ```
+
+## Submit Distributed Training Job With Python Code
+
+
+- `paddle.job.dist_train()` will call the Job Server API `/v1/packages` to upload the trainer package and save them on CephFS, and then call `/v1/trainer/job` to submit the PaddlePaddle distributed job.
+- `/v1/trainer/job` will start a building job for preparing the runtime Docker image. When the building job is finished, Job Server will submit the PaddlePaddle distributed job to Kubernetes.
+- *NOTE*: For the first version, we will not prepare the runtime Docker image, instead, the package is uploaded to Paddle Cloud, and Paddle Cloud will mount the package in a temporary folder into the base Docker image. We will not support custom Python dependencies in the first version as well.
+
+You can call `paddle.job.dist_train` and provide distributed training configuration as the parameters:
+```python
+paddle.job.dist_train(
+  trainer=dist_trainer(),
+  paddle_job=PaddleJob(
+    job_name = "paddle-cloud",
+    entry_point = "python %s"%__file__,
+    trainer_package = "/example/word2vec",
+    image = "yancey1989/paddle-job",
+    trainers = 10,
+    pservers = 3,
+    trainer_cpu = 1,
+    trainer_gpu = 1,
+    trainer_mem = "10G",
+    pserver_cpu = 1,
+    pserver_mem = "2G"
+  ))
+```
+
+The parameter `trainer` of `paddle.job.dist_train` is a function and you can implement it as follows:
+```python
+def dist_trainer():
+  def trainer_creator():
+    trainer = paddle.v2.trainer.SGD(...)
+    trainer.train(...)
+  return trainer_creator
+```
+
+The pseudo code of `paddle.job.dist_train` is as follows:
+```python
+def dist_train(trainer, paddle_job):
+  # if the code is running on cloud, set PADDLE_ON_CLOUD=YES
+  if os.getenv("RUNNING_ON_CLOUD", "NO") == "NO":
+    #submit the paddle job
+    paddle_job.submit()
+  else:
+    #start the training
+    trainer()
+```
+### PaddleJob Parameters
+parameter | type | explanation
+ --- | --- | ---
+job_name | str | the unique name for the training job
+entry_point | str | entry point for startup trainer process
+trainer_package | str | trainer package file path which user have the access right
+image|str|the [base image](#base-docker-image) for building the [runtime image](#runtime-docker-image)
+pservers|int| Parameter Server process count
+trainers|int| Trainer process count
+pserver_cpu|int| CPU count for each Parameter Server process
+pserver_mem|str| memory allocated for each Parameter Server process, a plain integer using one of these suffixes: E, P, T, G, M, K
+trainer_cpu|int| CPU count for each Trainer process
+trainer_mem|str| memory allocated for each Trainer process, a plain integer using one of these suffixes: E, P, T, G, M, K
+trainer_gpu|int| GPU count for each Trainer process, if you only want CPU, do not set this parameter
+
+### Deploy Parameter Server, Trainer and Master Process
+  - Deploy PaddlePaddle Parameter Server processes, it's a Kubernetes ReplicaSet.
+  - Deploy PaddlePaddle Trainer processes, it's a Kubernetes Job.
+  - Deploy PaddlePaddle Master processes, it's a Kubernetes ReplicaSet.
+
+## Job Server
+
+- RESTful API
+
+  Job server provides RESTful HTTP API for receiving the trainer package and displaying
+  PaddlePaddle job related informations.
+  - `POST   /v1/package` receive the trainer package and save them on CephFS
+  - `POST   /v1/trainer/job` submit a trainer job
+  - `GET    /v1/jobs/` list all jobs
+  - `GET    /v1/jobs/` the status of a job
+  - `DELETE /v1/jobs/` delete a job
+  - `GET    /v1/version` job server version
+
+- Build Runtime Docker Image on Kubernetes
+
+  `paddle.job.dist_train` will upload the trainer package to Job Server, save them on the distributed filesystem, and then start up a job for building the runtime Docker image that gets scheduled by Kubernetes to run during training.
+
+  There are some benefits for building runtime Docker image on JobServer:
+  - On Paddle Cloud, users will run the trainer code in a Jupyter Notebook which is a Kubernetes Pod, if we want to execute `docker build` in the Pod, we should mount the host's `docker.sock` to the Pod, user's code will connect the host's Docker Engine directly, it's not safe.
+  - Users only need to upload the training package files, does not need to install docker engine, docker registry as dependencies.
+  - If we want to change another image type, such as RKT, users do not need to care about it.
+
+- Deploy Parameter Server, Trainer and Master Processes
+
+  `POST /v1/trainer/job` receives the distributed training parameters, and deploy the job as follows:
+  - Deploy PaddlePaddle Parameter Server processes, it's a Kubernetes ReplicaSet.
+  - Deploy PaddlePaddle Trainer processes, it's a Kubernetes Job.
+  - Deploy PaddlePaddle Master processes, it's a Kubernetes ReplicaSet.
diff --git a/doc_cn/_sources/design/evaluator.md.txt b/doc_cn/_sources/design/evaluator.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..11cc129d56905a9ee666da92fbe6f8559c6d325a
--- /dev/null
+++ b/doc_cn/_sources/design/evaluator.md.txt
@@ -0,0 +1,58 @@
+## Evaluator Design
+
+### Problem Statement
+
+During training or inference, we provide an evaluation function to measure the model performance, for example, accuracy, precision, etc. In the operator based framework design, the data passes through the network pipeline batch by batch. As a result, inside the operator, we only calculate the metrics for one minibatch. Thus, we need to provide a mechanism to calculate the metrics for each N pass/batch the user wants.
+
+### Evaluator Design
+Currently, every operation is expressed in the graph. We divide the evaluator process into three steps.
+
+1. Initialize the metric state and add it into the block.
+
+2. Calculate the concerned metrics for every mini-batch. The single evaluator operator is only responsible for calculating the necessary statistics for one mini-batch. For example, the accuracy operator only calculates the accuracy for a minibatch data if run once.
+
+
+3. Merge the mini-batch statistics to form the evaluation result for multiple mini-batches. When it comes to distributed training/Multi-GPU training, aggregate the value from different devices.
+
+### Implementation
+This design is shown in the Python API. 
+Each metric operator needs to caculate the metric statistic and return the batch-aware states. Python side is responsible for accumulating the states for each pass. 
+
+    
+```python
+class Evaluator(object):
+    """
+    Evaluator Base class.
+    """
+    def __init__(self, name, **kwargs):
+       """
+       Different evaluator may has different metric states. E.g, Accuracy need two variables, total and right sample counts.
+       Auc need four variables, `true_positives`,
+         `true_negatives`, `false_positives` and `false_negatives`. So every evaluator should create its needed variables and append to main_program
+
+       The initialization of Evaluator should be responsible for:
+       create metric states and append to the main_program
+       """ 
+       pass
+
+    def _update_ops(self, input, label, **kwargs)
+       """
+       Add mini-batch evaluator caculate operators to the main_program.
+       Add increment operator to accumulate the metric states.
+       """
+    
+
+    def reset(self, executor, reset_program=None):
+      """
+      Reset metric states at the begin of each pass/user specified batch number.
+      Execute the reset_program to reset the states.
+      """
+      
+
+    def eval(self, executor, eval_program=None):
+      """
+      Merge the mini-batch statistics to form the evaluation result for multiple mini-batches.
+      Execute the eval_program and return the result.
+      """
+      return eval_result
+```
diff --git a/doc_cn/_sources/design/executor.md.txt b/doc_cn/_sources/design/executor.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..b5fb6c5c3c1da3c112ce63878322083dd5c42b70
--- /dev/null
+++ b/doc_cn/_sources/design/executor.md.txt
@@ -0,0 +1,23 @@
+# Executor Design Doc
+
+## Motivation
+
+We use executor to do the runtime evaluation of a `ProgramDesc`.
+
+## Overview
+
+An executor takes a `ProgramDesc`, a `block_id` and a `Scope`.  The `ProgramDesc` is a list of blocks and each block contains the protobuf definition of all the parameters and operators. The `block_id` specifies the entrance block. And the `Scope` is the container of all the variable instance, which is persistent throughout different runs.
+
+### What does executor do?
+
+It evaluates all the operators in the `block_id`th block of a `ProgramDesc`.
+
+### What does executor NOT do?
+
+It does not do runtime optimization, meaning intelligently parse the dependency of each op a choose which one to be run and in which order they should be run.
+
+It does not do graph partitioning, meaning dividing the `ProgramDesc` into several small pieces and executing them on different devices.
+
+## Implementation
+
+`Executor` evaluates a `ProgramDesc`. Essentially, it instantiates Variables and Operators, then run all the operators in sequence. [[code]](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/executor.cc)
diff --git a/doc_cn/_sources/design/file_manager/README.md.txt b/doc_cn/_sources/design/file_manager/README.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..3df10d801e568834729f902aace483d033340e2d
--- /dev/null
+++ b/doc_cn/_sources/design/file_manager/README.md.txt
@@ -0,0 +1,87 @@
+# FileManager设计文档
+## 目标
+在本文档中，我们设计说明了名为FileManager系统，方便用户上传自己的训练数据以进行分布式训练
+
+主要功能包括：
+
+- 提供常用的命令行管理命令管理文件和目录
+- 支持大文件的断点上传、下载  
+
+## 名词解释
+- PFS：是`Paddlepaddle cloud File System`的缩写，是对用户文件存储空间的抽象，与之相对的是local filesystem。目前我们用CephFS来搭建。
+- [CephFS](http://docs.ceph.com/docs/master/cephfs/)：一个POSIX兼容的文件系统。
+- Chunk：逻辑划上文件分块的单位。
+
+## 模块
+### 架构图
+
+
+### PFSClient
+- 功能： 详细设计[link](./pfs/pfsclient.md)
+	- 提供用户管理文件的命令
+	- 需要可以跨平台执行
+
+- 双向验证   
+	PFSClient需要和Ingress之间做双向验证[tls](#tls)，所以用户需要首先在`cloud.paddlepaddle.org`上注册一下，申请用户空间，并且把系统生成的CA(certificate authority)、Key、CRT(CA signed certificate)下载到本地，然后才能使用PFSClient。
+		
+### [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/)
+- 功能：  
+	提供七层协议的反向代理、基于粘性会话的负载均衡功能。
+	
+- 透传用户身份的办法  
+	Ingress需要把PFSClient的身份信息传给PFSServer，配置的方法参考[link](http://www.integralist.co.uk/posts/clientcertauth.html#3)
+
+### PFSServer
+PFSServer提供RESTful API接口，接收处理PFSClient端的文件管理请求，并且把结果返回PFSClient端。
+
+RESTful API
+
+- /api/v1/files
+	- `GET /api/v1/files`: Get metadata of files or directories.
+	- `POST /api/v1/files`: Create files or directories.
+	- `PATCH /api/v1/files`: Update files or directories.
+	- `DELETE /api/v1/files`: Delete files or directories.
+
+- /api/v1/file/chunks
+	- `GET /api/v1/storage/file/chunks`: Get chunks's metadata of a file.
+
+- /api/v1/storage/files
+	- `GET /api/v1/storage/files`: Download files or directories.
+	- `POST /api/v1/storage/files`: Upload files or directories.
+
+- /api/v1/storage/file/chunks
+	- `GET /api/v1/storage/file/chunks`: Download chunks's data.
+	- `POST /api/v1/storage/file/chunks`: Upload chunks's data.
+
+## 文件传输优化
+
+### 分块文件传输
+用户文件可能是比较大的，上传到Cloud或者下载到本地的时间可能比较长，而且在传输的过程中也可能出现网络不稳定的情况。为了应对以上的问题，我们提出了Chunk的概念，一个Chunk由所在的文件偏移、数据、数据长度及校验值组成。文件的上传和下载都是通过对Chunk的操作来实现的。由于Chunk比较小（默认256K），完成一个传输动作完成的时间也比较短，不容易出错。PFSClient需要在传输完毕最后一个Chunk的时候检查destination文件的MD5值是否和source文件一致。
+
+一个典型的Chunk如下所示：
+
+```
+type Chunk struct {
+	fileOffset int64
+	checksum uint32
+	len     uint32
+	data    []byte
+}
+```  
+
+### 生成sparse文件
+当destination文件不存在或者大小和source文件不一致时，可以用[Fallocate](https://Go.org/pkg/syscall/#Fallocate)生成sparse文件，然后就可以并发写入多个Chunk。
+
+### 覆盖不一致的部分
+文件传输的的关键在于需要PFSClient端对比source和destination的文件Chunks的checksum是否保持一致，不一致的由PFSClient下载或者传输Chunk完成。这样已经传输成功的部分就不用重新传输了。
+
+## 用户使用流程
+参考[link](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/cluster_train/data_dispatch.md)
+
+## 框架生成
+用[swagger](https://github.com/swagger-api/swagger-codegen)生成PFSClient和PFSServer的框架部分，以便我们可以把更多的精力放到逻辑本身上。
+
+## 参考文档
+- [TLS complete guide](https://github.com/k8sp/tls/blob/master/tls.md)
+- [aws.s3](http://docs.aws.amazon.com/cli/latest/reference/s3/)
+- [linux man document](https://linux.die.net/man/)
diff --git a/doc_cn/_sources/design/file_manager/pfs/pfsclient.md.txt b/doc_cn/_sources/design/file_manager/pfs/pfsclient.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..56bc70c54bbc92b78d66e04fb495b1300cf8ebe0
--- /dev/null
+++ b/doc_cn/_sources/design/file_manager/pfs/pfsclient.md.txt
@@ -0,0 +1,129 @@
+# PFSClient
+
+## Description
+The `pfs` command is a Command Line Interface to manage your files on PaddlePaddle Cloud
+
+## Synopsis
+```
+paddle [options] pfs  [parameters]
+```
+
+## Options
+```
+--profile (string)
+	Use a specific profile from your credential file.
+
+--help (string)
+	Display more information about command
+
+--version
+	Output version information and exit
+
+--debug
+	Show detailed debugging log	
+	
+--only-show-errors (boolean) 
+	Only errors and warnings are displayed. All other output is suppressed.
+```
+
+## Path Arguments
+When using a command, we need to specify path arguments. There are two path argument type: `localpath` and `pfspath`.  
+
+A `pfspath` begin with `/pfs`, eg: `/pfs/$DATACENTER/home/$USER/folder`.
+
+[Here](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/cluster_train/data_dispatch.md#上传训练文件) is how to config datacenters.
+
+## order of Path Arguments
+Commonly, if there are two path arguments, the first is the source, and the second is the destination.
+
+## Subcommonds
+- rm - remove files or directories
+
+```
+Synopsis:
+	rm [-r] [-v]  ...
+
+Options:
+	-r 
+		Remove directories and their contents recursively 
+	-v      
+		Cause rm to be verbose, showing files after they are removed.
+	
+Examples:
+	paddle pfs rm /pfs/$DATACENTER/home/$USER/file
+	paddle pfs rm -r /pfs/$DATACENTER/home/$USER/folder
+```
+- mv - move (rename) files
+
+```
+Synopsis:
+	mv [-f | -n] [-v]  
+	mv [-f | -n] [-v]  ... 
+	mv [-f | -n] [-v]   
+	mv [-f | -n] [-v]  ...  
+	mv [-f | -n] [-v]   
+	mv [-f | -n] [-v]  ...  
+	
+Options:
+	-f      
+		Do not prompt for confirmation before overwriting the destination path.  (The -f option overrides previous -n options.)
+	-n      
+		Do not overwrite an existing file.  (The -n option overrides previous -f options.)
+	-v      
+		Cause mv to be verbose, showing files after they are moved.
+		
+Examples:
+	paddle pfs mv ./text1.txt /pfs/$DATACENTER/home/$USER/text1.txt
+```
+- cp - copy files or directories
+
+```
+Synopsis:
+	cp [-r] [-f | -n] [-v] [--preserve--links]  
+	cp [-r] [-f | -n] [-v] [--preserve--links]  ... 
+	cp [-r] [-f | -n] [-v] [--preserve--links]   
+	cp [-r] [-f | -n] [-v] [--preserve--links]  ... 
+	cp [-r] [-f | -n] [-v] [--preserve--links]   
+	cp [-r] [-f | -n] [-v] [--preserve--links]  ... 
+
+Options:
+	-r
+   		Copy directories recursively
+   	-f      
+		Do not prompt for confirmation before overwriting the destination path.  (The -f option overrides previous -n options.)
+	-n      
+		Do not overwrite an existing file.  (The -n option overrides previous -f options.)
+	-v      
+		Cause cp to be verbose, showing files after they are copied.
+	--preserve--links
+	   Reserve links when copy links
+	   
+Examples:
+	paddle pfs cp ./file /pfs/$DATACENTER/home/$USER/file
+	paddle pfs cp /pfs/$DATACENTER/home/$USER/file ./file
+```
+- ls- list files
+
+```
+Synopsis:
+	ls [-r]  ...
+	
+Options:
+	-R
+   		List directory(ies) recursively
+
+Examples:
+	paddle pfs ls  /pfs/$DATACENTER/home/$USER/file
+	paddle pfs ls  /pfs/$DATACENTER/home/$USER/folder
+```
+
+- mkdir - mkdir directory(ies)
+Create intermediate directory(ies) as required.
+
+```
+Synopsis:
+	mkdir  ...
+
+Examples:
+	paddle pfs mkdir  /pfs/$DATACENTER/home/$USER/folder
+```
diff --git a/doc_cn/_sources/design/float16.md.txt b/doc_cn/_sources/design/float16.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..1ea95ed6b5d6792171569b6ff76d09be92fcb13e
--- /dev/null
+++ b/doc_cn/_sources/design/float16.md.txt
@@ -0,0 +1,105 @@
+# Design Doc: float16
+
+## Why float16
+Half precision (float16) is a binary floating-point format that occupies 16 bits in memory. float16 is half the size of traditional 32-bit single precision format (float) and has lower precision and smaller range. 
+
+When high precision computation is not required, using float16 data type could potentially 
+
+- reduce storage space, memory bandwidth, and power usages; 
+- increase the chance of data fitting into a smaller cache of lower latency; 
+- provide arithmetic speed up if supported by hardware. 
+
+## Survey of current float16 support
+A brief survey of float16 support on different compilers, hardwares, and libraries can be found below. Interested readers can refer to [link1](https://github.com/PaddlePaddle/Paddle/issues/4853) and [link2](https://github.com/Xreki/Xreki.github.io/blob/master/multi_data_types_in_dl_framework/ppt/float16_and_quantized_type.md) for more info.
+
+The goal of float16 is to serve as a key for the executor to find and run the correct version of compute method specialized for float16 in operator kernel. It should be compatible with various natively supported float16 implementations including `__half` for cuda, `float16_t` for ARM, and `Eigen::half` for Eigen to make writing customized float16 kernels easier. 
+
+### Compiler
+- nvcc supports `__half` data type after CUDA 7.5.
+- `__fp16` or `float16_t` is supported as storage type for gcc >= 6.1 and clang >= 3.4.
+- `__fp16` or `float16_t` is supported as arithmetic type for gcc >= 7.1 and clang >= 3.9.
+
+### Hardware
+- `__half` is supported on GPU with compute capability >= 5.3.
+- `__fp16` is supported as storage type for ARMv7-A, ARMv8-A, and above.
+- `__fp16` is supported as arithmetic type after ARMv8.2-A (currently, the only microarchitecture implementing ARMv8.2-A is ARM Cortex-A75, which is announced in May 2017. There seems to be no application processors currently available on market that adopts this architecture. It is reported that Qualcomm Snapdragon 845 uses Cortex-A75 design and will be available in mobile devices in early 2018).
+
+### Libraries
+- [Eigen](https://github.com/RLovelett/eigen) >= 3.3 supports float16 calculation on both GPU and CPU using the `Eigen::half` class. It is mostly useful for Nvidia GPUs because of the overloaded arithmetic operators using cuda intrinsics. It falls back to using software emulation on CPU for calculation and there is no special treatment to ARM processors.
+- [ARM compute library](https://github.com/ARM-software/ComputeLibrary) >= 17.02.01 supports NEON FP16 kernels (requires ARMv8.2-A CPU).
+
+### CUDA version issue
+There are currently three versions of CUDA that supports `__half` data type, namely, CUDA 7.5, 8.0, and 9.0. 
+CUDA 7.5 and 8.0 define `__half` as a simple struct that has a `uint16_t` data (see [`cuda_fp16.h`](https://github.com/ptillet/isaac/blob/9212ab5a3ddbe48f30ef373f9c1fb546804c7a8c/include/isaac/external/CUDA/cuda_fp16.h)) as follows:
+```
+typedef struct __align__(2) {
+   unsigned short x;
+} __half;
+
+typedef __half half;
+```
+This struct does not define any overloaded arithmetic operators. So you have to directly use `__hadd` instead of `+` to correctly add two half types:
+```
+__global__ void Add() {
+  half a, b, c;
+  c = __hadd(a, b); // correct
+  c = a + b; // compiler error: no operator "+" matches these operands
+}
+```
+CUDA 9.0 provides a major update to the half data type. The related code can be found in the updated [`cuda_fp16.h`](https://github.com/ptillet/isaac/blob/master/include/isaac/external/CUDA/cuda_fp16.h) and the newly added [`cuda_fp16.hpp`](https://github.com/ptillet/isaac/blob/master/include/isaac/external/CUDA/cuda_fp16.hpp).
+
+Essentially, CUDA 9.0 renames the original `__half` type in 7.5 and 8.0 as `__half_raw`, and defines a new `__half` class type that has constructors, conversion operators, and also provides overloaded arithmetic operators such as follows:
+```
+typedef struct __CUDA_ALIGN__(2) {
+    unsigned short x;
+} __half_raw;
+
+
+struct __CUDA_ALIGN__(2) __half {
+protected:
+    unsigned short __x;
+public:
+    // constructors and conversion operators from/to 
+    // __half_raw and other built-in data types
+}
+
+typedef __half half;
+
+__device__ __forceinline__ 
+__half operator+(const __half &lh, const __half &rh) { 
+    return __hadd(lh, rh); 
+}
+
+// Other overloaded operators
+``` 
+This new design makes `c = a + b` work correctly for CUDA half data type. 
+
+## Implementation
+The float16 class holds a 16-bit `uint16_t` data internally.
+```
+struct float16 {
+  uint16_t x;
+};
+``` 
+
+float16 supports the following features:
+  - constructors / assignment operators that take input from primitive data types including bool, integers of various length, float, and double. 
+  - constructors / assignment operators that take input from `__half` on cuda, `float16_t` on ARM, and `Eigen::half` on Eigen.
+  - conversion operators to primitive data types and half precision data types on cuda, ARM and Eigen. 
+  - overloaded arithmetic operators for cuda, arm, and non-arm cpu, respectively. These operators will take advantage of the cuda and ARM intrinsics on the corresponding hardware. 
+  
+To support the above features, two fundamental conversion functions are provided:
+```
+float16 float_to_half_rn(float f);  // convert to half precision in round-to-nearest-even mode
+float half_to_float(float16 h);
+```
+which provides one-to-one conversion between float32 and float16. These twos functions will do different conversion routines based on the current hardware. CUDA/ARM instrinsics will be used when the corresonding hardware is available. If the hardware or compiler level does not support float32 to float16 conversion, software emulation will be performed to do the conversion.
+
+## To do
+After float16 class is available, some of the future items are below:
+
+- Update pybind/tensor_py.h to bind c++ float16 with numpy float16. 
+
+- Modify `GetKernelType()` method in `framework/operator.h` to make it compatible with float16.
+
+- Create a type-casting operator that can convert the data type in tensor between float16 and other types.
diff --git a/doc_cn/_sources/design/functions_operators_layers.md.txt b/doc_cn/_sources/design/functions_operators_layers.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..984b59f4c6971dfb6f46dfe342f2751f392c0e88
--- /dev/null
+++ b/doc_cn/_sources/design/functions_operators_layers.md.txt
@@ -0,0 +1,100 @@
+# Design Doc: Functions, Operators, and Layers
+
+In a DL system, we can compose one or more fine grained operators into a coarse grained one.  For example, the FC layer can be composed of a multiplication operator and an add operator.
+
+Historically, some fine grained operations are known as operators, and some coarse level ones are known as layers.  But we need a well-defined separation.
+
+In general, operators are those very fine grained operations, e.g., mul and add. In the implementation, we can write them as C++ functions:
+
+```c++
+template  T add(T x, T y) { return x + y; }
+template  T mul(T x, T y) { return x * y; }
+```
+
+Then we can wrap them into operators which are C++ classes and can be created from Python bindings by name.  A C macro can do this. For example, the following macro invocation
+
+```c++
+#define MAKE_FUNCTION_OPERATOR(mul);
+```
+
+generates
+
+```c++
+template  class mulOp : public OperatorBase {...};
+REGISTER_OP(mulOp, "mul");
+```
+
+so that in Python we can create operator mul by:
+
+```python
+X1 = Var()
+X2 = Var()
+Y = Var()
+paddle.cpp.create_operator("mul", input=[X1, X2], output=Y)
+```
+
+Also, at the same time, we can compose a coarse level C++ operator class by composing functions `mul` and `add`:
+
+```c++
+template 
+class FCOp : public OperatorBase {
+ public:
+  void Run(...) {
+    add(mul(Input("X"), Input("W")), Input("b");
+  }
+};
+REGISTER_OP(FCOp, "fc");
+```
+
+We need to support such composition in Python as well.  To do so, we need a higher level Python wrapping of operator creation than `paddle.cpp.create_operator`.  This higher level operator API should be compatible with the layer API.
+
+Let's explain using an example.  Suppose that we are going to compose the FC using mul and add in Python, we'd like to have Python functions `mul` and `add` defined in module `operator`:
+
+```python
+def operator.mul(X1, X2):
+    O = Var()
+    paddle.cpp.create_operator("mul", input={X1, Y1}, output=O)
+    return O
+
+def operator.add(X1, X2):
+    O = Var()
+    paddle.cpp.create_operator("add", input={X1, X2}, output=O)
+    return O
+```
+
+Above code snippets are automatically generated.  Given them, users can define
+
+```python
+def layer.fc(X):
+    W = Var()
+    b = Var()
+    return operator.add(operator.mul(X, W), b)
+```
+
+If we don't have `operator.mul` and `operator.add`, the definiton of `layer.fc` would be complicated:
+
+```python
+def layer.fc(X):
+    W = Var()
+    b = Var()
+    O1 = Var()
+    paddle.cpp.create_operator("mul", input=[X, W], output=O1)
+    O2 = Var()
+    paddle.cpp.create_operator("add", input=[O1, b], output=O2)
+    return O2
+```
+
+We'd like to have Python bindings to operators in package `paddle.operator`, and Python compositions of operators in package `paddle.layer`.  So we have the following concepts in above illustrative example:
+
+
+| C++ functions/functors | mul          | add          |             |          |
+|------------------------|--------------|--------------|-------------|----------|
+| C++ operator class     | mulOp        | addOp        | FCOp        |          |
+| Python binding         | operator.mul | operator.add | operator.fc |          |
+| Python function        |              |              |             | layer.fc |
+
+
+This is how we differentiate layer and operators in PaddlePaddle:
+
+- those defined in C++ and have a lightweighted Python wrapper in module `operators` are operators; whereas
+- those who don't have C++ implementations but a Python implementation that compose C++ operators are known as layers.
diff --git a/doc_cn/_sources/design/gan_api.md.txt b/doc_cn/_sources/design/gan_api.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..fb41df8615f73d9fd4c32995eab265833eac1a55
--- /dev/null
+++ b/doc_cn/_sources/design/gan_api.md.txt
@@ -0,0 +1,253 @@
+# Design for GAN
+
+GAN (General Adversarial Net [https://arxiv.org/abs/1406.2661]) is an important model for unsupervised learning and widely used in many areas. 
+
+It applies several important concepts in machine learning system design, including building and running subgraphs, dependency tracing, different optimizers in one executor and so forth.
+
+In our GAN design, we wrap it as a user-friendly easily customized python API to design different models. We take the conditional DC-GAN (Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks [https://arxiv.org/abs/1511.06434]) as an example due to its good performance on image generation.
+
+
    
+
    
+Figure 1. The overall running logic of GAN. The black solid arrows indicate the forward pass; the green dashed arrows indicate the backward pass of generator training; the red dashed arrows indicate the backward pass of the discriminator training. The BP pass of the green (red) arrow should only update the parameters in the green (red) boxes. The diamonds indicate the data providers. d\_loss and g\_loss marked in red and green are the two targets we would like to run.
+
    
+
+The operators, layers and functions required/optional to build a GAN demo is summarized in https://github.com/PaddlePaddle/Paddle/issues/4563.
+
+
    
+
    
+Figure 2. Photo borrowed from the original DC-GAN paper.
+
    
+
+## The Conditional-GAN might be a class. 
+This design we adopt the popular open source design in https://github.com/carpedm20/DCGAN-tensorflow and https://github.com/rajathkmp/DCGAN. It contains following data structure:
+
+- DCGAN(object): which contains everything required to build a GAN model. It provides following member functions methods as API:
+
+- __init__(...): Initialize hyper-parameters (like conv dimension and so forth), and declare model parameters of discriminator and generator as well.
+
+- generator(z, y=None): Generate a fake image from input noise z. If the label y is provided, the conditional GAN model will be chosen.
+Returns a generated image.
+
+- discriminator(image):
+Given an image, decide if it is from a real source or a fake one. 
+Returns a 0/1 binary label.
+
+- build_model(self):
+build the whole GAN model, define training loss for both generator and discrimator.
+
+## Discussion on Engine Functions required to build GAN
+- Trace the tensor and variable dependency in the engine executor. (Very critical, otherwise GAN can'be be trained correctly)
+- Different optimizers responsible for optimizing different loss.
+
+To be more detailed, we introduce our design of DCGAN as following:
+
+### Class member Function: Initializer
+- Set up hyper-parameters, including condtional dimension, noise dimension, batch size and so forth.
+- Declare and define all the model variables. All the discriminator parameters are included in the list self.theta_D and all the generator parameters are included in the list self.theta_G.
+```python
+class DCGAN(object):
+  def __init__(self, y_dim=None):
+  
+    # hyper parameters  
+    self.y_dim = y_dim # conditional gan or not
+    self.batch_size = 100
+    self.z_dim = z_dim # input noise dimension
+
+    # define parameters of discriminators
+    self.D_W0 = pd.Variable(shape=[3,3, 1, 128], data=pd.gaussian_normal_randomizer())
+    self.D_b0 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.D_W1 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer())
+    self.D_b1 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.D_W2 = pd.Varialble(np.random.rand(128, 1))
+    self.D_b2 = pd.Variable(np.zeros(128))
+    self.theta_D = [self.D_W0, self.D_b0, self.D_W1, self.D_b1, self.D_W2, self.D_b2]
+
+    # define parameters of generators
+    self.G_W0 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer())
+    self.G_b0 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.G_W1 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer())
+    self.G_b1 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.G_W2 = pd.Varialble(np.random.rand(128, 1))
+    self.G_b2 = pd.Variable(np.zeros(128))
+    self.theta_G = [self.G_W0, self.G_b0, self.G_W1, self.G_b1, self.G_W2, self.G_b2]
+```
+
+### Class member Function: Generator
+- Given a noisy input z, returns a fake image.
+- Concatenation, batch-norm, FC operations required;
+- Deconv layer required, which is missing now...
+```python
+class DCGAN(object):
+  def generator(self, z, y = None):
+    # input z: the random noise
+    # input y: input data label (optional)
+    # output G_im: generated fake images
+    
+    if not self.y_dim:
+      z = pd.layer.concat(1, [z, y])
+      
+    G_h0 = pd.layer.fc(z, self.G_w0, self.G_b0)
+    G_h0_bn = pd.layer.batch_norm(G_h0)
+    G_h0_relu = pd.layer.relu(G_h0_bn)
+    
+    G_h1 = pd.layer.deconv(G_h0_relu, self.G_w1, self.G_b1)
+    G_h1_bn = pd.layer.batch_norm(G_h1)
+    G_h1_relu = pd.layer.relu(G_h1_bn)
+    
+    G_h2 = pd.layer.deconv(G_h1_relu, self.G_W2, self.G_b2))
+    G_im = pd.layer.tanh(G_im)
+    return G_im
+```
+
+### Class member function: Discriminator
+- Given a noisy input z, returns a fake image.
+- Concatenation, Convolution, batch-norm, FC, Leaky-ReLU operations required;
+```python
+class DCGAN(object):
+  def discriminator(self, image):
+    # input image: either generated images or real ones
+    # output D_h2: binary logit of the label
+
+    D_h0 = pd.layer.conv2d(image, w=self.D_w0, b=self.D_b0)
+    D_h0_bn = pd.layer.batchnorm(h0)
+    D_h0_relu = pd.layer.lrelu(h0_bn)
+    
+    D_h1 = pd.layer.conv2d(D_h0_relu, w=self.D_w1, b=self.D_b1)
+    D_h1_bn = pd.layer.batchnorm(D_h1)
+    D_h1_relu = pd.layer.lrelu(D_h1_bn)
+    
+    D_h2 = pd.layer.fc(D_h1_relu, w=self.D_w2, b=self.D_b2)
+    return D_h2
+```
+
+### Class member function: Build the model
+- Define data readers as placeholders to hold the data;
+- Build generator and discriminators;
+- Define two training losses for discriminator and generator, respectively. 
+If we have execution dependency engine to back-trace all tensors, the module building our GAN model will be like this:
+```python
+class DCGAN(object):
+  def build_model(self):
+    if self.y_dim:
+        self.y = pd.data(pd.float32, [self.batch_size, self.y_dim])
+    self.images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    self.faked_images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    self.z = pd.data(tf.float32, [None, self.z_size])
+    
+    # step 1: generate images by generator, classify real/fake images with discriminator
+    if self.y_dim: # if conditional GAN, includes label
+        self.G = self.generator(self.z, self.y)
+        self.D_t = self.discriminator(self.images)
+        # generated fake images
+        self.sampled = self.sampler(self.z, self.y)
+        self.D_f = self.discriminator(self.G)
+    else: # original version of GAN
+        self.G = self.generator(self.z)
+        self.D_t = self.discriminator(self.images)
+        # generate fake images
+        self.sampled = self.sampler(self.z)
+        self.D_f = self.discriminator(self.images)
+    
+    # step 2: define the two losses
+    self.d_loss_real = pd.reduce_mean(pd.cross_entropy(self.D_t, np.ones(self.batch_size))
+    self.d_loss_fake = pd.reduce_mean(pd.cross_entropy(self.D_f, np.zeros(self.batch_size))
+    self.d_loss = self.d_loss_real + self.d_loss_fake
+    
+    self.g_loss = pd.reduce_mean(pd.cross_entropy(self.D_f, np.ones(self.batch_szie))
+```
+
+If we do not have dependency engine but blocks, the module building our GAN model will be like this:
+```python
+class DCGAN(object):
+  def build_model(self, default_block):
+    # input data in the default block
+    if self.y_dim:
+        self.y = pd.data(pd.float32, [self.batch_size, self.y_dim])
+    self.images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    # self.faked_images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    self.z = pd.data(tf.float32, [None, self.z_size])
+
+    # step 1: generate images by generator, classify real/fake images with discriminator
+    with pd.default_block().g_block():
+      if self.y_dim: # if conditional GAN, includes label
+        self.G = self.generator(self.z, self.y)
+        self.D_g = self.discriminator(self.G, self.y)
+      else: # original version of GAN
+        self.G = self.generator(self.z)
+        self.D_g = self.discriminator(self.G, self.y)
+      self.g_loss = pd.reduce_mean(pd.cross_entropy(self.D_g, np.ones(self.batch_szie))
+    
+    with pd.default_block().d_block():
+      if self.y_dim: # if conditional GAN, includes label
+        self.D_t = self.discriminator(self.images, self.y)
+        self.D_f = self.discriminator(self.G, self.y)
+      else: # original version of GAN
+        self.D_t = self.discriminator(self.images)
+        self.D_f = self.discriminator(self.G)
+
+      # step 2: define the two losses
+      self.d_loss_real = pd.reduce_mean(pd.cross_entropy(self.D_t, np.ones(self.batch_size))
+      self.d_loss_fake = pd.reduce_mean(pd.cross_entropy(self.D_f, np.zeros(self.batch_size))
+      self.d_loss = self.d_loss_real + self.d_loss_fake
+```
+Some small confusion and problems with this design:
+- D\_g and D\_f are actually the same thing, but has to be written twice; i.e., if we want to run two sub-graphs conceptually, the same codes have to be written twice if they are shared by the graph.
+- Requires ability to create a block anytime, rather than in if-else or rnn only;
+
+## Main function for the demo:
+Generally, the user of GAN just need to the following things:
+- Define an object as DCGAN class;
+- Build the DCGAN model;
+- Specify two optimizers for two different losses with respect to different parameters.
+```python
+# pd for short, should be more concise.
+from paddle.v2 as pd
+import numpy as np
+import logging
+
+if __name__ == "__main__":
+    # dcgan class in the default graph/block
+    # if we use dependency engine as tensorflow
+    # the codes, will be slightly different like:
+    # dcgan = DCGAN()
+    # dcgan.build_model()
+    with pd.block() as def_block:
+      dcgan = DCGAN()
+      dcgan.build_model(def_block)
+
+    # load mnist data
+    data_X, data_y = self.load_mnist()
+    
+    # Two subgraphs required!!!
+    with pd.block().d_block():
+      d_optim = pd.train.Adam(lr = .001, beta= .1)
+      d_step = d_optim.minimize(dcgan.d_loss, dcgan.theta_D)
+    with pd.block.g_block():
+      g_optim = pd.train.Adam(lr = .001, beta= .1)
+      g_step = pd.minimize(dcgan.g_loss, dcgan.theta_G)
+
+    # executor
+    sess = pd.executor()
+    
+    # training
+    for epoch in xrange(10000):
+      for batch_id in range(N / batch_size):
+        idx = ...
+        # sample a batch
+        batch_im, batch_label = data_X[idx:idx+batch_size], data_y[idx:idx+batch_size]
+        # sample z
+        batch_z = np.random.uniform(-1., 1., [batch_size, z_dim])
+
+        if batch_id % 2 == 0:
+          sess.run(d_step, 
+                   feed_dict = {dcgan.images: batch_im,
+                                dcgan.y: batch_label,
+                                dcgan.z: batch_z})
+        else:
+          sess.run(g_step,
+                   feed_dict = {dcgan.z: batch_z})
+```
+
+# More thinking about dependency engine v.s. block design:
+- What if we just want to run an intermediate result? Do we need to run the whole block/graph?
+- Should we call eval() to get the fake images in the first stage? And then train the discriminator in the second stage?
diff --git a/doc_cn/_sources/design/graph.md.txt b/doc_cn/_sources/design/graph.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..7519a65df835a39fe14f6ef45530afff170191ff
--- /dev/null
+++ b/doc_cn/_sources/design/graph.md.txt
@@ -0,0 +1,70 @@
+# Design Doc: Computations as a Graph
+
+A primary goal of the refactorization of PaddlePaddle is a more flexible representation of deep learning computation, in particular, a graph of operators and variables, instead of sequences of layers as before.
+
+This document explains that the construction of a graph as three steps:
+
+- construct the forward part
+- construct the backward part
+- construct the optimization part
+
+## The Construction of a Graph
+
+Let us take the problem of image classification as a simple example.  The application program that trains the model looks like:
+
+```python
+x = layer.data("images")
+l = layer.data("label")
+y = layer.fc(x)
+cost = layer.mse(y, l)
+optimize(cost)
+train(cost, reader=mnist.train())
+```
+
+### Forward Part
+
+The first four lines of above program build the forward part of the graph.
+
+![](images/graph_construction_example_forward_only.png)
+
+In particular, the first line `x = layer.data("images")` creates variable x and a Feed operator that copies a column from the minibatch to x.  `y = layer.fc(x)` creates not only the FC operator and output variable y, but also two parameters, W and b, and the initialization operators.
+
+Initialization operators are kind of "run-once" operators -- the `Run` method increments a class data member counter so to run at most once.  By doing so, a parameter wouldn't be initialized repeatedly, say, in every minibatch.
+
+In this example, all operators are created as `OpDesc` protobuf messages, and all variables are `VarDesc`.  These protobuf messages are saved in a `BlockDesc` protobuf message.
+
+### Backward Part
+
+The fifth line `optimize(cost)` calls two functions, `ConstructBackwardGraph` and `ConstructOptimizationGraph`.
+
+`ConstructBackwardGraph` traverses the forward graph in the `BlockDesc` protobuf message and builds the backward part.
+
+![](images/graph_construction_example_forward_backward.png)
+
+According to the chain rule of gradient computation, `ConstructBackwardGraph` would
+
+1. create a gradient operator G for each operator F,
+1. make all inputs, outputs, and outputs' gradient of F as inputs of G,
+1. create gradients for all inputs of F, except for those who don't have gradients, like x and l, and
+1. make all these gradients as outputs of G.
+
+### Optimization Part
+
+For each parameter, like W and b created by `layer.fc`, marked as double circles in above graphs, `ConstructOptimizationGraph` creates an optimization operator to apply its gradient.  Here results in the complete graph:
+
+![](images/graph_construction_example_all.png)
+
+## Block and Graph
+
+The word block and graph are interchangable in the desgin of PaddlePaddle.  A [Block](https://github.com/PaddlePaddle/Paddle/pull/3708) is a metaphore of the code and local variables in a pair of curly braces in programming languages, where operators are like statements or instructions.  A graph of operators and variables is a representation of the block.
+
+A Block keeps operators in an array `BlockDesc::ops`
+
+```protobuf
+message BlockDesc {
+  repeated OpDesc ops = 1;
+  repeated VarDesc vars = 2;
+}
+```
+
+in the order that they appear in user programs, like the Python program at the beginning of this article.  We can imagine that in `ops`,  we have some forward operators, followed by some gradient operators, and then some optimization operators.
diff --git a/doc_cn/_sources/design/graph_survey.md.txt b/doc_cn/_sources/design/graph_survey.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..6c6db08f463ae0a2b94fc4546f123a1d7c151870
--- /dev/null
+++ b/doc_cn/_sources/design/graph_survey.md.txt
@@ -0,0 +1,232 @@
+## Survey on Graph
+
+Neural network framework often provides symbolic API for users to write network topology conveniently. This doc manily focus on symbolic API in most popular neural network frameworks, and try to find out how to parse symbolic configuration to a portable file, such as protobuf or json.
+
+### Mxnet
+
+The core concept of symbolic API is `Symbol`. Mxnet implements `Symbol` class in C++, and export to Python using C-API. Please refer to the comments in Mxnet:
+
+
+`Symbol` is help class used to represent the operator node in Graph.
+`Symbol` acts as an interface for building graphs from different components like Variable, Functor and Group. `Symbol` is also exported to python front-end (while Graph is not) to enable quick test and deployment. Conceptually, symbol is the final operation of a graph and thus including all the information required (the graph) to evaluate its output value.
+
+
+A simple network topology wrote by Symbol is as follows:
+
+```python
+def get_symbol(num_classes=10, **kwargs):
+    data = mx.symbol.Variable('data')
+    data = mx.symbol.Flatten(data=data)
+    fc1  = mx.symbol.FullyConnected(data = data, name='fc1', num_hidden=128)
+    act1 = mx.symbol.Activation(data = fc1, name='relu1', act_type="relu")
+    fc2  = mx.symbol.FullyConnected(data = act1, name = 'fc2', num_hidden = 64)
+    act2 = mx.symbol.Activation(data = fc2, name='relu2', act_type="relu")
+    fc3  = mx.symbol.FullyConnected(data = act2, name='fc3', num_hidden=num_classes)
+    mlp  = mx.symbol.SoftmaxOutput(data = fc3, name = 'softmax')
+    return mlp
+```
+
+
+
+Varible here is actually a Symbol. Every basic Symbol will correspond to one Node, and every Node has its own NodeAttr. There is a op field in NodeAttr class, when a Symbol represents Variable(often input data), the op field is null.
+
+Symbol contains a data member, std::vector outputs, and NodeEntry cantains a poniter to Node. We can follow the Node pointer to get all the Graph.
+
+And Symbol can be saved to a Json file.
+
+Here is a detailed example:
+
+```
+>>> import mxnet as mx
+>>> data = mx.symbol.Variable('data')
+>>> print data.debug_str()
+Variable:data
+
+>>> data = mx.symbol.Flatten(data=data)
+>>> print data.debug_str()
+Symbol Outputs:
+	output[0]=flatten0(0)
+Variable:data
+--------------------
+Op:Flatten, Name=flatten0
+Inputs:
+	arg[0]=data(0) version=0
+
+>>> fc1  = mx.symbol.FullyConnected(data = data, name='fc1', num_hidden=128)
+>>> print fc1.debug_str()
+Symbol Outputs:
+	output[0]=fc1(0)
+Variable:data
+--------------------
+Op:Flatten, Name=flatten0
+Inputs:
+	arg[0]=data(0) version=0
+Variable:fc1_weight
+Variable:fc1_bias
+--------------------
+Op:FullyConnected, Name=fc1
+Inputs:
+	arg[0]=flatten0(0)
+	arg[1]=fc1_weight(0) version=0
+	arg[2]=fc1_bias(0) version=0
+Attrs:
+	num_hidden=128
+
+```
+
+
+### TensorFlow
+
+
+The core concept of symbolic API is `Tensor`. Tensorflow defines `Tensor` in Python. Please refer to the comments in TensorFlow:
+
+A `Tensor` is a symbolic handle to one of the outputs of an `Operation`. It does not hold the values of that operation's output, but instead provides a means of computing those values in a TensorFlow [Session](https://www.tensorflow.org/api_docs/python/tf/Session).
+
+A simple example is as follows:
+
+```python
+  # Build a dataflow graph.
+  c = tf.constant([[1.0, 2.0], [3.0, 4.0]])
+  d = tf.constant([[1.0, 1.0], [0.0, 1.0]])
+  e = tf.matmul(c, d)
+
+  # Construct a `Session` to execute the graph.
+  sess = tf.Session()
+
+  # Execute the graph and store the value that `e` represents in `result`.
+  result = sess.run(e)
+```
+
+  
+The main method of `Tensor` is as follows: 
+ 
+ 
+```python
+@property
+def op(self):
+  """The `Operation` that produces this tensor as an output."""
+  return self._op
+
+@property
+def dtype(self):
+   """The `DType` of elements in this tensor."""
+  return self._dtype
+
+@property
+def graph(self):
+  """The `Graph` that contains this tensor."""
+  return self._op.graph
+
+@property
+def name(self):
+  """The string name of this tensor."""
+  if not self._op.name:
+    raise ValueError("Operation was not named: %s" % self._op)
+  return "%s:%d" % (self._op.name, self._value_index)
+
+@property
+def device(self):
+  """The name of the device on which this tensor will be produced, or None."""
+  return self._op.device
+```
+
+
+Tensor can be taken as target to run by session. Tensor contains all the information of Graph, and tracks data dependency.
+
+
+Here is a detailed example:
+
+
+```
+>>> import tensorflow as tf
+>>> c = tf.constant([[1.0, 2.0], [3.0, 4.0]])
+>>> print c.graph
+
+>>> d = tf.constant([[1.0, 1.0], [0.0, 1.0]])
+>>> print d.graph
+
+>>> e = tf.matmul(c, d)
+>>> print e.graph
+
+```
+
+### Dynet
+
+
+The core concept of symbolic API is `Expression`, and Dynet defines `Expression` class in C++.
+
+
+A simple example is as follows:
+
+```cpp
+ComputationGraph cg;
+Expression W = parameter(cg, pW);
+
+Expression in = input(cg, xs[i]);
+Expression label = input(cg, ys[i]);
+Expression pred = W * in;
+Expression loss = square(pred - label);
+```
+
+The input data and parameter are also represented by Expression. Every basci Expression corresponds to a Node. And input data is also a Node. 
+
+Expression has a data member ComputationGraph, and ComputationGraph will be modified in users' configuring process. Expression can be a running target, beacuse Expression contains all dependency.
+
+
+Here is a detailed example:
+
+write topology in C++
+
+```
+ComputationGraph cg;
+Expression W = parameter(cg, pW);
+cg.print_graphviz();
+
+Expression pred = W * xs[i];
+cg.print_graphviz();
+
+Expression loss = square(pred - ys[i]);
+cg.print_graphviz();
+```
+
+compile and print
+
+```
+# first print
+digraph G {
+  rankdir=LR;
+  nodesep=.05;
+  N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"];
+}
+# second print
+digraph G {
+  rankdir=LR;
+  nodesep=.05;
+  N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"];
+  N1 [label="v1 = v0 * -0.98"];
+  N0 -> N1;
+}
+# third print
+digraph G {
+  rankdir=LR;
+  nodesep=.05;
+  N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"];
+  N1 [label="v1 = v0 * -0.98"];
+  N0 -> N1;
+  N2 [label="v2 = -1.88387 - v1"];
+  N1 -> N2;
+  N3 [label="v3 = -v2"];
+  N2 -> N3;
+  N4 [label="v4 = square(v3)"];
+  N3 -> N4;
+}
+```
+
+### Conclusion
+
+
+Actually, Symbol/Tensor/Expression in Mxnet/TensorFlow/Dynet are the same level concepts. We use a unified name Expression here, this level concept has following features:
+
+- Users wirte topoloy with symbolic API, and all return value is Expression, including input data and parameter.
+- Expression corresponds with a global Graph, and Expression can also be composed.
+- Expression tracks all dependency and can be taken as a run target
diff --git a/doc_cn/_sources/design/if_else_op.md.txt b/doc_cn/_sources/design/if_else_op.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..26d140f06db4ecefa86be015eaa731ffddc6910c
--- /dev/null
+++ b/doc_cn/_sources/design/if_else_op.md.txt
@@ -0,0 +1,51 @@
+# The `IfElse` Operator
+
+PaddlePaddle's `IfElse` operator differs from TensorFlow's:
+
+- the TensorFlow version takes a scalar boolean value as the condition so that the whole mini-batch goes to either the true or the false branch, whereas
+- the PaddlePaddle version takes a vector of boolean value as the condition, and instances corresponding to true values go to the true branch, those corresponding to false values go to the false branch.
+
+## Example
+
+The following PaddlePaddle program shows the usage of the IfElse operator:
+
+```python
+import paddle as pd
+
+x = minibatch([10, 20, 30]) # shape=[None, 1]
+y = var(1) # shape=[1], value=1
+z = minibatch([10, 20, 30]) # shape=[None, 1]
+cond = larger_than(x, 15) # [false, true, true]
+
+ie = pd.ifelse()
+with ie.true_block():
+    d = pd.layer.add(x, y)
+    ie.output(d, pd.layer.softmax(d))
+with ie.false_block():
+    d = pd.layer.fc(z)
+    ie.output(d, d+1)
+o1, o2 = ie(cond)
+```
+
+A challenge to implement the `IfElse` operator is to infer those variables to be split, or, say, to identify the variable of the mini-batch or those derived from the mini-batch.
+
+An equivalent C++ program is as follows:
+
+```c++
+namespace pd = paddle;
+
+int x = 10;
+int y = 1;
+int z = 10;
+bool cond = false;
+int o1, o2;
+if (cond) {
+  int d = x + y;
+  o1 = z;
+  o2 = pd::layer::softmax(z);
+} else {
+  int d = pd::layer::fc(z);
+  o1 = d;
+  o2 = d+1;
+}
+```
diff --git a/doc_cn/_sources/design/infer_var_type.md.txt b/doc_cn/_sources/design/infer_var_type.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..d9d5397becba2ef1806d9341cd49cd9aabbf4a6a
--- /dev/null
+++ b/doc_cn/_sources/design/infer_var_type.md.txt
@@ -0,0 +1,78 @@
+# Design Doc: InferVarType
+
+## The Problem Posed
+
+The variable in our design can hold variant types. Such as `LoDTensor` and `SelectedRows`. An operator should be able to inference the variable types of its output.
+
+For example, a `lookup table` operator takes two `LoDTensor`; one is a float tensor as the embedding table, the other is an int tensor as word ID. The gradient operator of `lookup table` will generate a `SelectedRows` as its output. A `sum` operator can take both `LoDTensor` and `SelectedRows` as its inputs and will generate a `LoDTensor` if any of its inputs is `LoDTensor`, otherwise, the `sum` operator will generate `SelectedRows` as its output.
+
+The variable type will be constant at runtime. Every variable's type can either be set by the user (input data and parameter) or be inferred by the operator in compile time.
+
+## Proposed Solution
+
+The `InferVarType` is a compile-time function which is registered to each operator. The inferface of that function is:
+
+
+```c++
+using InferVarTypeFN = std::function<
+    void (const OpDescBind& /*op_desc*/, BlockDescBind* /*block*/)>;
+```
+
+It takes an operator description as its input and will write the output variable type and store them in block description.
+
+The `InferVarTypeFN` will be registered in `OpInfo`, to replace `infer_var_type_` field. The `OpInfo` should be
+
+```cpp
+struct OpInfo {
+  InferVarTypeFN infer_var_type_;
+  ...
+};
+```
+
+The default `InferVarType` will set output type as `LoDTensor`. It can be done by `GetInferVarType()`.
+
+```cpp
+void DefaultInferVarType(const OpDescBind& op_desc, BlockDescBind* block) {
+  // set the output type of variable as `LoDTensor`.
+  // ...
+}
+
+struct OpInfo {
+  InferVarTypeFN infer_var_type_;
+  InferVarTypeFN GetInferVarType() const {
+    if (infer_var_type_) {
+      return infer_var_type_;
+    } else {
+      return DefaultInferVarType;
+    }
+  }
+};
+```
+
+## Register InferVarType
+
+We provide a thin base class for registering an `InferVarTypeFN`. To use a base class will ease the implementation of registry since we can detect the registry entry is an `InferVarTypeFN` or not.
+
+```cpp
+class VarTypeInferer {
+public:
+  virtual void operator()(const OpDescBind& op_desc, BlockDescBind* block) const = 0;
+}
+```
+
+Operator developers can write the specialize `VarTypeInferer` as follow.
+
+```cpp
+class SpecialVarTypeInferer : public VarTypeInferer {
+public:
+  virtual void operator()(const OpDescBind& op_desc, BlockDescBind* block) const {
+    // .. own logic
+  }
+}
+```
+
+Then user can register the `InferVarType` just like `GradOpDescMaker` and `OpInfoMaker`.
+
+```
+REGISTER_OPERATOR(some_op, OpType, SpecialVarTypeInferer, ...);
+```
diff --git a/doc_cn/_sources/design/model_format.md.txt b/doc_cn/_sources/design/model_format.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..e29129fddf775939c9f7a8b49d850d523e6e5a45
--- /dev/null
+++ b/doc_cn/_sources/design/model_format.md.txt
@@ -0,0 +1,36 @@
+# Design Doc: Model Format
+
+## Motivation
+
+A model is an output of the training process. One complete model consists of two parts, the **topology** and the **parameters**. In order to support industrial deployment, the model format must be self-complete and must not expose any training source code.
+
+As a result, In PaddlePaddle, the **topology** is represented as a  [ProgramDesc](https://github.com/PaddlePaddle/Paddle/blob/1c0a4c901c9fc881d120249c703b15d1c50dae7d/doc/design/program.md), which describes the model structure. The **parameters** contain all the trainable weights in the model. We must support large size parameters and efficient serialization/deserialization of parameters. 
+
+## Implementation
+
+The topology is saved as a plain text in a detailed self-contain protobuf file. 
+
+The parameters are saved as a binary file. As we all know, the protobuf message has a limit of [64M size](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.io.coded_stream#CodedInputStream.SetTotalBytesLimit.details). We have done a [benchmark experiment](https://github.com/PaddlePaddle/Paddle/pull/4610), which shows that protobuf is not fit for the task.
+
+As a result, we design a particular format for tensor serialization. By default, an arbitrary tensor in Paddle is a [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md), and has a description information proto of [LoDTensorDesc](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L99). We save the DescProto as the byte string header. It contains all the necessary information, such as the `dims`, and the `LoD` information in [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/1c0a4c901c9fc881d120249c703b15d1c50dae7d/paddle/framework/lod_tensor.md). A tensor stores values in a continuous memory buffer. For speed we dump the raw memory to disk and save it as the byte string content. So, the binary format of one tensor is, 
+
+The table below shows a tensor's byte view in detail. Note that all the signed values are written in the little-endian format.
+
+|field name  | type | description |
+| --- | --- | --- |
+| version | uint32_t | Version of saved file. Always 0 now. |
+| tensor desc length | uint32_t | TensorDesc(Protobuf message) length in bytes. |
+| tensor desc | void* | TensorDesc protobuf binary message |
+| tensor data | void* | Tensor's data in binary format. The length of `tensor_data` is decided by `TensorDesc.dims()` and `TensorDesc.data_type()` |
+| lod_level | uint64_t | Level of LoD |
+| length of lod[0] | uint64_t | [Optional] length of lod[0] in bytes. |
+| data of lod[0] | uint64_t*  | [Optional] lod[0].data() |
+| ... | ... | ... |
+
+
+
+## Summary
+
+- We introduce a model format.
+- The model represented by its forward-pass computation procedure is saved in a **ProgramDesc** protobuf message.
+- A bunch of specified format binary tensors describe the **parameters**.
diff --git a/doc/_sources/design/multi_language_interface/why_plain_c.md.txt b/doc_cn/_sources/design/multi_language_interface/00.why_plain_c.md.txt
similarity index 94%
rename from doc/_sources/design/multi_language_interface/why_plain_c.md.txt
rename to doc_cn/_sources/design/multi_language_interface/00.why_plain_c.md.txt
index a3f41ca7b93de8a55d927c88812802ef12246182..a1443093342c5a3ed698fb6b52a751dfc7cb5319 100644
--- a/doc/_sources/design/multi_language_interface/why_plain_c.md.txt
+++ b/doc_cn/_sources/design/multi_language_interface/00.why_plain_c.md.txt
@@ -58,32 +58,32 @@ typedef void* paddle_matrix;
 typedef int paddle_error;
 
 extern "C"
-paddle_error paddle_matrix_shape(paddle_matrix matrix,
-                                 uint64_t* width,
-                                 uint64_t* height);
+paddle_error paddle_matrix_get_shape(paddle_matrix matrix,
+                                     uint64_t* width,
+                                     uint64_t* height);
 ```
 而在CPP里面实现这个C的接口，文件 `paddle_matrix.cpp`
 
 ```cpp
-#include "paddle/math/matrix.hpp"
+#include "paddle/math/matrix.h"
 extern "C"
 paddle_error paddle_matrix_shape(paddle_matrix matrix,
                                  uint64_t *width,
                                  uint64_t *height) {
-  auto m = (paddle::math::matrix*)(matrix);
+  auto m = (paddle::capi::CMatrix*)(matrix);
   *width = m->width();
   *height = m->height();
 }
 ```
 
-其中`paddle/math/matrix.hpp`文件内容为:
+其中`paddle/capi/CMatrix.hpp`文件内容为:
 
 ```cpp
 namespace paddle {
 namespace math {  
 
-class Matrix {
-  //...
+class CMatrix {
+  std::shared_ptr mat;
 };
 
 }  // namespace math
@@ -113,6 +113,6 @@ class Matrix {
 | 手写多语言绑定 | 不使用SWIG | 使用SWIG需要多语言绑定的开发人员熟练掌握SWIG配置，社区参与困难。SWIG生成的代码不能保证多语言代码风格的一致性 |
 
 
-## 简单实现
+## 实现
 
-TBD
+参考[Inference implementation](01.inference_implementation.md)
diff --git a/doc_cn/_sources/design/multi_language_interface/01.inference_implementation.md.txt b/doc_cn/_sources/design/multi_language_interface/01.inference_implementation.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..9820284523246a062581f322616d196f575c9d29
--- /dev/null
+++ b/doc_cn/_sources/design/multi_language_interface/01.inference_implementation.md.txt
@@ -0,0 +1,131 @@
+# C-API 模型推断实现文档
+
+本文档描述Paddle C-API的实现细节。Paddle C-API是多语言API的基础部分。Paddle需要暴露的API很多。先实现模型推断的API，通过模型推断API的实现作为一个样例，来进行讨论。至于为什么需要C-API，请参考[Why Plain C](./00.why_plain_c.md)。
+
+## Table of Contents
+   * [C-API 模型推断实现文档](#c-api-模型推断实现文档)
+      * [暴露接口原则](#暴露接口原则)
+      * [目录结构](#目录结构)
+      * [实现方式](#实现方式)
+         * [capi.h](#capih)
+         * [具体某种类型的头文件](#具体某种类型的头文件)
+         * [capi_private.h](#capi_privateh)
+         * [具体某种类型的实现文件](#具体某种类型的实现文件)
+         * [libpaddle_capi_shared.{so, dylib}](#libpaddle_capi_sharedso-dylib)
+         * [libpaddle_capi_whole.a](#libpaddle_capi_wholea)
+         * [examples](#examples)
+      * [编译选项](#编译选项)
+
+
+## 暴露接口原则
+
+1. 所有的接口均为C接口。即使用`extern "C"`
+2. 除构造某种类型的函数(`paddle_matrix_create`等)，其他函数均返回`paddle_error`。且调用时不能抛出异常或出现运行时错误。
+3. 所有类型名为`paddle_类型名`，所有与类型相关的函数，函数名为`paddle_类型名_函数名`
+4. 如果某一个Paddle Core概念(GradientMachine/Matrix)需要被暴露到其他语言，那么
+	* 为了暴露的接口尽量简单。只暴露概念的接口，而不暴露概念的实现。即暴露`GradientMachine`或者`Matrix`但不暴露`RecurrentGradientMachine`和`CpuSparseMatrix`。
+	* 暴露这个概念必要函数。`必要`是指，即完成某一个任务的最少函数。
+5. 不在`capi`接口层做过多封装。
+	* 如果某一个Paddle概念必须要暴露，但是又过于琐碎。不在`capi`这一层进行封装，而是直接修改Paddle Core。让Paddle核心中，这一概念不再琐碎。
+
+
+## 目录结构
+
+```text
+Paddle
+  `-- paddle
+        `-- capi
+              `-- examples  # The example project for C-API.
+              `-- tests  # unittests for C-API
+              `-- capi.h  # C-API header file.
+              `-- capi_private.h  # The shared header file between implementation sources.
+              `-- matrix.{h, cpp}
+              `-- gradient_machine.{h, cpp}
+              `-- ...
+```
+
+
+Paddle的C-API目录结构如上图表所示。这个目录中除了`capi_private.h`之外的所有头文件，均会被安装到include/paddle路径下。C-API生成的二进制文件会被安装到`lib`目录下。即，安装后的目录结构为
+
+```text
+`-- include
+      `-- paddle
+             `-- capi.h
+             `-- matrix.h
+             `-- gradient_machine.h
+             `-- ...
+`-- lib
+     `-- libpaddle_capi_shared.{so, dylib}  # In mac, dynamic libary's file name extention is `dylib`
+     `-- libpaddle_capi_whole.a  # static library for all symbols of Paddle.
+```
+
+## 实现方式
+
+下面分别介绍某一类文件的实现方式。
+
+### capi.h
+
+`capi.h`是用户使用C-API时所唯一需要引入的头文件。在`capi.h`中，引入了类型的头文件，`matrix.h`, `gradient_machine.h`。在引入其他类型的头文件时，使用相对路径的引用方式。即`#include "matrix.h"`
+
+### 具体某种类型的头文件
+
+具体某种类型的头文件，即例如`matrix.h`，`gradient_machine.h`等。在这些头文件中，包含了某种类型的类型定义和暴露的全部函数。
+
+这个头文件不假设其他文件的引用顺序，即使用户直接引用某种类型的头文件，也不应该报错(虽然不鼓励这样)。如果某一个类型需要引用另一个类型，例如`gradient_machine`需要引用`matrix`，则直接引入另一种类型的头文件，即`#include "matrix.h"`。
+
+### capi_private.h
+
+`capi_prviate.h`是各个实现中共享的头文件，他主要包含了实际暴露的类型结构。在用户使用C-API时，Paddle的类型全部退化成`void *`，即`typedef paddle_matrix void*`。但，对于每种C-API暴露的类型，均是在`capi_private.h`中实现的结构体。
+
+```cpp
+struct CMatrix {
+   int type = MatrixType;
+   std::shared_ptr mat;
+};
+```
+
+通常，这个结构体包含两个项目。
+
+* `type`是一个类型的标志。对于每种类型，type字段均不尽相同。这样，即使C-API接受的类型全是`void *`，我们也可以确定每一个参数的类型。
+
+  ```cpp
+  void some_c_api_function(void* some_instance) {
+     int* type = (int *) some_instance;
+     switch (*type) {
+       case MatrixType:
+         CMatrix* mat = (CMatrix *) some_instance;
+         ...
+       ...
+     }
+  }
+  ```
+* 这个结构体中的另一个项目是，Paddle Core中这一类型接口的智能指针(shared_ptr)。
+	* 使用智能指针的原因是: 用户可以安全的释放某个C-API的实例，而不必在意Paddle Core是否还在使用这个实例。
+	* 例如，用户通过C-API获得了神经网络的参数实例。当用户使用完这个参数后，直接删除这个参数即可。即便Paddle Core中的模型还在使用这个参数，这个参数也不会一并删除。
+
+### 具体某种类型的实现文件
+
+具体某种类型的实现文件，即`matrix.cpp`, `gradient_machine.cpp`等文件。在这些文件中，使用C++ 11实现了C-API的接口，并且使用`extern "C"`导出这些接口。在实现过程中，对输入参数的安全性进行了必要的判断，并将C-API接口的参数转发给`Paddle Core`。
+
+### libpaddle\_capi_shared.{so, dylib}
+
+`libpaddle_capi_shared`是C-API导出的动态库。这个动态库的连接参数与Paddle的其他二进制(例如`paddle_trainer`)类似。用户可以直接使用这个动态库来引入Paddle C-API。具体使用方法为`-lpaddle_capi_shared`。
+
+### libpaddle\_capi_whole.a
+
+`libpaddle_capi_whole`是C-API导出的静态库。这个静态库包含了Paddle的全部符号。他是将`libpaddle_gserver.a`, `libpaddle_math.a`, `libpaddle_capi.a`等全部静态库中的目标文件全部打包后产生的文件。具体使用方法为`--whole-archive -lpaddle_capi_whole --no-whole-archive`。
+
+
+### examples
+
+在样例中，使用`C99`开发了模型预测的样例代码。具体请参考[example/README.md](../../../paddle/capi/examples/README.md)。
+
+## 编译选项
+
+C-API的编译选项默认关闭，打开这个编译选项，需要在cmake的时候，设置
+
+```bash
+cmake ${YOUR_SOURCE_ROOT} -DWITH_C_API=ON -DWITH_PYTHON=OFF -DWITH_SWIG_PY=OFF
+```
+
+编译C-API的时候推荐Paddle不嵌入Python解释器，也不生成`SWIG`接口，具体原因参考[Why Plain C](./00.why_plain_c.md)。
diff --git a/doc_cn/_sources/design/ops/rnn.md.txt b/doc_cn/_sources/design/ops/rnn.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..2f4854793fa1f0b02e4dc17b51a48a972be61c06
--- /dev/null
+++ b/doc_cn/_sources/design/ops/rnn.md.txt
@@ -0,0 +1,153 @@
+# RNNOp design
+
+This document describes the RNN (Recurrent Neural Network) operator and how it is implemented in PaddlePaddle. The RNN op requires that all instances in a mini-batch have the same length. We will have a more flexible dynamic RNN operator in the future.
+
+## RNN Algorithm Implementation
+
+
    
+
+
    
+
+The above diagram shows an RNN unrolled into a full network.
+
+There are several important concepts here:
+
+- *step-net*: the sub-graph that runs at each step.
+- *memory*, $h_t$, the state of the current step.
+- *ex-memory*, $h_{t-1}$, the state of the previous step.
+- *initial memory value*, the memory of the first (initial) step.
+
+### Step-scope
+
+There could be local variables defined in each step-net.  PaddlePaddle runtime realizes these variables in *step-scopes* which are created for each step.
+
+
    
+
    
+Figure 2 illustrates the RNN's data flow
+
    
+
+Please be aware that every step runs the same step-net.  Each step does the following:
+
+1. Creates the step-scope.
+2. Initializes the local variables including step-outputs, in the step-scope.
+3. Runs the step-net, which uses the above mentioned variables.
+
+The RNN operator will compose its output from step outputs in each of the step scopes.
+
+### Memory and Ex-memory
+
+Let's give more details about memory and ex-memory using a simple example:
+
+$$
+h_t = U h_{t-1} + W x_t
+$$,
+
+where $h_t$ and $h_{t-1}$ are the memory and ex-memory (previous memory) of step $t$ respectively.
+
+In the implementation, we can make an ex-memory variable either "refer to" the memory variable of the previous step,
+or copy the memory value of the previous step to the current ex-memory variable.
+
+### Usage in Python
+
+For more information on Block, please refer to the [design doc](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/block.md).
+
+We can define an RNN's step-net using a Block:
+
+```python
+import paddle as pd
+
+X = some_op() # x is some operator's output and is a LoDTensor
+a = some_op()
+
+# declare parameters
+W = pd.Variable(shape=[20, 30])
+U = pd.Variable(shape=[20, 30])
+
+rnn = pd.create_rnn_op(output_num=1)
+with rnn.stepnet():
+    x = rnn.add_input(X)
+    # declare a memory (rnn's step)
+    h = rnn.add_memory(init=a)
+    # h.pre_state(), the previous memory of rnn
+    new_state = pd.add_two( pd.matmul(W, x) + pd.matmul(U, h.pre_state()))
+    # update current memory
+    h.update(new_state)
+    # indicate that h variables in all step scopes should be merged
+    rnn.add_outputs(h)
+
+out = rnn()
+```
+
+Python API functions in above example:
+
+- `rnn.add_input`: indicates that the parameter is a variable that will be segmented into step-inputs.
+- `rnn.add_memory`: creates a variable used as the memory.
+- `rnn.add_outputs`: marks the variables that will be concatenated across steps into the RNN output.
+
+### Nested RNN and LoDTensor
+
+An RNN whose step-net includes other RNN operators is known as an *nested RNN*.
+
+For example, we could have a 2-level RNN, where the top level corresponds to paragraphs, and the lower level corresponds to sentences. Each step of the higher level RNN also receives an input from the corresponding step of the lower level, and additionally the output from the previous time step at the same level.
+
+The following figure illustrates feeding in text into the lower level, one sentence at a step, and the feeding in step outputs to the top level. The final top level output is about the whole text.
+
+
    
+
+
    
+
+```python
+import paddle as pd
+
+W = pd.Variable(shape=[20, 30])
+U = pd.Variable(shape=[20, 30])
+
+W0 = pd.Variable(shape=[20, 30])
+U0 = pd.Variable(shape=[20, 30])
+
+# a is output of some op
+a = some_op()
+
+# chapter_data is a set of 128-dim word vectors
+# the first level of LoD is sentence
+# the second level of LoD is a chapter
+chapter_data = pd.Variable(shape=[None, 128], type=pd.lod_tensor, level=2)
+
+def lower_level_rnn(paragraph):
+    '''
+    x: the input
+    '''
+    rnn = pd.create_rnn_op(output_num=1)
+    with rnn.stepnet():
+        sentence = rnn.add_input(paragraph, level=0)
+        h = rnn.add_memory(shape=[20, 30])
+        h.update(
+            pd.matmul(W, sentence) + pd.matmul(U, h.pre_state()))
+        # get the last state as sentence's info
+        rnn.add_outputs(h)
+    return rnn
+
+top_level_rnn = pd.create_rnn_op(output_num=1)
+with top_level_rnn.stepnet():
+    paragraph_data = rnn.add_input(chapter_data, level=1)
+    low_rnn = lower_level_rnn(paragraph_data)
+    paragraph_out = low_rnn()
+
+    h = rnn.add_memory(init=a)
+    h.update(
+        pd.matmul(W0, paragraph_data) + pd.matmul(U0, h.pre_state()))
+    top_level_rnn.add_outputs(h)
+
+# output the last step
+chapter_out = top_level_rnn(output_all_steps=False)
+```
+
+In the above example, the construction of the `top_level_rnn` calls  `lower_level_rnn`.  The input is an LoD Tensor. The top level RNN segments input text data into paragraphs, and the lower level RNN segments each paragraph into sentences.
+
+By default, the `RNNOp` will concatenate the outputs from all the time steps.
+If the `output_all_steps` is set to False, it will only output the final time step.
+
+
+
    
+
+
    
diff --git a/doc_cn/_sources/design/ops/sequence_decoder.md.txt b/doc_cn/_sources/design/ops/sequence_decoder.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..9db5fb8e9a9f89b004bf71ddc064cd976c0d0bee
--- /dev/null
+++ b/doc_cn/_sources/design/ops/sequence_decoder.md.txt
@@ -0,0 +1,229 @@
+# Design: Sequence Decoder Generating LoDTensors
+In tasks such as machine translation and visual captioning,
+a [sequence decoder](https://github.com/PaddlePaddle/book/blob/develop/08.machine_translation/README.md) is necessary to generate sequences, one word at a time.
+
+This documentation describes how to implement the sequence decoder as an operator.
+
+## Beam Search based Decoder
+The [beam search algorithm](https://en.wikipedia.org/wiki/Beam_search) is necessary when generating sequences. It is a heuristic search algorithm that explores the paths by expanding the most promising node in a limited set.
+
+In the old version of PaddlePaddle, the C++ class `RecurrentGradientMachine` implements the general sequence decoder based on beam search, due to the complexity involved, the implementation relies on a lot of special data structures that are quite trivial and hard to be customized by users.
+
+There are a lot of heuristic tricks in the sequence generation tasks, so the flexibility of sequence decoder is very important to users.
+
+During the refactoring of PaddlePaddle, some new concepts are proposed such as:  [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md) and [TensorArray](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/tensor_array.md) that can better support the sequence usage, and they can also help make the implementation of beam search based sequence decoder **more transparent and modular** .
+
+For example, the RNN states, candidates IDs and probabilities of beam search can be represented all as `LoDTensors`;
+the selected candidate's IDs in each time step can be stored in a `TensorArray`, and `Packed` to the sentences translated.
+
+## Changing LoD's absolute offset to relative offsets
+The current `LoDTensor` is designed to store levels of variable-length sequences. It stores several arrays of integers where each represents a level.
+
+The integers in each level represent the begin and end (not inclusive) offset of a sequence **in the underlying tensor**,
+let's call this format the **absolute-offset LoD** for clarity.
+
+The relative-offset LoD can retrieve any sequence very quickly but fails to represent empty sequences, for example, a two-level LoD is as follows
+```python
+[[0, 3, 9]
+ [0, 2, 3, 3, 3, 9]]
+```
+The first level tells that there are two sequences:
+- the first's offset is `[0, 3)`
+- the second's offset is `[3, 9)`
+
+while on the second level, there are several empty sequences that both begin and end at `3`.
+It is impossible to tell how many empty second-level sequences exist in the first-level sequences.
+
+There are many scenarios that rely on empty sequence representation, for example in machine translation or visual captioning, one instance has no translation or the empty candidate set for a prefix.
+
+So let's introduce another format of LoD,
+it stores **the offsets of the lower level sequences** and is called **relative-offset** LoD.
+
+For example, to represent the same sequences of the above data
+
+```python
+[[0, 3, 6]
+ [0, 2, 3, 3, 3, 9]]
+```
+
+the first level represents that there are two sequences,
+their offsets in the second-level LoD is `[0, 3)` and `[3, 5)`.
+
+The second level is the same with the relative offset example because the lower level is a tensor.
+It is easy to find out the second sequence in the first-level LoD has two empty sequences.
+
+The following examples are based on relative-offset LoD.
+
+## Usage in a simple machine translation model
+Let's start from a simple machine translation model that is simplified from the [machine translation chapter](https://github.com/PaddlePaddle/book/tree/develop/08.machine_translation) to draw a blueprint of what a sequence decoder can do and how to use it.
+
+The model has an encoder that learns the semantic vector from a sequence, and a decoder which uses the sequence encoder to generate new sentences.
+
+**Encoder**
+```python
+import paddle as pd
+
+dict_size = 8000
+source_dict_size = dict_size
+target_dict_size = dict_size
+word_vector_dim = 128
+encoder_dim = 128
+decoder_dim = 128
+beam_size = 5
+max_length = 120
+
+# encoder
+src_word_id = pd.data(
+    name='source_language_word',
+    type=pd.data.integer_value_sequence(source_dict_dim))
+src_embedding = pd.embedding(size=source_dict_size, size=word_vector_dim)
+
+src_word_vec = pd.lookup(src_embedding, src_word_id)
+
+encoder_out_seq = pd.gru(input=src_word_vec, size=encoder_dim)
+
+encoder_ctx = pd.last_seq(encoder_out_seq)
+# encoder_ctx_proj is the learned semantic vector
+encoder_ctx_proj = pd.fc(
+    encoder_ctx, size=decoder_dim, act=pd.activation.Tanh(), bias=None)
+```
+
+**Decoder**
+
+```python
+def generate():
+    decoder = pd.while_loop()
+    with decoder.step():
+        decoder_mem = decoder.memory(init=encoder_ctx)  # mark the memory
+        generated_ids = decoder.memory() # TODO init to batch_size s
+        generated_scores = decoder.memory() # TODO init to batch_size 1s or 0s
+
+        target_word = pd.lookup(trg_embedding, gendrated_ids)
+        # expand encoder_ctx's batch to fit target_word's lod
+        # for example
+        # decoder_mem.lod is
+        # [[0 1 3],
+        #  [0 1 3 6]]
+        # its tensor content is [a1 a2 a3 a4 a5]
+        # which means there are 2 sentences to translate
+        #   - the first sentence has 1 translation prefixes, the offsets are [0, 1)
+        #   - the second sentence has 2 translation prefixes, the offsets are [1, 3) and [3, 6)
+        # the target_word.lod is
+        # [[0, 1, 6]
+        #  [0, 2, 4, 7, 9 12]]
+        # which means 2 sentences to translate, each has 1 and 5 prefixes
+        # the first prefix has 2 candidates
+        # the following has 2, 3, 2, 3 candidates
+        # the encoder_ctx_expanded's content will be
+        # [a1 a1 a2 a2 a3 a3 a3 a4 a4 a5 a5 a5]
+        encoder_ctx_expanded = pd.lod_expand(encoder_ctx, target_word)
+        decoder_input = pd.fc(
+            act=pd.activation.Linear(),
+            input=[target_word, encoder_ctx],
+            size=3 * decoder_dim)
+        gru_out, cur_mem = pd.gru_step(
+            decoder_input, mem=decoder_mem, size=decoder_dim)
+        scores = pd.fc(
+            gru_out,
+            size=trg_dic_size,
+            bias=None,
+            act=pd.activation.Softmax())
+        # K is an config
+        topk_scores, topk_ids = pd.top_k(scores, K)
+        topk_generated_scores = pd.add_scalar(topk_scores, generated_scores)
+
+        selected_ids, selected_generation_scores = decoder.beam_search(
+            topk_ids, topk_generated_scores)
+
+        # update the states
+        decoder_mem.update(cur_mem)  # tells how to update state
+        generated_ids.update(selected_ids)
+        generated_scores.update(selected_generation_scores)
+
+        decoder.output(selected_ids)
+        decoder.output(selected_generation_scores)
+
+translation_ids, translation_scores = decoder()
+```
+The `decoder.beam_search` is an operator that, given the candidates and the scores of translations including the candidates,
+returns the result of the beam search algorithm.
+
+In this way, users can customize anything on the input or output of beam search, for example:
+
+1. Make the corresponding elements in `topk_generated_scores` zero or some small values, beam_search will discard this candidate.
+2. Remove some specific candidate in `selected_ids`.
+3. Get the final `translation_ids`, remove the translation sequence in it.
+
+The implementation of sequence decoder can reuse the C++ class:  [RNNAlgorithm](https://github.com/Superjom/Paddle/blob/68cac3c0f8451fe62a4cdf156747d6dc0ee000b3/paddle/operators/dynamic_recurrent_op.h#L30),
+so the python syntax is quite similar to that of an  [RNN](https://github.com/Superjom/Paddle/blob/68cac3c0f8451fe62a4cdf156747d6dc0ee000b3/doc/design/block.md#blocks-with-for-and-rnnop).
+
+Both of them are two-level `LoDTensors`:
+
+- The first level represents `batch_size` of (source) sentences.
+- The second level represents the candidate ID sets for translation prefix.
+
+For example, 3 source sentences to translate, and has 2, 3, 1 candidates.
+
+Unlike an RNN, in sequence decoder, the previous state and the current state have different LoD and shape, and an `lod_expand` operator is used to expand the LoD of the previous state to fit the current state.
+
+For example, the previous state:
+
+* LoD is `[0, 1, 3][0, 2, 5, 6]`
+* content of tensor is `a1 a2 b1 b2 b3 c1`
+
+the current state is stored in `encoder_ctx_expanded`:
+
+* LoD is `[0, 2, 7][0 3 5 8 9 11 11]`
+* the content is
+  - a1 a1 a1 (a1 has 3 candidates, so the state should be copied 3 times for each candidates)
+  - a2 a2
+  - b1 b1 b1
+  - b2
+  - b3 b3
+  - None (c1 has 0 candidates, so c1 is dropped)
+
+The benefit from the relative offset LoD is that the empty candidate set can be represented naturally.
+
+The status in each time step can be stored in `TensorArray`, and `Pack`ed to a final LoDTensor. The corresponding syntax is:
+
+```python
+decoder.output(selected_ids)
+decoder.output(selected_generation_scores)
+```
+
+The `selected_ids` are the candidate ids for the prefixes, and will be `Packed` by `TensorArray` to a two-level `LoDTensor`, where the first level represents the source sequences and the second level represents generated sequences.
+
+Packing the `selected_scores` will get a `LoDTensor` that stores scores of each translation candidate.
+
+Packing the `selected_generation_scores` will get a `LoDTensor`, and each tail is the probability of the translation.
+
+## LoD and shape changes during decoding
+
    
+  
+
    
+
+According to the image above, the only phase that changes the LoD is beam search.
+
+## Beam search design
+The beam search algorithm will be implemented as one method of the sequence decoder and has 3 inputs:
+
+1. `topk_ids`, the top K candidate ids for each prefix.
+2. `topk_scores`, the corresponding scores for `topk_ids`
+3. `generated_scores`, the score of the prefixes.
+
+All of these are LoDTensors, so that the sequence affiliation is clear. Beam search will keep a beam for each prefix and select a smaller candidate set for each prefix.
+
+It will return three variables:
+
+1. `selected_ids`, the final candidate beam search function selected for the next step.
+2. `selected_scores`, the scores for the candidates.
+3. `generated_scores`, the updated scores for each prefix (with the new candidates appended).
+
+## Introducing the LoD-based `Pack` and `Unpack` methods in `TensorArray`
+The `selected_ids`, `selected_scores` and `generated_scores` are LoDTensors that exist at each time step,
+so it is natural to store them in arrays.
+
+Currently, PaddlePaddle has a module called `TensorArray` which can store an array of tensors. It is better to store the results of beam search in a `TensorArray`.
+
+The `Pack` and `UnPack` in `TensorArray` are used to pack tensors in the array to an `LoDTensor` or split the `LoDTensor` to an array of tensors.
+It needs some extensions to support the packing or unpacking an array of `LoDTensors`.
diff --git a/doc_cn/_sources/design/optimizer.md.txt b/doc_cn/_sources/design/optimizer.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..202b4b65103c0b7c536a9cb466c4120ce134d8c3
--- /dev/null
+++ b/doc_cn/_sources/design/optimizer.md.txt
@@ -0,0 +1,91 @@
+## Optimizer Design
+
+### The Problem
+
+A PaddlePaddle program, or a block, is a sequence of operators operating variables.  A training program needs to do three kinds of works:
+
+1. the forward pass, which computes intermediate results and the cost(s),
+1. the backward pass, which derives gradients from intermediate results and costs, and
+1. the optimization pass, which update model parameters to optimize the cost(s).
+
+These works rely on three kinds of operators:
+
+1. forward operators,
+1. gradient operators, and
+1. optimization operators.
+
+It's true that users should be able to create all these operators manually by calling some low-level API, but it would be much more convenient if they could only describe the forward pass and let PaddlePaddle create the backward and optimization operators automatically.
+
+In this design, we propose a high-level API that automatically derives the optimisation pass and operators from the forward pass.
+
+
+### High-level Python API to describe the training process
+
+1. User write code to describe the network:
+
+	```python
+	images = layer.data("images")
+	labels = layer.data("labels")
+	w1 = pd.var("w1")
+	b1 = pd.var("b1")
+	hidden = layer.fc(images, w=w1, b=b1)
+	cost = layer.mse(hidden, labels)
+	```
+
+	The above code snippet will create forward operators in [Block](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/block.md).
+
+
+2. Users create a certain kind of Optimizer with some argument.
+
+	```python
+	optimizer = AdagradOptimizer(learing_rate=0.001)
+	```
+
+3. Users use the optimizer to `minimize` a certain `cost` through updating parameters in parameter_list.
+
+	```python
+	opt_op_list = optimizer.minimize(cost, parameter_list=[w1, b1])
+	```
+	The above code snippet will create gradient and optimization operators in Block. The return value of `minimize()` is list of optimization operators that will be run by session.
+
+4. Users use Session/Executor to run this opt_op_list as target to do training.
+
+	```python
+	sess.run(target= opt_op_list, ...)
+	```
+
+#### Optimizer Python interface:
+
+```python
+class Optimizer(object):
+    """Optimizer Base class.
+
+    """
+
+    def __init__(self):
+        pass
+
+    def create_optimization_pass(self, parameters_and_grads):
+        """Add optimization operators to update gradients to variables.
+
+        Args:
+          parameters_and_grads: a list of (variable, gradient) pair to update.
+
+        Returns:
+          optmization_op_list: a list of optimization operator that will update parameter using gradient.
+        """
+        return None
+
+    def minimize(self, loss, parameter_list):
+        """Add operations to minimize `loss` by updating `parameter_list`.
+
+        This method combines interface `append_backward_ops()` and
+        `create_optimization_pass()` into one.
+        """
+        params_grads = self.create_backward_pass(loss, parameter_list)
+        update_ops = self.create_optimization_pass(params_grads)
+        return update_ops
+
+```
+
+Users can inherit the Optimizer above to create their own Optimizer with some special logic, such as AdagradOptimizer.
diff --git a/doc_cn/_sources/design/parameter_average.md.txt b/doc_cn/_sources/design/parameter_average.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..2c4edee9fe31d502ea62b9fe5c8757c0a4c5e79f
--- /dev/null
+++ b/doc_cn/_sources/design/parameter_average.md.txt
@@ -0,0 +1,72 @@
+# Averaging Parameter in PaddlePaddle
+
+## Why Averaging
+In a large scale machine learning setup where the size of the training data is huge, it could take us a large number of iterations over the training data before we can achieve the optimal values of parameters of our model. Looking at the problem setup, it is desirable if we can obtain the optimal values of parameters by going through the data in as few passes as we can.
+
+Polyak and Juditsky (1992) showed that the test performance of simple average of parameters obtained by Stochastic Gradient Descent (SGD) is as good as that of parameter values that are obtained by training the model over and over again, over the training dataset.
+
+Hence, to accelerate the speed of Stochastic Gradient Descent, Averaged Stochastic Gradient Descent (ASGD) was proposed in Polyak and Juditsky (1992). For ASGD, the running average of parameters obtained by SGD, is used as the estimator for 
     . The averaging is done as follows:
+
+
    
+
+We propose averaging for any optimizer similar to how ASGD performs it, as mentioned above.
+
+### How to perform Parameter Averaging in PaddlePaddle
+
+Parameter Averaging in PaddlePaddle works in the following way during training :
+1. It will take in an instance of a normal optimizer as an input, e.g. RMSPropOptimizer
+2. The optimizer itself is responsible for updating the parameters.
+3. The ParameterAverageOptimizer maintains a separate copy of the parameters for itself:
+    1. In concept, the values of this copy are the average of the values of the parameters in the most recent N batches.
+    2. However, saving all the N instances of the parameters in memory is not feasible.
+    3. Therefore, an approximation algorithm is used.
+
+Hence, overall we have have two copies of the parameters: one for the optimizer itself, and one for the ParameterAverageOptimizer. The former should be used in back propagation, while the latter should be used during testing and should be saved.
+
+During the testing/ saving the model phase, we perform the following steps:
+1. Perform the delayed operations.
+2. Save current values of the parameters to a temporary variable.
+3. Replace the values of the parameters with the averaged values.
+4. Perform testing and/or save the parameters.
+5. Restore the values of the parameters once done.
+
+### How to implement Averaging of Parameter in PaddlePaddle
+
+We can add the ParameterAverageOptimizer op to the graph through Python API. Using this approach, we manually add this op to the graph and direct the output of the optimizer op to this op during training.
+
+	**Advantages**:
+    - Allows for greater flexibility to the users of PaddlePaddle. Using this approach, the users can plug different optimizers into ParameterAverageOptimizer by passing in the optimizer to the op.
+    - Makes it easy for the users to customize and extend the framework.
+
+	**Disadvantages**:
+    - Implementation requires re-writing the averaging methodology in Python.  
+
+### Low-Level implementation
+
+In the new design, we propose to create a new operation for averaging parameter updates (ParameterAverageOptimizer). For now, we can add an op that takes in the following as input:
+- the optimizer
+- the window_size to keep the updates
+
+The ParameterAverageOptimizer op can be like any other operator with its own CPU/GPU implementation either using Eigen or separate CPU and GPU kernels. As the initial implementation, we can implement the kernel using Eigen following the abstraction pattern implemented for [Operators](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/rmsprop_op.h). We also want to support the case when the Trainer/Optimizer runs on the GPU while ParameterAverageOptimizer runs on a CPU.
+
+The idea of building an op for averaging is in sync with the refactored PaddlePaddle philosophy of using operators to represent any computation unit. The way the op will be added to the computation graph will be decided by the [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md#layer-function) in Python API.
+
+### Python API implementation for ParameterAverageOptimizer
+
+Based on Polyak and Juditsky (1992), we can generalize the averaging of updates to any optimizer. The input to the op would be the following:
+- Any optimizer (RMSProp , AdaGrad etc.)
+- A window size. The op keeps accumulating updated parameter values over a window of N batches and takes an average. Move the averaged value to a buffer when window is full to avoid loss of precision.
+
+Using the ParameterAverageOptimizer op, any user can add the operation to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support averaging. As per the PaddlePaddle [Python API design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md), the layer functions are responsible for creating operators, operator parameters and variables. Since ParameterAverageOptimizer will be an operator, it makes sense to create it in the layer functions.
+We will have a wrapper written in Python that will support the functionality and implement the actual core computation in C++ core as we have done for other [Optimizers](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/rmsprop_op.cc)
+
+#### Creation of the ParameterAverageOptimizer operator
+There are two ways for creating the ParameterAverageOptimizer op:
+1. We create the op immediately while building the computation graph.
+2. We add the op in a lazy manner, just before the backward pass, similar to the way the optimization ops are added.
+
+The proposal is to add the op immediately while building the computation graph.
+
+#### High-level API
+
+In PaddlePaddle Python API, users will primarily rely on [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md#layer-function) to create neural network layers. Hence, we also need to provide parameter average functionality in layer functions.
diff --git a/doc_cn/_sources/design/parameters_in_cpp.md.txt b/doc_cn/_sources/design/parameters_in_cpp.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..a7ac3f17c44ca94a669a8f1e283b291bceb42317
--- /dev/null
+++ b/doc_cn/_sources/design/parameters_in_cpp.md.txt
@@ -0,0 +1,41 @@
+# Design Doc: The C++ Class `Parameters`
+
+`Parameters` is a concept we designed in PaddlePaddle V2 API. `Parameters` is a container of parameters, which makes PaddlePaddle capable of  sharing parameter between topologies. We described usages of `Parameter` in [api.md](./api.md).
+
+We used Python to implement Parameters when designing V2 API before. There are several defects for the current implementation:
+* We just use `memcpy` to share Parameters between topologies, but this is very inefficient. 
+* We did not support sharing Parameters while training. We just trigger `memcpy` when start training.
+
+It is necessary that we implement Parameters in CPP side. However, it could result a code refactoring for PaddlePaddle, because PaddlePaddle was designed for training only one topology before, i.e., each GradientMachine contains its Parameter as a data member. In current PaddlePaddle implementation, there are three concepts associated with `Parameters`:
+
+1. `paddle::Parameter`. A `Parameters` is a container for `paddle::Parameter`.
+It is evident that we should use `paddle::Parameter` when developing `Parameters`.
+However, the `Parameter` class contains many functions and does not have a clear interface.
+It contains `create/store Parameter`, `serialize/deserialize`, `optimize(i.e SGD)`, `randomize/zero`.
+When we developing `Parameters`, we only use `create/store Parameter` functionality.
+We should extract functionalities of Parameter into many classes to clean PaddlePaddle CPP implementation.
+
+2. `paddle::GradientMachine` and its sub-classes, e.g., `paddle::MultiGradientMachine`, `paddle::NeuralNetwork`.
+We should pass `Parameters` to `paddle::GradientMachine` when `forward/backward` to avoid `memcpy` between topologies.
+Also, we should handle multi-GPU/CPU training, because `forward` and `backward` would perform on multi-GPUs and multi-CPUs.
+`Parameters` should dispatch the parameter value to each device, and gather the parameter gradient from each device.
+
+3. `paddle::ParameterUpdater`. The ParameterUpdater is used to update parameters in Paddle. 
+So `Parameters` should be used by `paddle::ParameterUpdater`, and `paddle::ParameterUpdater` should optimize `Parameters` (by SGD).
+
+
+The step by step approach for implementation Parameters in PaddlePaddle C++ core is listed below. Each step should be a PR and could be merged into PaddlePaddle one by one.
+
+1. Clean `paddle::Parameter` interface. Extract the functionalities of `paddle::Parameter` to prepare for the implementation of Parameters.
+
+2. Implementation a `Parameters` class. It just stores the `paddle::Parameter` inside. Make `GradientMachine` uses `Parameters` as a class member.
+
+3. Make `Parameters` support Multi-CPU and Multi-GPU training to prepare for sharing `Parameter` between topologies.
+Because we need share `Parameters` between topologies, it is `Parameters`'s response to exchange Parameters between GPUs.
+`GradientMachine` should not handle how to exchange Parameters because `GradientMachine` only used to train one topology and we need to support train many topologies in Paddle, i.e., there could be many GradientMachines use one `Parameters`.
+   * We should use a global function to exchange Parameters between GPUs, not a member function in `Parameters`. The `MultiGradientMachine` invoke this function, which uses `Parameters` as this function inputs.
+   * The MultiGradientMachine contains many functionalities. Extracting the Parameters exchanging logic could make MultiGradientMachine clearer and simpler.
+
+4. Make `Parameters` as an argument for `forward/backward` function, not a data member for `GradientMachine`. For example, `forward` could be `forward(const Parameters& params, ...)` and `backward` could be `backward(Parameters* params, ...)`. After this step, Paddle could share `Parameters` between topologies.
+
+5. `ParameterUpdater` is invoked by `GradientMachine` and `Trainer`, but it updates `Parameters`. In the end of this code refactoring, we could change `ParameterUpdater` directly uses `Parameters` to make `ParameterUpdater`'s implementation clear.
diff --git a/doc_cn/_sources/design/program.md.txt b/doc_cn/_sources/design/program.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..bd2456787c4e336d357a65255a8274a7c9e465cc
--- /dev/null
+++ b/doc_cn/_sources/design/program.md.txt
@@ -0,0 +1,139 @@
+# Design Doc: PaddlePaddle Programs
+
+## Compile and Execution
+
+A PaddlePaddle program consists of two parts -- the first generates a `ProgramDesc` protobuf message that describes the program, and the second runs this message using a C++ class `Executor`.
+
+A simple example PaddlePaddle program can be found in [graph.md](./graph.md):
+
+```python
+x = layer.data("images")
+l = layer.data("label")
+y = layer.fc(x)
+cost = layer.mse(y, l)
+optimize(cost)
+train(cost, reader=mnist.train())
+```
+
+The first five lines of the following PaddlePaddle program generates, or, compiles, the `ProgramDesc` message.  The last line runs it.
+
+## Programs and Blocks
+
+The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.
+
+- program: some nested blocks
+- [block](./block.md):
+  - some local variable definitions, and
+  - a sequence of operators
+
+The concept of block comes from usual programs.  For example, the following C++ program has three blocks:
+
+```c++
+int main() { // block 0
+  int i = 0;
+  if (i < 10) { // block 1
+    for (int j = 0; j < 10; j++) { // block 2
+    }
+  }
+  return 0;
+}
+```
+
+The following PaddlePaddle program has three blocks:
+
+```python
+import paddle as pd  // block 0
+
+x = minibatch([10, 20, 30]) # shape=[None, 1]
+y = var(1) # shape=[1], value=1
+z = minibatch([10, 20, 30]) # shape=[None, 1]
+cond = larger_than(x, 15) # [false, true, true]
+
+ie = pd.ifelse()
+with ie.true_block():  // block 1
+    d = pd.layer.add_scalar(x, y)
+    ie.output(d, pd.layer.softmax(d))
+with ie.false_block():  // block 2
+    d = pd.layer.fc(z)
+    ie.output(d, d+1)
+o1, o2 = ie(cond)
+```
+
+## `BlockDesc` and `ProgramDesc`
+
+All protobuf messages are defined in `framework.proto`.
+
+`BlockDesc` is straight-forward -- it includes local variable definitions, `vars`, and a sequence of operators, `ops`.
+
+```protobuf
+message BlockDesc {
+  required int32 parent = 1;
+  repeated VarDesc vars = 2;
+  repeated OpDesc ops = 3;
+}
+```
+
+The parent ID indicates the parent block so that operators in a block can refer to variables defined locally and also those defined in their ancestor blocks.
+
+All hierarchical blocks in a program are flattened and stored in an array. The block ID is the index of the block in this array.
+
+```protobuf
+message ProgramDesc {
+  repeated BlockDesc blocks = 1;
+}
+```
+
+
+### Global Block
+
+The global block is the first one in the above array.
+
+## Operators that Use Blocks
+
+In the above example, the operator `IfElseOp` has two blocks -- the true branch and the false branch.
+
+The definition of `OpDesc` shows that an operator could have some attributes:
+
+```protobuf
+message OpDesc {
+  AttrDesc attrs = 1;
+  ...
+}
+```
+
+and an attribute could be of type block, which is, in fact, a block ID as described above:
+
+```
+message AttrDesc {
+  required string name = 1;
+
+  enum AttrType {
+    INT = 1,
+    STRING = 2,
+    ...
+    BLOCK = ...
+  }
+  required AttrType type = 2;
+
+  optional int32 block = 10; // when type == BLOCK
+  ...
+}
+```
+
+## InferShape
+
+With this design, the InferShape function should take the following parameters:
+
+```c++
+void InferShape(int current_block,
+                int current_operator,
+                ProgramDesc* program // might change VarDesc values.
+                ) {
+  ...
+}
+```
+
+where
+
+- `current_block` indices into `ProgramDesc::blocks`,
+- `current_operator` indices into `BlockDesc::ops`.
diff --git a/doc_cn/_sources/design/prune.md.txt b/doc_cn/_sources/design/prune.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..4a5cf10c79a554779137f0cce5494fdd96ef6b7a
--- /dev/null
+++ b/doc_cn/_sources/design/prune.md.txt
@@ -0,0 +1,63 @@
+# Prune
+
+## Motivation
+
+We want to support running inference, training and checkpointing in one `ProgramDesc`. We implement 
+`void Prune(const ProgramDesc* input, ProgramDesc* output)` function, which takes a `ProgramDesc`
+and generate a pruned `ProgramDesc`.
+
+## Challenge
+
+Pruning need to support both variables and operators being evaluation targets. Consider the following
+different situations.
+
+```python
+# Case 1: run foward pass.
+cost_np = session.run(target=cost)
+# Case 2: run backward passing.
+opts_np, _ = session.run(target=[cost, opt])
+# Case 3: run checkpointing
+_ = session.run(target=checkpoint)
+```
+
+## Solution
+
+To support evaluation of operators, we add `is_target` field in the `OpDesc`.
+
+```c++
+message OpDesc {
+  required string type = 3;
+  repeated Var inputs = 1;
+  repeated Var outputs = 2;
+  repeated Attr attrs = 4;
+  optional bool is_target = 5 [ default = false ];
+};
+```
+
+To support evaluation of variables, we add [fetch_op](https://github.com/PaddlePaddle/Paddle/pull/4599).
+For each variable in the `target`, we insert a `fetch_op` into the `ProgramDesc` with `variable` being
+`fetch_op`'s input. Then we also set `fetch_op` is a target.
+
+### Algorithm
+
+If an operator needs to be run, it must fall into one of the following cases:
+
+1. It is the target.
+2. It is depended by some other ops, meaning its output is some other op's input.
+
+The first case can be checked by `op_desc.is_traget()` . The second case can be implement as
+
+```c++
+bool HasDependentVar(const OpDesc& op_desc, const std::set& dependent_vars) {
+  for (auto& var : op_desc.outputs()) {
+    for (auto& argu : var.arguments()) {
+      if (dependent_vars.count(argu) != 0) {
+        return true;
+      }
+    }
+  }
+  return false;
+}
+```
+
+Then the whole algorithm can be implemented as the following [code](https://github.com/tonyyang-svail/Paddle/blob/prune_impl/paddle/framework/prune.cc).
diff --git a/doc_cn/_sources/design/python_api.md.txt b/doc_cn/_sources/design/python_api.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..cb5fdc765b7126fc66a1c8978d4b96c0dc5a9f2c
--- /dev/null
+++ b/doc_cn/_sources/design/python_api.md.txt
@@ -0,0 +1,284 @@
+# Design Doc: Python API
+
+Due to the refactorization of the PaddlePaddle core, we need Python classes to construct corresponding protobuf messages that describe a DL program.
+
+| Python classes | Protobuf messages |
+| --- | --- |
+| Program | ProgramDesc |
+| Block | BlockDesc |
+| Operator | OpDesc |
+| Variable | VarDesc |
+
+Please be aware that these Python classes need to maintain some construction-time information, which are not part of the protobuf messages.
+
+## Core Concepts
+
+### Program
+
+A `ProgramDesc` describes a [DL program](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/program.md), which is composed of an array of `BlockDesc`s.  The `BlockDesc`s in a `ProgramDesc` can have a tree-like hierarchical structure. However, the `ProgramDesc` onlys stores a flattened array of `BlockDesc`s. A `BlockDesc` refers to its parent block by its index in the array.  For example, operators in the step block of an RNN operator need to be able to access variables in its ancestor blocks.
+
+Whenever we create a block, we need to set its parent block to the current block, hence the Python class `Program` needs to maintain a data member `current_block`.
+
+```python
+class Program(objects):
+    def __init__(self):
+        self.desc = core.NewProgram() # a C++ ProgramDesc pointer.
+        self.blocks = vector()
+        self.blocks.append(Block(self, -1)) # the global block
+        self.current_block = 0          # initialized to the global block
+
+    def global_block():
+        return self.blocks[0]
+
+    def current_block():
+        return self.get_block(self.current_block)
+
+    def rollback():
+        self.current_block = self.current_block().parent_idx
+
+    def create_block():
+        new_block_idx = len(self.block)
+        self.blocks.append(Block(self, self.current_block))
+        self.current_block = new_block_idx
+        return current_block()
+```
+
+`Program` is an accessor to the protobuf message `ProgramDesc`, which is created in C++ space, because the InferShape function is in C++, which manipulates `VarDesc` messages, which are in turn members of `BlockDesc`, which is a member of `ProgramDesc`.
+
+`Program` creates the first block as the global block in its constructor.  All parameters and their initializer operators are in the global block.
+
+### Block
+
+A [Block](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/block.md) includes
+
+1. a map from variable names to an instance of the Python `Variable` class, and
+1. a list of `Operator` instances.
+
+```python
+class Block(objects):
+    def __init__(self, program, parent_idx):
+        self.desc = core.NewBlock(program.desc)
+        self.program = program
+        self.vars = map()
+        self.ops = vector()
+        self.parent_idx = parent_idx
+
+    def create_var(self, ...):
+        return Variable(self, ...)
+
+    def _create_global_var(self, ...):
+        program.global_block().create_var(...)
+
+    def create_parameter(self, name, ...):
+        # Parameter is a subclass of variable. See Parameter section for details.
+        self.vars[name] = Parameter(self._create_global_var(...), ...)
+        return self.vars[name]
+
+    def append_operator(self, ...):
+        self.ops.append(Operator(self, ...))
+
+    def prepend_operator(self, ...): # Parameter's ctor prepands initialize operators.
+       self.ops.prepend(Operator(self, ...))
+```
+
+`create_parameter` is necessary because parameters are global variables, defined in the global block, but can be created in some sub-blocks. For example, an FC layer in the step block of an RNN operator.
+
+`prepend_operator` is necessary because the constructor of `Parameter` needs to create the initialize (or load) operator of the parameter, and would like to put it in the *preamble* of the global block.
+
+### Operator
+
+The `Operator` class fills in the `OpDesc` message and calls the C++ function `InferShape` to infer the output shapes from the input shapes.
+
+```python
+class Operator(object):
+    def __init__(self,
+                 block,  # Block
+                 type,   # string
+                 inputs, # dict
+                 outputs,# dict
+                 attrs   # dict
+                 ):
+        self.desc = core.NewOpDesc(block.desc, type, inputs, outputs, attrs)
+        core.infer_shape(self.desc, inputs, outputs)
+
+    def type(self):
+        return self.desc.type()
+```
+
+`Operator` creates the `OpDesc` message in C++ space, so that it can call the `InferShape` function, which is in C++.
+
+### Variable
+
+Operators take Variables as its inputs and outputs.
+
+```python
+class Variable(object):
+    def __init__(self,
+                 block=None,      # Block
+                 name=None,       # string
+                 shape,           # tuple
+                 dtype="float32", # string
+                 lod_level=None   # int
+                 ):
+        if name is None:
+            name = unique_name_generator()
+        self.name = name
+        self.block = block
+        self.desc = core.NewVarDesc(block.desc, name, shape, lod_level)
+        self.writer = None
+```
+
+Please be aware of `self.writer`, that tracks operator who creates the variable.  It possible that there are more than one operators who write a variable, but in Python space, each write to a variable is represented by a Variable class.  This is guaranteed by the fact that **`core.NewVarDesc` must NOT create a new `VarDesc` message if its name already exists in the specified block**.
+
+### Parameter
+
+A parameter is a global variable with an initializer (or load) operator.
+
+```python
+class Parameter(Variable):
+    def __init__(self,
+                 block=None,      # Block
+                 name=None,       # string
+                 shape,           # tuple
+                 dtype="float32", # string
+                 lod_level=None   # int
+                 trainable,       # bool
+                 initialize_op_attrs,
+                 optimize_op_attrs):
+        super(Parameter, self).__init__(block, name, shape, dtype, lod_level)
+        self.trainable = trainable
+        self.optimize_op_attrs = optimize_op_attrs
+        block.prepend(Operator(block,  # Block
+                               initialize_op_attrs['type'],   # string
+                               None,   # no inputs
+                               self,   # output is the parameter
+                               initialize_op_attrs)
+```
+
+When users create a parameter, they can call
+
+```python
+program.create_parameter(
+  ...,
+  init_attr={
+    type: "uniform_random",
+    min: -1.0,
+    max: 1.0,
+  })
+)
+```
+
+In above example, `init_attr.type` names an initialize operator.  It can also name the load operator
+
+```python
+init_attr={
+ type: "load",
+ filename: "something.numpy",
+}
+```
+
+`optimize_op_attrs` is not in the `VarDesc` message, but kept in the Python instance, as it will be used in the Python space when creating the optimize operator's `OpDesc`, and will be in the `OpDesc` message.
+
+## Layer Function
+
+A layer is a Python function that creates some operators and variables. Layers simplify the work of application programmers.
+
+Layer functions take `Variable` and configuration parameters as its input and return the output variable(s).
+
+For example, `FullyConnected` take one or more variable as its input. The input could be input data or another layer's output. There are many configuration options for a `FullyConnected` layer, such as layer size, activation, parameter names, initialization strategies of parameters, and so on. The `FullyConnected` layer will return an output variable.
+
+
+### Necessity for reusing code between layer functions
+
+There are a lot of code that can be reused. Such as
+
+* Give the default value of configuration. e.g., default initialize strategy for parameters is uniform random with `min = -1.0`, `max = 1.0`. and default initialize strategy for bias is to fill zero.
+* Append the activation operator.
+* Create a temporary variable.
+* Create parameter.
+* Generate a unique name.
+* Add a bias.
+* ...
+
+A mechanism to reuse code between layer functions is necessary. It will be around [150 lines of code](https://github.com/PaddlePaddle/Paddle/pull/4724/files#diff-823b27e07e93914ada859232ae23f846R12) if we write a `FullyConnected` layer without any helper functions.
+
+
+
+### Comparision between global functions and helper class
+
+The `FullyConnected` layer will be as follow when we provide global functions:
+
+```python
+def fc_layer(input, size, param_attr=None, bias_attr=None, act=None, name=None):
+  if name is None:
+    name = unique_name("fc")
+  input = multiple_input(input)
+  param_attr = default_param_attr(param_attr)
+  param_attr = multiple_param_attr(param_attr, len(input))
+
+  # mul
+  mul_results = []
+  for ipt, attr in zip(input, param_attr):
+    shape = ipt.shape[1:] + [size]
+    w = g_program.global_block().create_parameter(shape, ipt.dtype, name, attr)
+    tmp = create_tmp_var(name)
+    g_program.current_block().append_op("mul", {ipt, w}, {tmp})
+  mul_results.append(tmp)
+
+  # add sum
+  ...
+  # add bias
+  ...
+  # add activation
+  ...
+  return out
+```
+
+We can provide many helpers functions for layer developers. However, there are several disadvantages for global helper functions:
+
+1. We need a namespace for these methods, then layer developers can quickly figure out what method they can use.
+2. Global functions will force layer developers to pass its parameter time by time.
+
+So we provide a helper class, `LayerHelper`, to share code between layer functions. The `FullyConnected` Layer will be as follow.
+
+```python
+def fc_layer(input, size, param_attr=None, bias_attr=None, act=None, name=None):
+  helper = LayerHelper(locals())  # pass all parameter to LayerHelper
+
+  mul_results = []
+  for ipt, param in helper.iter_multiple_input_and_param():
+    w = helper.create_parameter(shape=ipt.shape[1:] + [size], dtype = ipt.dtype)
+    tmp = helper.create_tmp_variable()
+    helper.append_op('mul', {ipt, w}, {tmp})
+    mul_results.append(tmp)
+
+  pre_bias = helper.add_sum(mul_results)
+  pre_activation = helper.add_bias(pre_bias)
+  return helper.add_activation(pre_activation)
+```
+
+We not only use the fewer lines of code to write `fc_layer` but also make the code clearer to understand. At the same time, layer developers can figure out what function they can invoke by typing `helper.` in a python editor.
+
+
+### Implementation of layer helper
+
+We just keep all parameters of a layer function as a dictionary in layer helper as a private data member. Every method of layer helper will look up the dictionary after it is invoked. In that way, we can implement a layer helper for all layer functions even some layer does not contain some operator. For example, The `activation` is used by the FullyConnected layer or convolution layers, but a cross-entropy layer does not use it. The example code of `add_activation` are:
+
+```python
+class LayerHelper(object):
+  def __init__(self, **kwargs):  # kwargs is short for `keyword arguments`
+    self.kwargs = kwargs
+
+  def add_activation(self, input_var):
+    act = self.kwargs.get("act", None)  # default value is None
+    if act is None:  # do nothing if no act
+      return input_var
+
+    tmp = self.create_tmp_var(self)
+    self.append_op(type=act, input=input_var, output=tmp)
+    return tmp
+```
+
+## Optimizer
+
+[Optimizer Design Doc](./optimizer.md)
diff --git a/doc_cn/_sources/design/reader/README.md.txt b/doc_cn/_sources/design/reader/README.md.txt
index f21f7af520df5171798326818ecb97c3bcd14a12..2cd4b6225b61cf374458e40afabad7745f61ba71 100644
--- a/doc_cn/_sources/design/reader/README.md.txt
+++ b/doc_cn/_sources/design/reader/README.md.txt
@@ -1,25 +1,25 @@
 # Python Data Reader Design Doc
 
-At training and testing time, PaddlePaddle programs need to read data. To ease the users' work to write data reading code, we define that
+During the training and testing phases, PaddlePaddle programs need to read data. To help the users write code that performs reading input data, we define the following:
 
-- A *reader* is a function that reads data (from file, network, random number generator, etc) and yields data items.
-- A *reader creator* is a function that returns a reader function.
-- A *reader decorator* is a function, which accepts one or more readers, and returns a reader.
-- A *batch reader* is a function that reads data (from *reader*, file, network, random number generator, etc) and yields a batch of data items.
+- A *reader*: A function that reads data (from file, network, random number generator, etc) and yields the data items.
+- A *reader creator*: A function that returns a reader function.
+- A *reader decorator*: A function, which takes in one or more readers, and returns a reader.
+- A *batch reader*: A function that reads data (from *reader*, file, network, random number generator, etc) and yields a batch of data items.
 
-and provide function which converts reader to batch reader, frequently used reader creators and reader decorators.
+and also provide a function which can convert a reader to a batch reader, frequently used reader creators and reader decorators.
 
 ## Data Reader Interface
 
-Indeed, *data reader* doesn't have to be a function that reads and yields data items. It can be any function with no parameter that creates a iterable (anything can be used in `for x in iterable`):
+*Data reader* doesn't have to be a function that reads and yields data items. It can just be any function without any parameters that creates an iterable (anything can be used in `for x in iterable`) as follows:
 
 ```
 iterable = data_reader()
 ```
 
-Element produced from the iterable should be a **single** entry of data, **not** a mini batch. That entry of data could be a single item, or a tuple of items. Item should be of [supported type](http://www.paddlepaddle.org/doc/ui/data_provider/pydataprovider2.html?highlight=dense_vector#input-types) (e.g., numpy 1d array of float32, int, list of int)
+The item produced from the iterable should be a **single** entry of data and **not** a mini batch. The entry of data could be a single item or a tuple of items. Item should be of one of the [supported types](http://www.paddlepaddle.org/doc/ui/data_provider/pydataprovider2.html?highlight=dense_vector#input-types) (e.g., numpy 1d array of float32, int, list of int etc.)
 
-An example implementation for single item data reader creator:
+An example implementation for single item data reader creator is as follows:
 
 ```python
 def reader_creator_random_image(width, height):
@@ -29,7 +29,7 @@ def reader_creator_random_image(width, height):
     return reader
 ```
 
-An example implementation for multiple item data reader creator:
+An example implementation for multiple item data reader creator is as follows:
 ```python
 def reader_creator_random_image_and_label(width, height, label):
     def reader():
@@ -40,9 +40,10 @@ def reader_creator_random_image_and_label(width, height, label):
 
 ## Batch Reader Interface
 
-*batch reader* can be any function with no parameter that creates a iterable (anything can be used in `for x in iterable`). The output of the iterable should be a batch (list) of data items. Each item inside the list must be a tuple.
+*Batch reader* can be any function without any parameters that creates an iterable (anything can be used in `for x in iterable`). The output of the iterable should be a batch (list) of data items. Each item inside the list should be a tuple.
+
+Here are some valid outputs:
 
-Here are valid outputs:
 ```python
 # a mini batch of three data items. Each data item consist three columns of data, each of which is 1.
 [(1, 1, 1),
@@ -52,26 +53,28 @@ Here are valid outputs:
 # a mini batch of three data items, each data item is a list (single column).
 [([1,1,1],),
 ([2,2,2],),
-([3,3,3],),
+([3,3,3],)]
 ```
 
 Please note that each item inside the list must be a tuple, below is an invalid output:
 ```python
  # wrong, [1,1,1] needs to be inside a tuple: ([1,1,1],).
- # Otherwise it's ambiguous whether [1,1,1] means a single column of data [1, 1, 1],
- # or three column of datas, each of which is 1.
+ # Otherwise it is ambiguous whether [1,1,1] means a single column of data [1, 1, 1],
+ # or three columns of data, each of which is 1.
 [[1,1,1],
 [2,2,2],
 [3,3,3]]
 ```
 
-It's easy to convert from reader to batch reader:
+It is easy to convert from a reader to a batch reader:
+
 ```python
 mnist_train = paddle.dataset.mnist.train()
 mnist_train_batch_reader = paddle.batch(mnist_train, 128)
 ```
 
-Also easy to create custom batch reader:
+It is also straight forward to create a custom batch reader:
+
 ```python
 def custom_batch_reader():
     while True:
@@ -85,7 +88,8 @@ mnist_random_image_batch_reader = custom_batch_reader
 
 ## Usage
 
-batch reader, mapping from item(s) read to data layer, batch size and number of total pass will be passed into `paddle.train`:
+Following is how we can use the reader with PaddlePaddle:
+The batch reader, a mapping from item(s) to data layer, the batch size and the number of total passes will be passed into `paddle.train` as follows:
 
 ```python
 # two data layer is created:
@@ -99,13 +103,13 @@ paddle.train(batch_reader, {"image":0, "label":1}, 128, 10, ...)
 
 ## Data Reader Decorator
 
-*Data reader decorator* takes a single or multiple data reader, returns a new data reader. It is similar to a [python decorator](https://wiki.python.org/moin/PythonDecorators), but it does not use `@` syntax.
+The *Data reader decorator* takes in a single reader or multiple data readers and returns a new data reader. It is similar to a [python decorator](https://wiki.python.org/moin/PythonDecorators), but it does not use `@` in the syntax.
 
-Since we have a strict interface for data readers (no parameter, return a single data item). Data reader can be used flexiable via data reader decorators. Following are a few examples:
+Since we have a strict interface for data readers (no parameters and return a single data item), a data reader can be used in a flexible way using data reader decorators. Following are a few examples:
 
 ### Prefetch Data
 
-Since reading data may take time and training can not proceed without data. It is generally a good idea to prefetch data.
+Since reading data may take some time and training can not proceed without data, it is generally a good idea to prefetch the data.
 
 Use `paddle.reader.buffered` to prefetch data:
 
@@ -117,9 +121,9 @@ buffered_reader = paddle.reader.buffered(paddle.dataset.mnist.train(), 100)
 
 ### Compose Multiple Data Readers
 
-For example, we want to use a source of real images (reusing mnist dataset), and a source of random images as input for [Generative Adversarial Networks](https://arxiv.org/abs/1406.2661).
+For example, if we want to use a source of real images (say reusing mnist dataset), and a source of random images as input for [Generative Adversarial Networks](https://arxiv.org/abs/1406.2661).
 
-We can do:
+We can do the following :
 
 ```python
 def reader_creator_random_image(width, height):
@@ -139,13 +143,13 @@ false_reader = reader_creator_bool(False)
 
 reader = paddle.reader.compose(paddle.dataset.mnist.train(), data_reader_creator_random_image(20, 20), true_reader, false_reader)
 # Skipped 1 because paddle.dataset.mnist.train() produces two items per data entry.
-# And we don't care second item at this time.
+# And we don't care about the second item at this time.
 paddle.train(paddle.batch(reader, 128), {"true_image":0, "fake_image": 2, "true_label": 3, "false_label": 4}, ...)
 ```
 
 ### Shuffle
 
-Given shuffle buffer size `n`, `paddle.reader.shuffle` will return a data reader that buffers `n` data entries and shuffle them before a data entry is read.
+Given the shuffle buffer size `n`, `paddle.reader.shuffle` returns a data reader that buffers `n` data entries and shuffles them before a data entry is read.
 
 Example:
 ```python
@@ -154,21 +158,21 @@ reader = paddle.reader.shuffle(paddle.dataset.mnist.train(), 512)
 
 ## Q & A
 
-### Why reader return only a single entry, but not a mini batch?
+### Why does a reader return only a single entry, and not a mini batch?
 
-Always returning a single entry make reusing existing data readers much easier (e.g., if existing reader return not a single entry but 3 entries, training code will be more complex because it need to handle cases like batch size 2).
+Returning a single entry makes reusing existing data readers much easier (for example, if an existing reader returns 3 entries instead if a single entry, the training code will be more complicated because it need to handle cases like a batch size 2).
 
-We provide function `paddle.batch` to turn (single entry) reader into batch reader.
+We provide a function: `paddle.batch` to turn (a single entry) reader into a batch reader.
 
-### Why do we need batch reader, isn't train take reader and batch_size as arguments sufficient?
+### Why do we need a batch reader, isn't is sufficient to give the reader and batch_size as arguments during training ?
 
-In most of the case, train taking reader and batch_size as arguments would be sufficent. However sometimes user want to customize order of data entries inside a mini batch. Or even change batch size dynamically.
+In most of the cases, it would be sufficient to give the reader and batch_size as arguments to the train method. However sometimes the user wants to customize the order of data entries inside a mini batch, or even change the batch size dynamically. For these cases using a batch reader is very efficient and helpful.
 
-### Why use a dictionary but not a list to provide mapping?
+### Why use a dictionary instead of a list to provide mapping?
 
-We decided to use dictionary (`{"image":0, "label":1}`) instead of list (`["image", "label"]`) is because that user can easily resue item (e.g., using `{"image_a":0, "image_b":0, "label":1}`) or skip item (e.g., using `{"image_a":0, "label":2}`).
+Using a dictionary (`{"image":0, "label":1}`) instead of a list (`["image", "label"]`) gives the advantage that the user can easily reuse the items (e.g., using `{"image_a":0, "image_b":0, "label":1}`) or even skip an item (e.g., using `{"image_a":0, "label":2}`).
 
-### How to create custom data reader creator
+### How to create a custom data reader creator ?
 
 ```python
 def image_reader_creator(image_path, label_path, n):
@@ -192,7 +196,7 @@ paddle.train(paddle.batch(reader, 128), {"image":0, "label":1}, ...)
 
 ### How is `paddle.train` implemented
 
-An example implementation of paddle.train could be:
+An example implementation of paddle.train is:
 
 ```python
 def train(batch_reader, mapping, batch_size, total_pass):
diff --git a/doc_cn/_sources/design/refactor/distributed_architecture.md.txt b/doc_cn/_sources/design/refactor/distributed_architecture.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..d9fe7d6bbb0eeb73fcdca3ee749a4f10bcdda682
--- /dev/null
+++ b/doc_cn/_sources/design/refactor/distributed_architecture.md.txt
@@ -0,0 +1,144 @@
+# Design Doc: Distributed Training Architecture
+
+## Abstract
+
+PaddlePaddle version 0.10.0 uses the "trainer-parameter server" architecture. We run multiple instances of trainers (where each trainer runs the same model) and parameter servers for distributed training. This architecture serves well, but has few limitations:
+
+1. There is a need to write special code that handles tasks which should only be run on a single trainer. E.g., initializing the model, saving the model etc.
+
+2. Model parallelism is hard: It would need all the if-else branches conditioned on the trainer ID to partition the model onto the trainers, and eventually manually writing out the inter-model-shard communication code to communicate between different trainers.
+
+3. The user can not directly specify the parameter update rule: This would need to modify the parameter server code and compile a new binary. This makes things more complicated for researchers: A lot of extra effort is required to make this work. Besides, the training job submission program may not allow running arbitrary binaries.
+
+This design doc discusses PaddlePaddle's new distributed training architecture that addresses the above mentioned limitations.
+
+## Analysis
+
+The assumption is that the user writes the trainer program in either Python or C++.
+
+### Limitation 1
+
+There are two basic functionalities in the trainer program:
+
+1. The training logic such as loading / saving the model and printing out the logs.
+2. The neural network definition such as the definition of the data layer, the fully connected layer, the cost function and the
+  optimizer.
+
+When we train using PaddlePaddle v0.10.0 in a distributed fashion, multiple instances of the same Python code are run on different nodes, hence both: the
+training logic as well as the neural network computation logic, is replicated.
+
+The tasks that only need to be run once belong to the training logic. Hence if we only replicate the neural network computation part, and do **not**
+replicate the training logic, the limitation mentioned above can be avoided.
+
+### Limitation 2
+
+Model parallelism means that a single model is partitioned into different components and each node runs one of the component separately. This comes at the extra cost of managing the
+inter-model-shard communication between nodes.
+
+PaddlePaddle should ideally be able to modify the neural network computation and figure out the support for model parallelism automatically. However, the
+computation is only specified in Python code which sits outside of PaddlePaddle, hence PaddlePaddle can not support the feature in this setup.
+
+Similar to how a compiler uses an intermediate representation (IR) so that the programmer does not need to manually optimize their code for most of the cases, we can have an intermediate representation in PaddlePaddle as well. The compiler optimizes the IR as follows:
+
+
+
+PaddlePaddle can support model parallelism by converting the IR so that the user no longer needs to manually perform the computation and operations in the Python component:
+
+
+
+The IR for PaddlePaddle after refactoring is called a `Block`, it specifies the computation dependency graph and the variables used in the computation.
+
+### Limitation 3
+
+The user can not directly specify the parameter update rule for the parameter server in the Python module, since the parameter server does not use the same computation definition as the trainer. Instead, the update rule is baked inside the parameter server. The user can not specify the update rule explicitly.
+
+This could be fixed by making the parameter server run the same computation definition as the trainer (the user's Python module). For a detailed explanation, refer to this document -
+[Design Doc: Operation Graph Based Parameter Server](./parameter_server.md)
+
+## Distributed Training Architecture
+
+The revamped distributed training architecture can address the above discussed limitations. Below is the illustration of how it does so:
+
+
+
+The major components in the architecture are: *PaddlePaddle Python*, *PaddlePaddle converter* and *PaddlePaddle runtime*.
+
+### PaddlePaddle Python
+
+PaddlePaddle Python is the Python library that user's Python code invokes, to read the data. build the neural network topology, start training, etc.
+
+```Python
+paddle.init()
+input = paddle.op.recordIO("/home/data/mnist.recordio") # file stored on the cluster
+img, label = input[0], input[1]
+hidden = paddle.layer.fc(input=img, size=200, act=paddle.activation.Tanh())
+prediction = paddle.layer.fc(input=img, size=10, act=paddle.activation.Softmax())
+cost = paddle.layer.classification_cost(input=prediction, label=label)
+optimizer = paddle.optimizer.SGD(cost, learning_rate=0.01)
+session = paddle.session.NewRemote(num_trainer=3, num_ps=2, GPU_per_trainer=1)
+for i in range(1000):
+	_, cost_val = session.eval(targets=[cost, optimizer])
+	print cost_val
+```
+
+The above code is what a typical Python trainer code is, the neural network topology is built using the helper functions such as `paddle.layer.fc`. Training is done by calling `session.eval` iteratively.
+
+#### session.eval
+
+As shown in the graph, `session.eval` sends the IR and the evaluation inputs or targets to the PaddlePaddle cluster for evaluation.
+The targets can be any variable in the computation graph. When the target is say, the `optimizer` variable, the neural network will be optimized once. When the target is the `cost` variable, `session.eval` returns the cost value. Based on what the target is, an appropriate action is taken.
+
+The Python `session` is a wrapper of the C++ `Session` class. For more information about `Session`, refer to this document - [Design Doc: Session](./session.md).
+
+### PaddlePaddle Converter
+
+The PaddlePaddle converter automatically converts the IR in the request (IR and evaluation inputs/targets) from PaddlePaddle Python to partitioned IRs and dispatches the new IRs and evaluation inputs/targets to different PaddlePaddle runtimes. Below are the steps that are followed :
+
+1. Add a `feed` OP that feeds the eval inputs, and a `fetch` OP that fetches the eval targets to the IR.
+
+2. Extract a new computation (sub)graph with the `feed` and `fetch` OPs as the boundary. The runtime does not need to run the OP that is not dependent on the `fetch` OP.
+
+3. Optimize the computation graph.
+
+4. Place the OPs in the graph onto different devices on different PaddlePaddle runtime according to a placement algorithm and the device constraints specified by the user.
+
+5. Partition the graph according to runtime boundaries and add `send` / `recv` OP pair on the runtime boundaries.
+
+6. Dispatch the partitioned graph to different PaddlePaddle runtimes.
+
+7. PaddlePaddle runtimes with the `fetch` OP reports evaluation results back to the converter, the converter reports the evaluation results back to the PaddlePaddle Python.
+
+The output IRs will be cached to optimize the conversion latency.
+
+
+#### Placement Algorithm
+
+Our first implementation will only support "trainer-parameter server" placement: the parameters, initializers, and optimizers are all placed on the PaddlePaddle runtimes with the parameter server role. Everything else will be placed on the PaddlePaddle runtimes with the trainer role. This has the same functionality as the "trainer-parameter server" architecture of PaddlePaddle v0.10.0, but is more generic and flexible.
+
+In the future, a more general placement algorithm should be implemented, which makes placements according to the input IR, and a model of device computation time and device communication time. Model parallelism requires the generic placement algorithm.
+
+
+### PaddlePaddle Runtime
+
+The PaddlePaddle runtime owns multiple devices (e.g., CPUs, GPUs) and runs the IR. The runtime does not need to do OP placement since it is already done by the converter.
+
+
+### Local Training Architecture
+
+The local training architecture will be the same as the distributed training architecture, the difference is that everything runs locally, and there is just one PaddlePaddle runtime:
+
+
+
+
+### Training Data
+
+In PaddlePaddle v0.10.0, training data is typically read with a [data reader](../reader/README.md) from Python. This approach is no longer efficient when training in a distributed fashion since the Python process no longer runs on the same node with the trainer processes. The Python reader will need to read from the distributed filesystem (assuming it has the required access) and send to the trainers, doubling the network traffic.
+
+When doing distributed training, the user can still use Python data reader: the training data are sent with `session.eval`. However this should be used for debugging purpose only. The users are encouraged to use the read data OPs.
+
+
+## References:
+
+[1] [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45166.pdf)
+
+[2] [TensorFlow: A System for Large-Scale Machine Learning](https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf)
diff --git a/doc_cn/_sources/design/refactor/parameter_server.md.txt b/doc_cn/_sources/design/refactor/parameter_server.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..fa3c5d7990213cf2b0d236e66e592dd2699da876
--- /dev/null
+++ b/doc_cn/_sources/design/refactor/parameter_server.md.txt
@@ -0,0 +1,106 @@
+# Design Doc: Operation Graph Based Parameter Server
+
+## Abstract
+
+We propose an approach to implement the parameter server. In this
+approach, there is no fundamental difference between the trainer and
+the parameter server: they both run subgraphs, but subgraphs of
+different purposes.
+
+## Background
+
+The previous implementations of the parameter server does not run a
+subgraph. parameter initialization, optimizer computation, network
+communication and checkpointing are implemented twice on both the
+trainer and the parameter server.
+
+It would be great if we can write code once and use them on both the
+trainer and the parameter server: reduces code duplication and
+improves extensibility. Given that after the current refactor, we are
+representing everything as a computing graph on the
+trainer. Representing everything as a computing graph on the parameter
+server becomes a natural extension.
+
+## Design
+
+### Graph Converter
+
+The *graph converter* converts the user-defined operation (OP) graph
+into subgraphs to be scheduled on different nodes with the following
+steps:
+
+1. OP placement: the OPs will be placed on different nodes according
+   to heuristic that minimizes estimated total computation
+   time. Currently we will use a simple heuristic that puts parameter
+   varable on parameter server workers and everything else on trainer
+   workers.
+
+1. Add communication OPs to enable the communication between nodes.
+
+We will need these OPs: *Send*, *Recv*, *Enqueue*, *Dequeue*.
+
+Below is an example of converting the user defined graph to the
+subgraphs for the trainer and the parameter server:
+
+
+
+After converting:
+
+
+
+1. The parameter variable W and it's optimizer subgraph are placed on the parameter server.
+1. Operators are added to the subgraphs.
+   - *Send* sends data to the connected *Recv* operator.  The
+	 scheduler on the receive node will only schedule *Recv* operator
+	 to run when the *Send* operator has ran (the *Send* OP will mark
+	 the *Recv* OP runnable automatically).
+   - *Enueue* enqueues the input variable, it can block until space
+     become available in the queue.
+   - *Dequeue* outputs configurable numbers of tensors from the
+     queue. It will block until the queue have the required number of
+     tensors.
+
+
+### Benefits
+
+- Model parallelism become easier to implement: it's an extension to
+  the trainer - parameter server approach. we already have the
+  communication OPs, but need to extend the graph converter's
+  placement functionality.
+
+- User-defined optimizer is easier to add - user can now express it as
+  a subgraph.
+
+- No more duplication logic inside the trainer and the parameter
+  server mentioned in the background section.
+
+### Challenges
+
+- It might be hard for the graph converter to cut a general graph
+  (without any hint for which subgraph is the optimizer). We may need
+  to label which subgraph inside the OP graph is the optimizer.
+
+- It's important to balance the parameter shards of on multiple
+  parameter server. If a single parameter is very big (some
+  word-embedding, fully connected, softmax layer), we need to
+  automatically partition the single parameter onto different
+  parameter servers when possible (only element-wise optimizer depends
+  on the parameter variable).
+
+### Discussion
+
+- In the "Aync SGD" figure, the "W" variable on the parameter server
+  could be read and wrote concurrently, what is our locking strategy?
+  E.g., each variable have a lock cpp method to be invoked by every
+  OP, or, have a lock OP.
+
+- Can the Enqueue OP be implemented under our current tensor design
+  (puts the input tensor into the queue tensor)?
+
+- *Dequeue* OP will have variable numbers of output (depends on the
+  `min_count` attribute), does our current design support it? (similar
+  question for the *Add* OP)
+
+
+### References:
+[1] [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45166.pdf)
diff --git a/doc_cn/_sources/design/refactor/session.md.txt b/doc_cn/_sources/design/refactor/session.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..1d9a26683c14f54e3b5fe41675cd03b5620646b8
--- /dev/null
+++ b/doc_cn/_sources/design/refactor/session.md.txt
@@ -0,0 +1,180 @@
+# Design Doc: Session
+
+## Abstract
+
+The *session* object encapsulates the environment in which the
+computation graph is executed.
+
+We will have the *local* session and *remote* session, they offer the
+same [interface](#interface). The local session encapsulates the local
+runtime environment and the remote session encapsulates the cluster
+runtime environment.
+
+The local runtime environment contains:
+
+1. computation devices (i.e., CPU, GPU) handles, and
+1. the [scope](../scope.md) which holds all variables.
+
+The remote runtime environment contains:
+
+1. computation devices (i.e., CPU and GPU on node 0, 1) in a cluster,
+   and
+1. the distributed [scope](../scope.md) in a cluster which holds all
+   variables.
+
+The user can create a remote session on Paddle Cloud and evaluate the
+computation graph with it. In this way, the user can control the
+remote computation resource in a cluster from his local computer.
+
+
+## Background
+
+The current design has an implicit global session in which
+`paddle.eval()` is executed. The pain point is:
+
+Since the user is not able to explicitly switch between runtime
+environments, the user cannot run a topology in two independent
+environments.
+
+For example, in reinforcement learning, the user may want to have a
+stale model for inference and a fresh model for training, and only
+replace the stale model with the fresh model periodically.
+
+Furthermore, we have no concept that encapsulates a remote environment
+that executes a computation graph.
+
+We need the session object to address above issues.
+
+
+## Session
+
+A session is an object that owns the runtime environment. All
+computations are executed through `session.eval()`.
+
+
+### Interface
+
+```python
+eval(
+    targets,
+    feed_dict=None,
+)
+```
+
+Evaluates the target Operations or Variables in `targets`.
+
+- *targets*: the evaluation targets. Can be a single Operation or
+  Variable, or a list with the Operations or Variables as
+  elements. The value returned by `eval()` has the same shape as the
+  `target` argument.
+
+  The PaddlePaddle program is represented by
+  the [ProgramDesc](../design/program.md), `eval()` will infer the
+  ProgramDesc from the given targets and run the PaddlePaddle
+  program. Please
+  see
+  [this graph](./distributed_architecture.md#local-training-architecture) for
+  the detailed illustration for the local session
+  and
+  [this graph](./distributed_architecture.md#distributed-training-architecture) for
+  the detailed illustration for the remote session.
+
+- *feed_dict*: a dictionary that contains the tensors which override
+  the edges of the computation graph.
+
+  feed_dict not only can provide the input data, it can override any
+  OP's input as well:
+
+  ```python
+  a = pd.constant(2.0, name="a")
+  b = pd.variable(name="b")
+  c = pd.mul(a,b)
+  sess.eval(targets=c, feed_dict={"b":3.0}) # returns 6.0
+  ```
+
+```python
+close()
+```
+
+Closes the session and releases the scope that the session owns.
+
+
+### Create a Local Session
+
+```python
+session(
+    devices=None
+)
+```
+
+Creates a new session. One session owns one global scope, so creating
+multiple sessions will create different scopes.
+
+- *devices*: a single `string` or a list of `string` of device names,
+  the corresponding devices will be the computation devices for
+  `eval()`. If not specified, all available devices (e.g., all GPUs)
+  will be used. The user doesn't need to specify the CPU device since
+  it will be always used. Multiple sessions can use the same device.
+
+
+#### Example
+
+```Python
+a = paddle.constant(1.0)
+b = paddle.constant(2.0)
+c = a + b
+sess = paddle.session(devices=["gpu:0", "gpu:1", "fpga:0"])
+sess.eval(c)
+sess.close()
+```
+
+### Create a Remote Session
+
+```python
+create_cloud_job(
+    name,
+    num_trainer,
+    mem_per_trainer,
+    gpu_per_trainer,
+    cpu_per_trainer,
+    num_ps,
+    mem_per_ps,
+    cpu_per_ps,
+)
+```
+
+Creates a Paddle Cloud job. Fails if the job name exists.
+
+```python
+get_cloud_job(
+    name
+)
+```
+
+Gets a Paddle Cloud job.
+
+```python
+remote_session(
+    job
+)
+```
+
+- *job*: the Paddle Cloud job.
+
+#### Example
+
+```Python
+reader = paddle.reader.recordio("/pfs/home/peter/mnist-train-*") # data stored on Paddle Cloud
+image = reader.column(0)
+label = reader.column(1)
+fc1 = paddle.op.fc(image, size=256, act="sigmoid")
+fc2 = paddle.op.fc(fc1, size=10, act="softmax")
+cost = paddle.op.cross_entropy(fc2, label)
+opt = paddle.optimizer.sgd(cost)
+
+job = paddle.create_cloud_job("test", 3, "1G", 1, 1, 2, "1G", 1)
+sess = paddle.remote_ession(job)
+for i in range(1000):
+    sess.eval(opt)
+sess.close()
+```
diff --git a/doc_cn/_sources/design/refactorization.md.txt b/doc_cn/_sources/design/refactorization.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..f93d6155e1764386b01d2f0df3f141ab75cd55d4
--- /dev/null
+++ b/doc_cn/_sources/design/refactorization.md.txt
@@ -0,0 +1,249 @@
+# Design Doc: Refactorization Overview
+
+The goals of refactoring include:
+
+1. Making it easy for external contributors to write new elementary computation operations.
+1. Making the codebase clean and readable.
+1. Designing a new computation representation -- a computation graph of operators and variables.
+1. Implementing auto-scalability and auto fault recoverable distributed computing with the help of computation graphs.
+
+## Computation Graphs
+
+1. PaddlePaddle represents the computation, training and inference of Deep Learning models, by computation graphs.
+
+  1. Please refer to [computation graphs](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/graph.md) for a concrete example.
+
+1. Users write Python programs to describe the graphs and run them (locally or remotely).
+
+1. A graph is composed of *variables* and *operators*.
+
+1. The description of graphs must be serializable/deserializable, so that:
+
+   1. It can be sent to the cloud for distributed execution, and
+   1. It can be sent to clients for mobile or enterprise deployment.
+
+1. The Python program does two things
+
+   1. *Compilation* runs a Python program to generate a protobuf message representation of the graph and send it to
+      1. the C++ library `libpaddle.so` for local execution,
+      1. the master process of a distributed training job for training, or
+      1. the server process of a Kubernetes serving job for distributed serving.
+   1. *Execution* executes the graph by constructing instances of class [`Variable`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24) and [`OperatorBase`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L70), according to the protobuf message.
+
+## Description and Realization of Computation Graph
+
+At compile time, the Python program generates a protobuf message representation of the graph, or a description of the graph.
+
+At runtime, the C++ program realizes the graph and runs it.
+
+| | Representation (protobuf messages) | Realization (C++ class objects) |
+|---|---|---|
+|Data|[VarDesc](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L107)|[Variable](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24)|
+|Operation|[OpDesc](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L35)|[Operator](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L64)|
+|Block|BlockDesc|Block|
+
+The word *graph* is interchangeable with *block* in this document.  A graph consists of computation steps and local variables similar to a C++/Java program block, or a pair of parentheses(`{` and `}`).
+
+## Compilation and Execution
+
+1. Run a Python program to describe the graph.  In particular, the Python application program does the following:
+
+   1. Create `VarDesc` to represent local/intermediate variables,
+   1. Create operators and set attributes,
+   1. Validate attribute values,
+   1. Infer the type and the shape of variables,
+   1. Plan memory-reuse for variables,
+   1. Generate the backward graph
+   1. Add optimization operators to the computation graph.
+   1. Optionally, split the graph for distributed training.
+
+1. The invocation of `train` or [`infer`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/inference.py#L108) methods in the Python program does the following:
+
+   1. Create a new Scope instance in the [scope hierarchy](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md) for each run of a block,
+      1. realize local variables defined in the BlockDesc message in the new scope,
+      1. a scope is similar to the stack frame in programming languages,
+
+   1. Create an instance of class `Block`, in which,
+      1. realize operators in the BlockDesc message,
+
+   1. Run the Block by calling
+      1. `Block::Eval(vector* targets)` for forward and backward computations, or
+      1. `Block::Eval(vector* targets)` for optimization.
+
+
+## Intermediate Representation (IR)
+
+```text
+Compile Time -> IR -> Runtime
+```
+
+### Benefits of IR
+
+- Optimization
+  ```text
+  Compile Time -> IR -> Optimized IR -> Runtime
+  ```
+- Automatically send partitioned IR to different nodes.
+  - Automatic Data Parallelism
+    ```text
+    Compile Time
+    |-> Single GPU IR
+        |-> [trainer-IR-0, trainer-IR-1, pserver-IR]
+            |-> Node-0 (runs trainer-IR-0)
+            |-> Node-1 (runs trainer-IR-1)
+            |-> Node-2 (runs pserver-IR)
+    ```
+  - Automatic Model Parallelism (planned for future)
+
+---
+
+# Operator/OpWithKernel/OpKernel
+
+![class_diagram](http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/49caf1fb70820fb4a6c217634317c9306f361f36/op_op_with_kern_class_diagram.dot)
+
+---
+
+# Operator
+![class_diagram](http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/dd598e8f1976f5759f58af5e5ef94738a6b2e661/op.dot)
+
+* `Operator` is the fundamental building block of the user interface.
+    * Operator stores input/output variable names and attributes.
+    * The `InferShape` interface is used to infer the shape of the output variables based on the shapes of the input variables.
+    * Use `Run` to compute the `output` variables from the `input` variables.
+
+---
+
+# OpWithKernel/Kernel
+
+![class_diagram](http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/9d7f4eba185cf41c8e2fbfb40ae21890dbddcd39/op_with_kernel.dot)
+
+* `OpWithKernel` inherits `Operator`.
+* `OpWithKernel` contains a Kernel map.
+    * `OpWithKernel::Run` get device's kernel, and invoke `OpKernel::Compute`.
+    * `OpKernelKey` is the map key. Only device place now, but may be data type later.
+
+---
+
+# Why separate Kernel and Operator
+
+* Separate GPU and CPU code.
+    * Make Paddle capable of running without GPU.
+* Make one operator (which is a user interface) and create many implementations.
+    * For example, same multiplication op can have different implementations kernels such as FP16 kernel, FP32 kernel, MKL, eigen kernel.
+---
+
+# Libraries for Kernel development
+
+* `Eigen::Tensor` contains basic math and element-wise functions.
+    * Note that `Eigen::Tensor` has broadcast implementation.
+    * Limit the number of `tensor.device(dev) = ` in your code.
+* `thrust::transform` and `std::transform`.
+    * `thrust` has the same API as C++ standard library. Using `transform`, one can quickly implement customized element-wise kernels.
+    * `thrust`, in addition, supports more complex APIs, like `scan`, `reduce`, `reduce_by_key`.
+* Hand-writing `GPUKernel` and `CPU` code
+    * Do not write in header (`.h`) files. CPU Kernel should be in cpp source (`.cc`) and GPU kernels should be in cuda (`.cu`) files. (GCC cannot compile GPU code.)
+---
+# Operator Registration
+
+## Why is registration necessary?
+We need a method to build mappings between Op type names and Op classes.
+
+## How is registration implemented?
+Maintaining a map, whose key is the type name and the value is the corresponding Op constructor.
+
+---
+# The Registry Map
+
+### `OpInfoMap`
+
+`op_type(string)` -> `OpInfo`
+
+`OpInfo`:
+
+- **`creator`**: The Op constructor.
+- **`grad_op_type`**: The type of the gradient Op.
+- **`proto`**: The Op's Protobuf, including inputs, outputs and required attributes.
+- **`checker`**: Used to check attributes.
+
+---
+# Related Concepts
+
+### Op_Maker
+It's constructor takes `proto` and `checker`. They are completed during Op_Maker's construction. ([ScaleOpMaker](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/scale_op.cc#L37))
+
+### Register Macros
+```cpp
+REGISTER_OP(op_type, op_class, op_maker_class, grad_op_type, grad_op_class)
+REGISTER_OP_WITHOUT_GRADIENT(op_type, op_class, op_maker_class)
+```
+
+---
+# Registration Process
+1. Write an Op class and its gradient Op class, if required.
+2. Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator.
+3. Invoke the macro `REGISTER_OP`. This macro will
+	1. Call maker class to complete `proto` and `checker`
+	2. Using the completed `proto` and `checker`, it will add a new key-value pair to the `OpInfoMap`
+
+---
+# Backward Module (1/2)
+### Create Backward Operator
+- Mapping from forward Op to backward Op
+![backward](https://gist.githubusercontent.com/dzhwinter/a6fbd4623ee76c459f7f94591fd1abf0/raw/61026ab6e518e66bde66a889bc42557a1fccff33/backward.png)
+
+---
+# Backward Module (2/2)
+### Build Backward Network
+- **Input**: a graph of forward operators
+- **Output**: a graph of backward operators
+- **Corner cases in construction**
+	- Shared Variables => insert an `Add` operator to combine gradients
+	- No Gradient => insert a `fill_zero_grad` operator
+	- Recursive NetOp => call `Backward` recursively
+	- RNN Op => recursively call `Backward` on stepnet
+	- RNN Op => recursively call `Backward` on stepnet
+
+
+---
+# Scope, Variable, Tensor
+
+* `Tensor` is an n-dimension array with type.
+	* Only dims and data pointers are stored in `Tensor`.
+	* All operations on `Tensor` are written in `Operator` or global functions.
+	* Variable length Tensor design [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md)
+* `Variable` instances are the inputs and the outputs of an operator, not just `Tensor`.
+	* `step_scopes` in RNN is a variable and not a tensor.
+* `Scope` is where variables are stored.
+	* map
+	* `Scope` has a hierarchical structure. The local scope can get variables from its parent scope.
+
+---
+# Block (in design)
+## the difference between original RNNOp and Block
+- As an operator is more intuitive than `RNNOp`,
+- Offers a new interface `Eval(targets)` to deduce the minimal block to `Run`,
+- Fits the compile-time/ runtime separation design paradigm.
+  - During the compilation, `SymbolTable` stores `VarDesc`s and `OpDesc`s and serialize to a `BlockDesc`
+  - When graph executes, a Block with `BlockDesc` is passed. It then creates `Op` and `Var` instances and then invokes `Run`.
+
+---
+# Milestone
+- Take Paddle/books as the main line, the requirement of the models motivates framework refactoring,
+- Model migration
+  - Framework development gives **priority support** to model migration, for example,
+    - the MNIST demo needs a Python interface,
+    - the RNN models require the framework to support `LoDTensor`.
+  - Determine some timelines,
+  - Frequently used Ops need to be migrated first,
+  - Different models can be migrated in parallel.
+- Improve the framework at the same time
+- Accept imperfection, concentrate on solving the specific problem at the right price.
+
+---
+# Control the migration quality
+- Compare the performance of migrated models with old ones.
+- Follow the google C++ style guide.
+- Build the automatic workflow of generating Python/C++ documentations.
+  - The documentation of layers and ops should be written inside the code.
+  - Take the documentation quality into account when submitting pull requests.
+  - Preview the documentations, read and improve them from a user's perspective.
diff --git a/doc_cn/_sources/design/register_grad_op.md.txt b/doc_cn/_sources/design/register_grad_op.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..8d973eb53178c3e889c845144553a453e11f067c
--- /dev/null
+++ b/doc_cn/_sources/design/register_grad_op.md.txt
@@ -0,0 +1,92 @@
+# Design Doc: Gradient Operators Registration
+
+
+## The Problem Posed
+
+Currently, for each C++ operator class definition, a *gradient operator creator* function is registered, which takes as input a C++ operator instance and returns the corresponding gradient operator instance.
+
+However, we noticed two problems with the current design:
+
+1. As we decided to separate the *compilation* and the *execution* phases, we need to change the creator to take an `OpDesc` protobuf message in a `ProgramDesc` and inserts corresponding `OpDesc` messages into the `ProgramDesc` message.
+
+1. For some operators, the gradient computation can be written in terms of existing operators.  For example, the gradient of *minus* operator consists of two operators -- an *identity* operator followed by a *scale* operator.  Hence the registration mechanism needs to support mapping from an operator to a set of operators for the gradient computation.
+
+## The Current Implementation
+
+Instances of the C++ class `OpInfo` are stored an associative map whose key is the operator type. The `grad_op_type` indicates the associated gradient operator type. An operator can create the gradient operator by invoking `OpInfo::creator_` of the gradient operator. The pseudo code is as follows
+
+```cpp
+struct OpInfo {
+  std::function creator_;
+  std::string grad_op_type_;
+  ...
+};
+
+map OpInfoMap;
+
+OperatorBase* CreateGradientOperator(const OperatorBase& op) {
+  return OpInfoMap.at(op.Type()).creator_(...);
+}
+```
+
+## Proposed Solution
+
+The mapping relationship between an operator and its gradient operators is a function. The interface of this function is:
+
+```cpp
+// (OpDesc) --> vector
+std::function(const OpDescBind&)>;
+```
+
+The function takes an `OpDescBind` of the forward operator and returns one or many gradient operator descriptions. `OpDescBind` is a C++ wrapper for  the protobuf message `OpDesc` for rapid manipulation of `OpDesc`.
+
+The `GradOpDescMaker` will be registered in `OpInfo` and will replace the `grad_op_type_` field. The `OpInfo` should look like 
+
+```cpp
+struct OpInfo {
+  std::function>(const OpDescBind&)>  grad_op_maker_;
+  ...
+};
+```
+
+The `grad_op_maker_ ` is a `nullptr` if the operator does not have any associated gradient operators.
+
+We propose a base class called `GradOpDescMakerBase` to let operator developers generate `Gradient Operators` easily. The public interface of that class is
+
+```cpp
+class GradOpDescMakerBase {
+public:
+  GradOpDescMakerBase(const OpDescBind& );
+  virtual std::vector> operator()()const = 0;
+};
+```
+
+We can convert `GradOpDescMakerBase` to `std::function>(const OpDescBind&)>` by
+
+```cpp
+using GradOpMaker = ...;
+std::function(const OpDescBind&)> func;
+func = [] (const OpDescBind& fwd_op) {
+  GradOpMaker maker(fwd_op);
+  return maker();
+};
+```
+
+We can write many helper functions since the `GradOpDescMakerBase` is a class now. The basic helper functions get the variables of `Input`, `Output`, `InputGradient` and `OutputGradient` in the forwarding operator.
+
+We should change register macros at the same time. In the current solution, there is no difference between forwarding operators and backward operators. So `REGISTER_OP` just register one operator. If the `REGISTER_OPERATOR ` contains `OpProtoAndCheckerMaker` and `GradOpDescMaker`, we just list them in the same macro. It can be done by a macro contains `__VA_ARGS__`.
+
+The user interface should be
+
+```cpp
+vector MinusOpGradMaker(OpDesc) {...}
+REGISTER_OPERATOR(minus, MinusOp, MinusOpProtoAndCheckerMaker, SumOpGradMaker);
+// Developers can still manually implement gradient operator.
+REGISTER_OPERATOR(minus_grad, MinusGradOp);
+```
+
+The interface of current `REGISTER_OP` macro could not be changed. In `REGISTER_OP`, it will invoke `REGISTER_OPERATOR` two times and generate GradOpDescMaker inside.
+
+```cpp
+REGISTER_OP(minus, MinusOp, MinusOpProtoAndCheckerMaker, minus_grad, MinusGradOp);
+```
diff --git a/doc_cn/_sources/design/regularization.md.txt b/doc_cn/_sources/design/regularization.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..21280ac898feb4dd5e5a5d9e88d121e856850f0b
--- /dev/null
+++ b/doc_cn/_sources/design/regularization.md.txt
@@ -0,0 +1,72 @@
+# Regularization in PaddlePaddle
+
+## Introduction to Regularization
+A central problem in machine learning is how to design an algorithm that will perform well not just on the training data, but also on new data. A frequently faced problem is the problem of **overfitting**, where the model does not make reliable predictions on new unseen data. **Regularization** is the process of introducing additional information in order to prevent overfitting. This is usually done by adding extra penalties to the loss function that restricts the parameter spaces that an optimization algorithm can explore.
+
+### Parameter Norm Penalties
+Most common regularization approaches in deep learning are based on limiting the capacity of the models by adding a parameter norm penalty to the objective function `J`. This is given as follows:
+
+
    
+
+The parameter `alpha` is a hyperparameter that weights the relative contribution of the norm penalty term, `omega`, relative to the standard objective function `J`.
+
+The most commonly used norm penalties are the L2 norm penalty and the L1 norm penalty. These are given as follows:
+
+##### L2 Regularization:
+
    
+
+##### L1 Regularization
+
    
+
+A much more detailed mathematical background of regularization can be found [here](http://www.deeplearningbook.org/contents/regularization.html).
+
+## Regularization Survey
+
+A detailed survey of regularization in various deep learning frameworks can be found [here](https://github.com/PaddlePaddle/Paddle/wiki/Regularization-Survey). 
+
+## Proposal for Regularization in PaddlePaddle
+
+### Low-Level implementation
+
+In the new design, we propose to create new operations for regularization. For now, we can add 2 ops that correspond to the most frequently used regularizations:
+- L2_regularization_op
+- L1_regularization_op
+
+These ops can be like any other ops with their own CPU/GPU implementations either using Eigen or separate CPU and GPU kernels. As the initial implementation, we can implement their kernels using Eigen following the abstraction pattern implemented for [Activation Ops](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/accuracy_op.h). This abstraction pattern can make it very easy to implement new regularization schemes other than L1 and L2 norm penalties. 
+
+The idea of building ops for regularization is in sync with the refactored Paddle philosophy of using operators to represent any computation unit. The way these ops will be added to the computation graph, will be decided by the [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md#layer-function) in Python API. 
+
+### Computation Graph
+
+Below is an example of a really simple feed forward neural network.
+
+
    
+
+The Python API will modify this computation graph to add regularization operators. The modified computation graph will look as follows:
+
+
    
+   
+### Python API implementation for Regularization
+
+Using the low level ops, `L2_regularization_op` and `L1_regularization_op`, any user can add regularization to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support regularization. An example of such an API can be seen in [Keras](https://keras.io/regularizers/). As per the PaddlePaddle [Python API design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md), the layer functions are responsible for creating operators, operator parameters and variables. Since regularization is a property of parameters, it makes sense to create these in the layer functions. 
+
+#### Creation of Regularization ops
+There are two possibilities for creating the regularization ops:
+1. We create these ops immediately while building the computation graph. 
+2. We add these ops in a lazy manner, just before the backward, similar to the way the optimization ops are added. 
+
+The proposal is to add these ops in a lazy manner just before the backward pass. 
+
+#### Storage of Regularization attributes
+
+Since we want to create the regularization ops in a lazy manner, the regularization attributes (type of regularization and weight of regularization penalty) can be stored as attributes of the [`Parameter`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/framework.py#L421) class. This is because regularization is a property of the parameters and storing regularization properties with Parameters also allows for shared parameters. 
+
+#### High-level API
+
+In PaddlePaddle Python API, users will primarily rely on [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md#layer-function) to create neural network layers. Hence, we also need to provide regularization functionality in layer functions. The design of these APIs can be postponed for later right now. A good reference for these APIs can be found in [Keras](https://keras.io/regularizers/) and also by looking at Tensorflow in [`tf.contrib.layers`](https://www.tensorflow.org/api_guides/python/contrib.layers).
+
+
+
+
+
+    
diff --git a/doc_cn/_sources/design/releasing_process.md.txt b/doc_cn/_sources/design/releasing_process.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..14c081ea84282e52a2e36475c3c0ea755122d154
--- /dev/null
+++ b/doc_cn/_sources/design/releasing_process.md.txt
@@ -0,0 +1,68 @@
+# PaddlePaddle发行规范
+
+PaddlePaddle使用git-flow branching model做分支管理，使用[Semantic Versioning](http://semver.org/)标准表示PaddlePaddle版本号。
+
+PaddlePaddle每次发新的版本，遵循以下流程:
+
+1. 从`develop`分支派生出新的分支，分支名为`release/版本号`。例如，`release/0.10.0`
+1. 将新分支的版本打上tag，tag为`版本号rc.Patch号`。第一个tag为`0.10.0rc1`，第二个为`0.10.0rc2`，依次类推。
+1. 对这个版本的提交，做如下几个操作:
+	* 修改`python/setup.py.in`中的版本信息,并将`istaged`字段设为`True`。
+	* 编译这个版本的Docker发行镜像，发布到dockerhub。如果失败，修复Docker编译镜像问题，Patch号加一，返回第二步
+	* 编译这个版本的Ubuntu Deb包。如果失败，修复Ubuntu Deb包编译问题，Patch号加一，返回第二步。
+	* 使用Regression Test List作为检查列表，测试Docker镜像/ubuntu安装包的功能正确性
+		* 如果失败，记录下所有失败的例子，在这个`release/版本号`分支中，修复所有bug后，Patch号加一，返回第二步
+	* 编译这个版本的python wheel包，并发布到pypi。
+		* 由于pypi.python.org目前遵循[严格的命名规范PEP 513](https://www.python.org/dev/peps/pep-0513)，在使用twine上传之前，需要重命名wheel包中platform相关的后缀，比如将`linux_x86_64`修改成`manylinux1_x86_64`。
+		* pypi上的package名称为paddlepaddle和paddlepaddle_gpu，如果要上传GPU版本的包，需要修改build/python/setup.py中，name: "paddlepaddle_gpu"并重新打包wheel包：`python setup.py bdist_wheel`。
+		* 上传方法：
+			```
+			cd build/python
+			pip install twine
+			twine upload dist/[package to upload]
+			```
+1. 第三步完成后，将`release/版本号`分支合入master分支，并删除`release/版本号`分支。将master分支的合入commit打上tag，tag为`版本号`。同时再将`master`分支合入`develop`分支。最后删除`release/版本号`分支。
+1. 编译master分支的Docker发行镜像，发布到dockerhub。编译ubuntu的deb包，发布到github release页面
+1. 协同完成Release Note的书写
+
+
+需要注意的是:
+
+* `release/版本号`分支一旦建立，一般不允许再从`develop`分支合入`release/版本号`。这样保证`release/版本号`分支功能的封闭，方便测试人员测试PaddlePaddle的行为。
+* 在`release/版本号`分支存在的时候，如果有bugfix的行为，需要将bugfix的分支同时merge到`master`, `develop`和`release/版本号`这三个分支。
+
+## PaddlePaddle 分支规范
+
+PaddlePaddle开发过程使用[git-flow](http://nvie.com/posts/a-successful-git-branching-model/)分支规范，并适应github的特性做了一些区别。
+
+* PaddlePaddle的主版本库遵循[git-flow](http://nvie.com/posts/a-successful-git-branching-model/)分支规范。其中:
+	* `master`分支为稳定(stable branch)版本分支。每一个`master`分支的版本都是经过单元测试和回归测试的版本。
+	* `develop`分支为开发(develop branch)版本分支。每一个`develop`分支的版本都经过单元测试，但并没有经过回归测试。
+	* `release/版本号`分支为每一次Release时建立的临时分支。在这个阶段的代码正在经历回归测试。
+
+* 其他用户的fork版本库并不需要严格遵守[git-flow](http://nvie.com/posts/a-successful-git-branching-model/)分支规范，但所有fork的版本库的所有分支都相当于特性分支。
+	* 建议，开发者fork的版本库使用`develop`分支同步主版本库的`develop`分支
+	* 建议，开发者fork的版本库中，再基于`develop`版本fork出自己的功能分支。
+	* 当功能分支开发完毕后，向PaddlePaddle的主版本库提交`Pull Reuqest`，进而进行代码评审。
+		* 在评审过程中，开发者修改自己的代码，可以继续在自己的功能分支提交代码。 
+
+* BugFix分支也是在开发者自己的fork版本库维护，与功能分支不同的是，BugFix分支需要分别给主版本库的`master`、`develop`与可能有的`release/版本号`分支，同时提起`Pull Request`。
+
+## PaddlePaddle回归测试列表
+
+本列表说明PaddlePaddle发版之前需要测试的功能点。
+
+### PaddlePaddle Book中所有章节
+
+PaddlePaddle每次发版本首先要保证PaddlePaddle Book中所有章节功能的正确性。功能的正确性包括验证PaddlePaddle目前的`paddle_trainer`训练和纯使用`Python`训练模型正确性。
+
+| | 新手入门章节 | 识别数字 | 图像分类 | 词向量 | 情感分析 | 语意角色标注 | 机器翻译 | 个性化推荐 |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| API.V2 + Docker + GPU  |  |  |  |  |  |  |  |  |
+| API.V2 + Docker + CPU  |  |  |  |  |  |  |  |  |
+| `paddle_trainer` + Docker + GPU |  |  |  |  |  |  |  |  |
+| `paddle_trainer` + Docker + CPU |  |  |  |  |  |  |  |  |
+| API.V2 + Ubuntu + GPU |  |  |  |  |  |  |  |  |
+| API.V2 + Ubuntu + CPU |  |  |  |  |  |  |  |  |
+| `paddle_trainer` + Ubuntu + GPU |  |  |  |  |  |  |  |  |
+| `paddle_trainer` + Ubuntu + CPU |  |  |  |  |  |  |  |  |
diff --git a/doc_cn/_sources/design/scope.md.txt b/doc_cn/_sources/design/scope.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..4da76eebb74abcd26ec2b8671399e6bc4fb58574
--- /dev/null
+++ b/doc_cn/_sources/design/scope.md.txt
@@ -0,0 +1,124 @@
+# Design of Scope in Paddle
+
+## Overview
+
+Scope is an important concept in programming languages, which defines a program region that a set of bindings between names and entities applies. In a specific scope, a valid name is uniquely associated with an entity, such as a variable. And in another scope, this name may refer to other entity or nothing at all. It clearly restricts the visibility and validity of names in a program. Hence **Scope** is introduced to PaddlePaddle to manage variables in context. But different from the original abstract concept, Scope now becomes an object with two important attributes:
+
+- Scope is an association of a name to variable.
+- Variables in a parent scope can be retrieved from local scope.
+
+A detailed explanation of these two attributes goes as following.
+
+
+## Scope is an association of a name to variable.
+
+Scope is an association of a name to variable. All variables belong to `Scope`. You need to specify a scope to run a Net, i.e., `net.Run(&scope)`. One net can run in different scopes and update different variable in the scope.
+
+
+1. Scope only contains a map of a name to variable.
+
+   All parameters, data, states in a Net should be variables and stored inside a scope. Each op should get inputs and outputs to do computation from a scope, such as data buffer, state (momentum) etc.
+
+1. Variable can only be created by Scope and a variable can only be got from Scope. User cannot create or get a variable outside a scope. This is a constraints of our framework, and will keep our framework simple and clear.
+
+1. Scope only contains methods that are used to Create and Get Variables. Scope do not contain Operators and have no information to run them.
+    `Net` is designed to drive the computation and Scope only contains a map of variables. There is no computation logic inside a `Scope`. Scope just handles the lifetime management of variables.
+    - `Create` is used to create a Variable by its name and add the mapping relation.
+    - `Get` is used to find a Variable by name.
+
+1. Every variable only belongs to one certain Scope.
+
+   Variable can not belong to many scopes. If you want to use variables from parent scope, you can use `parent scope`.
+
+1. Scope should destruct all Variables inside it when itself is destructed. User can never store `Variable` pointer somewhere else. 
+
+   Because Variable can only be got from Scope. When destroying Scope, we also need to destroy all the Variables in it. If user store `Variable` pointer to private data member or some global variable, the pointer will be an invalid pointer when associated `Scope` is destroyed.
+
+```cpp
+class Scope {
+ public:
+  Variable* Var(const std::string& name);
+  const Variable* FindVar(const std::string& name) const;
+
+ private:
+    std::unordered_map> vars_;
+};
+```
+
+
+## Parent scope and local scope
+
+Just like [scope](https://en.wikipedia.org/wiki/Scope_(computer_science)) in programming languages, `Scope` in the neural network can also be a local scope. There are two attributes about local scope.
+
+1.  We can create local variables in a local scope. When that local scope is destroyed, all local variables should also be destroyed.
+2.  Variables in a parent scope can be retrieved from local scopes of that parent scope, i.e., when user get a variable from a scope, it will try to search this variable in current scope. If there is no such variable in the local scope, `scope` will keep searching from its parent, until the variable is found or there is no parent.
+
+```cpp
+class Scope {
+ public:
+  Scope(const std::shared_ptr& scope): parent_(scope) {}
+
+  Variable* FindVar(const std::string& name) const {
+    auto it = vars_.find(name);
+    if (it != vars_.end()) {
+      return it->second.get();
+    } else if (parent_ != nullptr) {
+      return parent_->FindVar(name);
+    } else {
+      return nullptr;
+    }
+  }
+
+ private:
+  std::shared_ptr parent_ {nullptr};
+};
+```
+
+In `Scope` class, there is a private data member called `parent_`. `parent_` is a smart pointer to its parent scope. When user `Get` a variable by its `name`, the `name` will be searched inside the current scope. If the variable cannot be found locally and parent scope is not a `nullptr`, the variable will be searched inside that parent scope. `parent_` pointer's default value is `nullptr`. It means that the scope is a global scope when `parent_` is nullptr.
+
+A local scope is very useful when we implement Recurrent Neural Network. Each timestep of an RNN should be a `Net`. Each `Net` of timestep (`StepNet` for short) should use an independent local scope. Just like variables in a while loop is inside a local scope in programming languages. By using a single `StepNet` and changing local scope, we can implement an RNN easily.
+
+# Interface Design
+
+```cpp
+class Variable {
+ private:
+  Variable() = default;
+  friend class Scope;
+};
+
+class Scope {
+ private:
+  Scope(const std::shared_ptr& parent = nullptr);
+
+ public:
+  static std::shared_ptr Create(const std::shared_ptr& parent = nullptr);
+
+  // return nullptr if not found.
+  Variable* FindVar(const std::string& name) const;
+
+  // return if already contains same name variable.
+  Variable* Var(const std::string& name);
+
+ private:
+  std::shared_ptr parent_;
+  std::unordered_map> vars_;
+};
+```
+## Only scope can create a variable
+
+To ensure `only scope can create a variable`, we should mark `Variable`'s constructor as a private member function, and Scope is a friend class of Variable. And then only `Var` can construct `Variable`.
+
+## When scope destroyed, all variables inside this scope should be destroyed together
+
+The scope hold unique pointers for all variables. User can `FindVar` from scope, but he should not hold this pointer as a member variable. Because when scope is destroyed, all variables inside this scope will be destroyed together.
+
+## Sharing a parent scope
+
+Local scope contains a `parent_` pointer. It is a linked-list for scopes. Using a `shared_ptr` because when a local scope is using, its parents cannot be destroyed.
+
+Also, as the parent scope is a `shared_ptr`, we can only `Create()` a scope shared pointer. We cannot construct a scope variable, because it cannot be passed to other scope as `parent` pointer.
+
+## Orthogonal interface
+
+`FindVar` will return `nullptr` when `name` is not found. It can be used as `Contains` method. `Var` will return an `Error` when there is a name conflict locally. Combine `FindVar` and `Var`, we can implement `Var` easily.
diff --git a/doc_cn/_sources/design/selected_rows.md.txt b/doc_cn/_sources/design/selected_rows.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..1a98839a957612b91b2276b58818623ecc62d1d5
--- /dev/null
+++ b/doc_cn/_sources/design/selected_rows.md.txt
@@ -0,0 +1,74 @@
+# Design Doc: Selected Rows
+
+`SelectedRows` is a type of sparse tensor data type, which is designed to support `embedding` operators. The gradient of embedding table is a sparse tensor. Only a few rows are non-zero values in this tensor. It is straight-forward to represent a sparse tensor by the following sparse tensor data structure:
+
+```cpp
+class SelectedRows {
+ private:
+  vector rows_;
+  Tensor value_;
+  int height_;
+};
+```
+
+The field `height_` is the first dimension of `SelectedRows`. The `rows` are the indices of the non-zero rows of `SelectedRows`. The `value_` field is an N-dim tensor of shape `[rows.size() /* NUM_ROWS */, ...]`, which supplies values for each row. The dimension of `SelectedRows` satisfies `[height_] + value_.shape[1:]`.
+
+Suppose that a SelectedRows-typed variable `x` has many rows, but only two of them have values -- row 73 is `[1, 2]` and row 84 is `[3, 4]`, the `SelectedRows` representation would be:
+
+```
+x = SelectedRow {
+  rows = [73, 84],
+  value = [[1, 2], [3,4]]
+}
+```
+
+
+## SelectedRows in Protobuf
+
+`SelectedRows` is a type of `Variable`. `VarDesc` in protobuf should describe the `SelectedRows` information. Only the tensor dimension of a `SelectedRows` will be described in compile-time because the `rows_` and `value_` are dependent on the training data. 
+So we use `TensorDesc` to unify `data_type` and `dims`. A LodTensorDesc contains a `TensorDesc` and `lod_level`. The description of `SelectedRows` is a Tensor description.
+
+```proto
+message TensorDesc {
+  required DataType data_type = 1;
+  repeated int64 dims = 2; // [UNK, 640, 480] is saved as [-1, 640, 480]
+}
+
+message LodTensorDesc {
+  required TensorDesc tensor = 1;
+  optional int lod_level = 2;
+}
+
+message VarDesc {
+  required string name = 1;
+  enum VarType { 
+    LOD_TENSOR = 0;
+    SELECTED_ROWS = 1;
+  }
+  required VarType type = 2;
+  optional LodTensorDesc lod_desc = 3;
+  optional TensorDesc selected_rows_desc = 4;
+  optional bool persistable = 5 [ default = false ];
+}
+```
+
+## InferShape for Selected Rows
+
+Just like `LoD` information, `InferShape` method will infer the output tensor type as well. The operator should decide whether its output is a `SelectedRows` or `Dense` tensor.
+
+For example, the gradient operator of `TableLookup` will always generate `SelectedRows`. Its `InferShape` method should be like following
+
+```cpp
+void TableLookupGrad::InferShape(context) {
+  ...
+  context.SetDataType("Embedding.Grad", kSelectedRows);
+}
+```
+
+
+## Sparse Operators
+
+There are several operators that need to be written to support `SelectedRows`. These are:
+
+1. Operators which generate `SelectedRows` gradient. e.g. Gradient of `TableLookupOp`.
+2. Optimize operators which support `SelectedRows` gradient. e.g. `SGD` or `AdaGrad` for `SelectedRows`. However, there should be only one `SGD` operator. `OpWithKernel::Run` should select a suitable kernel for both `dense` tensor or `SelectedRows`.
diff --git a/doc_cn/_sources/design/simple_op_design.md.txt b/doc_cn/_sources/design/simple_op_design.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..c7aeed7f9b4637e1c29d530f37b42d12500af82f
--- /dev/null
+++ b/doc_cn/_sources/design/simple_op_design.md.txt
@@ -0,0 +1,202 @@
+## Interaction between C++ and Python
+
+Users employ API in Python to describe their own network, however, the network construction actually happens in C++. so Protobuf is introduced to send the message between Python and C++. 
+
+The Interaction between Python and C++ can be simplified as two steps:
+
+1. C++ tells Python how many Ops there are, and what parameter do users need to offer to initialize a new Op. Python then builds API for each Op at compile time.
+
+2. Users invoke APIs built by Python and provide necessary parameters. These parameters will be sent to C++ for finishing the Op construction task.
+
+### Message from C++ to Python
+
+We define a Protobuf message class `OpProto` to hold message needed in the first step. What should an `OpProto` contain? This question is equivalent to “What message do we need to offer, to build a Python API which is legal and user oriented and can use to describe a whole Op.”
+
+Following message are necessary:
+
+1. Op's name, and its simple comment.
+2. Input and output variable number; each variable's name, type, and comment.
+3. Op's attributes; each attribute includes name, type, comment, **default value** and **value range**.
+
+So `OpProto` can be defined as follows:
+
+```proto
+enum AttrType {
+	INT = 1;
+	FLOAT = 2;
+	STRING = 3;
+	INTS = 4;
+	FLOATS = 5;
+	STRINGS = 6;
+};
+
+message AttrValue {
+	AttrType type = 1;
+	optional int iv = 2;
+	optional float fv = 3;
+	optional string sv = 4;
+	repeated int ivs = 5;
+	repeated float fvs = 6;
+	repeated string svs = 7;
+};
+
+message AttrProto {
+	required string name = 1;
+	required string comment = 2;
+	required AttrType type = 3;
+};
+
+message VarProto {
+	required string name = 1;
+	required string comment = 2;
+	required bool is_tensor = 3;
+};
+
+message OpProto {
+	repeated VarProto inputs = 1;
+	repeated VarProto outputs = 2;
+	repeated AttrProto attrs = 3;
+	required string type = 4;
+	required string comment = 5;
+};
+```
+
+To generate Python code automatically:
+
+```python 
+def create_python_ops_creatation_functions():
+	op_protos = paddle.framework.OpRegistry.get_all_op_proto()
+	for type_name in op_protos:
+		op_proto = op_protos[type_name]
+		def __impl__(**kwargs):  # User must use key word args in Paddle API
+			inputs = [kwargs.get(ipt.name, "") for ipt in op_proto.inputs]
+			outputs = [kwargs.get(opt.name, "") for opt in op_proto.outputs]
+			attrs = [cast_to_op_attr(attr, kwargs.get(attr.name, None)) for attr in op_proto.attrs]
+			opdesc = （input, outputs, type_name, attrs）
+			return paddle.framework.OpRegistry.CreateOp(opdesc)
+		__impl__.__doc__ = create_doc_string(op_proto)
+		globals()[type_name] = __impl__
+
+create_python_ops_creatation_functions()
+```
+
+### Message from Python to C++
+
+To hold message needed in the above second step, we define Protobuf message class `OpDesc`. It is used to hold user-specified parameters in Op describing.
+
+```proto
+message OpDesc {
+	required string type = 1;	
+	repeated string inputs = 2;
+	repeated string outputs = 3;
+	map attrs = 4;
+};
+```
+
+## OpProto Register
+
+Every Op has its own `OpProto`. For using convenience, we need to register them and record all their messages. For each `Op` class, we define a corresponding `OpMaker` class, in whose constructor we implement the `OpProto`'s building process. `OpMaker`'s constructor will be invoked by another function `OpRegistry::RegisterOp()`.
+
+```cpp
+class OpProtoMaker {
+public:
+	OpProtoMaker(OpProto* proto): proto_(proto) {}
+protected:
+	OpProto* proto_;
+	void AddInput(const std::string& name, const std::string& desc) {...}
+	void AddAttr(const std::string& name, const std::string& desc, TypeId type) {...}
+	void AddComment(const std::string& comment) { ... }
+};
+
+class OpRegistry {
+public:
+	using OpCreator = std::function;
+	
+	template 
+	static void RegisterOp(const std::string& name) {
+		gCreators_[name] = [](const OpDesc& desc) {
+			return new OpType(desc);
+		};
+		OpProto& opProto = gProtos_[name];
+		OpMaker()(&opProto);
+	}
+
+	static map gCreators_;
+	static map gProtos_;
+};
+
+template 
+class OpRegister {
+  public:
+    OpRegister(std::string type) {
+        OpRegistry::RegisterOp(type);
+    }
+};
+
+#define REGISTER_OP(op_class, op_maker_class, type_name)         \
+    class op_class##Register {                                   \
+      private:                                                   \
+        const static OpRegister<#op_class, #op_maker_class> reg; \
+    };                                                           \
+    const Register op_class##Register::reg(#type_name);
+    
+class CosineOp {
+// ...
+}
+
+struct CosineOpProtoMaker : public OpProtoMaker {
+	CosineOpProtoMaker(OpProto* proto) : OpProtoMaker(proto) {
+		AddInput("input", "input of cosine op");
+		AddAttr("scale", "scale of cosine op", float).Default(1.0).GreaterThan(0.0);
+		AddType("cos");
+		AddComment("This is cos op");
+	}
+}
+
+REGISTER_OP(CosineOp, CosineOpProtoMaker, cos);
+```
+
+In `REGISTER_OP(CosineOp, CosineOpProtoMaker, cos)`, we register not only `CosineOp` but also `CosineOpProto`. As fields of `CosineOpProto`, the default value and value range of `scale` are also registered here. 
+
+## Python API
+
+Python  APIs are divided into two types, high-level API and low-level API.
+
+### High-Level API
+
+High-level API is called by users directly, so it should keep its style consistent with existing V2 APIs.
+
+Here is a sample about how a define a fc layer:
+
+```python
+hd = fc_layer(input=data, size=56, with_bias=True, activation="sigmoid");
+```
+
+`hd` is the output of `fc_layer` and it's a `variable`. It can be further sent into other layers as input.
+
+The definition of `fc_layer()`:
+
+```python
+def fc_layer(input, size, with_bias, activation):
+	attr_map = {"size":size}
+	check_attrs(attr_map)
+	w = make_variable('w')
+	if with_bias:
+		b = make_variable('b')
+	else:
+		b = None
+	fc_output = make_variable('fc_output');
+	fc_op(input, w, b, fc_output, attr_map)
+	act_output = make_variable('sigmod_output');
+	if activation == "sigmod":
+		sigmod_op(fc_output, act_output);
+	elif:
+		# ...
+	return act_output;
+```
+
+### Low Leval API
+
+In above sample, `fc_op` and `sigmod_op` are low-level API. They build `OpDesc` and invoke corresponding C++ code.
+
+*TODO*
diff --git a/doc_cn/_sources/design/tensor_array.md.txt b/doc_cn/_sources/design/tensor_array.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..37e4f7b90f94fa3eb015e733999cd84c96b2239c
--- /dev/null
+++ b/doc_cn/_sources/design/tensor_array.md.txt
@@ -0,0 +1,271 @@
+# Design for TensorArray
+This design doc presents the necessity of a new C++ class `TensorArray`.
+In addition to the very simple C++ implementation
+
+```c++
+class TensorArray {
+ public:
+  explicit TensorArray(const LoDTensor&);
+  explicit TensorArray(size_t size);
+
+ private:
+  vector values_;
+};
+```
+
+We also need to expose it to PaddlePaddle's Python API,
+because users would want to use it with our very flexible operators `WhileLoop`.
+An example for a RNN based on dynamic operators is 
+
+```python
+input = pd.data(...)
+num_steps = Var(12)
+
+TensorArray states(size=num_steps)
+TensorArray step_inputs(unstack_from=input)
+TensorArray step_outputs(size=num_steps)
+
+W = Tensor(...)
+U = Tensor(...)
+default_state = some_op()
+
+step = Var(1)
+
+wloop = paddle.create_whileloop(loop_vars=[step])
+with wloop.frame():
+    wloop.break_if(pd.equal(step, num_steps)
+    pre_state = states.read(step-1, default_state)
+    step_input = step_inputs.read(step)
+    state = pd.sigmoid(pd.matmul(U, pre_state) + pd.matmul(W, step_input))
+    states.write(step, state)
+    step_outputs.write(step, state) # output state
+    step.update(state+1)
+
+output = step_outputs.stack()
+```
+
+## Background
+Steps are one of the core concepts of RNN. In each time step of RNN, there should be several input segments, states, and output segments; all these components act like arrays, for example, call `states[step_id]` will get the state in `step_id`th time step.
+
+An RNN can be implemented with the following pseudocode
+
+```c++
+Array states;
+Array input_segments;
+Array output_segments;
+Parameter W, U;
+
+step = 1
+seq_len = 12
+while_loop {
+   if (step == seq_len) break;
+    states[step] = sigmoid(W * states[step-1] + U * input_segments[step]);
+    output_segments[step] = states[step] // take state as output
+   step++;
+}
+```
+According to the [RNN roadmap](https://github.com/PaddlePaddle/Paddle/issues/4561), there are several different RNNs that PaddlePaddle will eventually support.
+
+Currently, the basic RNN implementation supported by PaddlePaddle is the `recurrent_op` which takes tensors as input and splits them into `input_segments`.
+
+
+Since a tensor cannot store variable-length sequences directly, PaddlePaddle implements the tensor with level of details (`LoDTensor` for short).
+Segmenting the `LoDTensor` is much more complicated than splitting a tensor, that makes it necessary to refactor the `recurrent_op` with `LoDTensor` segmenting support.
+
+As the next step in RNN support, `dynamic_recurrent_op` should be introduced to handle inputs with variable-length sequences.
+
+The implementation is similar to `recurrent_op`. 
+The key difference is the way **the original input `LoDTensors` and outupts are split to get the `input_segments` and the `output_segments`.**
+
+
+Though it can't be built over `recurrent_op` or `dynamic_recurrent_op` directly,
+the logic behind splitting a tensor or a LoD tensor into `input_segments` remains the same.
+
+## Why `TensorArray`
+The logic behind splitting the inputs to segments, states and outputs is similar and can be shared in a seperate module.
+
+The array of `states`, `input_segments` and `output_segments` would be exposed to users when writing a dynamic RNN model similar to the above pseudo codes. 
+
+So there should be an array-like container, which can store the segments of a tensor or LoD tensor.
+
+**This container can store an array of tensors and provides several methods to split a tensor or a LoD tensor** .
+This is where the notion of `TensorArray` comes from.
+
+## Introduce TensorArray to uniform all the three RNNs
+TensorArray as a new concept is borrowed from TensorFlow, 
+it is meant to be used with dynamic iteration primitives such as `while_loop` and `map_fn`.
+
+This concept can be used to support our new design of dynamic operations, and help to refactor some existing variant-sentence-related layers, 
+such as `recurrent_op`, `RecurrentGradientMachine`.
+
+In [our design for dynamic RNN](https://github.com/PaddlePaddle/Paddle/pull/4401), 
+`TensorArray` is used to segment inputs and store states in all time steps.
+By providing some methods similar to a C++ array,
+the definition of some state-based dynamic models such as RNN can be more natural and highly flexible.
+
+## Dynamic-operations on TensorArray
+
+`TensorArray` will be used directly when defining dynamic models, so some operators listed below should be implemented
+
+```python
+# several helper operators for TensorArray
+def tensor_array_stack(ta, tensor):
+    '''
+    get a tensor array `ta`, return a packed `tensor`.
+    '''
+    pass
+
+def tensor_array_unstack(tensor, ta):
+    '''
+    get a `tensor`, unstack it and get a tensor array `ta`.
+    '''
+    pass
+
+def tensor_array_write(ta, index, tensor, data_shared):
+    '''
+    get a `tensor` and a scalar tensor `index`, write `tensor` into index-th
+    value of the tensor array `ta`.
+    `data_shared` is an attribute that specifies whether to copy or reference the tensors.
+    '''
+    pass
+
+def tensor_array_read(ta, index, tensor):
+    '''
+    get a tensor array `ta`, a scalar tensor `index`, read the index-th value of
+    `ta` and return as the `tensor`.
+    '''
+    pass
+
+def tensor_array_size(ta, tensor):
+    '''
+    get a tensor array `ta`, return the size of `ta` and return as the scalar `tensor`.
+    '''
+    pass
+```
+
+It is trivial for users to use so many low-level operators, so some helper methods should be proposed in python wrapper to make `TensorArray` easier to use, 
+for example
+
+```python
+class TensorArray:
+    def __init__(self, name):
+        self.name = name
+        self.desc = TensorArrayDesc()
+
+    def stack(self, name=None):
+        '''
+        Pack the values in a `TensorArray` into a tensor with rank one higher
+        than each tensor in `values`.
+        `stack` can be used to split tensor into time steps for RNN or whileloop.
+
+        @name: str
+            the name of the variable to output.
+        '''
+        tensor = Var(name)
+        tensor_array_stack(self.name, tensor)
+        return tensor
+
+    def unstack(self, input):
+        '''
+        Unpacks the given dimension of a rank-`R` tensor into rank-`(R-1)` tensors.
+        `unstack` can be used to concatenate all the time steps for RNN or whileloop.
+
+        @input: str
+            the name of input tensor
+        '''
+        tensor_array_unstack(tensor, self.name)
+
+    def write(self, index, value, data_shared=True):
+        '''
+        Write value into index of the TensorArray.
+        If `data_shared` is set to True, than the index-th value in TensorArray will
+        be shared with the tensor passed in.
+
+        @index: str
+            name of a scalar tensor
+        @value: str
+            name of a tensor
+        @data_shared: bool
+        '''
+        tensor_array_write(self.name, index, value, data_shared)
+
+    def read(self, index, output):
+        '''
+        Read the value at location `index` in the `TensorArray`.
+
+        @index: str
+            name of a scalar tensor
+        @output:
+            name of a output variable
+        '''
+        tensor_array_read(self.name, index, output)
+
+
+    def size(self, output):
+        '''
+        Return the number of values.
+
+        @output: str
+            name of a scalar tensor
+        '''
+        tensor_array_size(self.name, output)
+```
+
+## LoDTensor-related Supports
+The `RecurrentGradientMachine` in Paddle serves as a flexible RNN layer; it takes varience-length sequences as input, and output sequences too.
+
+Since each step of RNN can only take a tensor-represented batch of data as input, 
+some preprocess should be taken on the inputs such as sorting the sentences by their length in descending order and cut each word and pack to new batches.
+
+Such cut-like operations can be embedded into `TensorArray` as general methods called `unpack` and `pack`,
+these two operations are similar to `stack` and `unstack` except that they operate on variable-length sequences formated as a LoD tensor rather than a tensor.
+
+Some definitions are like
+
+```python
+def unpack(level):
+    '''
+    Split LodTensor in some `level` and generate batches, if set `sort_by_length`,
+    will sort by length.
+
+    Returns:
+        - a new `TensorArray`, whose values are LodTensors and represents batches
+          of data.
+        - an int32 Tensor, which stores the map from the new batch's indices to
+          original LoDTensor
+    '''
+    pass
+
+def pack(level, indices_map):
+    '''
+    Recover the original LoD-arranged LoDTensor with the values in a `TensorArray`
+    and `level` and `indices_map`.
+    '''
+    pass
+```
+
+With these two methods, a varience-length sentence supported RNN can be implemented like
+
+```c++
+// input is the varient-length data
+LodTensor sentence_input(xxx);
+TensorArray ta;
+Tensor indice_map;
+Tensor boot_state = xxx; // to initialize rnn's first state
+TensorArray::unpack(input, 1/*level*/, true/*sort_by_length*/, &ta, &indice_map);
+TessorArray step_outputs;
+TensorArray states;
+
+for (int step = 0; step = ta.size(); step++) {
+  auto state = states.read(step);
+  // rnnstep is a function which acts like a step of RNN
+  auto step_input = ta.read(step);
+  auto step_output = rnnstep(step_input, state);
+  step_outputs.write(step_output, true/*data_shared*/);
+}
+
+// rnn_output is the final output of an rnn
+LoDTensor rnn_output = ta.pack(ta, indice_map);
+```
+the code above shows that by embedding the LoDTensor-related preprocess operations into `TensorArray`,
+the implementation of a RNN that supports varient-length sentences is far more concise than `RecurrentGradientMachine` because the latter mixes all the codes together, hard to read and extend.
diff --git a/doc_cn/_sources/design/var_desc.md.txt b/doc_cn/_sources/design/var_desc.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..0b2958c1b10ef6a6ce51aa75f61e15a7f2d94b3f
--- /dev/null
+++ b/doc_cn/_sources/design/var_desc.md.txt
@@ -0,0 +1,69 @@
+## Background
+PaddlePaddle divides the description of neural network computation graph into two stages: compile time and runtime.
+
+PaddlePaddle use proto message to describe compile time graph because
+
+1. Computation graph should be able to be saved to a file.
+1. In distributed training, the graph will be serialized and send to multiple workers.
+
+The computation graph is constructed by Data Node and Operation Node. The concept to represent them is in the table below.
+
+| |compile time|runtime|
+|---|---|---|
+|Data|VarDesc(proto)|Variable(cpp)|
+|Operation|OpDesc(proto)|Operator(cpp)|
+
+
+## Definition of VarDesc
+
+A VarDesc should have a name, and value. The are two kinds of variable type in compile time, they are `LoDTensor` and `SelectedRows`. 
+
+```proto
+message VarDesc {
+  required string name = 1;
+  enum VarType {
+    LOD_TENSOR = 0;
+    SELECTED_ROWS = 1;
+  }
+  required VarType type = 2;
+  optional LoDTensorDesc lod_desc = 3;
+  optional TensorDesc selected_rows_desc = 4;
+  optional bool persistable = 5 [ default = false ];
+}
+```
+
+## Definition of TensorDesc
+
+```proto
+enum DataType {
+  BOOL = 0;
+  INT16 = 1;
+  INT32 = 2;
+  INT64 = 3;
+  FP16 = 4;
+  FP32 = 5;
+  FP64 = 6;
+}
+
+message TensorDesc {
+  required DataType data_type = 1;
+  repeated int64 dims = 2; // [UNK, 640, 480] is saved as [-1, 640, 480]
+}
+```
+
+A TensorDesc describes `SelectedRows` and `LoDTensor`. For details of `SelectedRows`, please reference [`SelectedRows`](./selected_rows.md).
+
+## Definition of LodTensorDesc
+
+```proto
+message LoDTensorDesc {
+  required TensorDesc tensor = 1;
+  optional int lod_level = 2;
+}
+```
+
+A LoDTensorDesc contains a tensor and a lod_level.
+
+## Definition of Variable in Python
+
+For Variable in Python, please reference [`Python API`](./python_api.md).
diff --git a/doc_cn/_sources/faq/build_and_install/index_cn.rst.txt b/doc_cn/_sources/faq/build_and_install/index_cn.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..f1677e216f31d79b53ac29a0afbf6fbb886a0dcd
--- /dev/null
+++ b/doc_cn/_sources/faq/build_and_install/index_cn.rst.txt
@@ -0,0 +1,111 @@
+###################
+编译安装与单元测试
+###################
+
+..  contents::
+
+1. 运行Docker GPU镜像出现 "CUDA driver version is insufficient"
+----------------------------------------------------------------
+
+用户在使用PaddlePaddle GPU的Docker镜像的时候，常常出现 `Cuda Error: CUDA driver version is insufficient for CUDA runtime version`, 原因在于没有把机器上CUDA相关的驱动和库映射到容器内部。
+具体的解决方法是：
+
+..  code-block:: bash
+
+    $ export CUDA_SO="$(\ls usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
+    $ export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
+    $ docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddlepaddle:latest-gpu
+
+更多关于Docker的安装与使用, 请参考 `PaddlePaddle Docker 文档 `_ 。
+
+
+2. CMake源码编译, 找到的PythonLibs和PythonInterp版本不一致
+----------------------------------------------------------------
+
+这是目前CMake寻找Python的逻辑存在缺陷，如果系统安装了多个Python版本，CMake找到的Python库和Python解释器版本可能有不一致现象，导致编译PaddlePaddle失败。正确的解决方法是，
+用户强制指定特定的Python版本，具体操作如下：
+
+    ..  code-block:: bash
+
+        cmake .. -DPYTHON_EXECUTABLE= -DPYTHON_LIBRARY=  -DPYTHON_INCLUDE_DIR=
+
+用户需要指定本机上Python的路径：````, ````, ````
+
+3. CMake源码编译，Paddle版本号为0.0.0
+--------------------------------------
+
+如果运行 :code:`paddle version`, 出现 :code:`PaddlePaddle 0.0.0`；或者运行 :code:`cmake ..`，出现
+
+..  code-block:: bash
+
+    CMake Warning at cmake/version.cmake:20 (message):
+      Cannot add paddle version from git tag
+
+那么用户需要拉取所有的远程分支到本机，命令为 :code:`git fetch upstream`，然后重新cmake即可。
+
+4. paddlepaddle\*.whl is not a supported wheel on this platform.
+------------------------------------------------------------------------
+
+出现这个问题的主要原因是，没有找到和当前系统匹配的paddlepaddle安装包。最新的paddlepaddle python安装包支持Linux x86_64和MacOS 10.12操作系统，并安装了python 2.7和pip 9.0.1。
+
+更新 :code:`pip` 包的方法是\:
+
+..  code-block:: bash
+
+    pip install --upgrade pip
+
+如果还不行，可以执行 :code:`python -c "import pip; print(pip.pep425tags.get_supported())"` 获取当前系统支持的python包的后缀，
+并对比是否和正在安装的后缀一致。
+
+如果系统支持的是 :code:`linux_x86_64` 而安装包是 :code:`manylinux1_x86_64` ，需要升级pip版本到最新；
+如果系统支持 :code:`manylinux1_x86_64` 而安装包（本地）是 :code:`linux_x86_64` ，可以重命名这个whl包为 :code:`manylinux1_x86_64` 再安装。
+
+5. 编译安装后执行 import paddle.v2 as paddle 报ImportError: No module named v2
+------------------------------------------------------------------------------------------
+先查看一下是否曾经安装过paddle v1版本，有的话需要先卸载：
+
+pip uninstall py_paddle paddle
+
+然后安装paddle的python环境, 在build目录下执行
+
+pip install python/dist/paddle*.whl && pip install ../paddle/dist/py_paddle*.whl
+
+6. 遇到“非法指令”或者是“illegal instruction”
+--------------------------------------------
+
+PaddlePaddle使用avx SIMD指令提高cpu执行效率，因此错误的使用二进制发行版可能会导致这种错误，请选择正确的版本。
+
+7.  python相关的单元测试都过不了
+--------------------------------
+
+如果出现以下python相关的单元测试都过不了的情况：
+
+..  code-block:: bash
+
+    24 - test_PyDataProvider (Failed)
+    26 - test_RecurrentGradientMachine (Failed)
+    27 - test_NetworkCompare (Failed)
+    28 - test_PyDataProvider2 (Failed)
+    32 - test_Prediction (Failed)
+    33 - test_Compare (Failed)
+    34 - test_Trainer (Failed)
+    35 - test_TrainerOnePass (Failed)
+    36 - test_CompareTwoNets (Failed)
+    37 - test_CompareTwoOpts (Failed)
+    38 - test_CompareSparse (Failed)
+    39 - test_recurrent_machine_generation (Failed)
+    40 - test_PyDataProviderWrapper (Failed)
+    41 - test_config_parser (Failed)
+    42 - test_swig_api (Failed)
+    43 - layers_test (Failed)
+
+并且查询PaddlePaddle单元测试的日志，提示：
+
+..  code-block:: bash
+
+    paddle package is already in your PYTHONPATH. But unittest need a clean environment.
+    Please uninstall paddle package before start unittest. Try to 'pip uninstall paddle'.
+
+解决办法是：
+
+* 卸载PaddlePaddle包 :code:`pip uninstall paddle`, 清理掉老旧的PaddlePaddle安装包，使得单元测试有一个干净的环境。如果PaddlePaddle包已经在python的site-packages里面，单元测试会引用site-packages里面的python包，而不是源码目录里 :code:`/python` 目录下的python包。同时，即便设置 :code:`PYTHONPATH` 到 :code:`/python` 也没用，因为python的搜索路径是优先已经安装的python包。
diff --git a/doc_cn/_sources/faq/cluster/index_cn.rst.txt b/doc_cn/_sources/faq/cluster/index_cn.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..e59c1e1a54a0c876d1e6e89f88030de59fb9fc1a
--- /dev/null
+++ b/doc_cn/_sources/faq/cluster/index_cn.rst.txt
@@ -0,0 +1,17 @@
+###############
+集群训练与预测
+###############
+
+..  contents::
+
+1. 集群多节点训练，日志中保存均为网络通信类错误
+------------------------------------------------
+
+集群多节点训练，日志报错为网络通信类错误，比如 :code:`Connection reset by peer` 等。
+此类报错通常是由于某一个节点的错误导致这个节点的训练进程退出，从而引发其他节点无法连接导致，可以参考下面的步骤排查：
+
+* 从 :code:`train.log` ， :code:`server.log` 找到最早报错的地方，查看是否是其他错误引发的报错（比如FPE，内存不足，磁盘空间不足等）。
+
+* 如果发现最早的报错就是网络通信的问题，很有可能是非独占方式执行导致的端口冲突，可以联系OP，看当前MPI集群是否支持resource=full参数提交，如果支持增加此参数提交，并更换job 端口。
+
+* 如果当前MPI集群并不支持任务独占模式，可以联系OP是否可以更换集群或升级当前集群。
diff --git a/doc_cn/_sources/faq/index_cn.rst.txt b/doc_cn/_sources/faq/index_cn.rst.txt
index df5e172252277a881480cd2816eb901b711abe6b..9929767cac212237b3e2c3a547ba9a3c9d5f0979 100644
--- a/doc_cn/_sources/faq/index_cn.rst.txt
+++ b/doc_cn/_sources/faq/index_cn.rst.txt
@@ -1,301 +1,11 @@
-####################
 FAQ
-####################
+====
 
-..  contents::
+..  toctree::
+  :maxdepth: 1
 
-1. 如何减少内存占用
----------------------------------
-
-神经网络的训练本身是一个非常消耗内存和显存的工作，经常会消耗数10GB的内存和数GB的显存。
-PaddlePaddle的内存占用主要分为如下几个方面\:
-
-* DataProvider缓冲池内存（只针对内存）
-* 神经元激活内存（针对内存和显存）
-* 参数内存 （针对内存和显存）
-* 其他内存杂项
-
-其中，其他内存杂项是指PaddlePaddle本身所用的一些内存，包括字符串分配，临时变量等等，暂不考虑在内。
-
-减少DataProvider缓冲池内存
-++++++++++++++++++++++++++
-
-PyDataProvider使用的是异步加载，同时在内存里直接随即选取数据来做Shuffle。即
-
-..  graphviz::
-
-    digraph {
-        rankdir=LR;
-        数据文件 -> 内存池 -> PaddlePaddle训练
-    }
-
-所以，减小这个内存池即可减小内存占用，同时也可以加速开始训练前数据载入的过程。但是，这
-个内存池实际上决定了shuffle的粒度。所以，如果将这个内存池减小，又要保证数据是随机的，
-那么最好将数据文件在每次读取之前做一次shuffle。可能的代码为
-
-..  literalinclude:: src/reduce_min_pool_size.py
-
-这样做可以极大的减少内存占用，并且可能会加速训练过程，详细文档参考 :ref:`api_pydataprovider2` 。
-
-神经元激活内存
-++++++++++++++
-
-神经网络在训练的时候，会对每一个激活暂存一些数据，如神经元激活值等。
-在反向传递的时候，这些数据会被用来更新参数。这些数据使用的内存主要和两个参数有关系，
-一是batch size，另一个是每条序列(Sequence)长度。所以，其实也是和每个mini-batch中包含
-的时间步信息成正比。
-
-所以做法可以有两种：
-
-* 减小batch size。 即在网络配置中 :code:`settings(batch_size=1000)` 设置成一个小一些的值。但是batch size本身是神经网络的超参数，减小batch size可能会对训练结果产生影响。
-* 减小序列的长度，或者直接扔掉非常长的序列。比如，一个数据集大部分序列长度是100-200,
-  但是突然有一个10000长的序列，就很容易导致内存超限，特别是在LSTM等RNN中。
-
-参数内存
-++++++++
-
-PaddlePaddle支持非常多的优化算法(Optimizer)，不同的优化算法需要使用不同大小的内存。
-例如使用 :code:`adadelta` 算法，则需要使用等于权重参数规模大约5倍的内存。举例，如果参数保存下来的模型目录
-文件为 :code:`100M`， 那么该优化算法至少需要 :code:`500M` 的内存。
-
-可以考虑使用一些优化算法，例如 :code:`momentum`。
-
-2. 如何加速PaddlePaddle的训练速度
----------------------------------
-
-加速PaddlePaddle训练可以考虑从以下几个方面\：
-
-* 减少数据载入的耗时
-* 加速训练速度
-* 利用分布式训练驾驭更多的计算资源
-
-减少数据载入的耗时
-++++++++++++++++++
-
-使用\ :code:`pydataprovider`\ 时，可以减少缓存池的大小，同时设置内存缓存功能，即可以极大的加速数据载入流程。
-:code:`DataProvider` 缓存池的减小，和之前减小通过减小缓存池来减小内存占用的原理一致。
-
-..  literalinclude:: src/reduce_min_pool_size.py
-
-同时 :code:`@provider` 接口有一个 :code:`cache` 参数来控制缓存方法，将其设置成 :code:`CacheType.CACHE_PASS_IN_MEM` 的话，会将第一个 :code:`pass` (过完所有训练数据即为一个pass)生成的数据缓存在内存里，在之后的 :code:`pass` 中，不会再从 :code:`python` 端读取数据，而是直接从内存的缓存里读取数据。这也会极大减少数据读入的耗时。
-
-
-加速训练速度
-++++++++++++
-
-PaddlePaddle支持Sparse的训练，sparse训练需要训练特征是 :code:`sparse_binary_vector` 、 :code:`sparse_vector` 、或者 :code:`integer_value` 的任一一种。同时，与这个训练数据交互的Layer，需要将其Parameter设置成 sparse 更新模式，即设置 :code:`sparse_update=True`
-
-这里使用简单的 :code:`word2vec` 训练语言模型距离，具体使用方法为\:
-
-使用一个词前两个词和后两个词，来预测这个中间的词。这个任务的DataProvider为\:
-
-..  literalinclude:: src/word2vec_dataprovider.py
-
-这个任务的配置为\:
-
-..  literalinclude:: src/word2vec_config.py
-
-
-利用更多的计算资源
-++++++++++++++++++
-
-利用更多的计算资源可以分为一下几个方式来进行\:
-
-* 单机CPU训练
-
-  * 使用多线程训练。设置命令行参数 :code:`trainer_count`。
-
-* 单机GPU训练
-
-  * 使用显卡训练。设置命令行参数 :code:`use_gpu`。
-  * 使用多块显卡训练。设置命令行参数 :code:`use_gpu` 和 :code:`trainer_count` 。
-
-* 多机训练
-
-  * 请参考 :ref:`cluster_train` 。
-
-
-3. 遇到“非法指令”或者是“illegal instruction”
---------------------------------------------
-
-PaddlePaddle使用avx SIMD指令提高cpu执行效率，因此错误的使用二进制发行版可能会导致这种错误，请选择正确的版本。
-
-4. 如何选择SGD算法的学习率
---------------------------
-
-在采用sgd/async_sgd进行训练时，一个重要的问题是选择正确的learning_rate。如果learning_rate太大，那么训练有可能不收敛，如果learning_rate太小，那么收敛可能很慢，导致训练时间过长。
-
-通常做法是从一个比较大的learning_rate开始试，如果不收敛，那减少学习率10倍继续试验，直到训练收敛为止。那么如何判断训练不收敛呢？可以估计出如果模型采用不变的输出最小的cost0是多少。
-
-如果训练过程的的cost明显高于这个常数输出的cost，那么我们可以判断为训练不收敛。举一个例子，假如我们是三分类问题，采用multi-class-cross-entropy作为cost，数据中0,1,2三类的比例为 :code:`0.2, 0.5, 0.3` , 那么常数输出所能达到的最小cost是 :code:`-(0.2*log(0.2)+0.5*log(0.5)+0.3*log(0.3))=1.03` 。如果训练一个pass（或者更早）后，cost还大于这个数，那么可以认为训练不收敛，应该降低学习率。
-
-
-5. 如何初始化参数
------------------
-
-默认情况下，PaddlePaddle使用均值0，标准差为 :math:`\frac{1}{\sqrt{d}}` 来初始化参数。其中 :math:`d` 为参数矩阵的宽度。这种初始化方式在一般情况下不会产生很差的结果。如果用户想要自定义初始化方式，PaddlePaddle目前提供两种参数初始化的方式\:
-
-* 高斯分布。将 :code:`param_attr` 设置成 :code:`param_attr=ParamAttr(initial_mean=0.0, initial_std=1.0)`
-* 均匀分布。将 :code:`param_attr` 设置成 :code:`param_attr=ParamAttr(initial_max=1.0, initial_min=-1.0)`
-
-比如设置一个全连接层的参数初始化方式和bias初始化方式，可以使用如下代码。
-
-..  code-block:: python
-
-    hidden = fc_layer(input=ipt, param_attr=ParamAttr(initial_max=1.0, initial_min=-1.0),
-                      bias_attr=ParamAttr(initial_mean=1.0, initial_std=0.0))
-
-上述代码将bias全部初始化为1.0, 同时将参数初始化为 :code:`[1.0, -1.0]` 的均匀分布。
-
-6. 如何共享参数
----------------
-
-PaddlePaddle的参数使用名字 :code:`name` 作为参数的ID，相同名字的参数，会共享参数。设置参数的名字，可以使用 :code:`ParamAttr(name="YOUR_PARAM_NAME")` 来设置。更方便的设置方式，是使得要共享的参数使用同样的 :code:`ParamAttr` 对象。
-
-简单的全连接网络，参数共享的配置示例为\:
-
-..  literalinclude:: ../../python/paddle/trainer_config_helpers/tests/configs/shared_fc.py
-
-这里 :code:`hidden_a` 和 :code:`hidden_b` 使用了同样的parameter和bias。并且softmax层的两个输入也使用了同样的参数 :code:`softmax_param`。
-
-7. \*-cp27mu-linux_x86_64.whl is not a supported wheel on this platform.
-------------------------------------------------------------------------
-
-出现这个问题的主要原因是，系统编译wheel包的时候，使用的 :code:`wheel` 包是最新的，
-而系统中的 :code:`pip` 包比较老。具体的解决方法是，更新 :code:`pip` 包并重新编译PaddlePaddle。
-更新 :code:`pip` 包的方法是\:
-
-..  code-block:: bash
-
-    pip install --upgrade pip
-
-8.  python相关的单元测试都过不了
---------------------------------
-
-如果出现以下python相关的单元测试都过不了的情况：
-
-..  code-block:: bash
-
-    24 - test_PyDataProvider (Failed)
-    26 - test_RecurrentGradientMachine (Failed)
-    27 - test_NetworkCompare (Failed)
-    28 - test_PyDataProvider2 (Failed)
-    32 - test_Prediction (Failed)
-    33 - test_Compare (Failed)
-    34 - test_Trainer (Failed)
-    35 - test_TrainerOnePass (Failed)
-    36 - test_CompareTwoNets (Failed)
-    37 - test_CompareTwoOpts (Failed)
-    38 - test_CompareSparse (Failed)
-    39 - test_recurrent_machine_generation (Failed)
-    40 - test_PyDataProviderWrapper (Failed)
-    41 - test_config_parser (Failed)
-    42 - test_swig_api (Failed)
-    43 - layers_test (Failed)
-
-并且查询PaddlePaddle单元测试的日志，提示：
-
-..  code-block:: bash
-
-    paddle package is already in your PYTHONPATH. But unittest need a clean environment.
-    Please uninstall paddle package before start unittest. Try to 'pip uninstall paddle'.
-
-解决办法是：
-
-* 卸载PaddlePaddle包 :code:`pip uninstall paddle`, 清理掉老旧的PaddlePaddle安装包，使得单元测试有一个干净的环境。如果PaddlePaddle包已经在python的site-packages里面，单元测试会引用site-packages里面的python包，而不是源码目录里 :code:`/python` 目录下的python包。同时，即便设置 :code:`PYTHONPATH` 到 :code:`/python` 也没用，因为python的搜索路径是优先已经安装的python包。
-
-
-9. 运行Docker GPU镜像出现 "CUDA driver version is insufficient"
-----------------------------------------------------------------
-
-用户在使用PaddlePaddle GPU的Docker镜像的时候，常常出现 `Cuda Error: CUDA driver version is insufficient for CUDA runtime version`, 原因在于没有把机器上CUDA相关的驱动和库映射到容器内部。
-具体的解决方法是：
-
-..  code-block:: bash
-
-    $ export CUDA_SO="$(\ls usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
-    $ export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
-    $ docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddlepaddle:latest-gpu
-
-更多关于Docker的安装与使用, 请参考 `PaddlePaddle Docker 文档 `_ 。
-
-
-10. CMake源码编译, 找到的PythonLibs和PythonInterp版本不一致
-----------------------------------------------------------------
-
-这是目前CMake寻找Python的逻辑存在缺陷，如果系统安装了多个Python版本，CMake找到的Python库和Python解释器版本可能有不一致现象，导致编译PaddlePaddle失败。正确的解决方法是，
-用户强制指定特定的Python版本，具体操作如下：
-
-    ..  code-block:: bash
-
-        cmake .. -DPYTHON_EXECUTABLE= -DPYTHON_LIBRARY=  -DPYTHON_INCLUDE_DIR=
-
-用户需要指定本机上Python的路径：````, ````, ````
-
-10. A protocol message was rejected because it was too big
-----------------------------------------------------------
-
-如果在训练NLP相关模型时，出现以下错误：
-
-..  code-block:: bash
-
-    [libprotobuf ERROR google/protobuf/io/coded_stream.cc:171] A protocol message was rejected because it was too big (more than 67108864 bytes).  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
-    F1205 14:59:50.295174 14703 TrainerConfigHelper.cpp:59] Check failed: m->conf.ParseFromString(configProtoStr)
-
-可能的原因是：传给dataprovider的某一个args过大，一般是由于直接传递大字典导致的。错误的define_py_data_sources2类似：
-
-..  code-block:: python
-
-     src_dict = dict()
-     for line_count, line in enumerate(open(src_dict_path, "r")):
-        src_dict[line.strip()] = line_count
-
-     define_py_data_sources2(
-        train_list,
-        test_list,
-        module="dataprovider",
-        obj="process",
-        args={"src_dict": src_dict})
-
-解决方案是：将字典的地址作为args传给dataprovider，然后在dataprovider里面根据该地址加载字典。即define_py_data_sources2应改为：
-
-..  code-block:: python
-
-     define_py_data_sources2(
-        train_list,
-        test_list,
-        module="dataprovider",
-        obj="process",
-        args={"src_dict_path": src_dict_path})
-
-完整源码可参考 `seqToseq `_ 示例。
-
-11. 如何指定GPU设备
--------------------
-
-例如机器上有4块GPU，编号从0开始，指定使用2、3号GPU：
-
-* 方式1：通过 `CUDA_VISIBLE_DEVICES `_ 环境变量来指定特定的GPU。
-
-..      code-block:: bash
-
-        env CUDA_VISIBLE_DEVICES=2,3 paddle train --use_gpu=true --trainer_count=2
-
-* 方式2：通过命令行参数 ``--gpu_id`` 指定。
-
-..      code-block:: bash
-
-        paddle train --use_gpu=true --trainer_count=2 --gpu_id=2
-
-
-12. 训练过程中出现 :code:`Floating point exception`, 训练因此退出怎么办?
-------------------------------------------------------------------------
-
-Paddle二进制在运行时捕获了浮点数异常，只要出现浮点数异常(即训练过程中出现NaN或者Inf)，立刻退出。浮点异常通常的原因是浮点数溢出、除零等问题。
-主要原因包括两个方面:
-
-* 训练过程中参数或者训练过程中的梯度尺度过大，导致参数累加，乘除等时候，导致了浮点数溢出。
-* 模型一直不收敛，发散到了一个数值特别大的地方。
-* 训练数据有问题，导致参数收敛到了一些奇异的情况。或者输入数据尺度过大，有些特征的取值达到数百万，这时进行矩阵乘法运算就可能导致浮点数溢出。
-
-主要的解决办法是减小学习律或者对数据进行归一化处理。
+  build_and_install/index_cn.rst
+  model/index_cn.rst
+  parameter/index_cn.rst
+  local/index_cn.rst
+  cluster/index_cn.rst
diff --git a/doc_cn/_sources/faq/local/index_cn.rst.txt b/doc_cn/_sources/faq/local/index_cn.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..b331d9d36e6a279881c3b1a5586835e7186957fb
--- /dev/null
+++ b/doc_cn/_sources/faq/local/index_cn.rst.txt
@@ -0,0 +1,213 @@
+###############
+本地训练与预测
+###############
+
+..  contents::
+
+1. 如何减少内存占用
+-------------------
+
+神经网络的训练本身是一个非常消耗内存和显存的工作，经常会消耗数10GB的内存和数GB的显存。
+PaddlePaddle的内存占用主要分为如下几个方面\:
+
+* DataProvider缓冲池内存（只针对内存）
+* 神经元激活内存（针对内存和显存）
+* 参数内存 （针对内存和显存）
+* 其他内存杂项
+
+其中，其他内存杂项是指PaddlePaddle本身所用的一些内存，包括字符串分配，临时变量等等，暂不考虑在内。
+
+减少DataProvider缓冲池内存
+++++++++++++++++++++++++++
+
+PyDataProvider使用的是异步加载，同时在内存里直接随即选取数据来做Shuffle。即
+
+..  graphviz::
+
+    digraph {
+        rankdir=LR;
+        数据文件 -> 内存池 -> PaddlePaddle训练
+    }
+
+所以，减小这个内存池即可减小内存占用，同时也可以加速开始训练前数据载入的过程。但是，这
+个内存池实际上决定了shuffle的粒度。所以，如果将这个内存池减小，又要保证数据是随机的，
+那么最好将数据文件在每次读取之前做一次shuffle。可能的代码为
+
+..  literalinclude:: src/reduce_min_pool_size.py
+
+这样做可以极大的减少内存占用，并且可能会加速训练过程，详细文档参考 :ref:`api_pydataprovider2` 。
+
+神经元激活内存
+++++++++++++++
+
+神经网络在训练的时候，会对每一个激活暂存一些数据，如神经元激活值等。
+在反向传递的时候，这些数据会被用来更新参数。这些数据使用的内存主要和两个参数有关系，
+一是batch size，另一个是每条序列(Sequence)长度。所以，其实也是和每个mini-batch中包含
+的时间步信息成正比。
+
+所以做法可以有两种：
+
+* 减小batch size。 即在网络配置中 :code:`settings(batch_size=1000)` 设置成一个小一些的值。但是batch size本身是神经网络的超参数，减小batch size可能会对训练结果产生影响。
+* 减小序列的长度，或者直接扔掉非常长的序列。比如，一个数据集大部分序列长度是100-200,
+  但是突然有一个10000长的序列，就很容易导致内存超限，特别是在LSTM等RNN中。
+
+参数内存
+++++++++
+
+PaddlePaddle支持非常多的优化算法(Optimizer)，不同的优化算法需要使用不同大小的内存。
+例如使用 :code:`adadelta` 算法，则需要使用等于权重参数规模大约5倍的内存。举例，如果参数保存下来的模型目录
+文件为 :code:`100M`， 那么该优化算法至少需要 :code:`500M` 的内存。
+
+可以考虑使用一些优化算法，例如 :code:`momentum`。
+
+2. 如何加速训练速度
+-------------------
+
+加速PaddlePaddle训练可以考虑从以下几个方面\：
+
+* 减少数据载入的耗时
+* 加速训练速度
+* 利用分布式训练驾驭更多的计算资源
+
+减少数据载入的耗时
+++++++++++++++++++
+
+使用\ :code:`pydataprovider`\ 时，可以减少缓存池的大小，同时设置内存缓存功能，即可以极大的加速数据载入流程。
+:code:`DataProvider` 缓存池的减小，和之前减小通过减小缓存池来减小内存占用的原理一致。
+
+..  literalinclude:: src/reduce_min_pool_size.py
+
+同时 :code:`@provider` 接口有一个 :code:`cache` 参数来控制缓存方法，将其设置成 :code:`CacheType.CACHE_PASS_IN_MEM` 的话，会将第一个 :code:`pass` (过完所有训练数据即为一个pass)生成的数据缓存在内存里，在之后的 :code:`pass` 中，不会再从 :code:`python` 端读取数据，而是直接从内存的缓存里读取数据。这也会极大减少数据读入的耗时。
+
+
+加速训练速度
+++++++++++++
+
+PaddlePaddle支持Sparse的训练，sparse训练需要训练特征是 :code:`sparse_binary_vector` 、 :code:`sparse_vector` 、或者 :code:`integer_value` 的任一一种。同时，与这个训练数据交互的Layer，需要将其Parameter设置成 sparse 更新模式，即设置 :code:`sparse_update=True`
+
+这里使用简单的 :code:`word2vec` 训练语言模型距离，具体使用方法为\:
+
+使用一个词前两个词和后两个词，来预测这个中间的词。这个任务的DataProvider为\:
+
+..  literalinclude:: src/word2vec_dataprovider.py
+
+这个任务的配置为\:
+
+..  literalinclude:: src/word2vec_config.py
+
+
+利用更多的计算资源
+++++++++++++++++++
+
+利用更多的计算资源可以分为以下几个方式来进行\:
+
+* 单机CPU训练
+
+  * 使用多线程训练。设置命令行参数 :code:`trainer_count`。
+
+* 单机GPU训练
+
+  * 使用显卡训练。设置命令行参数 :code:`use_gpu`。
+  * 使用多块显卡训练。设置命令行参数 :code:`use_gpu` 和 :code:`trainer_count` 。
+
+* 多机训练
+
+  * 请参考 :ref:`cluster_train` 。
+
+3. 如何指定GPU设备
+------------------
+
+例如机器上有4块GPU，编号从0开始，指定使用2、3号GPU：
+
+* 方式1：通过 `CUDA_VISIBLE_DEVICES `_ 环境变量来指定特定的GPU。
+
+..      code-block:: bash
+
+        env CUDA_VISIBLE_DEVICES=2,3 paddle train --use_gpu=true --trainer_count=2
+
+* 方式2：通过命令行参数 ``--gpu_id`` 指定。
+
+..      code-block:: bash
+
+        paddle train --use_gpu=true --trainer_count=2 --gpu_id=2
+
+
+4. 训练过程中出现 :code:`Floating point exception`, 训练因此退出怎么办?
+------------------------------------------------------------------------
+
+Paddle二进制在运行时捕获了浮点数异常，只要出现浮点数异常(即训练过程中出现NaN或者Inf)，立刻退出。浮点异常通常的原因是浮点数溢出、除零等问题。
+主要原因包括两个方面:
+
+* 训练过程中参数或者训练过程中的梯度尺度过大，导致参数累加，乘除等时候，导致了浮点数溢出。
+* 模型一直不收敛，发散到了一个数值特别大的地方。
+* 训练数据有问题，导致参数收敛到了一些奇异的情况。或者输入数据尺度过大，有些特征的取值达到数百万，这时进行矩阵乘法运算就可能导致浮点数溢出。
+
+这里有两种有效的解决方法：
+
+1. 设置 :code:`gradient_clipping_threshold` 参数，示例代码如下：
+
+..  code-block:: python
+
+optimizer = paddle.optimizer.RMSProp(
+    learning_rate=1e-3,
+    gradient_clipping_threshold=10.0,
+    regularization=paddle.optimizer.L2Regularization(rate=8e-4))
+
+具体可以参考  `nmt_without_attention  `_ 示例。
+
+2. 设置 :code:`error_clipping_threshold` 参数，示例代码如下：
+
+..  code-block:: python
+
+decoder_inputs = paddle.layer.fc(
+    act=paddle.activation.Linear(),
+    size=decoder_size * 3,
+    bias_attr=False,
+    input=[context, current_word],
+    layer_attr=paddle.attr.ExtraLayerAttribute(
+        error_clipping_threshold=100.0))
+
+完整代码可以参考示例 `machine translation `_ 。
+
+两种方法的区别：
+
+1. 两者都是对梯度的截断，但截断时机不同，前者在 :code:`optimzier` 更新网络参数时应用；后者在激活函数反向计算时被调用；
+2. 截断对象不同：前者截断可学习参数的梯度，后者截断回传给前层的梯度;
+
+除此之外，还可以通过减小学习率或者对数据进行归一化处理来解决这类问题。
+
+5.  如何调用 infer 接口输出多个layer的预测结果
+-----------------------------------------------
+
+* 将需要输出的层作为 :code:`paddle.inference.Inference()` 接口的 :code:`output_layer` 参数输入，代码如下：
+
+..  code-block:: python
+
+    inferer = paddle.inference.Inference(output_layer=[layer1, layer2], parameters=parameters)
+
+* 指定要输出的字段进行输出。以输出 :code:`value` 字段为例，代码如下：
+
+..  code-block:: python
+
+    out = inferer.infer(input=data_batch, field=["value"])
+
+需要注意的是：
+
+* 如果指定了2个layer作为输出层，实际上需要的输出结果是两个矩阵；
+* 假设第一个layer的输出A是一个 N1 * M1 的矩阵，第二个 Layer 的输出B是一个 N2 * M2 的矩阵；
+* paddle.v2 默认会将A和B 横向拼接，当N1 和 N2 大小不一样时，会报如下的错误：
+
+..      code-block:: python
+
+    ValueError: all the input array dimensions except for the concatenation axis must match exactly
+
+多个层的输出矩阵的高度不一致导致拼接失败，这种情况常常发生在：
+
+* 同时输出序列层和非序列层；
+* 多个输出层处理多个不同长度的序列;
+
+此时可以在调用infer接口时通过设置 :code:`flatten_result=False` , 跳过“拼接”步骤，来解决上面的问题。这时，infer接口的返回值是一个python list:
+
+* list 中元素的个数等于网络中输出层的个数；
+* list 中每个元素是一个layer的输出结果矩阵，类型是numpy的ndarray；
+* 每一个layer输出矩阵的高度，在非序列输入时：等于样本数；序列输入时等于：输入序列中元素的总数；宽度等于配置中layer的size；
diff --git a/doc_cn/_sources/faq/model/index_cn.rst.txt b/doc_cn/_sources/faq/model/index_cn.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..b47bbe05bdb39d1ade9434a7e54bf6ca88a91cc9
--- /dev/null
+++ b/doc_cn/_sources/faq/model/index_cn.rst.txt
@@ -0,0 +1,69 @@
+#########
+模型配置
+#########
+
+..  contents::
+
+1. 出现 :code:`Duplicated layer name` 错误怎么办
+--------------------------------------------------
+
+出现该错误的原因一般是用户对不同layer的参数 :code:`name` 设置了相同的取值。遇到该错误时，先找出参数 :code:`name` 取值相同的layer，然后将这些layer的参数 :code:`name` 设置为不同的值。
+
+2. :code:`paddle.layer.memory` 的参数 :code:`name` 如何使用
+-------------------------------------------------------------
+
+* :code:`paddle.layer.memory` 用于获取特定layer上一时间步的输出，该layer是通过参数 :code:`name` 指定，即，:code:`paddle.layer.memory` 会关联参数 :code:`name` 取值相同的layer，并将该layer上一时间步的输出作为自身当前时间步的输出。
+
+* PaddlePaddle的所有layer都有唯一的name，用户通过参数 :code:`name` 设定，当用户没有显式设定时，PaddlePaddle会自动设定。而 :code:`paddle.layer.memory` 不是真正的layer，其name由参数 :code:`memory_name` 设定，当用户没有显式设定时，PaddlePaddle会自动设定。:code:`paddle.layer.memory` 的参数 :code:`name` 用于指定其要关联的layer，需要用户显式设定。
+
+3. 两种使用 drop_out 的方法有何区别
+------------------------------------
+
+* 在PaddlePaddle中使用dropout有两种方式
+
+  * 在相应layer的 :code:`layer_atter` 设置 :code:`drop_rate`，以 :code:`paddle.layer.fc` 为例，代码如下：
+
+  ..  code-block:: python
+
+      fc = paddle.layer.fc(input=input, layer_attr=paddle.attr.ExtraLayerAttribute(drop_rate=0.5))
+
+  * 使用 :code:`paddle.layer.dropout`，以 :code:`paddle.layer.fc` 为例，代码如下：
+
+  ..  code-block:: python
+
+      fc = paddle.layer.fc(input=input)
+      drop_fc = paddle.layer.dropout(input=fc, dropout_rate=0.5)
+
+* :code:`paddle.layer.dropout` 实际上使用了 :code:`paddle.layer.add_to`，并在该layer里采用第一种方式设置 :code:`drop_rate` 来使用dropout的。这种方式对内存消耗较大。
+
+* PaddlePaddle在激活函数里实现dropout，而不是在layer里实现。
+
+* :code:`paddle.layer.lstmemory`、:code:`paddle.layer.grumemory`、:code:`paddle.layer.recurrent` 不是通过一般的方式来实现对输出的激活，所以不能采用第一种方式在这几个layer里设置 :code:`drop_rate` 来使用dropout。若要对这几个layer使用dropout，可采用第二种方式，即使用 :code:`paddle.layer.dropout`。
+
+4. 不同的 recurrent layer 的区别
+----------------------------------
+以LSTM为例，在PaddlePaddle中包含以下 recurrent layer：
+
+* :code:`paddle.layer.lstmemory`
+* :code:`paddle.networks.simple_lstm`
+* :code:`paddle.networks.lstmemory_group`
+* :code:`paddle.networks.bidirectional_lstm`
+
+按照具体实现方式可以归纳为2类：
+
+1. 由 recurrent_group 实现的 recurrent layer：
+
+  * 用户在使用这一类recurrent layer时，可以访问由recurrent unit在一个时间步内计算得到的中间值（例如：hidden states, memory cells等）；
+  * 上述的 :code:`paddle.networks.lstmemory_group` 是这一类的 recurrent layer ；
+
+2. 将recurrent layer作为一个整体来实现：
+
+  * 用户在使用这一类recurrent layer，只能访问它们的输出值；
+  * 上述的 :code:`paddle.networks.lstmemory_group` 、 :code:`paddle.networks.simple_lstm` 和 :code:`paddle.networks.bidirectional_lstm` 属于这一类的实现；
+
+将recurrent layer作为一个整体来实现， 能够针对CPU和GPU的计算做更多优化， 所以相比于recurrent group的实现方式， 第二类 recurrent layer 计算效率更高。 在实际应用中，如果用户不需要访问LSTM的中间变量，而只需要获得recurrent layer计算的输出，我们建议使用第二类实现。
+
+此外，关于LSTM, PaddlePaddle中还包含 :code:`paddle.networks.lstmemory_unit` 这一计算单元：
+
+  * 不同于上述介绍的recurrent layer , :code:`paddle.networks.lstmemory_unit` 定义了LSTM单元在一个时间步内的计算过程，它并不是一个完整的recurrent layer，也不能接收序列数据作为输入；
+  * :code:`paddle.networks.lstmemory_unit` 只能在recurrent_group中作为step function使用；
diff --git a/doc_cn/_sources/faq/parameter/index_cn.rst.txt b/doc_cn/_sources/faq/parameter/index_cn.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..6fa0c64413be1616a435640b0347904a49873349
--- /dev/null
+++ b/doc_cn/_sources/faq/parameter/index_cn.rst.txt
@@ -0,0 +1,201 @@
+#########
+参数设置
+#########
+
+..  contents::
+
+1. 如何选择SGD算法的学习率
+--------------------------
+
+在采用sgd/async_sgd进行训练时，一个重要的问题是选择正确的learning_rate。如果learning_rate太大，那么训练有可能不收敛，如果learning_rate太小，那么收敛可能很慢，导致训练时间过长。
+
+通常做法是从一个比较大的learning_rate开始试，如果不收敛，那减少学习率10倍继续试验，直到训练收敛为止。那么如何判断训练不收敛呢？可以估计出如果模型采用不变的输出最小的cost0是多少。
+
+如果训练过程的的cost明显高于这个常数输出的cost，那么我们可以判断为训练不收敛。举一个例子，假如我们是三分类问题，采用multi-class-cross-entropy作为cost，数据中0,1,2三类的比例为 :code:`0.2, 0.5, 0.3` , 那么常数输出所能达到的最小cost是 :code:`-(0.2*log(0.2)+0.5*log(0.5)+0.3*log(0.3))=1.03` 。如果训练一个pass（或者更早）后，cost还大于这个数，那么可以认为训练不收敛，应该降低学习率。
+
+2. 如何设置学习率退火（learning rate annealing）
+------------------------------------------------
+
+在相应的优化算法里设置learning_rate_schedule及相关参数，以使用Adam算法为例，代码如下：
+
+..  code-block:: python
+
+    optimizer = paddle.optimizer.Adam(
+        learning_rate=1e-3,
+        learning_rate_decay_a=0.5,
+        learning_rate_decay_b=0.75,
+        learning_rate_schedule="poly",)
+
+PaddlePaddle目前支持8种learning_rate_schedule，这8种learning_rate_schedule及其对应学习率计算方式如下：
+
+* "constant"
+  
+  lr = learning_rate
+
+* "poly"
+
+  lr = learning_rate * pow(1 + learning_rate_decay_a * num_samples_processed, -learning_rate_decay_b)
+
+  其中，num_samples_processed为已训练样本数，下同。
+
+* "caffe_poly"
+
+  lr = learning_rate * pow(1.0 - num_samples_processed / learning_rate_decay_a, learning_rate_decay_b)
+
+* "exp"
+
+  lr = learning_rate * pow(learning_rate_decay_a, num_samples_processed / learning_rate_decay_b)
+
+* "discexp"
+
+  lr = learning_rate * pow(learning_rate_decay_a, floor(num_samples_processed / learning_rate_decay_b))
+
+* "linear"
+
+  lr = max(learning_rate - learning_rate_decay_a * num_samples_processed, learning_rate_decay_b)
+
+* "manual"
+
+  这是一种按已训练样本数分段取值的学习率退火方法。使用该learning_rate_schedule时，用户通过参数 :code:`learning_rate_args` 设置学习率衰减因子分段函数，当前的学习率为所设置 :code:`learning_rate` 与当前的衰减因子的乘积。以使用Adam算法为例，代码如下：
+
+  ..  code-block:: python
+
+      optimizer = paddle.optimizer.Adam(
+          learning_rate=1e-3,
+          learning_rate_schedule="manual",
+          learning_rate_args="1000:1.0,2000:0.9,3000:0.8",)
+
+  在该示例中，当已训练样本数小于等于1000时，学习率为 :code:`1e-3 * 1.0`；当已训练样本数大于1000小于等于2000时，学习率为 :code:`1e-3 * 0.9`；当已训练样本数大于2000时，学习率为 :code:`1e-3 * 0.8`。
+
+* "pass_manual"
+
+  这是一种按已训练pass数分段取值的学习率退火方法。使用该learning_rate_schedule时，用户通过参数 :code:`learning_rate_args` 设置学习率衰减因子分段函数，当前的学习率为所设置 :code:`learning_rate` 与当前的衰减因子的乘积。以使用Adam算法为例，代码如下：
+
+  ..  code-block:: python
+
+      optimizer = paddle.optimizer.Adam(
+          learning_rate=1e-3,
+          learning_rate_schedule="pass_manual",
+          learning_rate_args="1:1.0,2:0.9,3:0.8",)
+
+  在该示例中，当已训练pass数小于等于1时，学习率为 :code:`1e-3 * 1.0`；当已训练pass数大于1小于等于2时，学习率为 :code:`1e-3 * 0.9`；当已训练pass数大于2时，学习率为 :code:`1e-3 * 0.8`。
+
+3. 如何初始化参数
+-----------------
+
+默认情况下，PaddlePaddle使用均值0，标准差为 :math:`\frac{1}{\sqrt{d}}` 来初始化参数。其中 :math:`d` 为参数矩阵的宽度。这种初始化方式在一般情况下不会产生很差的结果。如果用户想要自定义初始化方式，PaddlePaddle目前提供两种参数初始化的方式\:
+
+* 高斯分布。将 :code:`param_attr` 设置成 :code:`param_attr=ParamAttr(initial_mean=0.0, initial_std=1.0)`
+* 均匀分布。将 :code:`param_attr` 设置成 :code:`param_attr=ParamAttr(initial_max=1.0, initial_min=-1.0)`
+
+比如设置一个全连接层的参数初始化方式和bias初始化方式，可以使用如下代码。
+
+..  code-block:: python
+
+    hidden = fc_layer(input=ipt, param_attr=ParamAttr(initial_max=1.0, initial_min=-1.0),
+                      bias_attr=ParamAttr(initial_mean=1.0, initial_std=0.0))
+
+上述代码将bias全部初始化为1.0, 同时将参数初始化为 :code:`[1.0, -1.0]` 的均匀分布。
+
+4. 如何共享参数
+---------------
+
+PaddlePaddle的参数使用名字 :code:`name` 作为参数的ID，相同名字的参数，会共享参数。设置参数的名字，可以使用 :code:`ParamAttr(name="YOUR_PARAM_NAME")` 来设置。更方便的设置方式，是使得要共享的参数使用同样的 :code:`ParamAttr` 对象。
+
+简单的全连接网络，参数共享的配置示例为\:
+
+..  literalinclude:: ../../python/paddle/trainer_config_helpers/tests/configs/shared_fc.py
+
+这里 :code:`hidden_a` 和 :code:`hidden_b` 使用了同样的parameter和bias。并且softmax层的两个输入也使用了同样的参数 :code:`softmax_param`。
+
+5. 如何加载预训练参数
+------------------------
+
+* 对加载预训练参数的层，设置其参数属性 :code:`is_static=True`，使该层的参数在训练过程中保持不变。以embedding层为例，代码如下：
+
+..  code-block:: python
+
+    emb_para = paddle.attr.Param(name='emb', is_static=True)
+    paddle.layer.embedding(size=word_dim, input=x, param_attr=emb_para)
+
+
+* 从模型文件将预训练参数载入 :code:`numpy.array`，在创建parameters后，使用 :code:`parameters.set()` 加载预训练参数。PaddlePaddle保存的模型参数文件前16字节为头信息，用户将参数载入 :code:`numpy.array` 时须从第17字节开始。以embedding层为例，代码如下：
+
+..  code-block:: python
+
+    def load_parameter(file_name, h, w):
+        with open(file_name, 'rb') as f:
+            f.read(16)  # skip header.
+            return np.fromfile(f, dtype=np.float32).reshape(h, w)
+
+    parameters = paddle.parameters.create(my_cost)
+    parameters.set('emb', load_parameter(emb_param_file, 30000, 256))
+
+6. 存储的参数格式是什么，如何和明文进行相互转化
+--------------------------------------------------
+
+PaddlePaddle保存的模型参数文件内容由16字节头信息和网络参数两部分组成。头信息中，1~4字节表示PaddlePaddle版本信息，请直接填充0；5~8字节表示每个参数占用的字节数，当保存的网络参数为float类型时为4，double类型时为8；9~16字节表示保存的参数总个数。
+
+将PaddlePaddle保存的模型参数还原回明文时，可以使用相应数据类型的 :code:`numpy.array` 加载具体网络参数，此时可以跳过PaddlePaddle模型参数文件的头信息。若在PaddlePaddle编译时，未指定按照double精度编译，默认情况下按照float精度计算，保存的参数也是float类型。这时在使用 :code:`numpy.array` 时，一般设置 :code:`dtype=float32` 。示例如下：
+
+..  code-block:: python
+
+    def read_parameter(fname, width):
+        s = open(fname).read()
+        # skip header
+        vec = np.fromstring(s[16:], dtype=np.float32)
+        # width is the size of the corresponding layer
+        np.savetxt(fname + ".csv", vec.reshape(width, -1),
+                fmt="%.6f", delimiter=",")
+
+
+将明文参数转化为PaddlePaddle可加载的模型参数时，首先构造头信息，再写入网络参数。下面的代码将随机生成的矩阵转化为可以被PaddlePaddle加载的模型参数。
+
+..  code-block:: python
+
+    def gen_rand_param(param_file, width, height, need_trans):
+        np.random.seed()
+        header = struct.pack("iil", 0, 4, height * width)
+        param = np.float32(np.random.rand(height, width))
+        with open(param_file, "w") as fparam:
+            fparam.write(header + param.tostring())
+
+7. A protocol message was rejected because it was too big
+------------------------------------------------------------
+
+如果在训练NLP相关模型时，出现以下错误：
+
+..  code-block:: bash
+
+    [libprotobuf ERROR google/protobuf/io/coded_stream.cc:171] A protocol message was rejected because it was too big (more than 67108864 bytes).  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
+    F1205 14:59:50.295174 14703 TrainerConfigHelper.cpp:59] Check failed: m->conf.ParseFromString(configProtoStr)
+
+可能的原因是：传给dataprovider的某一个args过大，一般是由于直接传递大字典导致的。错误的define_py_data_sources2类似：
+
+..  code-block:: python
+
+     src_dict = dict()
+     for line_count, line in enumerate(open(src_dict_path, "r")):
+        src_dict[line.strip()] = line_count
+
+     define_py_data_sources2(
+        train_list,
+        test_list,
+        module="dataprovider",
+        obj="process",
+        args={"src_dict": src_dict})
+
+解决方案是：将字典的地址作为args传给dataprovider，然后在dataprovider里面根据该地址加载字典。即define_py_data_sources2应改为：
+
+..  code-block:: python
+
+     define_py_data_sources2(
+        train_list,
+        test_list,
+        module="dataprovider",
+        obj="process",
+        args={"src_dict_path": src_dict_path})
+
+完整源码可参考 `seqToseq `_ 示例。
+
+
diff --git a/doc_cn/_sources/getstarted/basic_usage/index_cn.rst.txt b/doc_cn/_sources/getstarted/basic_usage/index_cn.rst.txt
deleted file mode 100644
index 428f58830e0b10c024f31238b7404c6df193eecd..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/getstarted/basic_usage/index_cn.rst.txt
+++ /dev/null
@@ -1,108 +0,0 @@
-经典的线性回归任务
-==================
-
-PaddlePaddle是源于百度的一个深度学习平台。这份简短的介绍将向你展示如何利用PaddlePaddle来解决一个经典的线性回归问题。
-
-任务简介
---------
-
-我们展示如何用PaddlePaddle解决 `单变量的线性回归 `_ 问题。线性回归的输入是一批点 `(x, y)` ，其中 `y = wx + b + ε`， 而 ε 是一个符合高斯分布的随机变量。线性回归的输出是从这批点估计出来的参数 `w` 和 `b` 。
-
-一个例子是房产估值。我们假设房产的价格（y）是其大小（x）的一个线性函数，那么我们可以通过收集市场上房子的大小和价格，用来估计线性函数的参数w 和 b。
-
-准备数据
------------
-
-假设变量 `x` 和 `y` 的真实关系为： `y = 2x + 0.3 + ε`，这里展示如何使用观测数据来拟合这一线性关系。首先，Python代码将随机产生2000个观测点，作为线性回归的输入。下面脚本符合PaddlePaddle期待的读取数据的Python程序的模式。
-
-.. code-block:: python
-
-    # dataprovider.py
-    from paddle.trainer.PyDataProvider2 import *
-    import random
-
-    # 定义输入数据的类型: 2个浮点数
-    @provider(input_types=[dense_vector(1), dense_vector(1)],use_seq=False)
-    def process(settings, input_file):
-        for i in xrange(2000):
-            x = random.random()
-            yield [x], [2*x+0.3]
-
-训练模型
------------
-
-为了还原 `y = 2x + 0.3`，我们先从一条随机的直线 `y' = wx + b` 开始，然后利用观测数据调整 `w` 和 `b` 使得 `y'` 和 `y` 的差距不断减小，最终趋于接近。这个过程就是模型的训练过程，而 `w` 和 `b` 就是模型的参数，即我们的训练目标。
-
-在PaddlePaddle里，该模型的网络配置如下。
-
-.. code-block:: python
-
-    # trainer_config.py
-    from paddle.trainer_config_helpers import *
-
-    # 1. 定义数据来源，调用上面的process函数获得观测数据
-    data_file = 'empty.list'
-    with open(data_file, 'w') as f: f.writelines(' ')
-    define_py_data_sources2(train_list=data_file, test_list=None, 
-                            module='dataprovider', obj='process',args={})
-
-    # 2. 学习算法。控制如何改变模型参数 w 和 b
-    settings(batch_size=12, learning_rate=1e-3, learning_method=MomentumOptimizer())
-
-    # 3. 神经网络配置
-    x = data_layer(name='x', size=1)
-    y = data_layer(name='y', size=1)
-    # 线性计算网络层: ȳ = wx + b
-    ȳ = fc_layer(input=x, param_attr=ParamAttr(name='w'), size=1, act=LinearActivation(), bias_attr=ParamAttr(name='b'))
-    # 计算误差函数，即  ȳ 和真实 y 之间的距离
-    cost = mse_cost(input= ȳ, label=y)
-    outputs(cost)
-
-
-这段简短的配置展示了PaddlePaddle的基本用法：
-
-- 第一部分定义了数据输入。一般情况下，PaddlePaddle先从一个文件列表里获得数据文件地址，然后交给用户自定义的函数（例如上面的 `process`函数）进行读入和预处理从而得到真实输入。本文中由于输入数据是随机生成的不需要读输入文件，所以放一个空列表（`empty.list`）即可。
-
-- 第二部分主要是选择学习算法，它定义了模型参数改变的规则。PaddlePaddle提供了很多优秀的学习算法，这里使用一个基于momentum的随机梯度下降(SGD)算法，该算法每批量(batch)读取12个采样数据进行随机梯度计算来更新更新。
-
-- 最后一部分是神经网络的配置。由于PaddlePaddle已经实现了丰富的网络层，所以很多时候你需要做的只是定义正确的网络层并把它们连接起来。这里使用了三种网络单元：
-    
-    - **数据层**：数据层 `data_layer` 是神经网络的入口，它读入数据并将它们传输到接下来的网络层。这里数据层有两个，分别对应于变量 `x` 和 `y`。
-    - **全连接层**：全连接层 `fc_layer` 是基础的计算单元，这里利用它建模变量之间的线性关系。计算单元是神经网络的核心，PaddlePaddle支持大量的计算单元和任意深度的网络连接，从而可以拟合任意的函数来学习复杂的数据关系。
-    - **回归误差代价层**：回归误差代价层 `mse_cost` 是众多误差代价函数层的一种，它们在训练过程作为网络的出口，用来计算模型的误差，是模型参数优化的目标函数。
-
-定义了网络结构并保存为 `trainer_config.py` 之后，运行以下训练命令：
-
-.. code-block:: bash
-
-    paddle train --config=trainer_config.py --save_dir=./output --num_passes=30
-
-PaddlePaddle将在观测数据集上迭代训练30轮，并将每轮的模型结果存放在 `./output` 路径下。从输出日志可以看到，随着轮数增加误差代价函数的输出在不断的减小，这意味着模型在训练数据上不断的改进，直到逼近真实解：` y = 2x + 0.3 `
-
-模型检验
------------
-
-训练完成后，我们希望能够检验模型的好坏。一种常用的做法是用学习的模型对另外一组测试数据进行预测，评价预测的效果。在这个例子中，由于已经知道了真实答案，我们可以直接观察模型的参数是否符合预期来进行检验。
-
-PaddlePaddle将每个模型参数作为一个numpy数组单独存为一个文件，所以可以利用如下方法读取模型的参数。
-
-.. code-block:: python
-
-    import numpy as np
-    import os
-
-    def load(file_name):
-        with open(file_name, 'rb') as f:
-            f.read(16) # skip header for float type.
-            return np.fromfile(f, dtype=np.float32)
-        
-    print 'w=%.6f, b=%.6f' % (load('output/pass-00029/w'), load('output/pass-00029/b'))
-    # w=1.999743, b=0.300137
-
-.. image:: ./parameters.png
-     :align: center
-     :scale: 80 %
-
-从图中可以看到，虽然 `w` 和 `b` 都使用随机值初始化，但在起初的几轮训练中它们都在快速逼近真实值，并且后续仍在不断改进，使得最终得到的模型几乎与真实模型一致。
-
-这样，我们用PaddlePaddle解决了单变量线性回归问题， 包括数据输入、模型训练和最后的结果验证。
diff --git a/doc_cn/_sources/getstarted/build_and_install/build_from_source_cn.rst.txt b/doc_cn/_sources/getstarted/build_and_install/build_from_source_cn.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..c875c807b8ab2e420dec189ef32d41533f58fa6d
--- /dev/null
+++ b/doc_cn/_sources/getstarted/build_and_install/build_from_source_cn.rst.txt
@@ -0,0 +1,141 @@
+从源码编译
+======================
+
+.. _build_step:
+
+编译方法
+----------------
+
+PaddlePaddle主要使用 `CMake `_ 以及GCC, G++作为编译工具。
+我们推荐您使用PaddlePaddle Docker编译环境镜像完成编译，这样可以免去单独安装编译依赖的步骤，可选的不同编译环境Docker镜像
+可以在 `这里 `_ 找到。
+
+如果您选择不使用Docker镜像，则需要在本机安装下面章节列出的 `编译依赖`_ 之后才能开始编译的步骤。
+
+编译PaddlePaddle，需要执行：
+
+.. code-block:: bash
+
+   git clone https://github.com/PaddlePaddle/Paddle.git
+   cd Paddle
+   # 如果使用Docker编译环境，执行下面的命令编译CPU-Only的二进制
+   docker run -it -v $PWD:/paddle -e "WITH_GPU=OFF" -e "WITH_TESTING=OFF" paddlepaddle/paddle_manylinux_devel:cuda8.0_cudnn5 bash -x /paddle/paddle/scripts/docker/build.sh
+   # 如果不使用Docker编译环境，执行下面的命令
+   mkdir build
+   cd build
+   cmake -DWITH_GPU=OFF -DWITH_TESTING=OFF ..
+   make
+
+编译完成后会在build/python/dist目录下生成输出的whl包，可以选在在当前机器安装也可以拷贝到目标机器安装：
+
+.. code-block:: bash
+
+   pip install build/python/dist/*.whl
+
+
+.. _run_test:
+
+执行单元测试
+----------------
+
+如果您期望在编译完成后立即执行所有的单元测试，可以按照下面的方法：
+
+使用Docker的情况下，设置 :code:`RUN_TEST=ON` 和 :code:`WITH_TESTING=ON` 就会在完成编译之后，立即执行单元测试。
+开启 :code:`WITH_GPU=ON` 可以指定同时执行GPU上的单元测试。
+
+.. code-block:: bash
+
+   docker run -it -v $PWD:/paddle -e "WITH_GPU=OFF" -e "WITH_TESTING=ON" -e "RUN_TEST=ON" paddlepaddle/paddle_manylinux_devel:cuda8.0_cudnn5 bash -x /paddle/paddle/scripts/docker/build.sh
+
+如果不使用Docker，可以执行ctest命令即可：
+
+.. code-block:: bash
+
+   mkdir build
+   cd build
+   cmake -DWITH_GPU=OFF -DWITH_TESTING=OFF ..
+   make
+   ctest
+   # 指定执行其中一个单元测试 test_mul_op
+   ctest -R test_mul_op
+
+.. _compile_deps:
+
+编译依赖
+----------------
+
+PaddlePaddle编译需要使用到下面的依赖（包含但不限于），其他的依赖软件，会自动在编译时下载。
+
+.. csv-table:: PaddlePaddle编译依赖
+   :header: "依赖", "版本", "说明"
+   :widths: 10, 15, 30
+
+   "CMake", ">=3.5", ""
+   "GCC", "4.8.2", "推荐使用CentOS的devtools2"
+   "Python", "2.7.x", "依赖libpython2.7.so"
+   "pip", ">=9.0", ""
+   "numpy", "", ""
+   "SWIG", ">=2.0", ""
+   "Go", ">=1.8", "可选"
+
+
+.. _build_options:
+
+编译选项
+----------------
+
+PaddlePaddle的编译选项，包括生成CPU/GPU二进制文件、链接何种BLAS库等。
+用户可在调用cmake的时候设置它们，详细的cmake使用方法可以参考
+`官方文档 `_ 。
+
+在cmake的命令行中，通过使用 ``-D`` 命令设置该类编译选项，例如：
+
+..  code-block:: bash
+
+    cmake .. -DWITH_GPU=OFF
+
+..  csv-table:: 编译选项说明
+    :header: "选项", "说明", "默认值"
+    :widths: 1, 7, 2
+
+    "WITH_GPU", "是否支持GPU", "ON"
+    "WITH_C_API", "是否仅编译CAPI", "OFF"
+    "WITH_DOUBLE", "是否使用双精度浮点数", "OFF"
+    "WITH_DSO", "是否运行时动态加载CUDA动态库，而非静态加载CUDA动态库。", "ON"
+    "WITH_AVX", "是否编译含有AVX指令集的PaddlePaddle二进制文件", "ON"
+    "WITH_PYTHON", "是否内嵌PYTHON解释器", "ON"
+    "WITH_STYLE_CHECK", "是否编译时进行代码风格检查", "ON"
+    "WITH_TESTING", "是否开启单元测试", "ON"
+    "WITH_DOC", "是否编译中英文文档", "OFF"
+    "WITH_SWIG_PY", "是否编译PYTHON的SWIG接口，该接口可用于预测和定制化训练", "Auto"
+    "WITH_GOLANG", "是否编译go语言的可容错parameter server", "ON"
+    "WITH_MKL", "是否使用MKL数学库，如果为否则是用OpenBLAS", "ON"
+
+BLAS
++++++
+
+PaddlePaddle支持 `MKL `_ 和
+`OpenBlAS `_ 两种BLAS库。默认使用MKL。如果使用MKL并且机器含有AVX2指令集，
+还会下载MKL-DNN数学库，详细参考 `这里 `_ 。
+
+如果关闭MKL，则会使用OpenBLAS作为BLAS库。
+
+CUDA/cuDNN
++++++++++++
+
+PaddlePaddle在编译时/运行时会自动找到系统中安装的CUDA和cuDNN库进行编译和执行。
+使用参数 :code:`-DCUDA_ARCH_NAME=Auto` 可以指定开启自动检测SM架构，加速编译。
+
+PaddlePaddle可以使用cuDNN v5.1之后的任何一个版本来编译运行，但尽量请保持编译和运行使用的cuDNN是同一个版本。
+我们推荐使用最新版本的cuDNN。
+
+编译选项的设置
+++++++++++++++
+
+PaddePaddle通过编译时指定路径来实现引用各种BLAS/CUDA/cuDNN库。cmake编译时，首先在系统路径（ :code:`/usr/lib:/usr/local/lib` ）中搜索这几个库，同时也会读取相关路径变量来进行搜索。 通过使用 ``-D`` 命令可以设置，例如 
+
+..  code-block:: bash
+
+    cmake .. -DWITH_GPU=ON -DWITH_TESTING=OFF -DCUDNN_ROOT=/opt/cudnnv5
+
+**注意：这几个编译选项的设置，只在第一次cmake的时候有效。如果之后想要重新设置，推荐清理整个编译目录（** :code:`rm -rf` ）**后，再指定。**
diff --git a/doc_cn/_sources/getstarted/build_and_install/cmake/build_from_source_cn.rst.txt b/doc_cn/_sources/getstarted/build_and_install/cmake/build_from_source_cn.rst.txt
deleted file mode 100644
index be0c1ffa451b2901ec06621dd4d886f800b4562e..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/getstarted/build_and_install/cmake/build_from_source_cn.rst.txt
+++ /dev/null
@@ -1,43 +0,0 @@
-PaddlePaddle的编译选项
-======================
-
-PaddlePaddle的编译选项，包括生成CPU/GPU二进制文件、链接何种BLAS库等。用户可在调用cmake的时候设置它们，详细的cmake使用方法可以参考 `官方文档 `_ 。
-
-Bool型的编译选项
-----------------
-用户可在cmake的命令行中，通过使用 ``-D`` 命令设置该类编译选项，例如
-
-..  code-block:: bash
-
-    cmake .. -DWITH_GPU=OFF
-
-..  csv-table:: Bool型的编译选项
-    :widths: 1, 7, 2
-    :file: compile_options.csv
-
-BLAS/CUDA/Cudnn的编译选项
---------------------------
-BLAS
-+++++
-
-PaddlePaddle支持以下任意一种BLAS库：`MKL `_ ，`ATLAS `_ ，`OpenBlAS `_ 和 `REFERENCE BLAS `_ 。
-
-..  csv-table:: BLAS路径相关的编译选项
-    :widths: 1, 2, 7
-    :file: cblas_settings.csv
-
-CUDA/Cudnn
-+++++++++++
-
-PaddlePaddle可以使用cudnn v2之后的任何一个版本来编译运行，但尽量请保持编译和运行使用的cudnn是同一个版本。 我们推荐使用最新版本的cudnn v5.1。
-
-编译选项的设置
-++++++++++++++
-
-PaddePaddle通过编译时指定路径来实现引用各种BLAS/CUDA/Cudnn库。cmake编译时，首先在系统路径(/usr/lib\:/usr/local/lib)中搜索这几个库，同时也会读取相关路径变量来进行搜索。 通过使用 ``-D`` 命令可以设置，例如 
-
-..  code-block:: bash
-
-    cmake .. -DMKL_ROOT=/opt/mkl/ -DCUDNN_ROOT=/opt/cudnnv5
-
-注意：这几个编译选项的设置，只在第一次cmake的时候有效。如果之后想要重新设置，推荐清理整个编译目录（``rm -rf``）后，再指定。
diff --git a/doc_cn/_sources/getstarted/build_and_install/docker_install_cn.rst.txt b/doc_cn/_sources/getstarted/build_and_install/docker_install_cn.rst.txt
index da2d4234658b6ea4730346e721437cc1633c4362..f78b1fb0e11aa028a4b7abb5270740b97f8039e9 100644
--- a/doc_cn/_sources/getstarted/build_and_install/docker_install_cn.rst.txt
+++ b/doc_cn/_sources/getstarted/build_and_install/docker_install_cn.rst.txt
@@ -1,183 +1,139 @@
-PaddlePaddle的Docker容器使用方式
+使用Docker安装运行
 ================================
 
-PaddlePaddle目前唯一官方支持的运行的方式是Docker容器。因为Docker能在所有主要操作系统（包括Linux，Mac OS X和Windows）上运行。 请注意，您需要更改 `Dockers设置 `_ 才能充分利用Mac OS X和Windows上的硬件资源。
+使用Docker安装和运行PaddlePaddle可以无需考虑依赖环境即可运行。并且也可以在Windows的docker中运行。
+您可以在 `Docker官网 `_ 获得基本的Docker安装和使用方法。
 
+如果您在使用Windows，可以参考
+`这篇 `_
+教程，完成在Windows上安装和使用Docker。
 
-PaddlePaddle发布的Docker镜像使用说明
-------------------------------
+在了解Docker的基本使用方法之后，即可开始下面的步骤：
 
-我们把PaddlePaddle的编译环境打包成一个镜像，称为开发镜像，里面涵盖了
-PaddlePaddle需要的所有编译工具。把编译出来的PaddlePaddle也打包成一个镜
-像，称为生产镜像，里面涵盖了PaddlePaddle运行所需的所有环境。每次
-PaddlePaddle发布新版本的时候都会发布对应版本的生产镜像以及开发镜像。运
-行镜像包括纯CPU版本和GPU版本以及其对应的非AVX版本。我们会在
-`dockerhub.com `_ 提供最新
-的Docker镜像，可以在"tags"标签下找到最新的Paddle镜像版本。为了方便在国
-内的开发者下载Docker镜像，我们提供了国内的镜像服务器供大家使用。如果您
-在国内，请把文档里命令中的paddlepaddle/paddle替换成
-docker.paddlepaddle.org/paddle。
-
-1. 开发镜像：:code:`paddlepaddle/paddle:-dev`
-
-   这个镜像包含了Paddle相关的开发工具以及编译和运行环境。用户可以使用开发镜像代替配置本地环境，完成开发，编译，发布，
-   文档编写等工作。由于不同的Paddle的版本可能需要不同的依赖和工具，所以如果需要自行配置开发环境需要考虑版本的因素。
-   开发镜像包含了以下工具：
-   
-   - gcc/clang
-   - nvcc
-   - Python
-   - sphinx
-   - woboq
-   - sshd
-   很多开发者会使用远程的安装有GPU的服务器工作，用户可以使用ssh登录到这台服务器上并执行 :code:`docker exec`进入开发镜像并开始工作，
-   也可以在开发镜像中启动一个SSHD服务，方便开发者直接登录到镜像中进行开发:
-
-   以交互容器方式运行开发镜像：
+.. _docker_pull:
 
-   .. code-block:: bash
+获取PaddlePaddle的Docker镜像
+------------------------------
 
-      docker run -it --rm paddlepaddle/paddle:-dev /bin/bash
+执行下面的命令获取最新的PaddlePaddle Docker镜像
 
-   或者，可以以后台进程方式运行容器：
+  .. code-block:: bash
 
-   .. code-block:: bash
+     docker pull paddlepaddle/paddle
 
-      docker run -d -p 2202:22 -p 8888:8888 paddledev/paddle:-dev
+对于国内用户，我们提供了加速访问的镜像源：
 
-   然后用密码 :code:`root` SSH进入容器：
+  .. code-block:: bash
 
-   .. code-block:: bash
+     docker pull docker.paddlepaddle.org/paddle
 
-      ssh -p 2202 root@localhost
+下载GPU版本的Docker镜像：
 
-   SSH方式的一个优点是我们可以从多个终端进入容器。比如，一个终端运行vi，另一个终端运行Python。另一个好处是我们可以把PaddlePaddle容器运行在远程服务器上，并在笔记本上通过SSH与其连接。
+  .. code-block:: bash
 
-2. 生产镜像：根据CPU、GPU和非AVX区分了如下4个镜像：
+     docker pull paddlepaddle/paddle:latest-gpu
+     docker pull docker.paddlepaddle.org/paddle:latest-gpu
 
-   - GPU/AVX：:code:`paddlepaddle/paddle:-gpu`
-   - GPU/no-AVX：:code:`paddlepaddle/paddle:-gpu-noavx`
-   - CPU/AVX：:code:`paddlepaddle/paddle:`
-   - CPU/no-AVX：:code:`paddlepaddle/paddle:-noavx`
+选择下载使用不同的BLAS库的Docker镜像：
 
-   纯CPU镜像以及GPU镜像都会用到AVX指令集，但是2008年之前生产的旧电脑不支持AVX。以下指令能检查Linux电脑是否支持AVX：
+  .. code-block:: bash
 
-   .. code-block:: bash
-
-      if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
+     # 默认是使用MKL的镜像
+     docker pull paddlepaddle/paddle
+     # 使用OpenBLAS的镜像
+     docker pull paddlepaddle/paddle:latest-openblas
 
-   如果输出是No，就需要选择使用no-AVX的镜像
+下载指定版本的Docker镜像，可以从 `DockerHub网站 `_ 获取可选的tag，并执行下面的命令：
 
-   以上方法在GPU镜像里也能用，只是请不要忘记提前在物理机上安装GPU最新驱动。
-   为了保证GPU驱动能够在镜像里面正常运行，我们推荐使用[nvidia-docker](https://github.com/NVIDIA/nvidia-docker)来运行镜像。
+  .. code-block:: bash
 
-   .. code-block:: bash
+     docker pull paddlepaddle/paddle:[tag]
+     # 比如：
+     docker pull docker.paddlepaddle.org/paddle:0.10.0-gpu
 
-      nvidia-docker run -it --rm paddledev/paddle:0.10.0rc1-gpu /bin/bash
+.. _docker_run:
 
-   注意: 如果使用nvidia-docker存在问题，你也许可以尝试更老的方法，具体如下，但是我们并不推荐这种方法。：
-
-   .. code-block:: bash
-
-      export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
-      export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
-      docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:-gpu
-
-3. 运行以及发布您的AI程序
-
-   假设您已经完成了一个AI训练的python程序 :code:`a.py`，这个程序是您在开发机上使用开发镜像完成开发。此时您可以运行这个命令在开发机上进行测试运行：
+在Docker中执行PaddlePaddle训练程序
+------------------------------
 
-   .. code-block:: bash
+假设您已经在当前目录（比如在/home/work）编写了一个PaddlePaddle的程序 :code:`train.py` （可以参考
+`PaddlePaddleBook `_ 
+编写），就可以使用下面的命令开始执行训练：
 
-      docker run -it -v $PWD:/work paddle /work/a.py
+  .. code-block:: bash
 
-   如果要使用GPU，请运行：
+     cd /home/work
+     docker run -it -v $PWD:/work paddlepaddle/paddle /work/train.py
+ 
+上述命令中， :code:`-it` 参数说明容器已交互式运行； :code:`-v $PWD:/work`
+指定将当前路径（Linux中$PWD变量会展开为当前路径的绝对路径）挂载到容器内部的 :code:`/work`
+目录； :code:`paddlepaddle/paddle` 指定需要使用的容器； 最后 :code:`/work/train.py`
+为容器内执行的命令，即运行训练程序。
 
-   .. code-block:: bash
+当然，您也可以进入到Docker容器中，以交互式的方式执行或调试您的代码：
 
-      nvidia-docker run -it -v $PWD:/work paddle /work/a.py
+  .. code-block:: bash
+     docker run -it -v $PWD:/work paddlepaddle/paddle /bin/bash
+     cd /work
+     python train.py
 
+**注：PaddlePaddle Docker镜像为了减小体积，默认没有安装vim，您可以在容器中执行** :code:`apt-get install -y vim` **安装后，在容器中编辑代码。**
 
-   这里`a.py`包含的所有依赖假设都可以在Paddle的运行容器中。如果需要包含更多的依赖、或者需要发布您的应用的镜像，可以编写`Dockerfile`使用`FROM paddledev/paddle:`
-   创建和发布自己的AI程序镜像。
+.. _docker_run_book:
 
-运行PaddlePaddle Book
----------------------
-
-Jupyter Notebook是一个开源的web程序，大家可以通过它制作和分享带有代码、公式、图表、文字的交互式文档。用户可以通过网页浏览文档。
+使用Docker启动PaddlePaddle Book教程
+------------------------------
 
-PaddlePaddle Book是为用户和开发者制作的一个交互式的Jupyter Nodebook。
+使用Docker可以快速在本地启动一个包含了PaddlePaddle官方Book教程的Jupyter Notebook，可以通过网页浏览。
+PaddlePaddle Book是为用户和开发者制作的一个交互式的Jupyter Notebook。
 如果您想要更深入了解deep learning，PaddlePaddle Book一定是您最好的选择。
+大家可以通过它阅读教程，或者制作和分享带有代码、公式、图表、文字的交互式文档。
 
 我们提供可以直接运行PaddlePaddle Book的Docker镜像，直接运行：
 
-.. code-block:: bash
+  .. code-block:: bash
 
-    docker run -p 8888:8888 paddlepaddle/book
+     docker run -p 8888:8888 paddlepaddle/book
 
 然后在浏览器中输入以下网址：
 
-.. code-block:: text
+  .. code-block:: text
 
-    http://localhost:8888/
+     http://localhost:8888/
 
 就这么简单，享受您的旅程！
 
-通过Docker容器开发PaddlePaddle
-------------------------------
-
-开发人员可以在Docker开发镜像中开发PaddlePaddle。这样开发人员可以以一致的方式在不同的平台上工作 - Linux，Mac OS X和Windows。
-
-1. 制作PaddlePaddle开发镜像
-
-   PaddlePaddle每次发布新版本都会发布对应的开发镜像供开发者直接使用。这里介绍如生成造这个开发镜像。
-   生成Docker镜像的方式有两个，一个是直接把一个容器转换成镜像，另一个是创建Dockerfile并运行docker build指令按照Dockerfile生成镜像。第一个方法的好处是简单快捷，适合自己实验，可以快速迭代。第二个方法的好处是Dockerfile可以把整个生成流程描述很清楚，其他人很容易看懂镜像生成过程，持续集成系统也可以简单地复现这个过程。我们采用第二个方法。Dockerfile位于PaddlePaddle repo的根目录。生成生产镜像只需要运行：
-
-   .. code-block:: bash
-      
-      git clone https://github.com/PaddlePaddle/Paddle.git
-      cd Paddle
-      docker build -t paddle:dev .
+.. _docker_run_gpu:
 
-   docker build这个命令的-t指定了生成的镜像的名字，这里我们用paddle:dev。到此，PaddlePaddle开发镜像就被构建完毕了。
+使用Docker执行GPU训练
+------------------------------
 
-2. 制作PaddlePaddle生产镜像
+为了保证GPU驱动能够在镜像里面正常运行，我们推荐使用
+`nvidia-docker `_ 来运行镜像。
+请不要忘记提前在物理机上安装GPU最新驱动。
 
-   生产镜像的生成分为两步，第一步是运行：
+  .. code-block:: bash
 
-   .. code-block:: bash
-      
-      docker run -v $(pwd):/paddle -e "WITH_GPU=OFF" -e "WITH_AVX=OFF" -e "WITH_TEST=ON" paddle:dev
+     nvidia-docker run -it -v $PWD:/work paddledev/paddle:latest-gpu /bin/bash
 
-   以上命令会编译PaddlePaddle，生成运行程序，以及生成创建生产镜像的Dockerfile。所有生成的的文件都在build目录下。“WITH_GPU”控制生成的生产镜像是否支持GPU，“WITH_AVX”控制生成的生产镜像是否支持AVX，”WITH_TEST“控制是否生成单元测试。
+**注: 如果没有安装nvidia-docker，可以尝试以下的方法，将CUDA库和Linux设备挂载到Docker容器内：**
 
-   第二步是运行：
+  .. code-block:: bash
 
-   .. code-block:: bash
-      
-      docker build -t paddle:prod -f build/Dockerfile ./build
+     export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
+     export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
+     docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:latest-gpu
 
-   以上命令会按照生成的Dockerfile把生成的程序拷贝到生产镜像中并做相应的配置，最终生成名为paddle:prod的生产镜像。
+**关于AVX：**
 
-3. 运行单元测试
+AVX是一种CPU指令集，可以加速PaddlePaddle的计算。最新的PaddlePaddle Docker镜像默认
+是开启AVX编译的，所以，如果您的电脑不支持AVX，需要单独
+`编译 <./build_from_source_cn.rst>`_ PaddlePaddle为no-avx版本。
 
-   运行以下指令：
+以下指令能检查Linux电脑是否支持AVX：
 
    .. code-block:: bash
-      
-      docker run -it -v $(pwd):/paddle paddle:dev bash -c "cd /paddle/build && ctest"
-
-文档
-----
 
-Paddle的Docker开发镜像带有一个通过 `woboq code browser
-`_ 生成的HTML版本的C++源代码，便于用户浏览C++源码。
-
-只要在Docker里启动PaddlePaddle的时候给它一个名字，就可以再运行另一个Nginx Docker镜像来服务HTML代码：
-
-.. code-block:: bash
-
-   docker run -d --name paddle-cpu-doc paddle:-dev
-   docker run -d --volumes-from paddle-cpu-doc -p 8088:80 nginx
+      if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
 
-接着我们就能够打开浏览器在 http://localhost:8088/paddle/ 浏览代码。
+如果输出是No，就需要选择使用no-AVX的镜像
diff --git a/doc_cn/_sources/getstarted/build_and_install/index_cn.rst.txt b/doc_cn/_sources/getstarted/build_and_install/index_cn.rst.txt
index a24df6c518fad84a48061ecb34ee46cb312a4995..c9ba84c842b530162c92713046e64fdf82bd441b 100644
--- a/doc_cn/_sources/getstarted/build_and_install/index_cn.rst.txt
+++ b/doc_cn/_sources/getstarted/build_and_install/index_cn.rst.txt
@@ -6,24 +6,28 @@
 安装流程
 ++++++++
 
-PaddlePaddle提供数个预编译的二进制来进行安装，包括Docker镜像，ubuntu的deb安装包等。我们推荐使用Docker镜像来部署环境，同时欢迎贡献更多的安装包。
+PaddlePaddle提供pip和Docker的安装方式：
 
 .. toctree::
    :maxdepth: 1
-   
-   docker_install_cn.rst 
-   ubuntu_install_cn.rst
-
 
+   pip_install_cn.rst
+   docker_install_cn.rst
+   ../../howto/dev/build_cn.md
 
 编译流程
 ++++++++
 
 ..  warning::
 
-    编译流程主要推荐高级用户查看，普通用户请走安装流程。
+    建议直接使用上述安装流程，方便快速安装。只有在遇到需要独立定制的二进制时才需要编译。
 
 ..  toctree::
     :maxdepth: 1
 
-    cmake/build_from_source_cn.rst
+    build_from_source_cn.rst
+
+常见问题解答
+++++++++++
+
+`常见问题解答 `_
diff --git a/doc_cn/_sources/getstarted/build_and_install/pip_install_cn.rst.txt b/doc_cn/_sources/getstarted/build_and_install/pip_install_cn.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..b270e2c2f0b0cbfd6fb4b9b0750d207952f84d76
--- /dev/null
+++ b/doc_cn/_sources/getstarted/build_and_install/pip_install_cn.rst.txt
@@ -0,0 +1,86 @@
+使用pip安装
+================================
+
+PaddlePaddle可以使用常用的Python包管理工具
+`pip `_
+完成安装，并可以在大多数主流的Linux操作系统以及MacOS上执行。
+
+.. _pip_install:
+
+使用pip安装
+------------------------------
+
+
+执行下面的命令即可在当前机器上安装PaddlePaddle的运行时环境，并自动下载安装依赖软件。
+
+  .. code-block:: bash
+
+     pip install paddlepaddle
+
+
+如果需要安装支持GPU的版本，需要执行：
+
+  .. code-block:: bash
+
+     pip install paddlepaddle-gpu
+
+如果需要获取并安装最新的（开发分支）PaddlePaddle，可以从我们的CI系统中下载最新的whl安装包和c-api开发包并安装，
+您可以从下面的表格中找到需要的版本：
+
+如果在点击下面链接时出现如下登陆界面，点击“Log in as guest”即可开始下载：
+
+.. image:: paddleci.png
+   :scale: 50 %
+   :align: center
+
+..  csv-table:: 各个版本最新的whl包
+    :header: "版本说明", "cp27-cp27mu", "cp27-cp27m", "C-API"
+    :widths: 1, 3, 3, 3
+
+    "cpu_avx_mkl", "`paddlepaddle-0.10.0-cp27-cp27mu-linux_x86_64.whl `_", "`paddlepaddle-0.10.0-cp27-cp27m-linux_x86_64.whl `_", "`paddle.tgz `_"
+    "cpu_avx_openblas", "`paddlepaddle-0.10.0-cp27-cp27mu-linux_x86_64.whl `_", "`paddlepaddle-0.10.0-cp27-cp27m-linux_x86_64.whl `_", "暂无"
+    "cuda7.5_cudnn5_avx_mkl", "`paddlepaddle-0.10.0-cp27-cp27mu-linux_x86_64.whl `_", "`paddlepaddle-0.10.0-cp27-cp27m-linux_x86_64.whl `_", "`paddle.tgz `_"
+    "cuda8.0_cudnn5_avx_mkl", "`paddlepaddle-0.10.0-cp27-cp27mu-linux_x86_64.whl `_", "`paddlepaddle-0.10.0-cp27-cp27m-linux_x86_64.whl `_", "`paddle.tgz `_"
+    "cuda8.0_cudnn7_avx_mkl", "`paddlepaddle-0.10.0-cp27-cp27mu-linux_x86_64.whl `_", "`paddlepaddle-0.10.0-cp27-cp27m-linux_x86_64.whl `_", "`paddle.tgz `_"
+
+.. _pip_dependency:
+
+运行环境依赖
+------------------------------
+
+PaddlePaddle安装包由于不仅仅包含.py程序，而且包含了C++编写的部分，所以我们确保发布的二进制包可以支持主流的Linux操作系统，比如CentOS 6以上，Ubuntu 14.04以上，MacOS 10.12以上。
+
+PaddlePaddle发布的安装包会尽量对齐 `manylinux1 `_ 标准，通常使用CentOS 5作为编译环境。但由于CUDA库通常需要CentOS 6以上，而且CentOS 5即将停止维护，所以我们默认使用CentOS 6作为标准编译环境。
+
+.. csv-table:: PaddlePaddle环境依赖
+   :header: "依赖", "版本", "说明"
+   :widths: 10, 15, 30
+
+   "操作系统", "Linux, MacOS", "CentOS 6以上，Ubuntu 14.04以上，MacOS 10.12以上"
+   "Python", "2.7.x", "暂时不支持Python3"
+   "libc.so", "GLIBC_2.7", "glibc至少包含GLIBC_2.7以上的符号"
+   "libstdc++.so", "GLIBCXX_3.4.11, CXXABI_1.3.3", "至少包含GLIBCXX_3.4.11, CXXABI_1.3.3以上的符号"
+   "libgcc_s.so", "GCC_3.3", "至少包含GCC_3.3以上的符号"
+
+.. _pip_faq:
+
+安装常见问题和解决方法
+------------------------------
+
+- paddlepaddle*.whl is not a supported wheel on this platform.
+  
+  出现这个问题的主要原因是，没有找到和当前系统匹配的paddlepaddle安装包。请检查Python版本是否为2.7系列。另外最新的pip官方源中的安装包默认是manylinux1标准，需要使用最新的pip (>9.0.0) 才可以安装。可以使用下面的命令更新您的pip：
+
+    .. code-block:: bash
+
+       pip install --upgrade pip
+
+  如果仍然存在问题，可以执行：
+
+      .. code-block:: bash
+
+         python -c "import pip; print(pip.pep425tags.get_supported())"
+
+  获取当前系统支持的安装包格式，并检查和需安装的包是否匹配。pypi安装包可以在 `这个 `_ 链接中找到。
+
+  如果系统支持的是 linux_x86_64 而安装包是 manylinux1_x86_64 ，需要升级pip版本到最新； 如果系统支持 manylinux1_x86_64 而安装包（本地）是 linux_x86_64 ，可以重命名这个whl包为 manylinux1_x86_64 再安装。
diff --git a/doc_cn/_sources/getstarted/build_and_install/ubuntu_install_cn.rst.txt b/doc_cn/_sources/getstarted/build_and_install/ubuntu_install_cn.rst.txt
deleted file mode 100644
index 9e39ccb00f5d5655c30148900a3d76a22aacfc01..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/getstarted/build_and_install/ubuntu_install_cn.rst.txt
+++ /dev/null
@@ -1,71 +0,0 @@
-Ubuntu部署PaddlePaddle
-===================================
-
-PaddlePaddle提供了ubuntu 14.04 deb安装包。
-
-安装
-------
-
-安装包的下载地址是\: https://github.com/PaddlePaddle/Paddle/releases
-
-它包含四个版本\:
-
-* cpu版本: 支持主流x86处理器平台, 使用了avx指令集。
-
-* cpu-noavx版本：支持主流x86处理器平台，没有使用avx指令集。
-
-* gpu版本：支持主流x86处理器平台，支持nvidia cuda平台，使用了avx指令集。
-
-* gpu-noavx版本：支持主流x86处理器平台，支持nvidia cuda平台，没有使用avx指令集。
-
-下载完相关安装包后，执行:
-
-..  code-block:: shell
-
-    sudo apt-get install gdebi
-    gdebi paddle-*-cpu.deb
-
-或者:
-
-..  code-block:: shell
-
-    dpkg -i paddle-*-cpu.deb
-    apt-get install -f
-
-
-在 :code:`dpkg -i` 的时候如果报一些依赖未找到的错误是正常的，
-在 :code:`apt-get install -f` 里会继续安装 PaddlePaddle。
-
-安装完成后，可以使用命令 :code:`paddle version` 查看安装后的paddle 版本:
-
-..  code-block:: shell
-
-    PaddlePaddle 0.8.0b1, compiled with
-        with_avx: ON
-        with_gpu: OFF
-        with_double: OFF
-        with_python: ON
-        with_rdma: OFF
-        with_timer: OFF
-        with_predict_sdk:
-
-
-可能遇到的问题
---------------
-
-libcudart.so/libcudnn.so找不到
-++++++++++++++++++++++++++++++
-
-安装完成后，运行 :code:`paddle train` 报错\:
-
-..  code-block:: shell
-
-      0831 12:36:04.151525  1085 hl_dso_loader.cc:70] Check failed: nullptr != *dso_handle For Gpu version of PaddlePaddle, it couldn't find CUDA library: libcudart.so Please make sure you already specify its path.Note: for training data on Cpu using Gpu version of PaddlePaddle,you must specify libcudart.so via LD_LIBRARY_PATH.
-
-原因是未设置cuda运行时环境变量。 如果使用GPU版本的PaddlePaddle，请安装CUDA 7.5 和CUDNN 5到本地环境中，并设置：
-
-..  code-block:: shell
-
-    export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/lib:$LD_LIBRARY_PATH
-    export PATH=/usr/local/cuda/bin:$PATH
-
diff --git a/doc_cn/_sources/getstarted/concepts/use_concepts_cn.rst.txt b/doc_cn/_sources/getstarted/concepts/use_concepts_cn.rst.txt
index e63ca11102c8ce457afcc3c262fa5f159361c01d..c243083794bb3c4659242de99b3b2715af9d7c24 100644
--- a/doc_cn/_sources/getstarted/concepts/use_concepts_cn.rst.txt
+++ b/doc_cn/_sources/getstarted/concepts/use_concepts_cn.rst.txt
@@ -81,9 +81,9 @@ PaddlePaddle支持不同类型的输入数据，主要包括四种类型，和
 ..	code-block:: bash
 
     y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear())
-    cost = paddle.layer.mse_cost(input=y_predict, label=y)
+    cost = paddle.layer.square_error_cost(input=y_predict, label=y)
 
-其中，x与y为之前描述的输入层；而y_predict是接收x作为输入，接上一个全连接层；cost接收y_predict与y作为输入，接上均方误差层。
+其中，x与y为之前描述的输入层；而y_predict是接收x作为输入，接上一个全连接层；cost接收y_predict与y作为输入，接上平方误差层。
 
 最后一层cost中记录了神经网络的所有拓扑结构，通过组合不同的layer，我们即可完成神经网络的搭建。
 
@@ -111,7 +111,7 @@ PaddlePaddle支持不同类型的输入数据，主要包括四种类型，和
     # define training dataset reader
     def train_reader():
         train_x = np.array([[1, 1], [1, 2], [3, 4], [5, 2]])
-        train_y = np.array([-2, -3, -7, -7])
+        train_y = np.array([[-2], [-3], [-7], [-7]])
         def reader():
             for i in xrange(train_y.shape[0]):
                 yield train_x[i], train_y[i]
@@ -147,4 +147,4 @@ PaddlePaddle支持不同类型的输入数据，主要包括四种类型，和
 ..  literalinclude:: src/train.py
     :linenos:
 
-有关线性回归的实际应用，可以参考PaddlePaddle book的 `第一章节 `_。
\ No newline at end of file
+有关线性回归的实际应用，可以参考PaddlePaddle book的 `第一章节 `_。
diff --git a/doc_cn/_sources/getstarted/index_cn.rst.txt b/doc_cn/_sources/getstarted/index_cn.rst.txt
index 0cb27f802c40ef123fdc9c6799aad3b2a5f554c0..a9087be6f350c5656cabb0c64ba0f200d1c666cc 100644
--- a/doc_cn/_sources/getstarted/index_cn.rst.txt
+++ b/doc_cn/_sources/getstarted/index_cn.rst.txt
@@ -1,10 +1,61 @@
 新手入门
 ============
 
+.. _quick_install:
+
+快速安装
+++++++++
+
+PaddlePaddle支持使用pip快速安装，目前支持CentOS 6以上, Ubuntu 14.04以及MacOS 10.12，并安装有Python2.7。
+执行下面的命令完成快速安装：
+
+  .. code-block:: bash
+
+     pip install paddlepaddle
+
+如果需要安装支持GPU的版本，需要执行：
+
+  .. code-block:: bash
+
+     pip install paddlepaddle-gpu
+
+更详细的安装和编译方法参考：
+
 ..  toctree::
   :maxdepth: 1
 
   build_and_install/index_cn.rst
-  concepts/use_concepts_cn.rst
 
-- `深度学习入门课程 `_
+.. _quick_start:
+
+快速开始
+++++++++
+
+创建一个 housing.py 并粘贴此Python代码：
+
+  .. code-block:: python
+
+     import paddle.v2 as paddle
+
+     # Initialize PaddlePaddle.
+     paddle.init(use_gpu=False, trainer_count=1)
+
+     # Configure the neural network.
+     x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13))
+     y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear())
+
+     # Infer using provided test data.
+     probs = paddle.infer(
+         output_layer=y_predict,
+         parameters=paddle.dataset.uci_housing.model(),
+         input=[item for item in paddle.dataset.uci_housing.test()()])
+
+     for i in xrange(len(probs)):
+         print 'Predicted price: ${:,.2f}'.format(probs[i][0] * 1000)
+
+执行 :code:`python housing.py` 瞧！ 它应该打印出预测住房数据的清单。
+
+..  toctree::
+  :maxdepth: 1
+
+  concepts/use_concepts_cn.rst
diff --git a/doc_cn/_sources/howto/deep_model/rnn/hierarchical_layer_cn.rst.txt b/doc_cn/_sources/howto/deep_model/rnn/hierarchical_layer_cn.rst.txt
index 79048e92482851af6c2dd7d055868ebcaa7a298b..e05173c2006ff47ecb6ca5a4fe1502de750acc59 100644
--- a/doc_cn/_sources/howto/deep_model/rnn/hierarchical_layer_cn.rst.txt
+++ b/doc_cn/_sources/howto/deep_model/rnn/hierarchical_layer_cn.rst.txt
@@ -28,17 +28,17 @@ pooling 的使用示例如下，详细见 :ref:`api_v2.layer_pooling` 配置API
 
         seq_pool = pooling(input=layer,
                            pooling_type=pooling.Max(),
-                           agg_level=AggregateLevel.EACH_SEQUENCE)
+                           agg_level=AggregateLevel.TO_SEQUENCE)
         
 - `pooling_type` 目前支持两种，分别是：pooling.Max()和pooling.Avg()。
 
-- `agg_level=AggregateLevel.EACH_TIMESTEP` 时（默认值）：
+- `agg_level=AggregateLevel.TO_NO_SEQUENCE` 时（默认值）：
 
   - 作用：双层序列经过运算变成一个0层序列，或单层序列经过运算变成一个0层序列
   - 输入：一个双层序列，或一个单层序列
   - 输出：一个0层序列，即整个输入序列（单层或双层）的平均值（或最大值）
 
-- `agg_level=AggregateLevel.EACH_SEQUENCE` 时：
+- `agg_level=AggregateLevel.TO_SEQUENCE` 时：
 
   - 作用：一个双层序列经过运算变成一个单层序列
   - 输入：必须是一个双层序列
@@ -52,15 +52,15 @@ last_seq 的使用示例如下（ :ref:`api_v2.layer_first_seq` 类似），详
 ..	code-block:: bash
 
         last = last_seq(input=layer,
-                        agg_level=AggregateLevel.EACH_SEQUENCE)
+                        agg_level=AggregateLevel.TO_SEQUENCE)
         
-- `agg_level=AggregateLevel.EACH_TIMESTEP` 时（默认值）：
+- `agg_level=AggregateLevel.TO_NO_SEQUENCE` 时（默认值）：
 
   - 作用：一个双层序列经过运算变成一个0层序列，或一个单层序列经过运算变成一个0层序列
   - 输入：一个双层序列或一个单层序列
   - 输出：一个0层序列，即整个输入序列（双层或者单层）最后一个，或第一个元素。
 
-- `agg_level=AggregateLevel.EACH_SEQUENCE` 时：
+- `agg_level=AggregateLevel.TO_SEQUENCE` 时：
   - 作用：一个双层序列经过运算变成一个单层序列
   - 输入：必须是一个双层序列
   - 输出：一个单层序列，其中每个元素是双层序列中每个subseq最后一个（或第一个）元素。
@@ -74,9 +74,9 @@ expand 的使用示例如下，详细见 :ref:`api_v2.layer_expand` 配置API。
 
         ex = expand(input=layer1,
                     expand_as=layer2,
-                    expand_level=ExpandLevel.FROM_TIMESTEP)
+                    expand_level=ExpandLevel.FROM_NO_SEQUENCE)
         
-- `expand_level=ExpandLevel.FROM_TIMESTEP` 时（默认值）：
+- `expand_level=ExpandLevel.FROM_NO_SEQUENCE` 时（默认值）：
 
   - 作用：一个0层序列经过运算扩展成一个单层序列，或者一个双层序列
   - 输入：layer1必须是一个0层序列，是待扩展的数据；layer2 可以是一个单层序列，或者是一个双层序列，提供扩展的长度信息
diff --git a/doc_cn/_sources/howto/deep_model/rnn/hrnn_rnn_api_compare_cn.rst.txt b/doc_cn/_sources/howto/deep_model/rnn/hrnn_rnn_api_compare_cn.rst.txt
index 96e52b910a22576fd75c9d4e1bef6e2cf74bc84f..efdc44455ea4dc81a87b4d4fc8a81e78b15cb06a 100644
--- a/doc_cn/_sources/howto/deep_model/rnn/hrnn_rnn_api_compare_cn.rst.txt
+++ b/doc_cn/_sources/howto/deep_model/rnn/hrnn_rnn_api_compare_cn.rst.txt
@@ -81,7 +81,7 @@
   
   * 在本例中，我们将原始数据的每一组，通过\ :code:`recurrent_group`\ 进行拆解，拆解成的每一句话再通过一个LSTM网络。这和单层RNN的配置是等价的。
 
-* 与单层RNN的配置类似，我们只需要使用LSTM encode成的最后一个向量。所以对\ :code:`recurrent_group`\ 进行了\ :code:`last_seq`\ 操作。但和单层RNN不同，我们是对每一个子序列取最后一个元素，因此\ :code:`agg_level=AggregateLevel.EACH_SEQUENCE`\ 。
+* 与单层RNN的配置类似，我们只需要使用LSTM encode成的最后一个向量。所以对\ :code:`recurrent_group`\ 进行了\ :code:`last_seq`\ 操作。但和单层RNN不同，我们是对每一个子序列取最后一个元素，因此\ :code:`agg_level=AggregateLevel.TO_SEQUENCE`\ 。
 
 * 至此，\ :code:`lstm_last`\ 便和单层RNN配置中的\ :code:`lstm_last`\ 具有相同的结果了。
 
diff --git a/doc_cn/_sources/howto/deep_model/rnn/index_cn.rst.txt b/doc_cn/_sources/howto/deep_model/rnn/index_cn.rst.txt
index 9e805ca85191b793c8798a239927a318c70b96f5..9ecab5594cff47cde4700b7ce0f58013a960a16e 100644
--- a/doc_cn/_sources/howto/deep_model/rnn/index_cn.rst.txt
+++ b/doc_cn/_sources/howto/deep_model/rnn/index_cn.rst.txt
@@ -4,6 +4,7 @@ RNN相关模型
 ..  toctree::
   :maxdepth: 1
 
+  rnn_config_cn.rst
   recurrent_group_cn.md
   hierarchical_layer_cn.rst
   hrnn_rnn_api_compare_cn.rst
diff --git a/doc_cn/_sources/howto/deep_model/rnn/rnn_config_cn.rst.txt b/doc_cn/_sources/howto/deep_model/rnn/rnn_config_cn.rst.txt
index ac2bd0775f4ab2e0a0c37462e2c23001123b152b..63fa161fafed0f3a8ec8799af21304cbec62d813 100644
--- a/doc_cn/_sources/howto/deep_model/rnn/rnn_config_cn.rst.txt
+++ b/doc_cn/_sources/howto/deep_model/rnn/rnn_config_cn.rst.txt
@@ -5,36 +5,13 @@ RNN配置
 中配置循环神经网络（RNN）。PaddlePaddle
 高度支持灵活和高效的循环神经网络配置。 在本教程中，您将了解如何：
 
--  准备用来学习循环神经网络的序列数据。
 -  配置循环神经网络架构。
 -  使用学习完成的循环神经网络模型生成序列。
 
 我们将使用 vanilla 循环神经网络和 sequence to sequence
 模型来指导你完成这些步骤。sequence to sequence
-模型的代码可以在\ ``demo / seqToseq``\ 找到。
-
-准备序列数据
-------------
-
-PaddlePaddle
-不需要对序列数据进行任何预处理，例如填充。唯一需要做的是将相应类型设置为输入。例如，以下代码段定义了三个输入。
-它们都是序列，它们的大小是\ ``src_dict``\ ，\ ``trg_dict``\ 和\ ``trg_dict``\ ：
-
-.. code:: python
-
-    settings.input_types = [
-      integer_value_sequence(len(settings.src_dict)),
-      integer_value_sequence(len(settings.trg_dict)),
-      integer_value_sequence(len(settings.trg_dict))]
-
-在\ ``process``\ 函数中，每个\ ``yield``\ 函数将返回三个整数列表。每个整数列表被视为一个整数序列：
-
-.. code:: python
-
-    yield src_ids, trg_ids, trg_ids_next
-
-有关如何编写数据提供程序的更多细节描述，请参考 :ref:`api_pydataprovider2` 。完整的数据提供文件在
-``demo/seqToseq/dataprovider.py``\ 。
+模型的代码可以在 `book/08.machine_translation `_ 找到。
+wmt14数据的提供文件在 `python/paddle/v2/dataset/wmt14.py `_ 。
 
 配置循环神经网络架构
 --------------------
@@ -44,7 +21,7 @@ PaddlePaddle
 
 循环神经网络在每个时间步骤顺序地处理序列。下面列出了 LSTM 的架构的示例。
 
-.. image:: ../../../tutorials/sentiment_analysis/bi_lstm.jpg
+.. image:: src/bi_lstm.jpg
       :align: center
 
 一般来说，循环网络从 :math:`t=1` 到 :math:`t=T` 或者反向地从 :math:`t=T` 到 :math:`t=1` 执行以下操作。
@@ -85,19 +62,19 @@ vanilla
                    act=None,
                    rnn_layer_attr=None):
         def __rnn_step__(ipt):
-           out_mem = memory(name=name, size=size)
-           rnn_out = mixed_layer(input = [full_matrix_projection(ipt),
-                                          full_matrix_projection(out_mem)],
-                                 name = name,
-                                 bias_attr = rnn_bias_attr,
-                                 act = act,
-                                 layer_attr = rnn_layer_attr,
-                                 size = size)
+           out_mem = paddle.layer.memory(name=name, size=size)
+           rnn_out = paddle.layer.mixed(input = [paddle.layer.full_matrix_projection(input=ipt),
+                                                 paddle.layer.full_matrix_projection(input=out_mem)],
+                                        name = name,
+                                        bias_attr = rnn_bias_attr,
+                                        act = act,
+                                        layer_attr = rnn_layer_attr,
+                                        size = size)
            return rnn_out
-        return recurrent_group(name='%s_recurrent_group' % name,
-                               step=__rnn_step__,
-                               reverse=reverse,
-                               input=input)
+        return paddle.layer.recurrent_group(name='%s_recurrent_group' % name,
+                                            step=__rnn_step__,
+                                            reverse=reverse,
+                                            input=input)
 
 PaddlePaddle
 使用“Memory”（记忆模块）实现单步函数。\ **Memory**\ 是在PaddlePaddle中构造循环神经网络时最重要的概念。
@@ -119,7 +96,7 @@ Sequence to Sequence Model with Attention
 我们将使用 sequence to sequence model with attention
 作为例子演示如何配置复杂的循环神经网络模型。该模型的说明如下图所示。
 
-.. image:: ../../../tutorials/text_generation/encoder-decoder-attention-model.png
+.. image:: src/encoder-decoder-attention-model.png
       :align: center
 
 在这个模型中，源序列 :math:`S = \{s_1, \dots, s_T\}` 
@@ -140,43 +117,52 @@ Sequence to Sequence Model with Attention
 .. code:: python
 
     # 定义源语句的数据层
-    src_word_id = data_layer(name='source_language_word', size=source_dict_dim)
+    src_word_id = paddle.layer.data(
+        name='source_language_word',
+        type=paddle.data_type.integer_value_sequence(source_dict_dim))
     # 计算每个词的词向量
-    src_embedding = embedding_layer(
+    src_embedding = paddle.layer.embedding(
         input=src_word_id,
         size=word_vector_dim,
-        param_attr=ParamAttr(name='_source_language_embedding'))
+        param_attr=paddle.attr.ParamAttr(name='_source_language_embedding'))
     # 应用前向循环神经网络
-    src_forward = grumemory(input=src_embedding, size=encoder_size)
+    src_forward = paddle.networks.simple_gru(
+        input=src_embedding, size=encoder_size)
     # 应用反向递归神经网络（reverse=True表示反向循环神经网络）
-    src_backward = grumemory(input=src_embedding,
-                              size=encoder_size,
-                              reverse=True)
+    src_backward = paddle.networks.simple_gru(
+        input=src_embedding, size=encoder_size, reverse=True)
     # 将循环神经网络的前向和反向部分混合在一起
-    encoded_vector = concat_layer(input=[src_forward, src_backward])
+    encoded_vector = paddle.layer.concat(input=[src_forward, src_backward])
 
     # 投射编码向量到 decoder_size
-    encoder_proj = mixed_layer(input = [full_matrix_projection(encoded_vector)],
-                               size = decoder_size)
+    encoded_proj = paddle.layer.mixed(
+        size=decoder_size,
+        input=paddle.layer.full_matrix_projection(encoded_vector))
 
     # 计算反向RNN的第一个实例
-    backward_first = first_seq(input=src_backward)
+    backward_first = paddle.layer.first_seq(input=src_backward)
 
     # 投射反向RNN的第一个实例到 decoder size
-    decoder_boot = mixed_layer(input=[full_matrix_projection(backward_first)], size=decoder_size, act=TanhActivation())
+    decoder_boot = paddle.layer.mixed(
+       size=decoder_size,
+       act=paddle.activation.Tanh(),
+       input=paddle.layer.full_matrix_projection(backward_first))
 
 解码器使用 ``recurrent_group`` 来定义循环神经网络。单步函数和输出函数在
 ``gru_decoder_with_attention`` 中定义：
 
 .. code:: python
 
-    group_inputs=[StaticInput(input=encoded_vector,is_seq=True),
-                  StaticInput(input=encoded_proj,is_seq=True)]
-    trg_embedding = embedding_layer(
-        input=data_layer(name='target_language_word',
-                         size=target_dict_dim),
-        size=word_vector_dim,
-        param_attr=ParamAttr(name='_target_language_embedding'))
+    group_input1 = paddle.layer.StaticInput(input=encoded_vector, is_seq=True)
+    group_input2 = paddle.layer.StaticInput(input=encoded_proj, is_seq=True)
+    group_inputs = [group_input1, group_input2]
+    trg_embedding = paddle.layer.embedding(
+            input=paddle.layer.data(
+                name='target_language_word',
+                type=paddle.data_type.integer_value_sequence(target_dict_dim)),
+            size=word_vector_dim,
+            param_attr=paddle.attr.ParamAttr(name='_target_language_embedding'))
+        group_inputs.append(trg_embedding)
     group_inputs.append(trg_embedding)
 
     # 对于配备有注意力机制的解码器，在训练中，
@@ -185,9 +171,10 @@ Sequence to Sequence Model with Attention
     # StaticInput 意味着不同时间步的输入都是相同的值，
     # 否则它以一个序列输入，不同时间步的输入是不同的。
     # 所有输入序列应该有相同的长度。
-    decoder = recurrent_group(name=decoder_group_name,
-                              step=gru_decoder_with_attention,
-                              input=group_inputs)
+    decoder = paddle.layer.recurrent_group(
+            name=decoder_group_name,
+            step=gru_decoder_with_attention,
+            input=group_inputs)
 
 单步函数的实现如下所示。首先，它定义解码网络的\ **Memory**\ 。然后定义
 attention，门控循环单元单步函数和输出函数：
@@ -198,27 +185,32 @@ attention，门控循环单元单步函数和输出函数：
         # 定义解码器的Memory
         # Memory的输出定义在 gru_step 内
         # 注意 gru_step 应该与它的Memory名字相同
-        decoder_mem = memory(name='gru_decoder',
-                             size=decoder_size,
-                             boot_layer=decoder_boot)
+        decoder_mem = paddle.layer.memory(
+            name='gru_decoder', size=decoder_size, boot_layer=decoder_boot)
         # 计算 attention 加权编码向量
-        context = simple_attention(encoded_sequence=enc_vec,
-                                   encoded_proj=enc_proj,
-                                   decoder_state=decoder_mem)
+        context = paddle.networks.simple_attention(
+            encoded_sequence=enc_vec,
+            encoded_proj=enc_proj,
+            decoder_state=decoder_mem)
         # 混合当前词向量和attention加权编码向量
-        decoder_inputs = mixed_layer(inputs = [full_matrix_projection(context),
-                                               full_matrix_projection(current_word)],
-                                     size = decoder_size * 3)
+         decoder_inputs = paddle.layer.mixed(
+            size=decoder_size * 3,
+            input=[
+                paddle.layer.full_matrix_projection(input=context),
+                paddle.layer.full_matrix_projection(input=current_word)
+            ])
         # 定义门控循环单元循环神经网络单步函数
-        gru_step = gru_step_layer(name='gru_decoder',
-                                  input=decoder_inputs,
-                                  output_mem=decoder_mem,
-                                  size=decoder_size)
+         gru_step = paddle.layer.gru_step(
+            name='gru_decoder',
+            input=decoder_inputs,
+            output_mem=decoder_mem,
+            size=decoder_size)
         # 定义输出函数
-        out = mixed_layer(input=[full_matrix_projection(input=gru_step)],
-                          size=target_dict_dim,
-                          bias_attr=True,
-                          act=SoftmaxActivation())
+         out = paddle.layer.mixed(
+            size=target_dict_dim,
+            bias_attr=True,
+            act=paddle.activation.Softmax(),
+            input=paddle.layer.full_matrix_projection(input=gru_step))
         return out
 
 生成序列
@@ -238,41 +230,32 @@ attention，门控循环单元单步函数和输出函数：
    -  ``beam_size``: beam search 算法中的beam大小。
    -  ``max_length``: 生成序列的最大长度。
 
--  使用 ``seqtext_printer_evaluator``
-   根据索引矩阵和字典打印文本。这个函数需要设置：
-
-   -  ``id_input``: 数据的整数ID，用于标识生成的文件中的相应输出。
-   -  ``dict_file``: 用于将词ID转换为词的字典文件。
-   -  ``result_file``: 生成结果文件的路径。
-
 代码如下：
 
 .. code:: python
 
-    group_inputs=[StaticInput(input=encoded_vector,is_seq=True),
-                  StaticInput(input=encoded_proj,is_seq=True)]
+    group_input1 = paddle.layer.StaticInput(input=encoded_vector, is_seq=True)
+    group_input2 = paddle.layer.StaticInput(input=encoded_proj, is_seq=True)
+    group_inputs = [group_input1, group_input2]
     # 在生成时，解码器基于编码源序列和最后生成的目标词预测下一目标词。
     # 编码源序列（编码器输出）必须由只读Memory的 StaticInput 指定。
     # 这里， GeneratedInputs 自动获取上一个生成的词，并在最开始初始化为起始词，如 。
-    trg_embedding = GeneratedInput(
-        size=target_dict_dim,
-        embedding_name='_target_language_embedding',
-        embedding_size=word_vector_dim)
+    trg_embedding = paddle.layer.GeneratedInput(
+            size=target_dict_dim,
+            embedding_name='_target_language_embedding',
+            embedding_size=word_vector_dim)
     group_inputs.append(trg_embedding)
-    beam_gen = beam_search(name=decoder_group_name,
-                           step=gru_decoder_with_attention,
-                           input=group_inputs,
-                           bos_id=0, # Beginnning token.
-                           eos_id=1, # End of sentence token.
-                           beam_size=beam_size,
-                           max_length=max_length)
-
-    seqtext_printer_evaluator(input=beam_gen,
-                              id_input=data_layer(name="sent_id", size=1),
-                              dict_file=trg_dict_path,
-                              result_file=gen_trans_file)
-    outputs(beam_gen)
-
-注意，这种生成技术只用于类似解码器的生成过程。如果你正在处理序列标记任务，请参阅 :ref:`semantic_role_labeling` 了解更多详细信息。
-
-完整的配置文件在\ ``demo/seqToseq/seqToseq_net.py``\ 。
+    beam_gen = paddle.layer.beam_search(
+            name=decoder_group_name,
+            step=gru_decoder_with_attention,
+            input=group_inputs,
+            bos_id=0, # Beginnning token.
+            eos_id=1, # End of sentence token.
+            beam_size=beam_size,
+            max_length=max_length)
+
+    return beam_gen
+
+注意，这种生成技术只用于类似解码器的生成过程。如果你正在处理序列标记任务，请参阅 `book/06.understand_sentiment `_ 了解更多详细信息。
+
+完整的配置文件在 `book/08.machine_translation/train.py `_ 。
diff --git a/doc_cn/_sources/howto/dev/build_cn.md.txt b/doc_cn/_sources/howto/dev/build_cn.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..4a80a5245102fb992f513a749f6a02e1130188af
--- /dev/null
+++ b/doc_cn/_sources/howto/dev/build_cn.md.txt
@@ -0,0 +1,124 @@
+# 用Docker编译和测试PaddlePaddle
+
+## 需要的软硬件
+
+为了开发PaddlePaddle，我们需要
+
+1. 一台电脑，可以装的是 Linux, BSD, Windows 或者 MacOS 操作系统，以及
+1. Docker。
+
+不需要依赖其他任何软件了。即便是 Python 和 GCC 都不需要，因为我们会把所有编译工具都安装进一个 Docker image 里。
+
+## 总体流程
+
+1. 获取源码
+
+   ```bash
+   git clone https://github.com/paddlepaddle/paddle
+   ```
+
+2. 安装开发工具到 Docker image 里
+
+   ```bash
+   cd paddle; docker build -t paddle:dev .
+   ```
+
+   请注意这个命令结尾处的 `.`；它表示 `docker build` 应该读取当前目录下的 [`Dockerfile`文件](https://github.com/PaddlePaddle/Paddle/blob/develop/Dockerfile)，按照其内容创建一个名为 `paddle:dev` 的 Docker image，并且把各种开发工具安装进去。
+
+3. 编译
+
+   以下命令启动一个 Docker container 来执行 `paddle:dev` 这个 Docker image，同时把当前目录（源码树根目录）映射为 container 里的 `/paddle` 目录，并且运行 `Dockerfile` 描述的默认入口程序 [`build.sh`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build.sh)。这个脚本调用 `cmake` 和 `make` 来编译 `/paddle` 里的源码，结果输出到 `/paddle/build`，也就是本地的源码树根目录里的 `build` 子目录。
+
+   ```bash
+   docker run --rm -v $PWD:/paddle paddle:dev
+   ```
+
+   上述命令编译出一个 CUDA-enabled 版本。如果我们只需要编译一个只支持 CPU 的版本，可以用
+
+   ```bash
+   docker run --rm -e WITH_GPU=OFF -v $PWD:/paddle paddle:dev
+   ```
+
+4. 运行单元测试
+
+   用本机的第一个 GPU 来运行包括 GPU 单元测试在内的所有单元测试：
+
+   ```bash
+   NV_GPU=0 nvidia-docker run --rm -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest"
+   ```
+
+   如果编译的时候我们用了 `WITH_GPU=OFF` 选项，那么编译过程只会产生 CPU-based 单元测试，那么我们也就不需要 nvidia-docker 来运行单元测试了。我们只需要：
+
+   ```bash
+   docker run --rm -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest"
+   ```
+
+   有时候我们只想运行一个特定的单元测试，比如 `memory_test`，我们可以
+
+   ```bash
+   nvidia-docker run --rm -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest -V -R memory_test"
+   ```
+
+5. 清理
+
+   有时候我们会希望清理掉已经下载的第三方依赖以及已经编译的二进制文件。此时只需要：
+
+   ```bash
+   rm -rf build
+   ```
+
+## 为什么要 Docker 呀？
+
+- 什么是 Docker?
+
+  如果您没有听说 Docker，可以把它想象为一个类似 virtualenv 的系统，但是虚拟的不仅仅是 Python 的运行环境。
+
+- Docker 还是虚拟机？
+
+  有人用虚拟机来类比 Docker。需要强调的是：Docker 不会虚拟任何硬件，Docker container 里运行的编译工具实际上都是在本机的 CPU 和操作系统上直接运行的，性能和把编译工具安装在本机运行一样。
+
+- 为什么用 Docker?
+
+  把工具和配置都安装在一个 Docker image 里可以标准化编译环境。这样如果遇到问题，其他人可以复现问题以便帮助。
+
+  另外，对于习惯使用Windows和MacOS的开发者来说，使用Docker就不用配置交叉编译环境了。
+
+- 我可以选择不用Docker吗？
+
+  当然可以。大家可以用把开发工具安装进入 Docker image 一样的方式，把这些工具安装到本机。这篇文档介绍基于 Docker 的开发流程，是因为这个流程比其他方法都更简便。
+
+- 学习 Docker 有多难？
+
+  理解 Docker 并不难，大概花十分钟看一下[这篇文章](https://zhuanlan.zhihu.com/p/19902938)。这可以帮您省掉花一小时安装和配置各种开发工具，以及切换机器时需要新安装的辛苦。别忘了 PaddlePaddle 更新可能导致需要新的开发工具。更别提简化问题复现带来的好处了。
+
+- 我可以用 IDE 吗？
+
+  当然可以，因为源码就在本机上。IDE 默认调用 make 之类的程序来编译源码，我们只需要配置 IDE 来调用 Docker 命令编译源码即可。
+
+  很多 PaddlePaddle 开发者使用 Emacs。他们在自己的 `~/.emacs` 配置文件里加两行
+
+  ```emacs
+  (global-set-key "\C-cc" 'compile)
+  (setq compile-command
+   "docker run --rm -it -v $(git rev-parse --show-toplevel):/paddle paddle:dev")
+  ```
+
+  就可以按 `Ctrl-C` 和 `c` 键来启动编译了。
+
+- 可以并行编译吗？
+
+  是的。我们的 Docker image 运行一个 [Bash 脚本](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build.sh)。这个脚本调用 `make -j$(nproc)` 来启动和 CPU 核一样多的进程来并行编译。
+
+## 可能碰到的问题
+
+- Docker 需要 sudo
+
+  如果用自己的电脑开发，自然也就有管理员权限（sudo）了。如果用公用的电脑开发，需要请管理员安装和配置好 Docker。此外，PaddlePaddle 项目在努力开始支持其他不需要 sudo 的集装箱技术，比如 rkt。
+
+- 在 Windows/MacOS 上编译很慢
+
+  Docker 在 Windows 和 MacOS 都可以运行。不过实际上是运行在一个 Linux 虚拟机上。可能需要注意给这个虚拟机多分配一些 CPU 和内存，以保证编译高效。具体做法请参考[这个issue](https://github.com/PaddlePaddle/Paddle/issues/627)。
+
+- 磁盘不够
+
+  本文中的例子里，`docker run` 命令里都用了 `--rm` 参数，这样保证运行结束之后的 containers 不会保留在磁盘上。可以用 `docker ps -a` 命令看到停止后但是没有删除的 containers。`docker build` 命令有时候会产生一些中间结果，是没有名字的 images，也会占用磁盘。可以参考[这篇文章](https://zaiste.net/posts/removing_docker_containers/)来清理这些内容。
diff --git a/doc_cn/_sources/howto/dev/contribute_to_paddle_cn.md.txt b/doc_cn/_sources/howto/dev/contribute_to_paddle_cn.md.txt
index 775938612e8d213b92e2eb69dae805838dc5ae96..699390145226ec2b65fdf5122db187e1d30d669e 100644
--- a/doc_cn/_sources/howto/dev/contribute_to_paddle_cn.md.txt
+++ b/doc_cn/_sources/howto/dev/contribute_to_paddle_cn.md.txt
@@ -7,6 +7,7 @@
 - 确保编译器选项 `WITH_STYLE_CHECK` 已打开，并且编译能通过代码样式检查。
 - 所有代码必须具有单元测试。
 - 通过所有单元测试。
+- 请遵守[提交代码的一些约定](#提交代码的一些约定)。
 
 以下教程将指导您提交代码。
 ## [Fork](https://help.github.com/articles/fork-a-repo/)
@@ -83,7 +84,7 @@ no changes added to commit (use "git add" and/or "git commit -a")
 ➜  docker build -t paddle:dev .
 ```
 
-随后可以用这个开发镜像开build PaddlePaddle的源码。比如如果要build一个不依赖GPU，但是支持AVX指令集，并且包括unit tests的PaddlePaddle，可以：
+随后可以用这个开发镜像开始build PaddlePaddle的源码。比如如果要build一个不依赖GPU，但是支持AVX指令集，并且包括unit tests的PaddlePaddle，可以：
 
 ```bash
 ➜  docker run -v $(pwd):/paddle -e "WITH_GPU=OFF" -e "WITH_AVX=ON" -e "WITH_TEST=ON" paddle:dev
@@ -217,3 +218,22 @@ upstream
 ```
 
 至此，我们就完成了一次代码贡献的过程。
+
+## 提交代码的一些约定
+
+为了使评审人在评审代码时更好地专注于代码本身，请您每次提交代码时，遵守以下约定：
+1. 请保证Travis-CI 中单元测试能顺利通过。如果没过，说明提交的代码存在问题，评审人一般不做评审。
+2. 提交PUll Request前：
+   - 请注意commit的数量：
+     - 原因：如果仅仅修改一个文件但提交了十几个commit，每个commit只做了少量的修改，这会给评审人带来很大困扰。评审人需要逐一查看每个commit才能知道做了哪些修改，且不排除commit之间的修改存在相互覆盖的情况。
+     - 建议：每次提交时，保持尽量少的commit，可以通过`git commit --amend`补充上次的commit。对已经Push到远程仓库的多个commit，可以参考[squash commits after push](http://stackoverflow.com/questions/5667884/how-to-squash-commits-in-git-after-they-have-been-pushed)。
+   - 请注意每个commit的名称：应能反映当前commit的内容，不能太随意。
+3. 如果解决了某个Issue的问题，请在该PUll Request的**第一个**评论框中加上：`fix #issue_number`，这样当该PUll Request被合并后，会自动关闭对应的Issue。关键词包括：close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved，请选择合适的词汇。详细可参考[Closing issues via commit messages](https://help.github.com/articles/closing-issues-via-commit-messages)。
+
+此外，在回复评审人意见时，请您遵守以下约定：
+1. 评审人的每个意见都必须回复（这是开源社区的基本礼貌，别人帮了忙，应该说谢谢）：
+   - 对评审意见同意且按其修改完的，给个简单的`Done`即可；
+   - 对评审意见不同意的，请给出您自己的反驳理由。
+2. 如果评审意见比较多：
+   - 请给出总体的修改情况。
+   - 请采用[start a review](https://help.github.com/articles/reviewing-proposed-changes-in-a-pull-request/)进行回复，而非直接回复的方式。原因是每个回复都会发送一封邮件，会造成邮件灾难。
diff --git a/doc_cn/_sources/howto/dev/new_layer_cn.rst.txt b/doc_cn/_sources/howto/dev/new_layer_cn.rst.txt
index 9489a921c70ad6ee5709f46445554f5d9640162c..75037e693b32f923ee7dc9dfec322495fe4ce10a 100644
--- a/doc_cn/_sources/howto/dev/new_layer_cn.rst.txt
+++ b/doc_cn/_sources/howto/dev/new_layer_cn.rst.txt
@@ -37,7 +37,7 @@
 
    \frac{\partial c(y)}{\partial x} = \frac{\partial c(y)}{\partial y} \frac{\partial y}{\partial x}
 
-假设 :math:`z = f(W^T x + b)` ，那么
+假设 :math:`z = W^T x + b` ，那么
 
 .. math::
 
diff --git a/doc_cn/_sources/howto/dev/new_op_cn.md.txt b/doc_cn/_sources/howto/dev/new_op_cn.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..6cfc9536f20e88571a9845a50be0341fe4d9f78b
--- /dev/null
+++ b/doc_cn/_sources/howto/dev/new_op_cn.md.txt
@@ -0,0 +1,332 @@
+# 如何写新的Operator
+
+ - [概念简介](#概念简介)
+ - [实现C++类](#实现C++类)
+   - [定义ProtoMaker类](#定义ProtoMaker类)
+   - [定义Operator类](#定义Operator类)
+   - [定义OpKernel类](#定义OpKernel类)
+   - [注册Operator](#注册Operator)
+   - [编译](#编译)
+ - [绑定Python](#绑定Python)
+ - [实现单元测试](#实现单元测试)
+   - [前向Operator单测](#前向Operator单测)
+   - [反向Operator单测](#反向Operator单测)
+   - [编译和执行](#编译和执行)
+
+
+## 概念简介
+
+简单介绍需要用到基类，详细介绍请参考设计文档。
+
+- `framework::OperatorBase`: Operator(简写，Op)基类。
+- `framework::OpKernel`: Op计算函数的基类，称作Kernel。
+- `framework::OperatorWithKernel`：继承自OperatorBase，Op有计算函数，称作有Kernel。
+- `class OpProtoAndCheckerMaker`：描述该Op的输入、输出、属性、注释,主要用于Python API接口生成
+
+依据是否包含kernel，可以将Op分为两种：包含Kernel的Op和不包含kernel的Op，前者Op的定义继承自`OperatorBase`，后者继承自`OperatorWithKernel`。本教程主要介绍带Kernel的Op如何写，简单总结Op需要包含的内容如下：
+
+
+ 内容            | 定义位置
+--------------  | :----------------------
+OpProtoMake定义  | `.cc`文件，Backward Op不需要定义OpProtoMake
+Op定义           | `.cc`文件
+Kernel实现       | CPU、GPU共享Kernel实现在`.h`文件中，否则，CPU 实现在`.cc`文件中，GPU 实现在`.cu`文件中。
+注册Op           | Op注册实现在`.cc`文件；Kernel注册CPU实现在`.cc`文件中，GPU实现在`.cu`文件中
+
+
+实现新的op都添加至目录[paddle/operators](https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/operators)下，文件命名以`*_op.h`（如有） 、 `*_op.cc` 、`*_op.cu`（如有）结尾。**系统会根据文件名自动构建op和其对应的Python扩展。**
+
+
+下面以矩阵乘操作，即[MulOp](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc)为例来介绍如何写带Kernel的Operator。
+
+
+## 实现C++类
+
+
+### 1. 定义ProtoMaker类
+
+矩阵乘法的公式：$Out = X * Y$, 可见该计算由两个输入，一个输出组成。
+
+首先定义`ProtoMaker`来描述该Op的输入、输出，并添加注释：
+
+```cpp
+class MulOpMaker : public framework::OpProtoAndCheckerMaker {
+ public:
+  MulOpMaker(framework::OpProto *proto, framework::OpAttrChecker *op_checker)
+      : OpProtoAndCheckerMaker(proto, op_checker) {
+    AddInput("X", "(Tensor), 2D tensor of size (M x K)");
+    AddInput("Y", "(Tensor), 2D tensor of size (K x N)");
+    AddOutput("Out", "(Tensor), 2D tensor of size (M x N)");
+    AddComment(R"DOC(
+Two Element Mul Operator.
+The equation is: Out = X * Y
+)DOC");
+  }
+};
+```
+
+[`MulOpMaker`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc#L43)继承自`framework::OpProtoAndCheckerMaker`，构造函数含有2个参数：
+
+   - `framework::OpProto` ： 前者存储Op的输入输出和参数属性，将用于Python API接口的生成。
+   - `framework::OpAttrChecker` ：后者用于检查参数属性的合法性。
+
+构造函数里通过`AddInput`添加输入参数，通过`AddOutput`添加输出参数，通过`AddComment`添加Op的注释。这些函数会将对应内容添加到`OpProto`中。
+
+上面的代码在`MulOp`中添加两个输入`X`和`Y`，添加了一个输出`Out`，并解释了各自含义，命名请遵守[命名规范](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/name_convention.md)。
+
+
+再以[`ScaleOp`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/scale_op.cc#L37)为例：
+
+```cpp
+template 
+class ScaleOpMaker : public framework::OpProtoAndCheckerMaker {
+ public:
+  ScaleOpMaker(framework::OpProto *proto, framework::OpAttrChecker *op_checker)
+      : OpProtoAndCheckerMaker(proto, op_checker) {
+    AddInput("X", "The input tensor of scale operator.").NotInGradient();
+    AddOutput("Out", "The output tensor of scale operator.").NotInGradient();
+    AddComment(R"DOC(Scale operator
+The equation is: Out = scale*X
+)DOC");
+    AddAttr("scale", "scale of scale operator.").SetDefault(1.0);
+  }
+};
+```
+
+这个例子有两处不同：
+
+- `AddInput("X","...").NotInGradient()` : 表示`X`这个输入不参与`ScaleOp`对应的梯度Op计算之中，如果Op的某个输入不参与反向梯度的计算，请显示地调用`.NotInGradient()`进行设置。
+
+- `AddAttr("scale", "...").SetDefault(1.0);` : 增加`scale`系数，作为参数属性，并且设置默认值为1.0。
+
+
+### 2. 定义Operator类
+
+下面的点实现了MulOp的定义：
+
+```cpp
+class MulOp : public framework::OperatorWithKernel {
+ public:
+  using framework::OperatorWithKernel::OperatorWithKernel;
+
+ protected:
+  void InferShape(const framework::InferShapeContext &ctx) const override {
+    auto dim0 = ctx.Input("X")->dims();
+    auto dim1 = ctx.Input("Y")->dims();
+    PADDLE_ENFORCE_EQ(dim0.size(), 2,
+                      "input X(%s) should be a tensor with 2 dims, a matrix",
+                      ctx.op_.Input("X"));
+    PADDLE_ENFORCE_EQ(dim1.size(), 2,
+                      "input Y(%s) should be a tensor with 2 dims, a matrix",
+                      ctx.op_.Input("Y"));
+    PADDLE_ENFORCE_EQ(
+        dim0[1], dim1[0],
+        "First matrix's width must be equal with second matrix's height.");
+    ctx.Output("Out")->Resize({dim0[0], dim1[1]});
+  }
+};
+```
+
+[`MulOp`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc#L22)继承自`OperatorWithKernel`。`public`成员：
+
+```cpp
+using framework::OperatorWithKernel::OperatorWithKernel;
+```
+
+这句表示使用基类`OperatorWithKernel`的构造函数，也可写成：
+
+```cpp
+MulOp(const std::string &type, const framework::VariableNameMap &inputs,
+      const framework::VariableNameMap &outputs,
+      const framework::AttributeMap &attrs)
+  : OperatorWithKernel(type, inputs, outputs, attrs) {}
+```
+
+还需要重写`InferShape`接口。`InferShape`为const函数，不能修改Op的成员变量，参数为`const framework::InferShapeContext &ctx`，通过该参数可获取到输入输出以及属性。它的功能是：
+
+  - 1). 做检查， 尽早报错：检查输入数据维度、类型等是否合法。
+  - 2). 设置输出Tensor的形状。
+
+通常`OpProtoMaker`和`Op`类的定义写在`.cc`文件中，和下面将要介绍的注册函数一起放在`.cc`中
+
+### 3. 定义OpKernel类
+
+`MulKernel`继承自`framework::OpKernel`，带有下面两个模板参数:
+
+- `typename  Place`: 表示设备类型，不同设备(CPU、GPU)共享同一个Kernel时，需加该模板参数，不共享则不加，一个不共享的例子是[`OnehotCrossEntropyOpKernel`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43)。
+
+- `typename T` : 表示数据类型，如`float`, `double`等。
+
+需要为`MulKernel`类重写`Compute`接口。
+- `Compute`接受一个输入参数：`const framework::ExecutionContext& context`。
+- 与`InferShapeContext`相比，`ExecutionContext`增加了设备类型，同样可获取到输入输出和属性参数。
+- `Compute`函数里实现`OpKernel`的具体计算逻辑。
+
+下面是 `MulKernel` `Compute`的实现：
+
+  ```cpp
+  template 
+  class MulKernel : public framework::OpKernel {
+  public:
+  void Compute(const framework::ExecutionContext& context) const override {
+    auto* X = context.Input("X");
+    auto* Y = context.Input("Y");
+    auto* Z = context.Output("Out");
+    Z->mutable_data(context.GetPlace());
+    auto* device_context =
+        const_cast(context.device_context_);
+    math::matmul(*X, false, *Y, false, 1, Z, 0, device_context);
+  }
+  };
+  ```
+
+需要注意：**不同设备(CPU、GPU)共享一个Op定义，是否则共享同一个`OpKernel`，取决于`Compute`调用的函数是否支持不同设备。**
+
+`MulOp`的CPU、GPU实现共享同一个`Kernel`。`OpKernel`不共享的例子可以参考：[`OnehotCrossEntropyOpKernel`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43)。
+
+为了使`OpKernel`的计算过程书写更加简单，并且CPU、GPU的代码可以复用，我们通常借助 Eigen unsupported Tensor模块来实现`Compute`接口。关于在PaddlePaddle中如何使用Eigen库，请参考[使用文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/dev/use_eigen_cn.md)。
+
+
+到此，前向Op实现完成。接下来，需要在`.cc`文件中注册该op和kernel。
+反向Op类的定义，反向OpKernel的定义与前向Op类似，这里不再赘述。**但需注意反向Op没有`ProtoMaker`**。
+
+### 4. 注册Operator
+
+- 在`.cc`文件中注册前向、反向Op类，注册CPU Kernel。
+
+    ```cpp
+    namespace ops = paddle::operators;
+    REGISTER_OP(mul, ops::MulOp, ops::MulOpMaker, mul_grad, ops::MulOpGrad);
+    REGISTER_OP_CPU_KERNEL(mul, ops::MulKernel);
+    REGISTER_OP_CPU_KERNEL(mul_grad,
+                  ops::MulGradKernel);
+    ```
+
+   在上面的代码中：
+
+    - `REGISTER_OP` ： 注册`ops::MulOp`类，类型名为`mul`，该类的`ProtoMaker`为`ops::MulOpMaker`，注册`ops::MulOpGrad`，类型名为`mul_grad`。
+    - `REGISTER_OP_WITHOUT_GRADIENT` ： 用于注册没有反向的Op。
+    - `REGISTER_OP_CPU_KERNEL` ：注册`ops::MulKernel`类，并特化模板参数为`paddle::platform::CPUPlace`和`float`类型，同理，注册`ops::MulGradKernel`类。
+
+
+- 在 `.cu`文件中注册GPU Kernel。
+    - 请注意，如果GPU Kernel的实现基于Eigen unsupported模块，那么在 `.cu`的开始请加上宏定义 `#define EIGEN_USE_GPU`，代码示例如下：
+
+    ```cpp
+    // if use Eigen unsupported module before include head files
+    // #define EIGEN_USE_GPU
+
+    namespace ops = paddle::operators;
+    REGISTER_OP_GPU_KERNEL(mul, ops::MulKernel);
+    REGISTER_OP_GPU_KERNEL(mul_grad,
+                           ops::MulGradKernel);
+    ```
+
+### 5. 编译
+
+运行下面命令可以进行编译：
+
+```
+make mul_op
+```
+
+## 绑定Python
+
+系统会对新增的op自动绑定Python，并链接到生成的lib库中。
+
+## 实现单元测试
+
+单测包括对比前向Op不同设备(CPU、GPU)的实现、对比反向OP不同设备(CPU、GPU)的实现、反向Op的梯度测试。下面介绍介绍[`MulOp`的单元测试](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/tests/test_mul_op.py)。
+
+### 前向Operator单元测试
+
+前向Op单元测试继承自`unittest.TestCase`，并定义元类`__metaclass__ = OpTestMeta`。各项更加具体的单元测试在`OpTestMeta`里完成。测试前向Operator，需要：
+
+1. 在`setUp`函数定义输入、输出，以及相关的属性参数。
+2. 生成随机的输入数据。
+3. 在Python脚本中实现与前向operator相同的计算逻辑，得到输出值，与operator前向计算的输出进行对比。
+
+
+  ```python
+  import unittest
+  import numpy as np
+  from gradient_checker import GradientChecker, create_op
+  from op_test_util import OpTestMeta
+
+  class TestMulOp(unittest.TestCase):
+      __metaclass__ = OpTestMeta
+
+      def setUp(self):
+          self.type = "mul"
+          self.inputs = {
+              'X': np.random.random((32, 84)).astype("float32"),
+              'Y': np.random.random((84, 100)).astype("float32")
+          }
+          self.outputs = {'Out': np.dot(self.inputs['X'], self.inputs['Y'])}
+  ```
+
+上面的代码首先导入依赖的包，下面是对`setUp`函数中操作的重要变量的详细解释：
+
+- `self.type = "mul" ` : 定义类型，与operator注册时注册的类型一致。
+- `self.inputs` : 定义输入，类型为`numpy.array`，并初始化。
+- `self.outputs` : 定义输出，并在Python脚本中完成与operator同样的计算逻辑，返回Python端的计算结果。
+
+
+### 反向Operator单元测试
+
+反向Op单元测试继承自`GradientChecker`，而`GradientChecker`继承自`unittest.TestCase`，因此，**反向单元测试函数需要以`test_`开头**。
+
+```python
+class TestMulGradOp(GradientChecker):
+    def setUp(self):
+        self.op = create_op("mul")
+        self.inputs = {
+            'X': np.random.random((32, 84)).astype("float32"),
+            'Y': np.random.random((84, 100)).astype("float32")
+        }
+
+    def test_check_grad_normal(self):
+        # mul op will enlarge the relative error
+        self.check_grad(['X', 'Y'], 'Out', max_relative_error=0.5)
+
+    def test_check_grad_ingore_x(self):
+        self.check_grad(
+            ['Y'], 'Out', max_relative_error=0.5, no_grad_set=set("X"))
+
+    def test_check_grad_ingore_y(self):
+        self.check_grad(
+            ['X'], 'Out', max_relative_error=0.5, no_grad_set=set('Y'))
+```
+
+下面解释代码中一些关键的地方:
+
+- 调用`create_op("mul")`创建反向Op对应的前向Op。
+- `test_check_grad_normal`中调用`check_grad`使用数值法检测梯度正确性和稳定性。
+  - 第一个参数`["X", "Y"]` : 指定对输入变量`X`、`Y`做梯度检测。
+  - 第二个参数`"Out"` : 指定前向网络最终的输出目标变量`Out`。
+  - 第三个参数`max_relative_error`：指定检测梯度时能容忍的最大错误值。
+- `test_check_grad_ingore_x`和`test_check_grad_ingore_y`分支用来测试只需要计算一个输入梯度的情况。
+
+
+### 编译和执行单元测试
+
+`python/paddle/v2/framework/tests` 目录下新增的 `test_*.py` 单元测试会被自动加入工程进行编译。
+
+请注意，**不同于Op的编译测试，运行单元测试测时需要编译整个工程**，并且编译时需要打开`WITH_TESTING`, 即`cmake paddle_dir -DWITH_TESTING=ON`。编译成功后，执行下面的命令来运行单元测试：
+
+```bash
+make test ARGS="-R test_mul_op -V"
+```
+
+或者:
+
+```bash
+ctest -R test_mul_op
+```
+
+## 注意事项
+
+- 为每个Op创建单独的`*_op.h`（如有）、`*_op.cc`和`*_op.cu`（如有）。不允许一个文件中包含多个Op，这将会导致编译出错。
+- 注册Op时的类型名，需要和该Op的名字一样。即不允许在`A_op.cc`里面，注册`REGISTER_OP(B, ...)`等，这将会导致单元测试出错。
+- 如果Op没有实现GPU Kernel，请不要创建空的`*_op.cu`，这将会导致单元测试出错。
+- 如果多个Op依赖一些共用的函数，可以创建非`*_op.*`格式的文件来存放，如`gather.h`文件。
diff --git a/doc_cn/_sources/howto/dev/use_eigen_cn.md.txt b/doc_cn/_sources/howto/dev/use_eigen_cn.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..1367323b71277984834d9d4f0d9bea0f69478479
--- /dev/null
+++ b/doc_cn/_sources/howto/dev/use_eigen_cn.md.txt
@@ -0,0 +1,146 @@
+## 在Paddle中如何使用Eigen
+
+神经网络本质上是一个计算图，计算需要的数据存放在`Tensor`中，而计算过程是由`Operartor`来描述的。在执行时，`Operator`调用对应`OpKernel`中的`Compute`接口，实现对`Tensor`的操作。
+
+
+### Eigen Tensor模块
+
+Eigen Tensor模块对element-wise计算提供了强大的支持，并且书写一份代码，可以同时在CPU、GPU执行。但Eigen Tensor是一个正在开发中的模块，因此可能测试不够完备，文档较少。
+
+关于Eigen Tensor模块的详细介绍请参考[文档1](https://github.com/RLovelett/eigen/blob/master/unsupported/Eigen/CXX11/src/Tensor/README.md) 和[文档2](https://bitbucket.org/eigen/eigen/src/default/unsupported/Eigen/CXX11/src/Tensor/README.md)
+
+
+### paddle::framework::Tensor
+
+Paddle Tensor定义在framework目录下，其主要接口如下：
+
+```cpp
+class Tensor {
+ public:
+  /*! Return a pointer to mutable memory block. */
+  template 
+  inline T* data();
+  
+  /**
+   * @brief   Return a pointer to mutable memory block.
+   * @note    If not exist, then allocation.
+   */
+  template 
+  inline T* mutable_data(platform::Place place);
+  
+  /**
+   * @brief     Return a pointer to mutable memory block.
+   *
+   * @param[in] dims    The dimensions of the memory block.
+   * @param[in] place   The place of the memory block.
+   *
+   * @note      If not exist, then allocation.
+   */
+  template 
+  inline T* mutable_data(DDim dims, platform::Place place);
+  
+  /*! Resize the dimensions of the memory block. */
+  inline Tensor& Resize(const DDim& dims);
+  
+  /*! Return the dimensions of the memory block. */
+  inline const DDim& dims() const;
+
+ private:  
+  /*! holds the memory block if allocated. */
+  std::shared_ptr holder_;
+  
+  /*! points to dimensions of memory block. */
+  DDim dim_;
+};
+```
+
+`Placeholder`的作用是延迟分配内存，即我们可以先定义一个Tensor，然后使用Resize接口设置Tensor的大小，最后再调用mutable_data接口分配实际的内存。
+
+```cpp
+paddle::framework::Tensor t;
+paddle::platform::CPUPlace place;
+// set size first
+t.Resize({2, 3});
+// allocate memory on CPU later
+t.mutable_data(place);
+```
+
+### paddle::framework::Tensor使用样例
+下面以AddOp为例说明Tensor的使用过程：
+
+- InferShape
+
+在运行神经网络计算图时，我们先调用每个`Operator`的`InferShape`接口，根据输入Tensor的大小来设置输出Tensor的大小，`Resize`接口会被调用。
+
+```cpp
+void InferShape(const framework::InferShapeContext &ctx) const override {
+  PADDLE_ENFORCE_EQ(ctx.Input("X")->dims(),
+                    ctx.Input("Y")->dims(),
+                    "Two input of Add Op's dimension must be same.");
+  ctx.Output("Out")->Resize(ctx.Input("X")->dims());
+}
+```
+
+
+- Run
+
+`Operator`的`Run`接口最终会调用对应`OpKernel`的`Compute`接口，在这时真正的分配内存，`mutable_data`接口会被调用。
+
+```cpp
+void Compute(const framework::ExecutionContext& context) const override {
+  auto* input0 = context.Input("X");
+  auto* input1 = context.Input("Y");
+  auto* output = context.Output("Out");
+
+  output->mutable_data(context.GetPlace());
+
+  auto x = EigenVector::Flatten(*input0);
+  auto y = EigenVector::Flatten(*input1);
+  auto z = EigenVector::Flatten(*output);
+
+  auto place = context.GetEigenDevice();
+
+  z.device(place) = x + y;
+}
+```
+
+
+### paddle::framework::Tensor到EigenTensor的转换
+
+如上一小节所示，在具体的计算中，我们需要先把输入Tensor和输出Tensor转换为Eigen支持的格式。我们在[eigen.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/eigen.h)中提供了一些全局函数用来实现paddle::framework::Tensor到EigenTensor/EigenMatrix/EigenVector/EigenScalar的转换。
+
+以EigenTensor为例，做一个介绍
+
+```cpp
+Tensor t;
+float* p = t.mutable_data(make_ddim({1, 2, 3}), platform::CPUPlace());
+for (int i = 0; i < 1 * 2 * 3; i++) {
+  p[i] = static_cast(i);
+}
+
+EigenTensor::Type et = EigenTensor::From(t);
+```
+
+From是EigenTensor模板提供的一个接口，可以实现从paddle::framework::Tensor到对EigenTensor的转换。由于Tensor的rank是模板参数，因此在转换时需要显示的指定。
+
+在Eigen中，不同rank的Tensor是不同类型，Vector是rank为1的Tensor。需要额外注意的是，EigenVector::From方法是把paddle中的一维Tensor转为Eigen的一维Tensor，在这里用EigenVector来表示；而EigenVector::Flatten方法是把paddle中的一个Tensor进行reshape操作，压扁成为Eigen的一维Tensor，类型仍然为EigenVector。
+
+更多的转换方法请参考eigen_test.cc中的[单元测试](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/eigen_test.cc)。
+
+
+
+### 实现计算
+
+当需要完成计算时，我们需要等式左边的EigenTensor调用device接口。在这里需要注意的是，这里的EigenTensor之间的运算只是改变了原有Tensor中的数据，而不会改变原有Tensor的shape信息。
+
+```cpp
+auto x = EigenVector::Flatten(*input0);
+auto y = EigenVector::Flatten(*input1);
+auto z = EigenVector::Flatten(*output);
+auto place = context.GetEigenDevice();
+z.device(place) = x + y;
+```
+
+在这段代码中，input0/input1/output可以是任意维度的Tensor。我们调用了EigenVector的Flatten接口，把任意维度的Tensor转为了一维的EigenVector。而在计算结束之后，input0/input1/output的原有shape信息不变。如果想改变原有Tensor的shape信息，可以调用Resize接口进行改变。
+
+由于Eigen Tensor模块的文档较少，我们可以参考TensorFlow的[kernels](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/kernels)模块下的相关`OpKernel`的计算代码。
diff --git a/doc_cn/_sources/howto/dev/write_docs_cn.rst.txt b/doc_cn/_sources/howto/dev/write_docs_cn.rst.txt
index 5051a892304fdc8b0f1a19a7d4560d5ee007c47d..1bc947c260d7adb75ee5a2bb10e6b91bc0be2d4c 100644
--- a/doc_cn/_sources/howto/dev/write_docs_cn.rst.txt
+++ b/doc_cn/_sources/howto/dev/write_docs_cn.rst.txt
@@ -3,52 +3,109 @@
 ##################
 
 PaddlePaddle的文档包括英文文档 ``doc`` 和中文文档 ``doc_cn`` 两个部分。文档都是通过 `cmake`_ 驱动 `sphinx`_ 编译生成，生成后的文档分别存储在编译目录的 ``doc`` 和 ``doc_cn`` 两个子目录下。
+也可以利用PaddlePaddle 工具来编译文档，这个情况下所有的文件会存在整理过的的文件目录 .ppo_workspace/content 下
 
+如何构建文档
+============
 
-如何构建PaddlePaddle的文档
-==========================
+PaddlePaddle的文档构建有三种方式。
 
-PaddlePaddle的文档构建有直接构建和基于Docker构建两种方式。构建PaddlePaddle文档需要准备的环境相对较复杂，所以我们推荐使用基于Docker来构建PaddlePaddle的文档。
 
+使用PaddlePaddle.org工具
+--------------
+这个是目前推荐的使用方法。除了可以自动编译文档，也可以直接在网页预览文档。
 
-使用Docker构建PaddlePaddle的文档
---------------------------------
+文件工具是使用Docker，需要在系统里先安装好Docker工具包。Docker安装请参考Docker的官网。安装好Docker之后及可用以下命令启动工具
 
-使用Docker构建PaddlePaddle的文档，需要在系统里先安装好Docker工具包。Docker安装请参考 `Docker的官网 `_ 。安装好Docker之后可以使用源码目录下的脚本构建文档，即
+..  code-block:: bash
+
+    mkdir paddlepaddle # Create paddlepaddle working directory
+    cd paddlepaddle
+
+    # Clone the content repositories
+    git clone https://github.com/PaddlePaddle/Paddle.git
+    git clone https://github.com/PaddlePaddle/book.git
+    git clone https://github.com/PaddlePaddle/models.git
+    git clone https://github.com/PaddlePaddle/Mobile.git
+
+    # Please specify the working directory through -v
+    docker run -it -p 8000:8000 -v `pwd`:/var/content paddlepaddle/paddlepaddle.org:latest
+
+注意: PaddlePaddle.org 会在 -v (volume) 指定的内容存储库运行命令
+之后再用网页连到http://localhost:8000就可以在网页上生成需要的文档
+编译后的文件将被存储在工作目录 /.ppo_workspace/content。
+
+如果不想使用 Docker，你还可以通过运行Django框架直接激活工具的服务器。使用下面的命令来运行它。
+
+..  code-block:: bash
+
+    mkdir paddlepaddle # Create paddlepaddle working directory
+    cd paddlepaddle
 
-..	code-block:: bash
+    # Clone the content repositories and PaddlePaddle.org
+    git clone https://github.com/PaddlePaddle/Paddle.git
+    git clone https://github.com/PaddlePaddle/book.git
+    git clone https://github.com/PaddlePaddle/models.git
+    git clone https://github.com/PaddlePaddle/Mobile.git
+    git clone https://github.com/PaddlePaddle/PaddlePaddle.org.git
 
-	cd TO_YOUR_PADDLE_CLONE_PATH
-	cd paddle/scripts/tools/build_docs
-	bash build_docs.sh
+    # Please specify the PaddlePaddle working directory. In the current setting, it should be pwd
+    export CONTENT_DIR=
+    export ENV=''
+    cd PaddlePaddle.org/portal/
+    pip install -r requirements.txt
+    python manage.py runserver
 
-编译完成后，该目录下会生成如下两个子目录\:
+工具服务器将读取环境变量 CONTENT_DIR 搜索代码库。请指定的PaddlePaddle工作目录给环境变量 CONTENT_DIR。
+之后再用网页连到http://localhost:8000就可以在网页上生成需要的文档。
+编译后的文件将被存储在工作目录 /.ppo_workspace/content。
 
-* doc 英文文档目录
-* doc_cn 中文文档目录
+想了解更多PaddlePaddle.org工具的详细信息，可以 `点击这里 `_ 。
 
+使用Docker构建
+--------------
+
+使用Docker构建PaddlePaddle的文档，需要在系统里先安装好Docker工具包。Docker安装请参考 `Docker的官网 `_ 。安装好Docker之后可以使用源码目录下的脚本构建文档，即
+
+..  code-block:: bash
+
+    cd TO_YOUR_PADDLE_CLONE_PATH
+    cd paddle/scripts/tools/build_docs
+    sh build_docs.sh
+
+编译完成之后，会在当前目录生成两个子目录\: doc(英文文档目录)和 doc_cn(中文文档目录)。
 打开浏览器访问对应目录下的index.html即可访问本地文档。
 
-..	code-block:: bash
+直接构建
+--------
+
+如果提示正确，可以执行以下命令编译生成文档，即
 
-	open doc_cn/index.html
+..  code-block:: bash
 
+    cd TO_YOUR_PADDLE_CLONE_PATH
+    mkdir -p build
+    cd build
+    cmake .. -DCMAKE_BUILD_TYPE=Debug -DWITH_GPU=OFF -DWITH_MKL=OFF -DWITH_DOC=ON
+    make gen_proto_py
+    make paddle_docs paddle_docs_cn
 
-直接构建PaddlePaddle的文档
---------------------------
+编译完成之后，会在当前目录生成两个子目录\: doc(英文文档目录)和 doc_cn(中文文档目录)。
+打开浏览器访问对应目录下的index.html即可访问本地文档。
 
-TBD
 
-如何书写PaddlePaddle的文档
-==========================
+如何书写文档
+============
 
-TBD
+PaddlePaddle文档使用 `sphinx`_ 自动生成，用户可以参考sphinx教程进行书写。
 
-如何更新www.paddlepaddle.org文档
-================================
+如何更新www.paddlepaddle.org
+============================
 
-TBD
+更新的文档以PR的形式提交到github中，提交方式参见 `贡献文档 `_ 。
+目前PaddlePaddle的develop分支的文档是自动触发更新的，用户可以分别查看最新的 `中文文档 `_ 和
+`英文文档 `_ 。
 
 
-..	_cmake: https://cmake.org/
-..	_sphinx: http://www.sphinx-doc.org/en/1.4.8/
+..  _cmake: https://cmake.org/
+..  _sphinx: http://www.sphinx-doc.org/en/1.4.8/
diff --git a/doc_cn/_sources/howto/index_cn.rst.txt b/doc_cn/_sources/howto/index_cn.rst.txt
index 26449a6365843b526b3ac3111b337d2f17524c9d..991b9e2596a3b499846b963152c838d66260265d 100644
--- a/doc_cn/_sources/howto/index_cn.rst.txt
+++ b/doc_cn/_sources/howto/index_cn.rst.txt
@@ -19,8 +19,8 @@
 ..  toctree::
   :maxdepth: 1
 
-  dev/write_docs_cn.rst
   dev/contribute_to_paddle_cn.md
+  dev/write_docs_cn.rst
 
 模型配置
 --------
diff --git a/doc_cn/_sources/howto/optimization/cpu_profiling.md.txt b/doc_cn/_sources/howto/optimization/cpu_profiling.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..1775374cf6e518586c28bbd8e04946c74df7e4c5
--- /dev/null
+++ b/doc_cn/_sources/howto/optimization/cpu_profiling.md.txt
@@ -0,0 +1,197 @@
+This tutorial introduces techniques we use to profile and tune the
+CPU performance of PaddlePaddle.  We will use Python packages
+`cProfile` and `yep`, and Google's `perftools`.
+
+Profiling is the process that reveals performance bottlenecks,
+which could be very different from what's in the developers' mind.
+Performance tuning is done to fix these bottlenecks. Performance optimization
+repeats the steps of profiling and tuning alternatively.
+
+PaddlePaddle users program AI applications by calling the Python API, which calls
+into `libpaddle.so.` written in C++.  In this tutorial, we focus on
+the profiling and tuning of
+
+1. the Python code and
+1. the mixture of Python and C++ code.
+
+## Profiling the Python Code
+
+### Generate the Performance Profiling File
+
+We can use Python standard
+package, [`cProfile`](https://docs.python.org/2/library/profile.html),
+to generate Python profiling file.  For example:
+
+```bash
+python -m cProfile -o profile.out main.py
+```
+
+where `main.py` is the program we are going to profile, `-o` specifies
+the output file.  Without `-o`, `cProfile` would outputs to standard
+output.
+
+### Look into the Profiling File
+
+`cProfile` generates `profile.out` after `main.py` completes. We can
+use [`cprofilev`](https://github.com/ymichael/cprofilev) to look into
+the details:
+
+```bash
+cprofilev -a 0.0.0.0 -p 3214 -f profile.out main.py
+```
+
+where `-a` specifies the HTTP IP, `-p` specifies the port, `-f`
+specifies the profiling file, and `main.py` is the source file.
+
+Open the Web browser and points to the local IP and the specifies
+port, we will see the output like the following:
+
+```
+   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
+        1    0.284    0.284   29.514   29.514 main.py:1()
+     4696    0.128    0.000   15.748    0.003 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/executor.py:20(run)
+     4696   12.040    0.003   12.040    0.003 {built-in method run}
+        1    0.144    0.144    6.534    6.534 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/__init__.py:14()
+```
+
+where each line corresponds to Python function, and the meaning of
+each column is as follows:
+
+| column | meaning |
+| --- | --- |
+| ncalls | the number of calls into a function |
+| tottime | the total execution time of the function, not including the
+ execution time of other functions called by the function |
+| percall | tottime divided by ncalls |
+| cumtime | the total execution time of the function, including the execution time of other functions being called |
+| percall | cumtime divided by ncalls |
+| filename:lineno(function) | where the function is defined |
+
+### Identify Performance Bottlenecks
+
+Usually, `tottime` and the related `percall` time is what we want to
+focus on. We can sort above profiling file by tottime:
+
+```text
+     4696   12.040    0.003   12.040    0.003 {built-in method run}
+   300005    0.874    0.000    1.681    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/dataset/mnist.py:38(reader)
+   107991    0.676    0.000    1.519    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:219(__init__)
+     4697    0.626    0.000    2.291    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp)
+        1    0.618    0.618    0.618    0.618 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/__init__.py:1()
+```
+
+We can see that the most time-consuming function is the `built-in
+method run`, which is a C++ function in `libpaddle.so`.  We will
+explain how to profile C++ code in the next section.  At this 
+moment, let's look into the third function `sync_with_cpp`, which is a
+Python function.  We can click it to understand more about it:
+
+```
+Called By:
+
+   Ordered by: internal time
+   List reduced from 4497 to 2 due to restriction <'sync_with_cpp'>
+
+Function                                                                                                 was called by...
+                                                                                                             ncalls  tottime  cumtime
+/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp)  <-    4697    0.626    2.291  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp)
+/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp)  <-    4696    0.019    2.316  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:487(clone)
+                                                                                                                  1    0.000    0.001  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:534(append_backward)
+
+
+Called:
+
+   Ordered by: internal time
+   List reduced from 4497 to 2 due to restriction <'sync_with_cpp'>
+```
+
+The lists of the callers of `sync_with_cpp` might help us understand
+how to improve the function definition.
+
+## Profiling Python and C++ Code
+
+### Generate the Profiling File
+
+To profile a mixture of Python and C++ code, we can use a Python
+package, `yep`, that can work with Google's `perftools`, which is a
+commonly-used profiler for C/C++ code.
+
+In Ubuntu systems, we can install `yep` and `perftools` by running the
+following commands:
+
+```bash
+apt update
+apt install libgoogle-perftools-dev
+pip install yep
+```
+
+Then we can run the following command
+
+```bash
+python -m yep -v main.py
+```
+
+to generate the profiling file.  The default filename is
+`main.py.prof`.
+
+Please be aware of the `-v` command line option, which prints the
+analysis results after generating the profiling file.  By examining the
+ the print result, we'd know that if we stripped debug
+information from `libpaddle.so` at build time.  The following hints
+help make sure that the analysis results are readable:
+
+1. Use GCC command line option `-g` when building `libpaddle.so` so to
+   include the debug information.  The standard building system of
+   PaddlePaddle is CMake, so you might want to set
+   `CMAKE_BUILD_TYPE=RelWithDebInfo`.
+
+1. Use GCC command line option `-O2` or `-O3` to generate optimized
+   binary code. It doesn't make sense to profile `libpaddle.so`
+   without optimization, because it would anyway run slowly.
+
+1. Profiling the single-threaded binary file before the
+   multi-threading version, because the latter often generates tangled
+   profiling analysis result.  You might want to set environment
+   variable `OMP_NUM_THREADS=1` to prevents OpenMP from automatically
+   starting multiple threads.
+
+### Examining the Profiling File
+
+The tool we used to examine the profiling file generated by
+`perftools` is [`pprof`](https://github.com/google/pprof), which
+provides a Web-based GUI like `cprofilev`.
+
+We can rely on the standard Go toolchain to retrieve the source code
+of `pprof` and build it:
+
+```bash
+go get github.com/google/pprof
+```
+
+Then we can use it to profile `main.py.prof` generated in the previous
+section:
+
+```bash
+pprof -http=0.0.0.0:3213 `which python`  ./main.py.prof
+```
+
+Where `-http` specifies the IP and port of the HTTP service.
+Directing our Web browser to the service, we would see something like
+the following:
+
+![result](./pprof_1.png)
+
+### Identifying the Performance Bottlenecks
+
+Similar to how we work with `cprofilev`, we'd focus on `tottime` and
+`cumtime`.
+
+![kernel_perf](./pprof_2.png)
+
+We can see that the execution time of multiplication and the computing
+of the gradient of multiplication takes 2% to 4% of the total running
+time, and `MomentumOp` takes about 17%. Obviously, we'd want to
+optimize `MomentumOp`.
+
+`pprof` would mark performance critical parts of the program in
+red. It's a good idea to follow the hints.
diff --git a/doc_cn/_sources/howto/optimization/cpu_profiling_cn.md.txt b/doc_cn/_sources/howto/optimization/cpu_profiling_cn.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..14eba0e2f34b115f5cd24920b5b1af07ec953d00
--- /dev/null
+++ b/doc_cn/_sources/howto/optimization/cpu_profiling_cn.md.txt
@@ -0,0 +1,155 @@
+此教程会介绍如何使用Python的cProfile包、Python库yep、Google perftools来进行性能分析 (profiling) 与调优（performance tuning）。
+
+Profling 指发现性能瓶颈。系统中的瓶颈可能和程序员开发过程中想象的瓶颈相去甚远。Tuning 指消除瓶颈。性能优化的过程通常是不断重复地 profiling 和 tuning。
+
+PaddlePaddle 用户一般通过调用 Python API 编写深度学习程序。大部分 Python API 调用用 C++ 写的 libpaddle.so。所以 PaddlePaddle 的性能分析与调优分为两个部分:
+
+* Python 代码的性能分析
+* Python 与 C++ 混合代码的性能分析
+
+
+## Python代码的性能分析
+
+### 生成性能分析文件
+
+Python标准库中提供了性能分析的工具包，[cProfile](https://docs.python.org/2/library/profile.html)。生成Python性能分析的命令如下:
+
+```bash
+python -m cProfile -o profile.out main.py
+```
+
+其中 `main.py` 是我们要分析的程序，`-o`标识了一个输出的文件名，用来存储本次性能分析的结果。如果不指定这个文件，`cProfile`会打印到标准输出。
+
+### 查看性能分析文件
+
+`cProfile` 在main.py 运行完毕后输出`profile.out`。我们可以使用[`cprofilev`](https://github.com/ymichael/cprofilev)来查看性能分析结果。`cprofilev`是一个Python的第三方库。使用它会开启一个HTTP服务，将性能分析结果以网页的形式展示出来：
+
+```bash
+cprofilev -a 0.0.0.0 -p 3214 -f profile.out main.py
+```
+
+其中`-a`标识HTTP服务绑定的IP。使用`0.0.0.0`允许外网访问这个HTTP服务。`-p`标识HTTP服务的端口。`-f`标识性能分析的结果文件。`main.py`标识被性能分析的源文件。
+
+用Web浏览器访问对应网址，即可显示性能分析的结果：
+
+```
+   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
+        1    0.284    0.284   29.514   29.514 main.py:1()
+     4696    0.128    0.000   15.748    0.003 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/executor.py:20(run)
+     4696   12.040    0.003   12.040    0.003 {built-in method run}
+        1    0.144    0.144    6.534    6.534 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/__init__.py:14()
+```
+
+每一列的含义是:
+
+| 列名 | 含义 |
+| --- | --- |
+| ncalls | 函数的调用次数 |
+| tottime | 函数实际使用的总时间。该时间去除掉本函数调用其他函数的时间 |
+| percall | tottime的每次调用平均时间 |
+| cumtime | 函数总时间。包含这个函数调用其他函数的时间 |
+| percall | cumtime的每次调用平均时间 |
+| filename:lineno(function) | 文件名, 行号，函数名 |
+
+
+### 寻找性能瓶颈
+
+通常`tottime`和`cumtime`是寻找瓶颈的关键指标。这两个指标代表了某一个函数真实的运行时间。
+
+将性能分析结果按照tottime排序，效果如下:
+
+```text
+     4696   12.040    0.003   12.040    0.003 {built-in method run}
+   300005    0.874    0.000    1.681    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/dataset/mnist.py:38(reader)
+   107991    0.676    0.000    1.519    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:219(__init__)
+     4697    0.626    0.000    2.291    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp)
+        1    0.618    0.618    0.618    0.618 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/__init__.py:1()
+```
+
+可以看到最耗时的函数是C++端的`run`函数。这需要联合我们第二节`Python`与`C++`混合代码的性能分析来进行调优。而`sync_with_cpp`函数的总共耗时很长，每次调用的耗时也很长。于是我们可以点击`sync_with_cpp`的详细信息，了解其调用关系。
+
+```text
+Called By:
+
+   Ordered by: internal time
+   List reduced from 4497 to 2 due to restriction <'sync_with_cpp'>
+
+Function                                                                                                 was called by...
+                                                                                                             ncalls  tottime  cumtime
+/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp)  <-    4697    0.626    2.291  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp)
+/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp)  <-    4696    0.019    2.316  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:487(clone)
+                                                                                                                  1    0.000    0.001  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:534(append_backward)
+
+
+Called:
+
+   Ordered by: internal time
+   List reduced from 4497 to 2 due to restriction <'sync_with_cpp'>
+```
+
+通常观察热点函数间的调用关系，和对应行的代码，就可以了解到问题代码在哪里。当我们做出性能修正后，再次进行性能分析(profiling)即可检查我们调优后的修正是否能够改善程序的性能。
+
+
+
+## Python与C++混合代码的性能分析
+
+### 生成性能分析文件
+
+C++的性能分析工具非常多。常见的包括`gprof`, `valgrind`, `google-perftools`。但是调试Python中使用的动态链接库与直接调试原始二进制相比增加了很多复杂度。幸而Python的一个第三方库`yep`提供了方便的和`google-perftools`交互的方法。于是这里使用`yep`进行Python与C++混合代码的性能分析
+
+使用`yep`前需要安装`google-perftools`与`yep`包。ubuntu下安装命令为
+
+```bash
+apt update
+apt install libgoogle-perftools-dev
+pip install yep
+```
+
+安装完毕后，我们可以通过
+
+```bash
+python -m yep -v main.py
+```
+
+生成性能分析文件。生成的性能分析文件为`main.py.prof`。
+
+命令行中的`-v`指定在生成性能分析文件之后，在命令行显示分析结果。我们可以在命令行中简单的看一下生成效果。因为C++与Python不同，编译时可能会去掉调试信息，运行时也可能因为多线程产生混乱不可读的性能分析结果。为了生成更可读的性能分析结果，可以采取下面几点措施:
+
+1. 编译时指定`-g`生成调试信息。使用cmake的话，可以将CMAKE_BUILD_TYPE指定为`RelWithDebInfo`。
+2. 编译时一定要开启优化。单纯的`Debug`编译性能会和`-O2`或者`-O3`有非常大的差别。`Debug`模式下的性能测试是没有意义的。
+3. 运行性能分析的时候，先从单线程开始，再开启多线程，进而多机。毕竟单线程调试更容易。可以设置`OMP_NUM_THREADS=1`这个环境变量关闭openmp优化。
+
+### 查看性能分析文件
+
+在运行完性能分析后，会生成性能分析结果文件。我们可以使用[`pprof`](https://github.com/google/pprof)来显示性能分析结果。注意，这里使用了用`Go`语言重构后的`pprof`，因为这个工具具有web服务界面，且展示效果更好。
+
+安装`pprof`的命令和一般的`Go`程序是一样的，其命令如下:
+
+```bash
+go get github.com/google/pprof
+```
+
+进而我们可以使用如下命令开启一个HTTP服务:
+
+```bash
+pprof -http=0.0.0.0:3213 `which python`  ./main.py.prof
+```
+
+这行命令中，`-http`指开启HTTP服务。`which python`会产生当前Python二进制的完整路径，进而指定了Python可执行文件的路径。`./main.py.prof`输入了性能分析结果。
+
+访问对应的网址，我们可以查看性能分析的结果。结果如下图所示:
+
+![result](./pprof_1.png)
+
+
+### 寻找性能瓶颈
+
+与寻找Python代码的性能瓶颈类似，寻找Python与C++混合代码的性能瓶颈也是要看`tottime`和`cumtime`。而`pprof`展示的调用图也可以帮助我们发现性能中的问题。
+
+例如下图中，
+
+![kernel_perf](./pprof_2.png)
+
+在一次训练中，乘法和乘法梯度的计算占用2%-4%左右的计算时间。而`MomentumOp`占用了17%左右的计算时间。显然，`MomentumOp`的性能有问题。
+
+在`pprof`中，对于性能的关键路径都做出了红色标记。先检查关键路径的性能问题，再检查其他部分的性能问题，可以更有次序的完成性能的优化。
diff --git a/doc_cn/_sources/howto/usage/cluster/cluster_train_cn.md.txt b/doc_cn/_sources/howto/usage/cluster/cluster_train_cn.md.txt
index 274452fbf0c595ad7b4dbeffe85ad9038f12b458..2e98b3de3fe2284375f87e883ff4bac19255dbeb 100644
--- a/doc_cn/_sources/howto/usage/cluster/cluster_train_cn.md.txt
+++ b/doc_cn/_sources/howto/usage/cluster/cluster_train_cn.md.txt
@@ -1,145 +1,225 @@
-```eval_rst
-.. _cluster_train:
+# PaddlePaddle分布式训练
+
+* [概述](#概述)
+* [环境准备](#环境准备)
+* [启动参数说明](#启动参数说明)
+  * [启动参数服务器](#启动参数服务器)
+  * [启动计算节点](#启动计算节点)
+  * [准备数据集](#准备数据集)
+  * [准备训练程序](#准备训练程序)
+* [使用分布式计算平台或工具](#使用分布式计算平台或工具)
+  * [使用Fabric启动集群作业](#使用fabric启动集群作业)
+     * [准备一个Linux集群](#准备一个linux集群)
+     * [启动集群作业](#启动集群作业)
+     * [终止集群作业](#终止集群作业)
+     * [检查集群训练结果](#检查集群训练结果)
+     * [检查模型输出](#检查模型输出)
+  * [在OpenMPI集群中提交训练作业](#在openmpi集群中提交训练作业)
+     * [准备OpenMPI集群](#准备OpenMPI集群)
+     * [启动集群作业](#启动集群作业-1)
+  * [在Kubernetes集群中提交训练作业](#在kubernetes集群中提交训练作业)
+
+## 概述
+本文将介绍如何使用PaddlePaddle在不同的集群框架下完成分布式训练。分布式训练架构如下图所示：
+
+
+
+- 数据分片（Data shard): 用于训练神经网络的数据，被切分成多个部分，每个部分分别给每个trainer使用。
+- 计算节点（Trainer）: 每个trainer启动后读取切分好的一部分数据，开始神经网络的“前馈”和“后馈”计算，并和参数服务器通信。在完成一定量数据的训练后，上传计算得出的梯度（gradients），然后下载优化更新后的神经网络参数（parameters）。
+- 参数服务器（Parameter server）:每个参数服务器只保存整个神经网络所有参数的一部分。参数服务器接收从计算节点上传的梯度，并完成参数优化更新，再将更新后的参数下发到每个计算节点。
+
+这样，通过计算节点和参数服务器的分布式协作，可以完成神经网络的SGD方法的训练。PaddlePaddle可以同时支持同步随机梯度下降（SGD）和异步随机梯度下降。
+
+在使用同步SGD训练神经网络时，PaddlePaddle使用同步屏障（barrier），使梯度的提交和参数的更新按照顺序方式执行。在异步SGD中，则并不会等待所有trainer提交梯度才更新参数，这样极大地提高了计算的并行性：参数服务器之间不相互依赖，并行地接收梯度和更新参数，参数服务器也不会等待计算节点全部都提交梯度之后才开始下一步，计算节点之间也不会相互依赖，并行地执行模型的训练。可以看出，虽然异步SGD方式会提高参数更新并行度, 但是并不能保证参数同步更新，在任意时间某一台参数服务器上保存的参数可能比另一台要更新，与同步SGD相比，梯度会有噪声。
+
+## 环境准备
+
+1. 准备您的计算集群。计算集群通常由一组（几台到几千台规模）的Linux服务器组成。服务器之间可以通过局域网（LAN）联通，每台服务器具有集群中唯一的IP地址（或者可被DNS解析的主机名）。集群中的每台计算机通常被成为一个“节点”。
+1. 我们需要在集群的所有节点上安装 PaddlePaddle。 如果要启用GPU，还需要在节点上安装对应的GPU驱动以及CUDA。PaddlePaddle的安装可以参考[build_and_install](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/getstarted/build_and_install)的多种安装方式。我们推荐使用[Docker](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)安装方式来快速安装PaddlePaddle。
+
+安装完成之后，执行下面的命令可以查看已经安装的版本（docker安装方式可以进入docker容器执行：`docker run -it paddlepaddle/paddle:[tag] /bin/bash`）：
+```bash
+$ paddle version
+PaddlePaddle 0.10.0, compiled with
+    with_avx: ON
+    with_gpu: OFF
+    with_double: OFF
+    with_python: ON
+    with_rdma: OFF
+    with_timer: OFF
 ```
 
-# 运行分布式训练
+下面以`doc/howto/usage/cluster/src/word2vec`中的代码作为实例，介绍使用PaddlePaddle v2 API完成分布式训练。
 
-在本文中，我们将阐释如何在集群上运行分布式 Paddle 训练作业。我们将以[推荐系统](https://github.com/baidu/Paddle/tree/develop/demo/recommendation)为例创建分布式的单进程训练。
+## 启动参数说明
+### 启动参数服务器
+执行以下的命令启动一个参数服务器并等待和计算节点的数据交互
+```bash
+$ paddle pserver --port=7164 --ports_num=1 --ports_num_for_sparse=1 --num_gradient_servers=1
+```
 
-在本文中使用的[脚本](https://github.com/baidu/Paddle/tree/develop/paddle/scripts/cluster_train)通过 SSH 运行分布式作业。 它们还可以供那些运行更复杂的集群管理系统（如 MPI 和 [Kubernetes](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/howto/usage/k8s) ）的用户参考。
+如果希望可以在后台运行pserver程序，并保存输出到一个日志文件，可以运行：
+```bash
+$ stdbuf -oL /usr/bin/nohup paddle pserver --port=7164 --ports_num=1 --ports_num_for_sparse=1 --num_gradient_servers=1 &> pserver.log
+```
 
-## 前提条件
+| 参数  | 是否必选 | 默认值 | 说明 |
+| ------------- | ------------- | ------------- | ------------- |
+| port  | 必选 | 7164 | pserver监听的起始端口，根据ports_num决定
    总端口个数，从起始端口监听多个端口用于通信  |
+| ports_num  | 必选 | 1 | 监听的端口个数  |
+| ports_num_for_sparse  | 必选 | 1 | 用于稀疏类型参数通信的端口个数  |
+| num_gradient_servers  | 必选 | 1 | 当前训练任务pserver总数 |
+
+### 启动计算节点
+执行以下命令启动使用python编写的trainer程序（文件名为任意文件名，如train.py）
+```bash
+$ python train.py
+```
 
-1. 上述脚本使用 Python 库 [fabric](http://www.fabfile.org/) 来运行 SSH 命令。 我们使用 `pip` 来安装 fabric:
+trainer需要和pserver保持网络联通以完成训练。trainer启动需要传入端口、pserver地址等参数使trainer可以正确连接到pserver。这些参数可以通过环境变量（https://zh.wikipedia.org/wiki/环境变量 ）或编写程序时`paddle.init()`中传入参数。如果同时使用`paddle.init()`参数和环境变量，将会优先使用`paddle.init()`中传入的参数。
 
-   ```bash
-   pip install fabric
-   ```
+使用环境变量：
 
-2. 我们需要在集群的所有节点上安装 PaddlePaddle。 如果要启用GPU，需要在 `/usr/local/cuda` 中安装 CUDA; 否则 Paddle 将在运行时报错。
+```bash
+export PADDLE_INIT_USE_GPU=False
+export PADDLE_INIT_TRAINER_COUNT=1
+export PADDLE_INIT_PORT=7164
+export PADDLE_INIT_PORTS_NUM=1
+export PADDLE_INIT_PORTS_NUM_FOR_SPARSE=1
+export PADDLE_INIT_NUM_GRADIENT_SERVERS=1
+export PADDLE_INIT_TRAINER_ID=0
+export PADDLE_INIT_PSERVERS=127.0.0.1
+```
 
-3. 在 [`cluster_train/conf.py`] 中设置 `ROOT_DIR`， 该 ROOT_DIR 要在所有节点上存在。为了方便起见，我们通常在所有节点上创建一个 Unix 用户 `paddle`，并设置 `ROOT_DIR=/home/paddle`。这样，我们可以将 SSH 公钥写入 `/home/paddle/.ssh/authorized_keys`，以便用户 `paddle` 可以 SSH 到所有节点而不用密码。
+使用参数：
 
-## 准备工作空间
+```python
+paddle.init(
+        use_gpu=False,
+        trainer_count=1,
+        port=7164,
+        ports_num=1,
+        ports_num_for_sparse=1,
+        num_gradient_servers=1,
+        trainer_id=0,
+        pservers="127.0.0.1")
+```
 
-我们将放置依赖库、配置等文件的目录视为 *工作空间（workspace）*。
+| 参数  | 是否必选 | 默认 | 说明 |
+| ------------- | ------------- | ------------- | ------------- |
+| use_gpu  | 可选 | False | 是否启用GPU训练 |
+| trainer_count  | 必选 | 1 | 当前训练任务trainer总个数 |
+| port  | 必选 | 7164 | 连接到pserver的端口  |
+| ports_num  | 必选 | 1 | 连接到pserver的端口个数  |
+| ports_num_for_sparse  | 必选 | 1 | 和pserver之间用于稀疏类型参数通信的端口个数  |
+| num_gradient_servers  | 必选 | 1 | 当前训练任务pserver总数 |
+| trainer_id  | 必选 | 0 | 每个trainer的唯一ID，从0开始的整数 |
+| pservers  | 必选 | 127.0.0.1 | 当前训练任务启动的pserver的IP列表，多个IP使用“,”隔开 |
 
-这些 `train/test` 数据应该在启动集群作业之前准备好。 为了满足训练/测试数据放置在工作空间中不同目录的要求，PADDLE 根据在模型配置文件中使用的名为 `train.list/test.list` 的索引文件引用训练/测试数据，所以训练/测试数据也包含 train.list/test.list 两个列表文件。所有本地训练 demo 已经提供了脚本来帮助您创建这两个文件，并且集群作业中的所有节点将在正常情况下处理具有相同逻辑代码的文件。
 
-通常，你可以使用本地训练中的相同模型文件进行集群训练。请记住，在模型文件的 `setting`函数中设置的 `batch_size` 表示在集群作业**每个**节点中的 batch 大小，而不是使用同步 SGD 的总 batch 大小。
+### 准备数据集
 
-以下步骤基于 demo 目录中的 [demo/recommendation](https://github.com/PaddlePaddle/Paddle/tree/develop/demo/recommendation)。
+参考样例数据准备脚本[prepare.py](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/howto/usage/cluster/src/word2vec/prepare.py)，准备训练数据和验证数据集，我们使用paddle.dataset.imikolov数据集，并根据分布式训练并发数（trainer节点个数），在`prepare.py`开头部分指定`SPLIT_COUNT`将数据切分成多份。
 
-你只需完成 demo/recommendation 教程文档到 `Train` 的部分，之后你会得到训练/测试数据和模型配置文件。最后，只需使用 demo/recommendation 作为集群训练的工作空间。
+在线上系统中，通常会使用MapReduce任务的输出结果作为训练结果，这样训练文件的个数会比较多，而且个数并不确定。在trainer中可以使用下面取模的方法为每个trainer分配训练数据文件：
 
-最后，你的工作空间应如下所示：
-```
-.
-|-- common_utils.py
-|-- data
-|   |-- config.json
-|   |-- config_generator.py
-|   |-- meta.bin
-|   |-- meta_config.json
-|   |-- meta_generator.py
-|   |-- ml-1m
-|   |-- ml_data.sh
-|   |-- ratings.dat.test
-|   |-- ratings.dat.train
-|   |-- split.py
-|   |-- test.list
-|   `-- train.list
-|-- dataprovider.py
-|-- evaluate.sh
-|-- prediction.py
-|-- preprocess.sh
-|-- requirements.txt
-|-- run.sh
-`-- trainer_config.py
+```python
+import os
+train_list = []
+flist = os.listdir("/train_data/")
+for f in flist:
+  suffix = int(f.split("-")[1])
+  if suffix % TRAINER_COUNT == TRAINER_ID:
+    train_list.append(f)
 ```
-虽然这些文件并非都需要集群训练，但是也没有必要删除无用的文件。
 
-`trainer_config.py`
-表示模型配置文件。
+示例程序`prepare.py`会把训练集和测试集分别分割成多个文件（例子中为3个，后缀为`-00000`、`-00001`和`-00002`）:
+```
+train.txt
+train.txt-00000
+train.txt-00001
+train.txt-00002
+test.txt
+test.txt-00000
+test.txt-00001
+test.txt-00002
+```
 
-`train.list` 和 `test.list`
-文件索引。它存储当前节点所有训练/测试数据的所有相对或绝对文件路径。
+在进行分布式训练时，每个trainer进程需要能够读取属于自己的一份数据。在一些分布式系统中，系统会提供一个分布式存储服务，这样保存在分布式存储中的数据可以被集群中的每个节点读取到。如果不使用分布式存储，则需要手动拷贝属于每个trainer节点的训练数据到对应的节点上。
 
-`dataprovider.py`
-用于读取训练/测试样本。这与本地训练相同。
+对于不同的训练任务，训练数据格式和训练程序的`reader()`会大不相同，所以开发者需要根据自己训练任务的实际场景完成训练数据的分割和`reader()`的编写。
 
-`data`
-数据目录中的所有文件被 train.list/test.list 引用。
+### 准备训练程序
 
+我们会对每个训练任务都会在每个节点上创建一个工作空间（workspace），其中包含了用户的训练程序、程序依赖、挂载或下载的训练数据分片。
 
-## 准备集群作业配置
+最后，工作空间应如下所示：
+```
+.
+|-- my_lib.py
+|-- word_dict.pickle
+|-- train.py
+|-- train_data_dir/
+|   |-- train.txt-00000
+|   |-- train.txt-00001
+|   |-- train.txt-00002
+`-- test_data_dir/
+    |-- test.txt-00000
+    |-- test.txt-00001
+    `-- test.txt-00002
+```
 
-以下选项必须在 cluster_train/conf.py 中认真设置
+- `my_lib.py`：会被`train.py`调用的一些用户定义的库函数，比如PIL库等。
+- `word_dict.pickle`：在`train.py`中会使用到的字典数据文件。
+- `train.py`：训练程序，代码参考[api_train_v2_cluster.py](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/howto/usage/cluster/src/word2vec/prepare.py)。***注意：*** 对于本样例代码，在使用不同的分布式计算平台时，您可能需要修改`train.py`开头的部分（如下），以便获得训练数据的位置和获取环境变量配置：
 
-`HOSTS`  所有节点运行集群作业的主机名或 IP 。你还可以将用户和 ssh 端口附加到主机名上，例如 root@192.168.100.17:9090。
+  ```python
+  cluster_train_file = "./train_data_dir/train/train.txt"
+  cluster_test_file = "./test_data_dir/test/test.txt"
+  node_id = os.getenv("OMPI_COMM_WORLD_RANK")
+  if not node_id:
+      raise EnvironmentError("must provied OMPI_COMM_WORLD_RANK")
+  ```
 
-`ROOT_DIR` 用于放置 JOB 工作空间目录的工作空间 ROOT 目录
+- `train_data_dir`：包含训练数据的目录，可以是从分布式存储挂载过来的，也可以是在任务启动前下载到本地的。
+- `test_data_dir`：包含测试数据集的目录。
 
-`PADDLE_NIC` 集群通信通道的 NIC(Network Interface Card, 网络接口卡) 接口名称，例如以太网的 eth0，infiniband 的 ib0。
+## 使用分布式计算平台或工具
 
-`PADDLE_PORT` 集群通信通道的端口号
+PaddlePaddle可以使用多种分布式计算平台构建分布式计算任务，包括：
+- [Kubernetes](http://kubernetes.io) Google开源的容器集群的调度框架，支持大规模集群生产环境的完整集群方案。
+- [OpenMPI](https://www.open-mpi.org) 成熟的高性能并行计算框架。
+- [Fabric](http://www.fabfile.org) 集群管理工具。可以使用`Fabric`编写集群任务提交和管理脚本。
 
-`PADDLE_PORTS_NUM` 用于集群通信通道的端口数。 如果集群节点数量少（少于5〜6个节点），建议将其设置为较大，如2〜8，以获得更好的网络性能。
+对于不同的集群平台，会分别介绍集群作业的启动和停止方法。这些例子都可以在[cluster_train_v2](https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/scripts/cluster_train_v2)找到。
 
-`PADDLE_PORTS_NUM_FOR_SPARSE` 用于 sparse remote updater 集群通信信道的端口数。如果使用 sparse remote update，则可以像 `PADDLE_PORTS_NUM` 一样设置。
+在使用分布式计算平台进行训练时，任务被调度在集群中时，分布式计算平台通常会通过API或者环境变量提供任务运行需要的参数，比如节点的ID、IP和任务节点个数等。
 
-`LD_LIBRARY_PATH` 为集群作业设置额外的 LD_LIBRARY_PATH。你可以使用它来设置 CUDA 库的路径。
+### 使用Fabric启动集群作业
 
-默认配置如下：
+#### 准备一个Linux集群
+可以在`paddle/scripts/cluster_train_v2/fabric/docker_cluster`目录下，执行`kubectl -f ssh_servers.yaml`启动一个测试集群，并使用`kubectl get po -o wide`获得这些节点的IP地址。
 
-```python
-HOSTS = [
-        "root@192.168.100.17",
-        "root@192.168.100.18",
-        ]
-
-'''
-工作空间配置
-'''
-
-#工作空间根目录
-ROOT_DIR = "/home/paddle"
-
-'''
-网络配置
-'''
-#pserver NIC
-PADDLE_NIC = "eth0"
-#pserver 端口
-PADDLE_PORT = 7164
-#pserver 端口数
-PADDLE_PORTS_NUM = 2
-#pserver sparse ports num
-PADDLE_PORTS_NUM_FOR_SPARSE = 2
-
-#集群作业中所有进程的环境设置
-LD_LIBRARY_PATH="/usr/local/cuda/lib64:/usr/lib64"
-```
+#### 启动集群作业
 
-### 启动集群作业
-`paddle.py` 提供了自动化脚本来启动不同节点中的所有 PaddlePaddle 集群进程。默认情况下，所有命令行选项可以设置为```paddle.py``` 命令选项并且 `paddle.py` 将透明、自动地将这些选项应用到 PaddlePaddle 底层进程。
+`paddle.py` 提供了自动化脚本来启动不同节点中的所有 PaddlePaddle 集群进程。默认情况下，所有命令行选项可以设置为 `paddle.py` 命令选项并且 `paddle.py` 将透明、自动地将这些选项应用到 PaddlePaddle 底层进程。
 
 `paddle.py` 为方便作业启动提供了两个独特的命令选项。
 
-`job_dispatch_package`  设为本地 `workspace` 目录，它将被分发到 conf.py 中设置的所有节点。  它有助于帮助频繁修改和访问工作区文件的用户减少负担，否则频繁的多节点工作空间部署可能会很麻烦。
-`job_workspace`  设为已部署的工作空间目录，`paddle.py` 将跳过分发阶段直接启动所有节点的集群作业。它可以帮助减少分发延迟。
+-  `job_dispatch_package`  设为本地 `workspace` 目录，它将被分发到 `conf.py` 中设置的所有节点。它有助于帮助频繁修改和访问工作区文件的用户减少负担，否则频繁的多节点工作空间部署可能会很麻烦。
+-  `job_workspace`  设为已部署的工作空间目录，`paddle.py` 将跳过分发阶段直接启动所有节点的集群作业。它可以帮助减少分发延迟。
 
-`cluster_train/run.sh` 提供了命令样例来运行 `demo/recommendation` 集群工作，只需用你定义的目录修改 `job_dispatch_package` 和 `job_workspace`，然后：
+`cluster_train/run.sh` 提供了命令样例来运行 `doc/howto/usage/cluster/src/word2vec` 集群任务，只需用您定义的目录修改 `job_dispatch_package` 和 `job_workspace`，然后：
 ```
 sh run.sh
 ```
 
 集群作业将会在几秒后启动。
 
-### 终止集群作业
+#### 终止集群作业
 `paddle.py`能获取`Ctrl + C` SIGINT 信号来自动终止它启动的所有进程。只需中断 `paddle.py` 任务来终止集群作业。如果程序崩溃你也可以手动终止。
 
-### 检查集群训练结果
+#### 检查集群训练结果
 详细信息请检查 $workspace/log 里的日志，每一个节点都有相同的日志结构。
 
 `paddle_trainer.INFO`
@@ -149,11 +229,57 @@ sh run.sh
 提供 pserver 运行日志，有助于诊断分布式错误。
 
 `server.log`
-提供 pserver 进程的 stderr 和 stdout。训练失败时可以检查错误日志。
+提供 parameter server 进程的 stderr 和 stdout。训练失败时可以检查错误日志。
 
 `train.log`
 提供训练过程的 stderr 和 stdout。训练失败时可以检查错误日志。
 
-### 检查模型输出
+#### 检查模型输出
 运行完成后，模型文件将被写入节点 0 的 `output` 目录中。
 工作空间中的 `nodefile` 表示当前集群作业的节点 ID。
+
+### 在OpenMPI集群中提交训练作业
+
+#### 准备OpenMPI集群
+
+执行下面的命令以启动3个节点的OpenMPI集群和一个"head"节点：
+
+```bash
+paddle/scripts/cluster_train_v2/openmpi/docker_cluster
+kubectl create -f head.yaml
+kubectl create -f mpi-nodes.yaml
+```
+
+然后可以从head节点ssh无密码登录到OpenMPI的每个节点上。
+
+#### 启动集群作业
+
+您可以按照下面的步骤在OpenMPI集群中提交paddle训练任务：
+
+```bash
+# 获得head和node节点的IP地址
+kubectl get po -o wide
+# 将node节点的IP地址保存到machines文件中
+kubectl get po -o wide | grep nodes | awk '{print $6}' > machines
+# 拷贝必要的文件到head节点
+scp -i ssh/id_rsa.mpi.pub machines prepare.py train.py start_mpi_train.sh tutorial@[headIP]:~
+# ssh 登录到head节点
+ssh -i ssh/id_rsa.mpi.pub tutorial@[headIP]
+# --------------- 以下操作均在head节点中执行 ---------------
+# 准备训练数据
+python prepare.py
+# 拷贝训练程序和字典文件到每台MPI节点
+cat machines | xargs -i scp word_dict.pickle train.py start_mpi_train.sh machines {}:/home/tutorial
+# 创建日志目录
+mpirun -hostfile machines -n 3 mkdir /home/tutorial/logs
+# 拷贝训练数据到各自的节点
+scp train.txt-00000 test.txt-00000 [node1IP]:/home/tutorial
+scp train.txt-00001 test.txt-00001 [node2IP]:/home/tutorial
+scp train.txt-00002 test.txt-00002 [node3IP]:/home/tutorial
+# 启动训练任务
+mpirun -hostfile machines -n 3  /home/tutorial/start_mpi_train.sh
+```
+
+### 在Kubernetes集群中提交训练作业
+
+此部分的使用方法可以参考[here](../k8s/k8s_distributed_cn.md)。
diff --git a/doc_cn/_sources/howto/usage/cmd_parameter/arguments_cn.md.txt b/doc_cn/_sources/howto/usage/cmd_parameter/arguments_cn.md.txt
index f7aa525054468670f59309ddf9206af55bb77869..2dea231ca5487978d59a4d0a570431722ed6b3bf 100644
--- a/doc_cn/_sources/howto/usage/cmd_parameter/arguments_cn.md.txt
+++ b/doc_cn/_sources/howto/usage/cmd_parameter/arguments_cn.md.txt
@@ -63,7 +63,7 @@
 
 
 
-训练dot_period
+训练dot_period
 √√
 
 
diff --git a/doc_cn/_sources/howto/usage/k8s/k8s_distributed_cn.md.txt b/doc_cn/_sources/howto/usage/k8s/k8s_distributed_cn.md.txt
index 3121b3f59df650c0a22d0bd305a6f793b202d30e..a9bebf09558b06993119803458977abedbbfbdd0 100644
--- a/doc_cn/_sources/howto/usage/k8s/k8s_distributed_cn.md.txt
+++ b/doc_cn/_sources/howto/usage/k8s/k8s_distributed_cn.md.txt
@@ -213,7 +213,7 @@ I1116 09:10:17.123440    50 Util.cpp:130] Calling runInitFunctions
 I1116 09:10:17.123764    50 Util.cpp:143] Call runInitFunctions done.
 [WARNING 2016-11-16 09:10:17,227 default_decorators.py:40] please use keyword arguments in paddle config.
 [INFO 2016-11-16 09:10:17,239 networks.py:1282] The input order is [movie_id, title, genres, user_id, gender, age, occupation, rating]
-[INFO 2016-11-16 09:10:17,239 networks.py:1289] The output order is [__mse_cost_0__]
+[INFO 2016-11-16 09:10:17,239 networks.py:1289] The output order is [__square_error_cost_0__]
 I1116 09:10:17.392917    50 Trainer.cpp:170] trainer mode: Normal
 I1116 09:10:17.613910    50 PyDataProvider2.cpp:257] loading dataprovider dataprovider::process
 I1116 09:10:17.680917    50 PyDataProvider2.cpp:257] loading dataprovider dataprovider::process
diff --git a/doc_cn/_sources/index_cn.rst.txt b/doc_cn/_sources/index_cn.rst.txt
index 9279bac7f4b2898c18979630a8d6dfcb2dba70e0..ada51c2d73263898b2c748437f8eb0f30b537073 100644
--- a/doc_cn/_sources/index_cn.rst.txt
+++ b/doc_cn/_sources/index_cn.rst.txt
@@ -8,3 +8,4 @@ PaddlePaddle 文档
   howto/index_cn.rst
   api/index_cn.rst
   faq/index_cn.rst
+  mobile/index_cn.rst
diff --git a/doc_cn/_sources/mobile/cross_compiling_for_android_cn.md.txt b/doc_cn/_sources/mobile/cross_compiling_for_android_cn.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..424d7718c64438496cf0895397babd5408e1ca02
--- /dev/null
+++ b/doc_cn/_sources/mobile/cross_compiling_for_android_cn.md.txt
@@ -0,0 +1,168 @@
+# Android平台编译指南
+
+用户可通过如下两种方式，交叉编译Android平台上适用的PaddlePaddle库：
+- 基于Docker容器的编译方式
+- 基于Linux交叉编译环境的编译方式
+
+## 基于Docker容器的编译方式
+Docker能在所有主要操作系统（包括Linux，Mac OS X和Windows）上运行，因此，使用基于Docker容器的编译方式，用户可在自己熟悉的开发平台上编译Android平台上适用的PaddlePaddle库。
+
+### 构建PaddlePaddle的Android开发镜像
+我们把PaddlePaddle的交叉编译环境打包成一个镜像，称为开发镜像，里面涵盖了交叉编译Android版PaddlePaddle库需要的所有编译工具。
+
+```bash
+$ git clone https://github.com/PaddlePaddle/Paddle.git
+$ cd Paddle
+$ docker build -t username/paddle-android:dev . -f Dockerfile.android
+```
+
+### 编译PaddlePaddle C-API库
+构建好开发镜像后，即可使用开发镜像来编译Android版PaddlePaddle C-API库。
+Android的Docker开发镜像向用户提供两个可配置的参数：
+
+
++  +  +  +
+
+  
+  
+  
+  
+
+
+
+  
+  
+  
+  
+
+
+  
+  
+  
+
+
+
    




    ArgumentOptional ValuesDefault
    ANDROID_ABIarmeabi-v7a, arm64-v8aarmeabi-v7a
    ANDROID_API>= 2121
    
+
+- 编译`armeabi-v7a`，`Android API 21`的PaddlePaddle库
+  ```bash
+  $ docker run -it --rm -v $PWD:/paddle -e "ANDROID_ABI=armeabi-v7a" -e "ANDROID_API=21" username/paddle-android:dev
+  ```
+
+- 编译`arm64-v8a`，`Android API 21`的PaddlePaddle库
+  ```bash
+  $ docker run -it --rm -v $PWD:/paddle -e "ANDROID_ABI=arm64-v8a" -e "ANDROID_API=21" username/paddle-android:dev
+  ```
+
+执行上述`docker run`命令时，容器默认执行[paddle/scripts/docker/build_android.sh](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/docker/build_android.sh)脚本。该脚本中记录了交叉编译Android版PaddlePaddle库常用的CMake配置，并且会根据`ANDROID_ABI`和`ANDROID_API`自动构建独立工具链、进行编译和安装。由于arm64架构要求Android API不小于21。因此当`ANDROID_ABI=arm64-v8a`，`ANDROID_API<21`时，Docker容器中将默认使用`Android API 21`的编译工具链。用户可以参考下文**配置交叉编译参数**章节，根据个人的需求修改定制Docker容器所执行的脚本。编译安装结束之后，PaddlePaddle的C-API库将被安装到`$PWD/install_android`目录，所依赖的第三方库同时也被安装到`$PWD/install_android/third_party`目录。
+
+## 基于Linux交叉编译环境的编译方式
+本文档将以Linux x86-64平台为例，介绍交叉编译Android平台上适用的PaddlePaddle库的方法和步骤。
+
+### 准备交叉编译环境
+
+从源码交叉编译PaddlePaddle，用户需要提前准备好交叉编译环境。Android平台上使用的C/C++交叉编译工具链为[Android NDK](https://developer.android.com/ndk/downloads/index.html?hl=zh-cn)，用户可自行前往下载预编译好的版本，也可通过以下命令获取：
+
+```bash
+wget -q https://dl.google.com/android/repository/android-ndk-r14b-linux-x86_64.zip
+unzip -q android-ndk-r14b-linux-x86_64.zip
+```
+
+Android NDK中包含了所有Android API级别、所有架构（arm/arm64/x86/mips）需要用到的编译工具和系统库。用户可根据自己的编译目标架构、所需支持的最低Android API级别，构建[独立工具链](https://developer.android.google.cn/ndk/guides/standalone_toolchain.html?hl=zh-cn)。
+
+- 构建`armeabi-v7a`、 `Android API 21`的独立工具链：
+
+```bash
+your/path/to/android-ndk-r14b-linux-x86_64/build/tools/make-standalone-toolchain.sh \
+        --arch=arm --platform=android-21 --install-dir=your/path/to/arm_standalone_toolchain
+```
+
+此命令将在`your/path/to/arm_standalone_toolchain`目录生成一套独立编译工具链，面向架构为32位ARM架构，支持的最小的Android API级别为21，支持编译器`arm-linux-androideabi-gcc (GCC) 4.9`和`clang 3.8`。
+
+- 构建`arm64-v8a`、 `Android API 21`的独立工具链：
+```bash
+your/path/to/android-ndk-r14b-linux-x86_64/build/tools/make-standalone-toolchain.sh \
+        --arch=arm64 --platform=android-21 --install-dir=your/path/to/arm64_standalone_toolchain
+```
+
+此命令将在`your/path/to/arm64_standalone_toolchain`目录生成一套独立编译工具链，面向架构为64位ARM64架构，支持的最小Android API级别为21，支持编译器`arm-linux-androideabi-gcc (GCC) 4.9`和`clang 3.8`。
+
+注意：**PaddlePaddle要求使用的编译工具链所支持的Android API级别不小于21**。
+
+### 配置交叉编译参数
+
+CMake系统对交叉编译提供了支持[cmake-toolchains](https://cmake.org/cmake/help/v3.0/manual/cmake-toolchains.7.html#cross-compiling)。为了简化cmake配置，PaddlePaddle为交叉编译提供了工具链配置文档[cmake/cross_compiling/android.cmake](https://github.com/PaddlePaddle/Paddle/blob/develop/cmake/cross_compiling/android.cmake)，以提供一些默认的编译器和编译参数相关配置。注意，从CMake 3.7版本开始，CMake官方对Android平台的交叉编译提供了通用的支持。PaddlePaddle若检测到用户使用的CMake版本不低于3.7时，将会将用户传进来的配置参数传递CMake系统，交由CMake系统本身来处理。有关参数配置的详细说明见[cmake-toolchains](https://cmake.org/cmake/help/v3.7/manual/cmake-toolchains.7.html#cross-compiling)。
+
+交叉编译Android版本的PaddlePaddle库时，有一些必须配置的参数：
+- `CMAKE_SYSTEM_NAME`，CMake编译的目标平台，必须设置为`Android`。在设置`CMAKE_SYSTEM_NAME=Android`后，PaddlePaddle的CMake系统才认为是在交叉编译Android系统的版本，并自动编译宿主机版protoc可执行文件、目标机版protobuf库、以及Android所需`arm_soft_fp_abi`分支的目标机版OpenBLAS库。此外，还会强制设置一些PaddlePaddle参数的值（`WITH_GPU=OFF`、`WITH_AVX=OFF`、`WITH_PYTHON=OFF`、`WITH_RDMA=OFF`）。
+- `WITH_C_API`，必须设置为`ON`。在Android平台上只支持使用C-API来预测。
+- `WITH_SWIG_PY`，必须设置为`OFF`。在Android平台上不支持通过swig调用来训练或者预测。
+
+Android平台可选配置参数：
+
+- `ANDROID_STANDALONE_TOOLCHAIN`，独立工具链所在的绝对路径，或者相对于构建目录的相对路径。PaddlePaddle的CMake系统将根据该值自动推导和设置需要使用的交叉编译器、sysroot、以及Android API级别；否则，用户需要在cmake时手动设置这些值。无默认值。
+- `ANDROID_TOOLCHAIN`，目标工具链。可设置`gcc/clang`，默认值为`clang`。
+	- CMake 3.7以上，将会始终使用`clang`工具链；CMake 3.7以下，可设置`ANDROID_TOOLCHAIN=gcc`以使用`gcc`工具链。
+	- Android官方提供的`clang`编译器要求系统支持`GLIBC 2.15`以上。
+- `ANDROID_ABI`，目标架构ABI。目前支持`armeabi-v7a`和`arm64-v8a`，默认值为`armeabi-v7a`。
+- `ANDROID_NATIVE_API_LEVEL`，工具链的Android API级别。若没有显式设置，PaddlePaddle将根据`ANDROID_STANDALONE_TOOLCHAIN`的值自动推导得到。
+- `ANROID_ARM_MODE`，是否使用ARM模式。
+	- `ANDROID_ABI=armeabi-v7a`时，可设置`ON/OFF`，默认值为`ON`；
+	- `ANDROID_ABI=arm64-v8a`时，不需要设置。
+- `ANDROID_ARM_NEON`，是否使用NEON指令。
+	- `ANDROID_ABI=armeabi-v7a`时，可设置`ON/OFF`，默认值为`ON`；
+	- `ANDROID_ABI=arm64-v8a`时，不需要设置。
+
+其他配置参数：
+
+- `USE_EIGEN_FOR_BLAS`，是否使用Eigen库进行矩阵计算。可设置`ON/OFF`，默认值为`OFF`。
+- `HOST_C/CXX_COMPILER`，宿主机的C/C++编译器。在编译宿主机版protoc可执行文件和目标机版OpenBLAS库时需要用到。默认设置成环境变量`CC`的值；若环境变量`CC`没有设置，则设置成`cc`编译器。
+
+常用的cmake配置如下：
+
+```bash
+cmake -DCMAKE_SYSTEM_NAME=Android \
+      -DANDROID_STANDALONE_TOOLCHAIN=your/path/to/arm_standalone_toolchain \
+      -DANDROID_ABI=armeabi-v7a \
+      -DANDROID_ARM_NEON=ON \
+      -DANDROID_ARM_MODE=ON \
+      -DUSE_EIGEN_FOR_BLAS=ON \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_C_API=ON \
+      -DWITH_SWIG_PY=OFF \
+      ..
+```
+
+```
+cmake -DCMAKE_SYSTEM_NAME=Android \
+      -DANDROID_STANDALONE_TOOLCHAIN=your/path/to/arm64_standalone_toolchain \
+      -DANDROID_ABI=arm64-v8a \
+      -DUSE_EIGEN_FOR_BLAS=OFF \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_C_API=ON \
+      -DWITH_SWIG_PY=OFF \
+      ..
+```
+
+用户还可根据自己的需求设置其他编译参数。比如希望最小化生成的库的大小，可以设置`CMAKE_BUILD_TYPE`为`MinSizeRel`；若希望最快的执行速度，则可设置`CMAKE_BUILD_TYPE`为`Release`。亦可以通过手动设置`CMAKE_C/CXX_FLAGS_MINSIZEREL/RELEASE`来影响PaddlePaddle的编译过程。
+
+**性能TIPS**，为了达到最快的计算速度，在CMake参数配置上，有以下建议：
+- 设置`CMAKE_BUILD_TYPE`为`Release`
+- 使用`clang`编译工具链
+- `armeabi-v7a`时，设置`USE_EIGEN_BLAS=ON`，使用Eigen进行矩阵计算；`arm64-v8a`时，设置`USE_EIGEN_FOR_BLAS=OFF`，使用OpenBLAS进行矩阵计算
+
+### 编译和安装
+
+CMake配置完成后，执行以下命令，PaddlePaddle将自动下载和编译所有第三方依赖库、编译和安装PaddlePaddle预测库。
+
+```bash
+make
+make install
+```
+
+注意：如果你曾经在源码目录下编译过其他平台的PaddlePaddle库，请先使用`rm -rf`命令删除`third_party`目录和`build`目录，以确保所有的第三方依赖库和PaddlePaddle代码都是针对新的CMake配置重新编译的。
+
+执行完安装命令后，`your/path/to/install`目录中会包含`include`、`lib`和`third_party`目录，其中`include`中包含C-API的头文件，`lib`中包含若干个不同Android ABI的PaddlePaddle库，`third_party`中包含所依赖的所有第三方库。自此，PaddlePaddle的已经安装完成，用户可将`your/path/to/install`目录下的生成文件用于深度学习相关Android App中，调用方法见C-API文档。
diff --git a/doc_cn/_sources/mobile/cross_compiling_for_ios_cn.md.txt b/doc_cn/_sources/mobile/cross_compiling_for_ios_cn.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..9da48e7f2119ce901fbb3abab73400df27be16d2
--- /dev/null
+++ b/doc_cn/_sources/mobile/cross_compiling_for_ios_cn.md.txt
@@ -0,0 +1,117 @@
+# iOS平台编译指南
+交叉编译iOS平台上适用的PaddlePaddle库，需要在MacOS系统上进行。本文的将介绍在MacOS上，从源码交叉编译iOS平台上适用的PaddlePaddle库。
+
+## 准备交叉编译环境
+Apple官方为iOS开发提供了完整的交叉编译工具和集成开发环境，用户从App Store下载安装Xcode即可。也可自行前往官网下载，[Xcode](https://developer.apple.com/cn/xcode/)。安装完成之后，可在命令行执行`xcodebuild -version`，判断是否安装成功。
+
+```bash
+$ xcodebuild -version
+Xcode 9.0
+Build version 9A235
+```
+
+## 配置交叉编译参数
+
+PaddlePaddle为交叉编译提供了工具链配置文档[cmake/cross_compiling/ios.cmake](https://github.com/PaddlePaddle/Paddle/blob/develop/cmake/cross_compiling/ios.cmake)，以提供一些默认的编译器和编译参数配置。
+
+交叉编译iOS版本的PaddlePaddle库时，有一些必须配置的参数：
+
+- `CMAKE_SYSTEM_NAME`，CMake编译的目标平台，必须设置为`iOS`。在设置`CMAKE_SYSTEM_NAME=iOS`后，PaddlePaddle的CMake系统会自动编译所有的第三方依赖库，并且强制设置一些PaddlePaddle参数的值（`WITH_C_API=ON`、`WITH_GPU=OFF`、`WITH_AVX=OFF`、`WITH_PYTHON=OFF`、`WITH_RDMA=OFF`）。
+- `WITH_C_API`，是否编译C-API预测库，必须设置为ON。在iOS平台上只支持使用C-API来预测。
+- `WITH_SWIG_PY`，必须设置为ON。在iOS平台上不支持通过swig调用来训练或者预测。
+
+iOS平台可选配置参数：
+
+- `IOS_PLATFORM`，可设置为`OS/SIMULATOR`，默认值为`OS`。
+  - `OS`，构建目标为`arm`架构的iPhone或者iPad等物理设备。
+  - `SIMULATOR`，构建目标为`x86`架构的模拟器平台。
+- `IOS_ARCH`，目标架构。针对不同的`IOS_PLATFORM`，可设置的目标架构如下表所示，默认编译所有架构：
+
+    
+    +      +      +    
+    
+      
+      
+      
+    
+    
+    
+      
+      
+      
+    
+    
+      
+      
+    
+    
+    
    



    IOS_PLATFORMIOS_ARCH
    OSarmv7, armv7s, arm64 
    SIMULATORi386, x86_64 
    
+
+- `IOS_DEPLOYMENT_TARGET`，最小的iOS部署版本，默认值为`7.0`。
+- `IOS_ENABLE_BITCODE`，是否使能[Bitcode](https://developer.apple.com/library/content/documentation/IDEs/Conceptual/AppDistributionGuide/AppThinning/AppThinning.html#//apple_ref/doc/uid/TP40012582-CH35-SW3)，可设置`ON/OFF`，默认值为`ON`。
+- `IOS_USE_VECLIB_FOR_BLAS`，是否使用[vecLib](https://developer.apple.com/documentation/accelerate/veclib)框架进行BLAS矩阵计算，可设置`ON/OFF`，默认值为`OFF`。
+- `IOS_DEVELOPMENT_ROOT`，`Developer`目录，可显式指定为`/path/to/platform/Developer`。若未显式指定，PaddlePaddle将会根据`IOS_PLATFORM`自动选择`Xcode`对应`platform`的`Developer`目录。
+- `IOS_SDK_ROOT`，所使用`SDK`的根目录，可显式指定为`/path/to/platform/Developer/SDKs/SDK`。若未显式指定，PaddlePaddle将会自动选择`IOS_DEVELOPMENT_ROOT`目录下最新的`SDK`版本。
+
+其他配置参数：
+
+- `USE_EIGEN_FOR_BLAS`，是否使用Eigen库进行矩阵计算，在`IOS_USE_VECLIB_FOR_BLAS=OFF`时有效。可设置`ON/OFF`，默认值为`OFF`。
+- `HOST_C/CXX_COMPILER`，宿主机的C/C++编译器。默认值为环境变量`CC/CXX`的值；若环境变量`CC/CXX`未设置，则使用`cc/c++`编译器。
+
+常用的cmake配置如下：
+
+```bash
+cmake -DCMAKE_SYSTEM_NAME=iOS \
+      -DIOS_PLATFORM=OS \
+      -DIOS_ARCH="armv7;arm64" \
+      -DIOS_ENABLE_BITCODE=ON \
+      -DIOS_USE_VECLIB_FOR_BLAS=ON \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_C_API=ON \
+      -DWITH_TESTING=OFF \
+      -DWITH_SWIG_PY=OFF \
+      ..
+```
+
+```bash
+cmake -DCMAKE_SYSTEM_NAME=iOS \
+      -DIOS_PLATFORM=SIMULATOR \
+      -DIOS_ARCH="x86_64" \
+      -DIOS_USE_VECLIB_FOR_BLAS=ON \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_C_API=ON \
+      -DWITH_TESTING=OFF \
+      -DWITH_SWIG_PY=OFF \
+      ..
+```
+
+用户还可根据自己的需求设置其他编译参数。比如希望最小化生成库的大小，可以设置`CMAKE_BUILD_TYPE`为`MinSizeRel`；若希望得到最快的执行速度，则可设置`CMAKE_BUILD_TYPE`为`Release`。亦可以通过手动设置`CMAKE_C/CXX_FLAGS`来影响PaddlePaddle的编译过程。
+
+**性能TIPS**，为了达到最快的计算速度，在CMake参数配置上，有以下建议：
+
+- 设置`CMAKE_BUILD_TYPE`为`Release`
+- 设置`IOS_USE_VECLIB_FOR_BLAS=ON`，调用`vecLib`框架提供的BLAS函数进行矩阵计算。
+
+## 编译和安装
+
+CMake配置完成后，执行以下命令，PaddlePaddle将自动下载和编译所有第三方依赖库、编译和安装PaddlePaddle预测库。
+
+```
+$ make
+$ make install
+```
+
+注意：如果你曾在源码目录下编译过其他平台的PaddlePaddle库，请先使用`rm -rf`命令删除`third_party`目录和`build`目录，以确保所有的第三方依赖库和PaddlePaddle代码都是针对新的CMake配置重新编译的。
+
+执行完安装命令后，`your/path/to/install`目录中会包含以下内容：
+
+- `include`目录，其中包含所有C-API的头文件
+- `lib`目录，其中包含PaddlePaddle的C-API静态库
+- `third_party`目录，其中包含所依赖的所有第三方库
+
+注意，如果PaddlePaddle库需要同时支持真机和模拟器，则需要分别编译真机和模拟器版本，然后使用`lipo`工具合并fat库。
+
+自此，PaddlePaddle库已经安装完成，用户可将合成的fat库用于深度学习相关的iOS App中，调用方法见C-API文档。
diff --git a/doc_cn/_sources/mobile/cross_compiling_for_raspberry_cn.md.txt b/doc_cn/_sources/mobile/cross_compiling_for_raspberry_cn.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..f8ef9dc8031613831437745995268f3abc392f5b
--- /dev/null
+++ b/doc_cn/_sources/mobile/cross_compiling_for_raspberry_cn.md.txt
@@ -0,0 +1,62 @@
+# Raspberry Pi平台编译指南
+
+通常有两个方法来构建基于 Rasspberry Pi 的版本：
+
+1. 通过ssh等方式登录到Raspberry Pi系统上来构建。所需的开发工具和第三方库可以参考 [`/Dockerfile`](https://github.com/PaddlePaddle/Paddle/blob/develop/Dockerfile)。
+
+1. 另一个方法是交叉编译。这篇文档介绍在 Linux/x64 上交叉编译Raspberry Pi平台上适用的PaddlePaddle的方法和步骤。
+
+## 安装交叉编译器
+
+克隆下面 Github repo
+
+```bash
+git clone https://github.com/raspberrypi/tools.git
+```
+
+即可在 `./tools/tree/master/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64` 目录里找到交叉编译器 arm-linux-gnueabihf-gcc 4.8.3。运行该编译工具链需要一台 Linux x64 机器上以及 2.14版本以上的 glibc。
+
+## 配置交叉编译参数
+
+CMake[支持交叉编译](https://cmake.org/cmake/help/v3.0/manual/cmake-toolchains.7.html#cross-compiling)。PaddlePaddle for Raspberry Pi的配置信息在[cmake/cross_compiling/raspberry_pi.cmake](https://github.com/PaddlePaddle/Paddle/blob/develop/cmake/cross_compiling/raspberry_pi.cmake)。
+
+交叉编译Raspberry Pi版本PaddlePaddle库时，有一些必须配置的参数：
+
+- `CMAKE_SYSTEM_NAME`：CMake编译的目标平台，必须配置为`RPi`。在设置`CMAKE_SYSTEM_NAME=RPi`后，PaddlePaddle的CMake系统才认为在是在交叉编译Raspberry Pi系统的版本，并自动编译宿主机版protoc可执行文件、目标机版protobuf库、以及目标机版OpenBLAS库。
+
+- `RPI_TOOLCHAIN`：编译工具链所在的绝对路径，或者相对于构建目录的相对路径。PaddlePaddle的CMake系统将根据该值自动设置需要使用的交叉编译器；否则，用户需要在cmake时手动设置这些值。无默认值。
+
+- `RPI_ARM_NEON`：是否使用NEON指令。目前必须设置成`ON`，默认值为`ON`。
+
+- `HOST_C/CXX_COMPILER`，宿主机的C/C++编译器。在编译宿主机版protoc可执行文件和目标机版OpenBLAS库时需要用到。默认设置成环境变量`CC`的值；若环境变量`CC`没有设置，则设置成`cc`编译器。
+
+一个常用的CMake配置如下：
+
+```
+cmake -DCMAKE_SYSTEM_NAME=RPi \
+      -DRPI_TOOLCHAIN=your/path/to/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64 \
+      -DRPI_ARM_NEON=ON \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_GPU=OFF \
+      -DWITH_C_API=ON \
+      -DWITH_PYTHON=OFF \
+      -DWITH_SWIG_PY=OFF \
+      ..
+```
+
+其中`WITH_C_API=ON`表示需要构建推理库。
+
+用户还可根据自己的需求设置其他编译参数。比如希望最小化生成的库的大小，可以设置`CMAKE_BUILD_TYPE`为`MinSizeRel`；若希望最快的执行速度，则可设置`CMAKE_BUILD_TYPE`为`Release`。
+
+## 编译和安装
+
+CMake配置完成后，执行以下命令，PaddlePaddle将自动下载和编译所有第三方依赖库、编译和安装PaddlePaddle。
+
+```bash
+make
+make install
+```
+
+注意：如果你曾经在源码目录下编译过其他平台的PaddlePaddle库，请先使用`rm -rf`命令删除`third_party`目录和`build`目录，以确保所有的第三方依赖库和PaddlePaddle代码都是针对新的CMake配置重新编译的。
+
+执行完安装命令后，`your/path/to/install`目录中会包含`include`和`lib`目录，其中`include`中包含C-API的头文件，`lib`中包含一个Raspberry Pi版本的库。
diff --git a/doc_cn/_sources/mobile/index_cn.rst.txt b/doc_cn/_sources/mobile/index_cn.rst.txt
new file mode 100644
index 0000000000000000000000000000000000000000..1d99666e58b7043b85b0203ee0dfcd1957710161
--- /dev/null
+++ b/doc_cn/_sources/mobile/index_cn.rst.txt
@@ -0,0 +1,9 @@
+MOBILE
+======
+
+..  toctree::
+  :maxdepth: 1
+
+  cross_compiling_for_android_cn.md
+  cross_compiling_for_ios_cn.md
+  cross_compiling_for_raspberry_cn.md
diff --git a/doc_cn/_sources/survey/cluster_bootstrapping_tools.md.txt b/doc_cn/_sources/survey/cluster_bootstrapping_tools.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..1cd9962700bb49866f1ed6987abc28b27888a23f
--- /dev/null
+++ b/doc_cn/_sources/survey/cluster_bootstrapping_tools.md.txt
@@ -0,0 +1,71 @@
+# Cluster bootstrapping tool survey
+## Abstract
+In order to bring up a cluster from bare metal machine to a fully functional kubernetes cluster for Paddlepaddle to run, we need to utilize some tools. Here we are going to compare [Sextant](https://github.com/k8sp/sextant) and [Tectonic installer](https://github.com/coreos/tectonic-installer)
+
+## Basic assumptions
+Here are some basic assumptions before we move on to  details
+1. You are an administrator of a bare metal machine cluster, which means:
+  * you have full control to each of the machines.
+  * you have full control to the network which machines are connected to.
+2. Machines can be booted from network with PEX or iPXE
+3. You understand the [general procedure to bring up a cluster](#appendix-general-procedure-to-bring-up-a-cluster)
+
+if your cluster is able to mark above items with checkmarks, then keep reading.
+
+## Comparing Sextant and Tectonic installer
+### Sextant
+Sextant is an end2end solution to bring up a bare metal cluster to a fully functional k8s cluster, it integrates DHCP, name service, PEX, cloud-config-service, docker registry services altogether. 
+
+#### Pros
+1. End2End: basically all admin need to do is to config the cluster.yaml and power on the cluster.
+2. Offline cluster configuration: Sextant has 2 phases during working with it, config time and deploy time. when admin is configuring, it requires admin's machine has internet connectivity, which will download some images, etc. But in deploy time, it's completely OK to go offline since all dependencies are ready during config time.
+3. docker registry integrated.
+4. GPU machine took care of.
+
+### Cons
+1. k8s API server is not deployed with high availability in considering by default.
+2. No grouping support.
+3. No API interface, a one-off service.
+
+
+### Tectonic installer
+First of all, Tectonic is not free, it requires coreos.com account as a step of installation, and free user can only create less than 10 nodes.
+
+Tectonic is a suite of software which wraps around k8s and providing more utility regarding dev ops, ie, 
+Tectonic installer as it's named, it installs Tectonic to a bare metal cluster which means it's not totally an equivalent of Sextant. At the "booting a cluster" part, it mostly utilizes [Matchbox](https://github.com/coreos/matchbox), which is a general cluster bootstrapper.
+
+Matchbox's Approach is similar to Sexstant.
+
+### Pros
+1. supports grouping machines.
+2. supports running provisioning service in rtk. (not a big deal though).
+3. supports http/gRPC API interface.
+4. supports multi-template.
+
+### Cons
+1. Not an e2e solution to bring up a cluster, need a lot of extra work and other software.
+2. [Not fully supporting](https://github.com/coreos/matchbox/issues/550) centOS deployment yet.
+
+## Conclusion
+Sextant is a better solution overall for paddle cloud deploying to a bare metal cluster. It would be great if Sextant can also 1) deploy k8s api server with high availability by default; 2) not designed as a one-off service.
+
+
+
+## Appendix: General procedure to bring up a cluster
+It's physically impossible for a cluster admin to manually install OS and applications into cluster nodes one by one, here is what an admin would do in cloud industry:
+1. setup a bootstrap machine with static IP in the cluster, which has following services:
+  * DHCP: assigns ip address for rest of the nodes.
+  * name service: to map node name to a IP
+  * PXE related services: the booting related info will be delivered to newly booted machines as their IP is assigned via DHCP service, PXE service will provide further booting and installing info and image with TFTP and http protocol. 
+  * cluster config service: this is for providing cluster node with OS config via http
+  * optional docker registry: a built-in docker registry makes the whole cluster independent from connecting internet, and speeds up software distribution.
+2. New node powers on, it will
+  * broadcast the request for an IP address
+  * DHCP server assigns the IP address, and deliver the PXE booting related info to the node.
+  * cluster node will request config files with booting info delivered with DHCP via the TFTP service, and in most of the cases, the config file will point to a http service for the booting image.
+  * Since PXE is configured with initrd, it will utilize the cloud config service and do further installations like coreOS or K8s installations.
+  * then restart the node.
+
+For further understanding, following 2 links from Matchbox are some good readings:
+* [Machine lifecycle](https://github.com/coreos/matchbox/blob/master/Documentation/machine-lifecycle.md)
+* [PXE booting](https://github.com/coreos/matchbox/blob/master/Documentation/network-booting.md)
diff --git a/doc_cn/_sources/tutorials/image_classification/index_cn.md.txt b/doc_cn/_sources/tutorials/image_classification/index_cn.md.txt
deleted file mode 100644
index 87f465522a0fa21c8c03754b4be8dcb035c4de81..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/tutorials/image_classification/index_cn.md.txt
+++ /dev/null
@@ -1,205 +0,0 @@
-图像分类教程
-==========
-
-在本教程中，我们将使用CIFAR-10数据集训练一个卷积神经网络，并使用这个神经网络来对图片进行分类。如下图所示，卷积神经网络可以辨识图片中的主体，并给出分类结果。
-
    ![Image Classification](./image_classification.png)
    
-
-## 数据准备
-首先下载CIFAR-10数据集。下面是CIFAR-10数据集的官方网址：
-
-
-
-我们准备了一个脚本，可以用于从官方网站上下载CIFAR-10数据集，转为jpeg文件并存入特定的目录。使用这个脚本前请确认已经安装了pillow及相关依赖模块。可以参照下面的命令进行安装：
-
-1. 安装pillow
-
-```bash
-sudo apt-get install libjpeg-dev
-pip install pillow
-```
-
-2. 下载数据集
-
-```bash
-cd demo/image_classification/data/
-sh download_cifar.sh
-```
-
-CIFAR-10数据集包含60000张32x32的彩色图片。图片分为10类，每个类包含6000张。其中50000张图片作为训练集，10000张作为测试集。
-
-下图展示了所有的图片类别，每个类别中随机抽取了10张图片。
-
    ![Image Classification](./cifar.png)
    
-
-脚本运行完成后，我们应当会得到一个名为cifar-out的文件夹，其下子文件夹的结构如下
-
-
-```
-train
----airplane
----automobile
----bird
----cat
----deer
----dog
----frog
----horse
----ship
----truck
-test
----airplane
----automobile
----bird
----cat
----deer
----dog
----frog
----horse
----ship
----truck
-```
-
-cifar-out下包含`train`和`test`两个文件夹，其中分别包含了CIFAR-10中的训练集和测试集。这两个文件夹下各自有10个子文件夹，每个子文件夹下存储相应分类的图片。将图片按照上述结构存储好之后，我们就可以着手对分类模型进行训练了。
-
-## 预处理
-数据下载之后，还需要进行预处理，将数据转换为Paddle的格式。我们可以通过如下命令进行预处理工作：
-
-```
-cd demo/image_classification/
-sh preprocess.sh
-```
-
-其中`preprocess.sh` 调用 `./demo/image_classification/preprocess.py` 对图片进行预处理
-```sh
-export PYTHONPATH=$PYTHONPATH:../../
-data_dir=./data/cifar-out
-python preprocess.py -i $data_dir -s 32 -c 1
-```
-
-`./demo/image_classification/preprocess.py` 使用如下参数：
-
-- `-i` 或 `--input` 给出输入数据所在路径；
-- `-s` 或 `--size` 给出图片尺寸；
-- `-c` 或 `--color` 标示图片是彩色图或灰度图
-
-## 模型训练
-在开始训练之前，我们需要先创建一个模型配置文件。下面我们给出了一个配置示例。**注意**，这里的列出的和`vgg_16_cifar.py`文件稍有差别，因为该文件可适用于预测。
-
-```python
-from paddle.trainer_config_helpers import *
-data_dir='data/cifar-out/batches/'
-meta_path=data_dir+'batches.meta'
-args = {'meta':meta_path, 'mean_img_size': 32,
-        'img_size': 32, 'num_classes': 10,
-        'use_jpeg': 1, 'color': "color"}
-define_py_data_sources2(train_list=data_dir+"train.list",
-                        test_list=data_dir+'test.list',
-                        module='image_provider',
-                        obj='processData',
-                        args=args)
-settings(
-    batch_size = 128,
-    learning_rate = 0.1 / 128.0,
-    learning_method = MomentumOptimizer(0.9),
-    regularization = L2Regularization(0.0005 * 128))
-
-img = data_layer(name='image', size=3*32*32)
-lbl = data_layer(name="label", size=10)
-# small_vgg is predined in trainer_config_helpers.network
-predict = small_vgg(input_image=img, num_channels=3)
-outputs(classification_cost(input=predict, label=lbl))
-```
-
-在第一行中我们载入用于定义网络的函数。
-```python
-from paddle.trainer_config_helpers import *
-```
-
-之后定义的`define_py_data_sources2`使用Python数据提供器，其中 `args`将在`image_provider.py`进行使用，该文件负责产生图片数据并传递给Paddle系统
- - `meta`: 训练集平均值。
- - `mean_img_size`: 平均特征图的高度及宽度。
- - `img_size`：输入图片的高度及宽度。
- - `num_classes`：类别个数。
- - `use_jpeg`：处理过程中数据存储格式。
- - `color`：标示是否为彩色图片。
- 
- `settings`用于设置训练算法。在下面的例子中，learning rate被设置为0.1除以batch size，而weight decay则为0.0005乘以batch size。
- 
- ```python
-settings(
-    batch_size = 128,
-    learning_rate = 0.1 / 128.0,
-    learning_method = MomentumOptimizer(0.9),
-    regularization = L2Regularization(0.0005 * 128)
-)
-```
-
-`small_vgg`定义了网络结构。这里我们使用的是一个小的VGG网络。关于VGG卷积神经网络的描述可以参考：[http://www.robots.ox.ac.uk/~vgg/research/very_deep/](http://www.robots.ox.ac.uk/~vgg/research/very_deep/)。
-```python
-# small_vgg is predined in trainer_config_helpers.network
-predict = small_vgg(input_image=img, num_channels=3)
-```
-配置创建完毕后，可以运行脚本train.sh来训练模型。
-
-```bash
-config=vgg_16_cifar.py
-output=./cifar_vgg_model
-log=train.log
-
-paddle train \
---config=$config \
---dot_period=10 \
---log_period=100 \
---test_all_data_in_one_period=1 \
---use_gpu=1 \
---save_dir=$output \
-2>&1 | tee $log
-
-python -m paddle.utils.plotcurve -i $log > plot.png
-```
-- 这里我们使用的是GPU模式进行训练。如果你没有GPU环境，可以设置`use_gpu=0`。
-- `./demo/image_classification/vgg_16_cifar.py`是网络和数据配置文件。各项参数的详细说明可以在命令行参数相关文档中找到。
-- 脚本`plotcurve.py`依赖于python的`matplotlib`模块。因此如果这个脚本运行失败，也许是因为需要安装`matplotlib`。
-在训练完成后，训练及测试误差曲线图会被`plotcurve.py`脚本保存在 `plot.png`中。下面是一个误差曲线图的示例：
-
-
    ![Training and testing curves.](./plot.png)
    
-
-## 预测
-在训练完成后，模型及参数会被保存在路径`./cifar_vgg_model/pass-%05d`下。例如第300个pass的模型会被保存在`./cifar_vgg_model/pass-00299`。
-
-要对一个图片的进行分类预测，我们可以使用`predict.sh`，该脚本将输出预测分类的标签：
-
-```
-sh predict.sh
-```
-
-predict.sh:
-```
-model=cifar_vgg_model/pass-00299/
-image=data/cifar-out/test/airplane/seaplane_s_000978.png
-use_gpu=1
-python prediction.py $model $image $use_gpu
-```
-
-## 练习
-在CUB-200数据集上使用VGG模型训练一个鸟类图片分类模型。相关的鸟类数据集可以从如下地址下载，其中包含了200种鸟类的照片（主要来自北美洲）。
-
-
-
-
-
-
-## 细节探究
-### 卷积神经网络
-卷积神经网络是一种使用卷积层的前向神经网络，很适合构建用于理解图片内容的模型。一个典型的神经网络如下图所示：
-
-![Convolutional Neural Network](./lenet.png)
-
-一个卷积神经网络包含如下层：
-
-- 卷积层：通过卷积操作从图片或特征图中提取特征
-- 池化层：使用max-pooling对特征图下采样
-- 全连接层：使输入层到隐藏层的神经元是全部连接的。
-
-卷积神经网络在图片分类上有着惊人的性能，这是因为它发掘出了图片的两类重要信息：局部关联性质和空间不变性质。通过交替使用卷积和池化处理， 卷积神经网络能够很好的表示这两类信息。
-
-关于如何定义网络中的层，以及如何在层之间进行连接，请参考Layer文档。
diff --git a/doc_cn/_sources/tutorials/index_cn.md.txt b/doc_cn/_sources/tutorials/index_cn.md.txt
deleted file mode 100644
index 6a27004d58d24cc466d930322be8cdbb2f434c74..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/tutorials/index_cn.md.txt
+++ /dev/null
@@ -1,13 +0,0 @@
-# 完整教程
-
-* [快速入门](quick_start/index_cn.rst)
-* [个性化推荐](rec/ml_regression_cn.rst)
-* [图像分类](image_classification/index_cn.md)
-* [情感分析](sentiment_analysis/index_cn.md)
-* [语义角色标注](semantic_role_labeling/index_cn.md)
-* [机器翻译](text_generation/index_cn.md)
-
-## 常用模型
-
-* [ResNet模型](imagenet_model/resnet_model_cn.md)
-* [词向量模型](embedding_model/index_cn.md)
diff --git a/doc_cn/_sources/tutorials/rec/ml_dataset_cn.md.txt b/doc_cn/_sources/tutorials/rec/ml_dataset_cn.md.txt
deleted file mode 100644
index 2207a776f0774e72aba15169e59258dd04583637..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/tutorials/rec/ml_dataset_cn.md.txt
+++ /dev/null
@@ -1,105 +0,0 @@
-```eval_rst
-.. _demo_ml_dataset:
-
-```
-
-# MovieLens数据集
-
-[MovieLens 数据集](http://grouplens.org/datasets/movielens/)由GroupLens Research实验室搜集整理。
-该数据集包含一些用户信息、电影信息以及电影评分\[1-5\]。根据数据量规模，该数据及有很多不同的版本。
-我们用[MovieLens 百万数据集](http://files.grouplens.org/datasets/movielens/ml-1m.zip)作为示例数据
-集，其中包含6,000位用户对4,000部电影的1,000,000条评价。该数据集于2003年2月发布。
-
-## 数据集特征
-
-在[ml-1m 数据集](http://files.grouplens.org/datasets/movielens/ml-1m.zip)中有许多的特征。在[ml-1m 数据集]
-(http://files.grouplens.org/datasets/movielens/ml-1m.zip)中的这些数据文件(含有".dat"的后缀)实际上是CSV文件，
-分隔符为"::"。以下我们翻译数据集网站中README文件的描述:
-
-### 评分文件描述(ratings.dat)
-
-
-所有的评分数据都包含在"ratings.dat"文件中，遵循如下的格式:
-
-用户ID::电影ID::评分::时间戳
-
-- 用户ID范围从1到6040
-- 电影ID范围从1到3952
-- 评分被调整为5星的规模(只允许整数的星级)
-- 时间戳表示为从1970-01-01(UTC)来的秒数，与time(2)的返回值一致
-- 每位用户至少有20条评分
-
-### 用户文件描述(users.dat)
-
-所有的用户信息都包含在"users.dat"文件中，遵循如下的格式:
-
-用户ID::性别::年龄::职业::邮编
-
-所有的人口统计学信息由用户自愿提供，没有进行正确性的检查。只有含有人
-口统计学信息的用户才被包含在数据集中。
-
-- 性别，用"M"表示男性，"F"表示女性
-- 年龄从下列列表范围中选取:
-
-	*   1:	"18岁以下"
-	*  18:	"18-24岁"
-	*  25:	"25-34岁"
-	*  35:	"35-44岁"
-	*  45:	"45-49岁"
-	*  50:	"50-55岁"
-	*  56:	"56+"
-
-- 职业从下面所列中选择:
-
-	*   0:  "其他"或不确定
-	*   1:  "学术/教育工作者"
-	*   2:  "艺术家"
-	*   3:  "文书工作/管理员"
-	*   4:  "大学生/研究生"
-	*   5:  "客户服务"
-	*   6:  "医生/医疗保健"
-	*   7:  "行政工作/管理人员"
-	*   8:  "农民"
-	*   9:  "操持家务者"
-	*  10:  "高中毕业生"
-	*  11:  "律师"
-	*  12:  "程序员"
-	*  13:  "退休人员"
-	*  14:  "销售/市场"
-	*  15:  "科学家"
-	*  16:  "自由职业者"
-	*  17:  "技术员/工程师"
-	*  18:  "推销员/手工艺者"
-	*  19:  "无业人士"
-	*  20:  "作家"
-
-### 电影文件描述(movies.dat)
-
-所有的电影信息都包含在"movies.dat"文件中，遵循如下的格式:
-
-电影ID::电影名称::电影类型
-
-- 电影名称（包括发行时间）与IMDB网站提供的一致
-- 电影类型如符合多种用管道符号|分割，选自下列类型:
-
-	*	动作片
-	*	冒险片
-	*	动画片
-	*	儿童片
-	*	喜剧片
-	*	犯罪片
-	*	纪录片
-	*	戏剧
-	*	奇幻片
-	*	黑色电影
-	*	恐怖片
-	*	音乐剧
-	*	悬疑片
-	*	浪漫片
-	*	科幻片
-	*	惊险电影
-	*	战争片
-	*	西部片
-
-- 由于意外的副本记录和测试记录，有些电影ID可能与实际电影不相符合
-- 电影大部分是手工输入数据，因此可能会有一些错误和不一致发生
diff --git a/doc_cn/_sources/tutorials/rec/ml_regression_cn.rst.txt b/doc_cn/_sources/tutorials/rec/ml_regression_cn.rst.txt
deleted file mode 100644
index 9278c9f603b648099f448963bc2246b8dc014ab7..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/tutorials/rec/ml_regression_cn.rst.txt
+++ /dev/null
@@ -1,349 +0,0 @@
-MovieLens数据集评分回归模型
-===========================
-
-这里我们在MovieLens数据集描述一种 **余弦相似度回归** 任务。
-该示例将展示paddle如何进行词向量嵌入，处理相似度回归，针对文本
-的单词级别的卷积神经网络，以及paddle如何处理多种类型的输入。
-需要注意的是，该模型网络只是用于进行demo展示paddle如何工作，而
-没有进行结构的微调。
-
-
-**我们非常欢迎您用PADDLEPADDLE构建更好的示例，如果您有好的建议来
-让这个示例变得更好，希望能让我们知晓。**
-
-数据准备
-`````````
-下载并解压数据集
-'''''''''''''''''
-这里我们使用 :ref:`demo_ml_dataset` 。
-要下载和解压数据集，只需要简单的运行下面的命令即可。
-
-.. code-block:: bash
-
-	cd demo/recommendation/data
-	./ml_data.sh
-
-:code:`demo/recommendation/data/ml-1m` 的目录结构为:
-
-.. code-block:: text
-
-	+--ml-1m
-		+--- movies.dat 	# 电影特征
-		+--- ratings.dat 	# 评分
-		+--- users.dat 		# 用户特征
-		+--- README 		# 数据集描述
-
-字段配置文件
-'''''''''''''
-**字段配置文件** 用来具体说明数据集的字段和文件格式，
-例如，说明每个特征文件具体字段是 **什么** 类型。
-
-ml-1m的字段配置文件在目录 :code:`demo/recommendation/data/config.json` 中。
-其具体说明了字段类型和文件名称:
-
-1) 用户文件中有四种类型的字段\: 编号，性别，年龄和职业；
-
-2) 文件名称为"users.dat"，文件的分隔符为"::"。
-
-.. include:: ../../../demo/recommendation/data/config.json
-   :code: json
-   :literal:
-
-准备数据
-`````````
-你需要安装python的第三方库。
-**强烈推荐使用VIRTUALENV来创造一个干净的python环境。**
-
-.. code-block:: bash
-
-	pip install -r requirements.txt
-
-预处理数据一般的命令为:
-
-.. code-block:: bash
-
-	cd demo/recommendation
-	./preprocess.sh
-
-下面介绍预处理过程具体的步骤。
-
-提取电影或用户的特征并生成python对象
-'''''''''''''''''''''''''''''''''''''
-
-在movielens 1m数据集中，电影和用户有许多的特征。
-评分文件的每一行仅仅提供电影或用户的编号来代表相应的电影或用户。
-我们首先处理电影或用户的特征文件，然后用pickle命令将特征( **Meta** )对象存储为文件。
-
-Meta配置文件
-.............
-
-**Meta配置文件** 用来具体描述 **如何** 解析数据集中的每一个字段。
-该文件可以从字段配置文件生成，或是手动编辑生成。文件的格式可以
-为json或yaml格式。解析器能通过文件的扩展名自动识别文件的格式。
-
-要将字段配置文件转化为meta配置文件，只需要运行：
-
-.. code-block:: bash
-
-	cd demo/recommendation/data
-	python config_generator.py config.json > meta_config.json
-
-生成的meta配置文件如下所示：
-
-.. include:: ../../../demo/recommendation/data/meta_config.json
-	:code: json
-	:literal:
-
-在meta文件中有两种特征\: 电影和用户。
-
-* 在电影文件movies.dat中
-	* 我们仅用"::"来分隔每一行
-	* pos 0 代表编号
-	* pos 1 特征：
-		* name是电影名
-		* 利用正则表达式来解析该特征
-		* 基于字母的词嵌入特征
-		* 是序列
-	* pos 2 特征：
-		* name是体裁
-		* type是one hot稠密向量
-		* dictionary由解析自动生成，每一个key由'|'分隔
-* 在用户文件users.dat中
-	* 我们仅用"::"来分隔每一行
-	* pos 0 代表编号
-	* pos 1 特征：
-		* name是性别
-		* 简单的基于字母的词嵌入
-	* pos 2 特征：
-		* name是年龄
-		* 是整个的词嵌入
-		* 嵌入编号会根据单词排序
-	* pos 3 特征：
-		* name是职业
-		* 简单的整个词嵌入
-
-
-Meta文件
-''''''''
-
-有了meta配置文件之后，我们可以生成 **Meta文件** ，该文件是python的pickle对象，
-存储着电影或用户信息。可以运行下面的命令来生成。
-
-.. code-block:: bash
-
-	python meta_generator.py ml-1m meta.bin --config=meta_config.json
-
-meta文件 :code:`meta.bin` 的结构如下：
-
-.. code-block:: text
-
-    +--+ movie
-    |      +--+ __meta__
-    |      |       +--+ raw_meta  # 每个特征的meta配置。列表
-    |      |       |       +
-    |      |       |       |     # 编号字段，我们用编号作为key 
-    |      |       |       +--+ {'count': 3883, 'max': 3952, 'is_key': True, 'type': 'id', 'min': 1}
-    |      |       |       |
-    |      |       |       |     # 电影名字段，嵌入特征字典
-    |      |       |       +--+ {'dict': [ ... ], 'type': 'embedding', 'name': 'title', 'seq': 'sequence'}
-    |      |       |       |
-    |      |       |       |     # 体裁字段，体裁字典
-    |      |       |       +--+ {'dict': [ ... ], 'type': 'one_hot_dense', 'name': 'genres'}
-    |      |       |
-    |      |       +--+ feature_map [1, 2] # a list for raw_meta index for feature field.
-    |      |                               # it means there are 2 features for each key.
-    |      |                               #    * 0 offset of feature is raw_meta[1], Title.
-    |      |                               #    * 1 offset of feature is raw_meta[2], Genres.
-    |      |
-    |      +--+ 1 # 电影1的特征
-    |      |    +
-    |      |    +---+ [[...], [...]] # title ids, genres dense vector
-    |      |
-    |      +--+ 2
-    |      |
-    |      +--+ ...
-    |
-    +--- user
-           +--+ __meta__
-           |       +
-           |       +--+ raw_meta
-           |       |       +
-           |       |       +--+ id field as user
-           |       |       |
-           |       |       +--+ {'dict': ['F', 'M'], 'type': 'embedding', 'name': 'gender', 'seq': 'no_sequence'}
-           |       |       |
-           |       |       +--+ {'dict': ['1', '18', '25', '35', '45', '50', '56'], 'type': 'embedding', 'name': 'age', 'seq': 'no_sequence'}
-           |       |       |
-           |       |       +--+ {'dict': [...], 'type': 'embedding', 'name': 'occupation', 'seq': 'no_sequence'}
-           |       |
-           |       +--+ feature_map [1, 2, 3]
-           |
-           +--+ 1 # 用户1的特征
-           |
-           +--+ 2
-           +--+ ...
-
-
-分割训练/测试文件
-''''''''''''''''''
-
-我们将 :code:`ml-1m/ratings.dat` 文件分割为训练和测试文件。分割文件的方法是：对于每位用户，我们将评分分成两部分。
-这样的话每位用户在测试文件中将与训练文件含有同样的信息。
-
-用 :code:`separate.py` 来分离训练和测试文件。
-
-.. code-block:: bash
-
-	python split.py ml-1m/ratings.dat --delimiter="::" --test_ratio=0.1
-
-这样就会生成两个文件：:code:`ml-1m/ratings.dat.train` 和 :code:`ml-1m/ratings.data.test` 。
-将他们移动到目录 :code:`data` ，然后进行随机打乱，再为paddle的训练过程提供文件列表。
-
-..  code-block:: bash
-
-    shuf ml-1m/ratings.dat.train > ratings.dat.train
-    cp ml-1m/ratings.dat.test .
-    echo "./data/ratings.dat.train" > train.list
-    echo "./data/ratings.dat.test" > test.list
-
-
-神经网络结构配置
-`````````````````
-
-训练器配置文件
-'''''''''''''''
-
-网络结构如下图所示：
-
-..  image:: rec_regression_network.png
-    :align: center
-    :alt: rec_regression_network
-
-该示例的神经网络配置文件 :code:`trainer_config.py` 如下所示：
-
-..  literalinclude:: ../../../demo/recommendation/trainer_config.py
-    :language: python
-    :lines: 15-
-
-在文件 :code:`trainer_config.py` 中，我们仅仅是将每个特征种类映射到一个特征向量中，以下
-展示了如何将每个特征映射到一个向量。
-
-* :code:`id` \: 仅仅是简单的嵌入，然后添加一个全连接层。
-* :code:`embedding` \:
-    - 如果是序列，则先做嵌入，然后再做一次文本卷积网络操作，
-      然后得到平均采样的结果。
-    - 如果不是序列，则先做嵌入，然后添加一个全连接层。
-* :code:`one_host_dense` \:
-    - 仅仅是两个全连接层。
-
-然后我们利用多输入的:code:`fc_layer` 全连接层将电影的每个特征结合成一个电影特征，
-并且对用户的特征做同样的操作，也得到一个用户特征。然后我们求这两个特征的余弦相似度。
-
-在这些网络中，我们用以下的一些:ref:`api_trainer_config` 中的接口。
-
-*  数据层， :ref:`api_trainer_config_helpers_layers_data_layer`
-*  全连接层， :ref:`api_trainer_config_helpers_layers_fc_layer`
-*  嵌入层， :ref:`api_trainer_config_helpers_layers_embedding_layer`
-*  文本投影层， :ref:`api_trainer_config_helpers_layers_context_projection`
-*  采样层， :ref:`api_trainer_config_helpers_layers_pooling_layer`
-*  余弦相似度层， :ref:`api_trainer_config_helpers_layers_cos_sim`
-*  文本卷积采样层， :ref:`api_trainer_config_helpers_network_text_conv_pool`
-*  声明Python数据源， :ref:`api_trainer_config_helpers_data_sources` 
-
-数据提供脚本
-'''''''''''''
-
-..  literalinclude:: ../../../demo/recommendation/dataprovider.py
-    :language: python
-    :lines: 15-
-
-数据提供脚本仅仅是读取meta.bin和评分文件，生成训练需要的样本。
-在脚本 :code:`dataprovider.py` 中，我们需要设置：
-
-* obj.slots\: 特征的类型和维度。
-* use_seq\: :code:`dataprovider.py` 中的数据是否为序列模式。
-* process\: 返回数据的每一条样本给 :code:`paddle` 。
-
-数据提供脚本的细节文档可以参考 :ref:`api_pydataprovider2` 。
-
-训练
-````
-
-准备好数据，配置了网络，编写好数据提供脚本后，现在我们可以开始paddle训练了。
-
-代码 :code:`run.sh` 如下：
-
-..  literalinclude:: ../../../demo/recommendation/run.sh
-    :language: bash
-    :lines: 16-
-
-该脚本仅仅是开始一个paddle训练过程，将日志写入文件 :code:`log.txt` ，然后
-打印在屏幕上。
-
-脚本 :code:`run.sh` 中的每一行命令，请参考页面 :ref:`cmd_line_index` 。
-这些参数的简短介绍如下：
-
-*  config\: 告诉paddle哪个文件是神经网络的配置文件。
-*  save_dir\: 告诉paddle将模型保存在: code:`./output` 中。
-*  use_gpu\: 是否使用GPU，默认为不使用。
-*  trainer_count\: 一台机器上面的线程数量。
-*  test_all_data_in_one_period\: 每一个测试周期测试一次所有数据。否则，
-   每个测试周期测试: code:`batch_size` 批次的数据。
-*  log_period\: 在训练了: code:`log_period` 批次后打印日志。
-*  dot_period\: 在每训练: code:`dot_period` 个批次后打印一个 :code:`.` 。
-*  num_passes\: 训练至多: code:`num_passes` 轮。
-
-如果训练过程启动成功的话，输出应该类似如下：
-
-..  code-block:: text
-
-    I0601 08:07:22.832059 10549 TrainerInternal.cpp:157]  Batch=100 samples=160000 AvgCost=4.13494 CurrentCost=4.13494 Eval:  CurrentEval:
-
-    I0601 08:07:50.672627 10549 TrainerInternal.cpp:157]  Batch=200 samples=320000 AvgCost=3.80957 CurrentCost=3.48421 Eval:  CurrentEval:
-
-    I0601 08:08:18.877369 10549 TrainerInternal.cpp:157]  Batch=300 samples=480000 AvgCost=3.68145 CurrentCost=3.42519 Eval:  CurrentEval:
-
-    I0601 08:08:46.863963 10549 TrainerInternal.cpp:157]  Batch=400 samples=640000 AvgCost=3.6007 CurrentCost=3.35847 Eval:  CurrentEval:
-
-    I0601 08:09:15.413025 10549 TrainerInternal.cpp:157]  Batch=500 samples=800000 AvgCost=3.54811 CurrentCost=3.33773 Eval:  CurrentEval:
-    I0601 08:09:36.058670 10549 TrainerInternal.cpp:181]  Pass=0 Batch=565 samples=902826 AvgCost=3.52368 Eval:
-    I0601 08:09:46.215489 10549 Tester.cpp:101]  Test samples=97383 cost=3.32155 Eval:
-    I0601 08:09:46.215966 10549 GradientMachine.cpp:132] Saving parameters to ./output/model/pass-00000
-    I0601 08:09:46.233397 10549 ParamUtil.cpp:99] save dir ./output/model/pass-00000
-    I0601 08:09:46.233438 10549 Util.cpp:209] copy trainer_config.py to ./output/model/pass-00000
-    I0601 08:09:46.233541 10549 ParamUtil.cpp:147] fileName trainer_config.py
-
-模型被保存在 :code:`output/` 目录中。你可以在任何时候用 :code:`Ctrl-C` 来停止训练。
-
-模型评估和预测
-```````````````
-
-在训练了几个轮次以后，你可以对模型进行评估，得到最好轮次下的模型。运行下面命令即可：
-
-.. code-block:: bash
-
-    ./evaluate.sh 
-
-你将看到如下的信息：
-
-.. code-block:: text
-
-    Best pass is 00009,  error is 3.06949, which means predict get error as 0.875998002281
-    evaluating from pass output/pass-00009
-
-然后，你可以预测任何用户对于任何一部电影的评价，运行下面命令即可：
-
-..  code-block:: bash
-
-    python prediction.py 'output/pass-00009/'
-
-预测程序将读取用户的输入，然后输出预测分数。用户预测的命令行界面如下：
-
-..  code-block:: text
-
-    Input movie_id: 9
-    Input user_id: 4
-    Prediction Score is 2.56
-    Input movie_id: 8
-    Input user_id: 2
-    Prediction Score is 3.13
diff --git a/doc_cn/_sources/tutorials/semantic_role_labeling/index_cn.md.txt b/doc_cn/_sources/tutorials/semantic_role_labeling/index_cn.md.txt
deleted file mode 100644
index f6061766c038a7bb6e4ae376685a10cd5669d2ed..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/tutorials/semantic_role_labeling/index_cn.md.txt
+++ /dev/null
@@ -1,201 +0,0 @@
-# 语义角色标注教程 #
-
-语义角色标注（Semantic role labeling, SRL）是浅层语义解析的一种形式，其目的是在给定的输入句子中发现每个谓词的谓词论元结构。 SRL作为很多自然语言处理任务中的中间步骤是很有用的，如信息提取、文档自动分类和问答。 实例如下 [1]:
-
- [ A0 He ] [ AM-MOD would ][ AM-NEG n’t ] [ V accept] [ A1 anything of value ] from [A2 those he was writing about ]. 
-
-- V: 动词
-- A0: 接受者
-- A1: 接受的东西
-- A2: 从……接受
-- A3: 属性
-- AM-MOD: 情态动词 
-- AM-NEG: 否定
-
-给定动词“accept”，句子中的组块将会扮演某些语义角色。这里，标签方案来自 Penn Proposition Bank。
-
-到目前为止，大多数成功的SRL系统是建立在某种形式的句法分析结果之上的，使用了基于句法结构的预定义特征模板。 本教程将介绍使用深度双向长短期记忆（DB-LSTM）模型[2]的端到端系统来解决SRL任务，这在很大程度上优于先前的最先进的系统。 这个系统将SRL任务视为序列标注问题。
-
-## 数据描述
-相关论文[2]采用 CoNLL-2005＆2012 共享任务中设置的数据进行训练和测试。由于数据许可的原因，演示采用 CoNLL-2005 的测试数据集，可以在网站上找到。
-
-用户只需执行以下命令就可以下载并处理原始数据：
-
-```bash
-cd data
-./get_data.sh
-```
-`data `目录会出现如下几个新的文件：
-```bash
-conll05st-release：the test data set of CoNll-2005 shared task 
-test.wsj.words：the Wall Street Journal data sentences
-test.wsj.props:  the propositional arguments
-feature: the extracted features from data set
-```
-
-## 训练
-### DB-LSTM
-请参阅情感分析的演示以了解有关长期短期记忆单元的更多信息。
-
-与在 Sentiment Analysis 演示中使用的 Bidirectional-LSTM 不同，DB-LSTM 采用另一种方法来堆叠LSTM层。首先，标准LSTM以正向处理该序列。该 LSTM 层的输入和输出作为下一个 LSTM 层的输入，并被反向处理。这两个标准 LSTM 层组成一对 LSTM。然后我们堆叠一对对的 LSTM 层后得到深度 LSTM 模型。
-
-下图展示了时间扩展的2层 DB-LSTM 网络。
-
    
-![pic](./network_arch.png)
-
    
-
-### 特征
-两个输入特征在这个流程中起着至关重要的作用：predicate（pred）和argument（arguments）。 还采用了两个其他特征：谓词上下文（ctx-p）和区域标记（mr）。 因为单个谓词不能精确地描述谓词信息，特别是当相同的词在句子中出现多于一次时。 使用谓词上下文，可以在很大程度上消除歧义。类似地，如果它位于谓词上下文区域中，则使用区域标记 mr = 1 来表示参数位置，反之则 mr = 0。这四个简单的特征是我们的SRL系统所需要的。上下文大小设置为1的一个样本的特征如下[2]所示：
-
    
-![pic](./feature.jpg)
-
    
-
-在这个示例中，相应的标记句子是：
-
-[ A1 A record date ] has [ AM-NEG n't ] been [ V set ] . 
-
-在演示中, 我们采用上面的特征模板, 包括：  `argument`, `predicate`, `ctx-p (p=-1,0,1)`, `mark` 并使用 `B/I/O` 方案来标记每个参数。这些特征和标签存储在 `feature` 文件中, 用`\t`分割。
-
-### 数据提供
-
-`dataprovider.py` 是一个包装数据的 Python 文件。 函数 `hook()` 定义了网络的数据槽。六个特征和标签都是索引槽。
-```
-def hook(settings, word_dict, label_dict, **kwargs):
-    settings.word_dict = word_dict
-    settings.label_dict = label_dict
-    #all inputs are integral and sequential type
-    settings.slots = [
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(predicate_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(2),
-        integer_value_sequence(len(label_dict))]
-```
-相应的数据迭代器如下：
-```
-@provider(init_hook=hook, should_shuffle=True, calc_batch_size=get_batch_size,
-          can_over_batch_size=False, cache=CacheType.CACHE_PASS_IN_MEM)
-def process(settings, file_name):
-    with open(file_name, 'r') as fdata:
-        for line in fdata:
-            sentence, predicate, ctx_n2, ctx_n1, ctx_0, ctx_p1, ctx_p2,  mark, label = \
-                line.strip().split('\t')
-
-            words = sentence.split()
-            sen_len = len(words)
-            word_slot = [settings.word_dict.get(w, UNK_IDX) for w in words]
-
-            predicate_slot = [settings.predicate_dict.get(predicate)] * sen_len
-            ctx_n2_slot = [settings.word_dict.get(ctx_n2, UNK_IDX)] * sen_len
-            ctx_n1_slot = [settings.word_dict.get(ctx_n1, UNK_IDX)] * sen_len
-            ctx_0_slot = [settings.word_dict.get(ctx_0, UNK_IDX)] * sen_len
-            ctx_p1_slot = [settings.word_dict.get(ctx_p1, UNK_IDX)] * sen_len
-            ctx_p2_slot = [settings.word_dict.get(ctx_p2, UNK_IDX)] * sen_len
-
-            marks = mark.split()
-            mark_slot = [int(w) for w in marks]
-
-            label_list = label.split()
-            label_slot = [settings.label_dict.get(w) for w in label_list]
-            yield word_slot, predicate_slot, ctx_n2_slot, ctx_n1_slot, \
-                  ctx_0_slot, ctx_p1_slot, ctx_p2_slot, mark_slot, label_slot
-```
-函数 `process` 返回8个特征list和1个标签list。
-
-### 神经网络配置
-
-`db_lstm.py` 是在训练过程中加载字典并定义数据提供程序模块和网络架构的神经网络配置文件。
-
-九个 `data_layer` 从数据提供程序加载实例。八个特征分别转换为向量，并由`mixed_layer`混合。 深度双向LSTM层提取softmax层的特征。目标函数是标签的交叉熵。
-
-### 训练 
-训练的脚本是 `train.sh`，用户只需执行:
-```bash
-  ./train.sh
-```
-`train.sh` 中的内容：
-```
-paddle train \
-  --config=./db_lstm.py \
-  --use_gpu=0 \
-  --log_period=5000 \
-  --trainer_count=1 \
-  --show_parameter_stats_period=5000 \
-  --save_dir=./output \
-  --num_passes=10000 \
-  --average_test_period=10000000 \
-  --init_model_path=./data \
-  --load_missing_parameter_strategy=rand \
-  --test_all_data_in_one_period=1 \
-2>&1 | tee 'train.log'
-```
-
--  \--config=./db_lstm.py : 网络配置文件
--  \--use_gpu=false: 使用 CPU 训练（如果已安装 PaddlePaddle GPU版本并想使用 GPU 训练可以设置为true，目前 crf_layer 不支持 GPU）
--  \--log_period=500: 每20个batch输出日志
--  \--trainer_count=1: 设置线程数（或 GPU 数）
--  \--show_parameter_stats_period=5000: 每100个batch显示参数统计
--  \--save_dir=./output: 模型输出路径
--  \--num_passes=10000: 设置数据遍历次数，一个pass意味着PaddlePaddle训练数据集中的所有样本被遍历一次
--  \--average_test_period=10000000:  每个 average_test_period 批次对平均参数进行测试
--  \--init_model_path=./data: 参数初始化路径
--  \--load_missing_parameter_strategy=rand: 随机初始不存在的参数
--  \--test_all_data_in_one_period=1: 在一个周期内测试所有数据
-
-
-训练后，模型将保存在目录`output`中。 我们的训练曲线如下：
-
    
-![pic](./src/curve.jpg)
-
    
-
-### 测试
-测试脚本是 `test.sh`, 执行:
-```bash
-  ./test.sh
-```
-`tesh.sh` 的主要部分：
-```
-paddle train \
-  --config=./db_lstm.py \
-  --model_list=$model_list \
-  --job=test \
-  --config_args=is_test=1 \
-```
-
-  - \--config=./db_lstm.py: 网络配置文件
-  - \--model_list=$model_list.list: 模型列表文件
-  - \--job=test: 指示测试任务
-  - \--config_args=is_test=1: 指示测试任务的标记
-  - \--test_all_data_in_one_period=1: 在一个周期内测试所有数据
-  
-
-### 预测
-预测脚本是 `predict.sh`，用户只需执行：
-```bash
-  ./predict.sh
-  
-```
-在`predict.sh`中，用户应该提供网络配置文件，模型路径，标签文件，字典文件，特征文件。
-```
-python predict.py 
-     -c $config_file \
-     -w $best_model_path \
-     -l $label_file \
-     -p $predicate_dict_file  \
-     -d $dict_file \
-     -i $input_file \
-     -o $output_file
-```
-
-`predict.py` 是主要的可执行python脚本，其中包括函数：加载模型，加载数据，数据预测。网络模型将输出标签的概率分布。 在演示中，我们使用最大概率的标签作为结果。用户还可以根据概率分布矩阵实现柱搜索或维特比解码。
-
-预测后，结果保存在 `predict.res` 中。
-
-## 引用
-[1] Martha Palmer, Dan Gildea, and Paul Kingsbury. The Proposition Bank: An Annotated Corpus of Semantic Roles , Computational Linguistics, 31(1), 2005. 
-
-[2] Zhou, Jie, and Wei Xu. "End-to-end learning of semantic role labeling using recurrent neural networks." Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.
diff --git a/doc_cn/_sources/tutorials/sentiment_analysis/index_cn.md.txt b/doc_cn/_sources/tutorials/sentiment_analysis/index_cn.md.txt
deleted file mode 100644
index 1323ec1a6abb2e7b5eeb2fbfff9cce5fe78a2c06..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/tutorials/sentiment_analysis/index_cn.md.txt
+++ /dev/null
@@ -1,325 +0,0 @@
-# 情感分析教程
-
-情感分析有许多应用场景。 一个基本的应用场景是区分给定文本的褒贬两极性，给定的文本可以是一个文档、句子、或者是一个小的文本片段。 一个简单的例子如：把用户在购物网站、旅游网站、团购网站（亚马逊、天猫、淘宝等）上发表的评论分成正面评论和负面评论两类。
-
-情感分析也常用于基于大量评论和个人博客来监控社会媒体。 例如，研究人员分析了几个关于消费者信心和政治观点的调查，结果发现它们与同时期的Twitter消息中的情绪词频率相关 [1]。 另一个例子是通过分析每日Twitter博客的文本内容来预测股票变动 [2]。
-
-另一方面，抓取产品的用户评论并分析他们的情感，有助于理解用户对不同公司，不同产品，甚至不同竞争对手产品的偏好。
-
-本教程将指导您完成长期短期记忆（LSTM）网络的训练过程，以分类来自[大型电影评论数据集](http://ai.stanford.edu/~amaas/data/sentiment/)（有时称为[互联网电影数据库 (IMDB)](http://ai.stanford.edu/~amaas/papers/wvSent_acl2011.pdf)）的句子的情感 。 此数据集包含电影评论及其相关联的类别标签，即正面和负面。
-
-## 数椐准备
-
-### IMDB 数椐介绍
-
-训练模型之前, 我们需要预处理数椐并构建一个字典。 首先, 你可以使用下面的脚本下载 IMDB 数椐集和[Moses](http://www.statmt.org/moses/)工具, 这是一个基于统计的机器翻译系统. 我们提供了一个数据预处理脚本，它不仅能够处理IMDB数据，还能处理其他用户自定义的数据。 为了使用提前编写的脚本，需要将标记的训练和测试样本移动到另一个路径，这已经在`get_imdb.sh`中完成。
-
-```
-cd demo/sentiment/data
-./get_imdb.sh
-```
-如果数椐获取成功，你将在目录```./demo/sentiment/data```中看到下面的文件：
-
-```
-aclImdb  get_imdb.sh  imdb  mosesdecoder-master
-```
-
-* aclImdb: 从外部网站上下载的原始数椐集。
-* imdb: 仅包含训练和测试数椐集。
-* mosesdecoder-master: Moses 工具。
-
-IMDB数据集包含25,000个已标注过的高极性电影评论用于训练，25,000个用于测试。负面的评论的得分小于等于4,正面的评论的得大于等于7，总评分10分。 运行完脚本 `./get_imdb.sh`后, 我们可以看到在目录 `aclImdb`中的数椐集的结构如下：
-
-```
-imdbEr.txt  imdb.vocab  README  test  train
-```
-* train: 训练数椐集。
-* test : 测试数椐集。
-* imdb.vocab: 字典文件。
-* imdbEr.txt: 字典imdb.vocab中每个切分单词的预期评级。
-* README: 数椐说明文档。
-
-测试集和训练集目录包含下面的文件:
-
-```
-labeledBow.feat  neg  pos  unsup  unsupBow.feat  urls_neg.txt  urls_pos.txt  urls_unsup.txt
-```
-
-* pos: 正面评价样本，包含12,500个txt文件，每个文件是一个电影评论。
-* neg: 负面评价样本，包含12,500个txt文件，每个文件是一个电影评论。
-* unsup: 未标记的评价样本，包含50,000个txt文件。
-* urls_xx.txt: 每个评论的网址。
-* xxBow.feat: 用于统计词频的Bow模型特征。
-
-### IMDB 数椐准备
-
-在这个例子中，我们只使用已经标注过的训练集和测试集，且默认在训练集上构建字典，而不使用IMDB数椐集中的imdb.vocab做为字典。训练集已经做了随机打乱排序而测试集没有。 Moses 工具中的脚本`tokenizer.perl` 用于切分单单词和标点符号。执行下面的命令就可以预处理数椐。
-
-```
-cd demo/sentiment/
-./preprocess.sh
-```
-preprocess.sh:
-
-```
-data_dir="./data/imdb"
-python preprocess.py -i data_dir
-```
-
-* data_dir: 输入数椐所在目录。
-* preprocess.py: 预处理脚本。
-
-运行成功后目录`demo/sentiment/data/pre-imdb` 结构如下:
-
-```
-dict.txt  labels.list  test.list  test_part_000  train.list  train_part_000
-```
-* test\_part\_000 and train\_part\_000: 所有标记的测试集和训练集， 训练集已经随机打乱。
-* train.list and test.list: 训练集和测试集文件列表。
-* dict.txt: 利用训练集生成的字典。
-* labels.txt: neg  0, pos 1, 含义：标签0表示负面的评论，标签1表示正面的评论。
-
-### 用户自定义数椐预处理
-
-如果你执行其它的用情感分析来分类文本的任务，可以按如下的结构来准备数椐. 我们提供了脚本来构建字典和预处理数椐。所以你只用按下面的结构来组织数椐就行了。
-
-```
-dataset
-|----train
-|    |----class1
-|    |    |----text_files
-|    |----class2
-|    |    |----text_files
-|    |    ...
-|----test
-|    |----class1
-|    |    |----text_files
-|    |----class2
-|    |    |----text_files
-|    |    ...
-```
-* dataset: 一级目录。
-* train, test: 二级目录。
-* class1,class2,...: 三级目录。
-* text_files: 文本格式的实例文件。
-
-所有同目录下的文本实例文件都是同级别的。 每个文本文件包含一个或者多个实例，每一行表示一个实例。 为了充分的随机打乱训练集, 在预处理含有多行数椐的文本文件时参数设置稍有不同, 执行`preprocess.sh`脚本时需要加上`-m True`参数。 tokenizer.perl 默认用来切分单记和标点符号，如果你不需要这个操作，在运行`preprocess.sh`时加上`-t False`参数即可。
-
-## 训练模型
-
-在这步任务中,我们使用了循环神经网络（RNN）的 LSTM 架构来训练情感分析模型。 引入LSTM模型主要是为了克服消失梯度的问题。 LSTM网络类似于具有隐藏层的标准循环神经网络, 但是隐藏层中的每个普通节点被一个记忆单元替换。 每个记忆单元包含四个主要的元素: 输入门, 具有自循环连接的神经元，忘记门和输出门。 更多的细节可以在文献中找到[4]。 LSTM架构的最大优点是它可以在长时间间隔内记忆信息，而没有短时记忆的损失。在有新的单词来临的每一个时间步骤内，存储在记忆单元区块的历史信息被更新用来迭代的学习单词以合理的序列程现。
-
-
    ![LSTM](src/lstm.png)
    
-
    图表 1. LSTM [3]
    
-
-情感分析是自然语言理解中最典型的问题之一。 它的目的是预测在一个序列中表达的情感态度。 通常, ，仅仅是一些关键词，如形容词和副词，在预测序列或段落的情感中起主要作用。然而有些评论上下文非常长，例如 IMDB的数椐集。 我们只所以使用LSTM来执行这个任务是因为其改进的设计并且具有门机制。 首先，它能够从词级到具有可变上下文长度的上下文级别来总结表示。 第二，它可以在句子级别利用可扩展的上下文, 而大多数方法只是利用n-gram级别的知识。第三，它直接学习段落表示，而不是组合上下文级别信息。
-
-在本演示中，我们提供两个网络，即双向LSTM和三层堆叠LSTM。
-
-#### 双向LSTM
-
-图2是双向LSTM网络，后面连全连接层和softmax层。
-
-
    ![BiLSTM](src/bi_lstm.jpg)
    
-
    图 2. Bidirectional-LSTM 
    
-
-#### Stacked-LSTM
-图3是三层LSTM结构。图的底部是word embedding(对文档处理后形成的单词向量)。 接下来，连接三个LSTM隐藏层，并且第二个是反向LSTM。然后提取隐藏LSTM层的所有时间步长的最大词向量作为整个序列的表示。 最后，使用具有softmax激活的全连接前馈层来执行分类任务。 更多内容可查看参考文献 [5]。
-
-
    ![StackedLSTM](src/stacked_lstm.jpg)
    
-
    图 3. Stacked-LSTM for sentiment analysis 
    
-
-**配置**
-
-进入`demo/sentiment` 目录 , `trainer_config.py` 是一个配置文件的例子, 其中包含算法和网络配置。第一行从`sentiment_net.py`中导出预定义的网络。
-
-trainer_config.py:
-
-```python
-from sentiment_net import *
-
-data_dir  = "./data/pre-imdb"
-# whether this config is used for test
-is_test = get_config_arg('is_test', bool, False)
-# whether this config is used for prediction
-is_predict = get_config_arg('is_predict', bool, False)
-dict_dim, class_dim = sentiment_data(data_dir, is_test, is_predict)
-
-################## Algorithm Config #####################
-
-settings(
-  batch_size=128,
-  learning_rate=2e-3,
-  learning_method=AdamOptimizer(),
-  regularization=L2Regularization(8e-4),
-  gradient_clipping_threshold=25
-)
-
-#################### Network Config ######################
-stacked_lstm_net(dict_dim, class_dim=class_dim,
-                 stacked_num=3, is_predict=is_predict)
-#bidirectional_lstm_net(dict_dim, class_dim=class_dim, is_predict=is_predict)
-```
-
-* **数椐定义**:
-   * get\_config\_arg(): 获取通过 `--config_args=xx` 设置的命令行参数。
-   * 定义训练数椐和测试数椐提供者, 这里使用了PaddlePaddle的Python接口来加载数椐。想了解更多细节可以参考PyDataProvider部分的文档
-
-* **算法配置**:
-   * 使用随机梯度下降（sgd）算法。
-   * 使用 adam 优化。
-   * 设置batch size大小为128。
-   * 设置平均sgd窗口。
-   * 设置全局学习率。
-* **网络配置**:
-   * dict_dim: 获取字典维度。
-   * class_dim: 设置类别数，IMDB有两个标签，即正面评价标签和负面评价标签。
-   * `stacked_lstm_net`: 预定义网络如图3所示，默认情况下使用此网络
-   * `bidirectional_lstm_net`: 预定义网络，如图2所示。
-
-**训练**
-
-首先安装PaddlePaddle。 然后使用下面的脚本 `train.sh` 来开启本地的训练。
-
-```
-cd demo/sentiment/
-./train.sh
-```
-
-train.sh:
-
-```
-config=trainer_config.py
-output=./model_output
-paddle train --config=$config \
-             --save_dir=$output \
-             --job=train \
-             --use_gpu=false \
-             --trainer_count=4 \
-             --num_passes=10 \
-             --log_period=20 \
-             --dot_period=20 \
-             --show_parameter_stats_period=100 \
-             --test_all_data_in_one_period=1 \
-             2>&1 | tee 'train.log'
-```
-
-* \--config=$config: 设置网络配置。
-* \--save\_dir=$output: 设置输出路径以保存训练完成的模型。
-* \--job=train: 设置工作模式为训练。
-* \--use\_gpu=false: 使用CPU训练，如果你安装GPU版本的PaddlePaddle，并想使用GPU来训练设置为true。
-* \--trainer\_count=4:设置线程数（或GPU个数）。
-* \--num\_passes=15: 设置pass，PaddlePaddle中的一个pass意味着对数据集中的所有样本进行一次训练。
-* \--log\_period=20: 每20个batch打印一次日志。
-* \--show\_parameter\_stats\_period=100: 每100个batch打印一次统计信息。
-* \--test\_all_data\_in\_one\_period=1: 每次测试都测试所有数据。
-
-如果运行成功，输出日志保存在路径 `demo/sentiment/train.log`中，模型保存在目录`demo/sentiment/model_output/`中。  输出日志说明如下：
-
-```
-Batch=20 samples=2560 AvgCost=0.681644 CurrentCost=0.681644 Eval: classification_error_evaluator=0.36875  CurrentEval: classification_error_evaluator=0.36875
-...
-Pass=0 Batch=196 samples=25000 AvgCost=0.418964 Eval: classification_error_evaluator=0.1922
-Test samples=24999 cost=0.39297 Eval: classification_error_evaluator=0.149406
-```
-- Batch=xx: 表示训练了xx个Batch。
-- samples=xx: 表示训练了xx个样本。。
-- AvgCost=xx: 从第0个batch到当前batch的平均损失。
-- CurrentCost=xx: 最新log_period个batch处理的当前损失。
-- Eval: classification\_error\_evaluator=xx: 表示第0个batch到当前batch的分类错误。
-- CurrentEval: classification\_error\_evaluator: 最新log_period个batch的分类错误。
-- Pass=0: 通过所有训练集一次称为一遍。 0表示第一次经过训练集。
-
-默认情况下，我们使用`stacked_lstm_net`网络，当传递相同的样本数时，它的收敛速度比`bidirectional_lstm_net`快。如果要使用双向LSTM，只需删除最后一行中的注释并把“stacked_lstm_net”注释掉。
-
-## 测试模型
-
-测试模型是指使用训练出的模型评估已标记的验证集。
-
-```
-cd demo/sentiment
-./test.sh
-```
-
-test.sh:
-
-```bash
-function get_best_pass() {
-  cat $1  | grep -Pzo 'Test .*\n.*pass-.*' | \
-  sed  -r 'N;s/Test.* error=([0-9]+\.[0-9]+).*\n.*pass-([0-9]+)/\1 \2/g' | \
-  sort | head -n 1
-}
-
-log=train.log
-LOG=`get_best_pass $log`
-LOG=(${LOG})
-evaluate_pass="model_output/pass-${LOG[1]}"
-
-echo 'evaluating from pass '$evaluate_pass
-
-model_list=./model.list
-touch $model_list | echo $evaluate_pass > $model_list
-net_conf=trainer_config.py
-paddle train --config=$net_conf \
-             --model_list=$model_list \
-             --job=test \
-             --use_gpu=false \
-             --trainer_count=4 \
-             --config_args=is_test=1 \
-             2>&1 | tee 'test.log'
-```
-
-函数`get_best_pass`依据分类错误率获得最佳模型进行测试。 在本示例中，我们默认使用IMDB的测试数据集作为验证。 与训练不同，它需要在这里指定`--job = test`和模型路径，即`--model_list = $model_list`。如果运行成功，日志将保存在“demo / sentiment / test.log”的路径中。例如，在我们的测试中，最好的模型是`model_output / pass-00002`，分类误差是0.115645，如下：
-
-```
-Pass=0 samples=24999 AvgCost=0.280471 Eval: classification_error_evaluator=0.115645
-```
-
-## 预测
-
-`predict.py`脚本提供了一个预测接口。在使用它之前请安装PaddlePaddle的python api。 预测IMDB的未标记评论的一个实例如下：
-
-```
-cd demo/sentiment
-./predict.sh
-```
-predict.sh:
-
-```
-#Note the default model is pass-00002, you shold make sure the model path
-#exists or change the mode path.
-model=model_output/pass-00002/
-config=trainer_config.py
-label=data/pre-imdb/labels.list
-cat ./data/aclImdb/test/pos/10007_10.txt | python predict.py \
-     --tconf=$config\
-     --model=$model \
-     --label=$label \
-     --dict=./data/pre-imdb/dict.txt \
-     --batch_size=1
-```
-
-* `cat ./data/aclImdb/test/pos/10007_10.txt` : 输入预测样本。
-* `predict.py` : 预测接口脚本。
-* `--tconf=$config` : 设置网络配置。
-* `--model=$model` : 设置模型路径。
-* `--label=$label` : 设置标签类别字典，这个字典是整数标签和字符串标签的一个对应。
-* `--dict=data/pre-imdb/dict.txt` : 设置字典文件。
-* `--batch_size=1` : 设置batch size。
-
-注意应该确保默认模型路径`model_output / pass-00002`存在或更改为其它模型路径。
-
-本示例的预测结果：
-
-```
-Loading parameters from model_output/pass-00002/
-./data/aclImdb/test/pos/10014_7.txt: predicting label is pos
-```
-我们真诚地感谢您的关注，并欢迎您来参与贡献。
-
-## 参考文档
-[1] Brendan O'Connor, Ramnath Balasubramanyan, Bryan R. Routledge, and Noah A. Smith. 2010. [From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series](http://homes.cs.washington.edu/~nasmith/papers/oconnor+balasubramanyan+routledge+smith.icwsm10.pdf). In ICWSM-2010. 
    
-[2] Johan Bollen, Huina Mao, Xiaojun Zeng. 2011. [Twitter mood predicts the stock market](http://arxiv.org/abs/1010.3003), Journal of Computational Science.
    
-[3] Alex Graves, Marcus Liwicki, Santiago Fernan- dez, Roman Bertolami, Horst Bunke, and Ju ̈rgen Schmidhuber. 2009. [A novel connectionist system for unconstrained handwriting recognition. IEEE Transactions on Pattern Analysis and Machine In- telligence](http://www.cs.toronto.edu/~graves/tpami_2009.pdf), 31(5):855–868.
    
-[4] Zachary C. Lipton, [A Critical Review of Recurrent Neural Networks for Sequence Learning](http://arxiv.org/abs/1506.00019v1), arXiv:1506.00019. 
    
-[5] Jie Zhou and Wei Xu; [End-to-end Learning of Semantic Role Labeling Using Recurrent Neural Networks](http://www.aclweb.org/anthology/P/P15/P15-1109.pdf); ACL-IJCNLP 2015. 
    
diff --git a/doc_cn/_sources/tutorials/text_generation/index_cn.md.txt b/doc_cn/_sources/tutorials/text_generation/index_cn.md.txt
deleted file mode 100644
index 41a87b926db399d692d677e5278e7d5a0b7b5594..0000000000000000000000000000000000000000
--- a/doc_cn/_sources/tutorials/text_generation/index_cn.md.txt
+++ /dev/null
@@ -1,339 +0,0 @@
-# 文本生成教程 #
-
-在语言生成领域中，“序列到序列”（sequence to sequence）的方法已被证明是一种强大的模型。它可以被应用于进行机器翻译（machine translation）、query改写（query rewriting）、图像描述（image captioning）等等。
-
-本篇教程将会指导你通过训练一个“序列到序列”的神经网络机器翻译（NMT）模型来将法语翻译成英语。
-
-我们遵循 [Neural Machine Translation by Jointly Learning to Align and Translate](http://arxiv.org/abs/1409.0473) 这篇文章，其中详细说明了模型架构，以及在WMT-14数据集上得到良好表现的训练过程。本篇教程在PaddlePaddle中重现了这一良好的训练结果。
-
-我们感谢@caoying的pull request，其中定义了模型架构和solver配置。
-
-## 数据准备 ##
-### 下载与解压缩 ###
-从该链接 [http://www-lium.univ-lemans.fr/~schwenk/cslm\_joint\_paper/](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/) 下载WMT-14数据集，然后解压，并将Develop和Test数据分别放入不同的文件夹。
-
-- **Train data**: [bitexts (选择过后的)](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/bitexts.tgz)
-- **Develop and Test data**: [dev 与 test 数据](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/dev+test.tgz)
-
-在Linux下，只需要简单地运行以下命令。否则你需要自己下载、解压、拆分到不同文件夹、并且分别重命名文件后缀。
-
-```bash
-cd demo/seqToseq/data
-./wmt14_data.sh
-```
-
-我们会发现数据集 `wmt14` 中包含如下表所示的3个文件夹。
-
------
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
    





    folder nameFrench-English parallel corpora filenumber of total filesize
    train_dataccb2_pc30.src, ccb2_pc30.trg, etc123.55G
    test_datantst1213.src, ntst1213.trg21636k
    gen_datantst14.src, ntst14.trg2864k
    
-
    
-
-- 每个文件夹都包含法语到英语的平行语料库
-- **XXX.src** 是原始法语文件；**XXX.trg** 是目标英语文件
-- **XXX.src** 和 **XXX.trg** 的行数应该一致
-- 每行都是一个法语或者英语的句子
-- **XXX.src** 和 **XXX.trg** 中任意第i行的句子之间都有着一一对应的关系
-
-### 用户自定义数据集 ###
-
-如果你想进行诸如语义转述（Paraphrasing）等其他“序列到序列”的任务，你只需要按照如下方式组织数据，并将它们放在`demo/seqToseq/data`目录下：
-
-    dataset
-      train
-        file1.src file1.trg
-        file2.src file2.trg
-        ......
-      test
-        file1.src file1.trg
-        file2.src file2.trg
-        ......
-      gen
-        file1.src file1.trg
-        file2.src file2.trg
-        ......
-  
-- 一级目录：数据集文件夹名称
-- 二级目录：train、test和gen这三个文件夹是固定的
-- 三级目录：源语言到目标语言的平行语料库文件
-  - **XXX.src** 是源语言的文件，**XXX.trg** 时目标语言的文件
-  - 文件中的每行都必须是一个句子
-  - **XXX.src** 和 **XXX.trg** 中任意第i行的句子之间都必须有着一一对应的关系
-
-## 数据预处理 ##
-### 预处理工作流程 ###
-- 将每个源语言到目标语言的平行语料库文件合并为一个文件：
-  - 合并每个 **XXX.src** 和 **XXX.trg** 文件为 **XXX**
-  - **XXX** 中的第i行 = **XXX.src** 中的第i行 + '\t' + **XXX.trg**中的第i行
-- 创建训练数据的“源字典”和“目标字典”，每个字典都有DICTSIZE个单词，包括：
-  - 词频最高的（DICTSIZE - 3）个单词
-  - 3个特殊符号
-  - ``：序列的开始
-  - ``：序列的结束
-  - ``：未包含在字典中的单词
-
-### 预处理命令和结果
-对数据集进行预处理的基本命令是：
-
-```python
-cd demo/seqToseq/
-python preprocess.py -i INPUT [-d DICTSIZE] [-m]
-```
-
-- `-i INPUT`：输入的原始数据集路径
-- `-d DICTSIZE`：指定的字典单词数，如果没有设置，字典会包含输入数据集中的所有单词
-- `-m --mergeDict`：合并 “源字典”和“目标字典”，使得两个字典有相同的上下文
-
-你将会看到如下消息：
-
-    concat parallel corpora for dataset
-    build source dictionary for train data
-    build target dictionary for train data
-    dictionary size is XXX
-
-然后你只需要运行以下命令：
-
-```python
-python preprocess.py -i data/wmt14 -d 30000
-```
-
-这将花费数分钟的时间，并且将预处理好的数据集存放在`demo/seqToseq/data/pre-wmt14`目录下。目录结构如下：
-
-    train test gen train.list test.list gen.list src.dict trg.dict# Text generation Tutorial #
-
-- **train, test, gen**：分别包含了法语到英语的平行语料库的训练数据、测试数据和生成数据。文件夹中的每个文件的每一行包含两部分，首先是法语序列，然后是对应的英语序列。
-- **train.list, test.list, gen.list**：分别为train，test，gen文件夹中的文件列表
-- **src.dict, trg.dict**：源（法语）/目标（英语）字典，每个字典包含总共30000个单词：29997个最高频单词和3个特殊符号
-
-## 模型训练 ##
-### 简介###
-
-神经网络机器翻译（NMT）旨在建立一个可以被协同调至最优翻译效果的单神经元网络。近期提出的NMT模型通常都属于编解码模型（encoder–decoder models）的一种。编解码模型将一个源语句编码为一个定长的向量，然后解码器通过这个向量生成一个目标语句。
-
-在这个任务中，我们使用了一个编解码模型的扩展，它同时学习排列(align)与翻译。每当模型在翻译过程中生成了一个单词，它就会在源语句中搜索出最相关信息的位置的集合。解码器根据上下文向量预测出一个目标单词，这个向量与源中搜索出的位置和所有之前生成的目标单词有关。如想了解更多详细的解释，可以参考 [Neural Machine Translation by Jointly Learning to Align and Translate](http://arxiv.org/abs/1409.0473)。
-
-这个模型对于编解码模型来说，最不同的特色是它并没有将输入语句编码为一个单独的定长向量。相反，它将输入语句编码为向量的序列，其中每个向量对应输入语句中的一个元素。然后在解码被翻译的语句时，会自适应地从这些向量中选择一个子集出来。这使得NMT模型得以解放出来，不必再将任意长度源语句中的所有信息压缩至一个定长的向量中。该模型在长语句翻译的场景下效果提升更加明显，在任意长度语句翻译的场景下都可以观察到其效果的提升。
-
    ![](./encoder-decoder-attention-model.png)
    
-
    Figure 1. Encoder-Decoder-Attention-Model
    
-
-### 使用PaddlePaddle训练模型 ###
-我们在训练之前需要常见一个模型配置文件，这里是一个例子`demo/seqToseq/translation/train.conf`。前三行import了定义network，job_mode和attention_mode的python函数。
-
-```python
-from seqToseq_net import *
-is_generating = False
-
-### Data Definiation
-train_conf = seq_to_seq_data(data_dir = "./data/pre-wmt14",
-                             is_generating = is_generating)
-
-### Algorithm Configuration
-settings(
-    learning_method = AdamOptimizer(),
-    batch_size = 50,
-    learning_rate = 5e-4)
-
-### Network Architecture
-gru_encoder_decoder(train_conf, is_generating)
-```
-
-1. **Data Definiation**：在示例中我们定义了一个序列到序列的训练和测试数据。它返回train_conf作为配置，其输入参数如下：
-  - data_dir：训练数据和测试数据的目录
-  - is_generating：这个配置是否用来生成，这里设置为False
-2. **Algorithm Configuration**：在示例中我们使用SGD训练算法（默认），和ADAM学习方法，指定batch_size为50，learning_rate为5e-4
-3. **Network Architecture**：在示例中我们使用attention版本的GRU编解码网络。它包括了一个双向的GRU作为编码器和解码器，它模拟了解码翻译过程中在源语句中的搜索。
-
-### 训练模型的命令与结果###
-写完模型配置之后，我们可以通过以下命令来训练模型：
-
-```bash
-cd demo/seqToseq/translation
-./train.sh
-```
-
-`train.sh` 的内容如下所示：
-
-```bash
-paddle train \
---config='translation/train.conf' \
---save_dir='translation/model' \
---use_gpu=false \
---num_passes=16 \
---show_parameter_stats_period=100 \
---trainer_count=4 \
---log_period=10 \
---dot_period=5 \
-2>&1 | tee 'translation/train.log'
-```
-- config: 设置神经网络的配置文件
-- save_dir: 设置保存模型的输出路径
-- use_gpu: 是否使用GPU训练，这里设置为使用CPU
-- num_passes: 设置passes的数量。paddle中的一条pass表示训练数据集中所有的样本一次
-- show_parameter_stats_period: 这里每隔100个batch显示一次参数统计信息
-- trainer_count: 设置CPU线程数或者GPU设备数
-- log_period: 这里每隔10个batch打印一次日志
-- dot_period: 这里每个5个batch打印一个点"."
-
-训练的损失函数默认每隔10个batch打印一次，你将会看到如下消息：
-
-    I0719 19:16:45.952062 15563 TrainerInternal.cpp:160]  Batch=10 samples=500 AvgCost=198.475 CurrentCost=198.475 Eval: classification_error_evaluator=0.737155  CurrentEval: classification_error_evaluator=0.737155
-    I0719 19:17:56.707319 15563 TrainerInternal.cpp:160]  Batch=20 samples=1000 AvgCost=157.479 CurrentCost=116.483 Eval: classification_error_evaluator=0.698392  CurrentEval: classification_error_evaluator=0.659065
-    .....
-- AvgCost：从第0个batch到当前batch的平均cost
-- CurrentCost:：当前batch的cost
-- classification\_error\_evaluator(Eval)：从第0个评估到当前评估中，每个单词的预测错误率
-- classification\_error\_evaluator(CurrentEval)：当前评估中，每个单词的预测错误率
-
-当classification\_error\_evaluator的值低于0.35时，模型就训练成功了。
-
-## 文本生成 ##
-### 简介###
-
-一般而言，NMT模型受制于源语句的编码，并且通过给出当前目标单词来预测下一个目标单词。在训练过程中，当前单词在相比之下总是被当作真值（ground truth）。在生成过程中，当前单词是解码器最后一步的输出，这来自于PaddlePaddle的内存中。
-
-而且，我们使用集束搜索（Beam Search）来生成序列。集束搜索使用广度优先搜索来构建搜索树。对于树的每一层，生成当前层的所有后继状态，并将它们按照启发代价（heuristic cost）升序排列。但是这种方法在每层只保存预设数量的最优状态（这个数量称为beam size）。
-
-### 预训练的模型 ###
-我们在拥有50个节点的集群中训练模型，每个节点有两个6核CPU。我们在5天里训练了16个pass，其中每条pass花费了7个小时。model_dir中有16个子目录，每个里面都包含202MB的全部的模型参数。然后我们发现pass-00012的模型有着最高的BLEU值27.77（参考文献[BLEU: a Method for Automatic Evaluation of Machine Translation](http://www.aclweb.org/anthology/P02-1040.pdf)）。要下载解压这个模型，只需在linux下运行如下命令：
-
-```bash
-cd demo/seqToseq/data
-./wmt14_model.sh
-```
-
-### 使用PaddlePaddle生成模型 ###
-在翻译法语句子之前，我们需要创建模型配置文件。这里是一个例子`demo/seqToseq/translation/gen.conf`。前三行import了定义network，job_mode和attention_mode的python函数。
-
-```python
-from seqToseq_net import *
-is_generating = True
-
-################## Data Definiation #####################
-gen_conf = seq_to_seq_data(data_dir = "./data/pre-wmt14",
-                           is_generating = is_generating,
-                           gen_result = "./translation/gen_result")
-
-############## Algorithm Configuration ##################
-settings(
-  learning_method = AdamOptimizer(),
-  batch_size = 1,
-  learning_rate = 0)
-
-################# Network configure #####################
-gru_encoder_decoder(gen_conf, is_generating)
-```
-
-1. **Data Definiation**：在示例中我们定义了一个序列到序列的生成数据。它返回gen_conf作为配置，其输入参数如下：
-  - data_dir：生成数据的目录
-  - is_generating：这个配置是否用来生成，这里设置为True
-  - gen_result：保存生成结果的文件
-2. **Algorithm Configuration**：在生成过程中我们使用SGD训练算法，并指定batch_size为1（每次生成1个序列），learning_rate为0
-3. **Network Architecture**：本质上与训练模型一样
-
-### 生成模型的命令与结果 ###
-写完模型配置之后，我们可以通过以下命令来进行从法语到英语的文本翻译：
-
-```bash
-cd demo/seqToseq/translation
-./gen.sh
-```
-
- `gen.sh` 的内容如下所示。与训练模型不同的是，这里有一些不同的参数需要指定：
-
-```bash
-paddle train \
---job=test \
---config='translation/gen.conf' \
---save_dir='data/wmt14_model' \
---use_gpu=true \
---num_passes=13 \
---test_pass=12 \
---trainer_count=1 \
-2>&1 | tee 'translation/gen.log'
-```
-- job：设置任务的模式为测试
-- save_dir：存储模型的路径
-- num_passes and test_pass：从test_pass到（num_passes - 1）加载模型参数，这里只加载 `data/wmt14_model/pass-00012`
-
-你将会看到这样的消息：
-
-    I0706 14:48:31.178915 31441 GradientMachine.cpp:143] Loading parameters from data/wmt14_model/pass-00012
-    I0706 14:48:40.012039 31441 Tester.cpp:125]  Batch=100 samples=100 AvgCost=0
-    I0706 14:48:48.898632 31441 Tester.cpp:125]  Batch=200 samples=200 AvgCost=0
-    ...
-
-然后在`demo/seqToseq/translation/gen_result`中的生成结果如下所示：
-
-    0
-    0       -11.1314         The   about the width of the seats while large controls are at stake 
-    1       -11.1519         The   on the width of the seats while large controls are at stake 
-    2       -11.5988         The   about the width of the seats while large controls are at stake . 
-
-    1
-    0       -24.4149         The dispute is between the major aircraft manufacturers about the width of the tourist seats on the  flights , paving the way for a  confrontation during the month of the Dubai  . 
-    1       -26.9524         The dispute is between the major aircraft manufacturers about the width of the tourist seats on the  flights , paving the way for a  confrontation during the month of Dubai ' s  . 
-    2       -27.9574         The dispute is between the major aircraft manufacturers about the width of the tourist seats on the  flights , paving the way for a  confrontation during the month of Dubai ' s Dubai  . 
-    ...
-
-- 这是集束搜索的结果，其中beam size是3
-- 第一行的“0”和第6行的“1”表示生成数据的序列id
-- 其他六行列出了集束搜索的结果
-  - 第二列是集束搜索的得分（从大到小）
-  - 第三列是生成的英语序列
-- 有两个特殊标识：
-  - ``：序列的结尾
-  - ``：不包含在字典中的单词
-
-### BLEU评估 ###
-对机器翻译的人工评估工作很广泛但也很昂贵。一篇论文 [BLEU: a Method for Automatic Evaluation of Machine Translation](http://www.aclweb.org/anthology/P02-1040.pdf) 展示了一种方法，当需要快速或者频繁的评估时，使用自动的替补来替代经验丰富的人工评判。[Moses](http://www.statmt.org/moses/) 是一个统计学的机器翻译系统，我们使用其中的 [multi-bleu.perl](https://github.com/moses-smt/mosesdecoder/blob/master/scripts/generic/multi-bleu.perl) 来做BLEU评估。运行以下命令来下载这个脚本：
-
-```bash
-cd demo/seqToseq/translation
-./moses_bleu.sh
-```
-
-由于标准的翻译结果已经下载到这里`data/wmt14/gen/ntst14.trg`，我们可以运行以下命令来做BLEU评估。
-
-```bash
-cd demo/seqToseq/translation
-./eval_bleu.sh FILE BEAMSIZE
-```
-
-- FILE：生成的结果文件
-- BEAMSIZE：集束搜索中的扩展广度
diff --git a/doc_cn/_sources/v1_api_tutorials/README.md.txt b/doc_cn/_sources/v1_api_tutorials/README.md.txt
new file mode 100644
index 0000000000000000000000000000000000000000..071b8da61fbcab3e88819273008b4526546202ad
--- /dev/null
+++ b/doc_cn/_sources/v1_api_tutorials/README.md.txt
@@ -0,0 +1,5 @@
+The tutorials in v1_api_tutorials are using v1_api currently, and will be upgraded to v2_api later.
+Thus, v1_api_tutorials is a temporary directory. We decide not to maintain it and will delete it in future.
+
+Please go to [PaddlePaddle/book](https://github.com/PaddlePaddle/book) and 
+[PaddlePaddle/models](https://github.com/PaddlePaddle/models) to learn PaddlePaddle.
diff --git a/doc_cn/_sources/tutorials/embedding_model/index_cn.md.txt b/doc_cn/_sources/v1_api_tutorials/embedding_model/index_cn.md.txt
similarity index 100%
rename from doc_cn/_sources/tutorials/embedding_model/index_cn.md.txt
rename to doc_cn/_sources/v1_api_tutorials/embedding_model/index_cn.md.txt
diff --git a/doc_cn/_sources/tutorials/imagenet_model/resnet_model_cn.md.txt b/doc_cn/_sources/v1_api_tutorials/imagenet_model/resnet_model_cn.md.txt
similarity index 100%
rename from doc_cn/_sources/tutorials/imagenet_model/resnet_model_cn.md.txt
rename to doc_cn/_sources/v1_api_tutorials/imagenet_model/resnet_model_cn.md.txt
diff --git a/doc_cn/_sources/tutorials/quick_start/index_cn.rst.txt b/doc_cn/_sources/v1_api_tutorials/quick_start/index_cn.rst.txt
similarity index 100%
rename from doc_cn/_sources/tutorials/quick_start/index_cn.rst.txt
rename to doc_cn/_sources/v1_api_tutorials/quick_start/index_cn.rst.txt
diff --git a/doc_cn/api/index_cn.html b/doc_cn/api/index_cn.html
index dc0562128d8fe5981d77937ccc2d598f69c40e9b..46915862017f3cf88804e5552b3015801244b787 100644
--- a/doc_cn/api/index_cn.html
+++ b/doc_cn/api/index_cn.html
@@ -64,7 +64,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -87,6 +87,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -110,9 +111,10 @@
           
 
         
diff --git a/doc_cn/api/v1/data_provider/dataprovider_cn.html b/doc_cn/api/v1/data_provider/dataprovider_cn.html
index 2dfe40a32bbcd15e37a2ba7a925cc187b2d8517b..2bcb6788e21d78faf7a3273473f610d066df3c8b 100644
--- a/doc_cn/api/v1/data_provider/dataprovider_cn.html
+++ b/doc_cn/api/v1/data_provider/dataprovider_cn.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -85,6 +85,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +109,10 @@
           
 
         
diff --git a/doc_cn/api/v1/data_provider/pydataprovider2_cn.html b/doc_cn/api/v1/data_provider/pydataprovider2_cn.html
index 21a73421c9fc0709a8fcb1084265e284c879f9f1..805b474f9b5f6b17e76da04375fa5654d06aa404 100644
--- a/doc_cn/api/v1/data_provider/pydataprovider2_cn.html
+++ b/doc_cn/api/v1/data_provider/pydataprovider2_cn.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -85,6 +85,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +109,10 @@
           
 
         
diff --git a/doc_cn/api/v1/index_cn.html b/doc_cn/api/v1/index_cn.html
index e57edf9bb2731afb400f49975fb5b73409933fdc..149a7eceddd046743d01a6284fb6982fc4755b08 100644
--- a/doc_cn/api/v1/index_cn.html
+++ b/doc_cn/api/v1/index_cn.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -85,6 +85,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +109,10 @@
           
 
         
@@ -196,16 +218,6 @@
 
 
    
diff --git a/doc_cn/api/v1/predict/swig_py_paddle_cn.html b/doc_cn/api/v1/predict/swig_py_paddle_cn.html
index caf7f28eaead7731cd956e030714a1df62318a74..242a456b3f7b4128d62a1ccaeb09b81e8c2bb331 100644
--- a/doc_cn/api/v1/predict/swig_py_paddle_cn.html
+++ b/doc_cn/api/v1/predict/swig_py_paddle_cn.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -85,6 +85,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +109,10 @@
           
 
         
diff --git a/doc_cn/api/v1/trainer_config_helpers/layers.html b/doc_cn/api/v1/trainer_config_helpers/layers.html
deleted file mode 100644
index 22c5c27ec55dfa373c5d28bbdf5dd3b31559d46d..0000000000000000000000000000000000000000
--- a/doc_cn/api/v1/trainer_config_helpers/layers.html
+++ /dev/null
@@ -1,3523 +0,0 @@
-
-
-
-
-  
-
-  
-  
-  
-  
-  Layers — PaddlePaddle  文档
-  
-
-  
-  
-
-  
-
-  
-  
-    
-
-  
-
-  
-  
-    
-  
-
-  
-  
-        
-        
-     
-
-  
-  
-  
-
-  
-
-  
-  
-
-
-
-
-
-  
-  
    
-    
-    
    
-      
    
-        Folk me on Github
-        
    
-          
-            English
-            
-            
-          
-          
-        
    
-        
-      
    
-      
    
-        
-        
-
-        
-
    
-  
    
-    
-    
-    
-  
    
-
            
-      
    
-    
    
-  
    
-  
-  
    
-
-    
-    
-    
-    
    
-
-      
-
- 
-
-
-
-
-
-
-
-
    
-  
      
-      
-    
    • Layers
      • 
-  
    
-
    
-      
-      
    
-        
    
-          
    
-           
    
-            
-  
    
-
    Layers
    
-
    
-
    Base
    
-
    
-
    LayerType
    
-
    
-
    
-class paddle.trainer_config_helpers.layers.LayerType
    
-
    Layer type enumerations.
    
-
    
-
    
-static is_layer_type(type_name)
    
-
    If type_name is a layer type.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:type_name (basestring) – layer type name. Because layer type enumerations are
-strings.
    返回:True if is a layer_type
    返回类型:bool
    
-
    
-
-
    
-
-
    
-
    
-
    LayerOutput
    
-
    
-
    
-class paddle.trainer_config_helpers.layers.LayerOutput(name, layer_type, parents=None, activation=None, num_filters=None, img_norm_type=None, size=None, outputs=None, reverse=None)
    
-
    LayerOutput is output for layer function. It is used internally by several
-reasons.
    
-
      
-
    • Check layer connection make sense.
      
-
      
-
        
-
      • FC(Softmax) => Cost(MSE Error) is not good for example.
        • 
-
      
-
      
-
      • 
-
    • Tracking layer connection.
      
-
      • 
-
    • Pass to layer methods as input.
      
-
      • 
-
    
-
---
-
-
-
-
    


    参数:
      
-
    • name (basestring) – Layer output name.
      • 
-
    • layer_type (basestring) – Current Layer Type. One of LayerType enumeration.
      • 
-
    • activation (BaseActivation.) – Layer Activation.
      • 
-
    • parents (list|tuple|collections.Sequence) – Layer’s parents.
      • 
-
    
-
    
-
    
-
    
-set_input(input)
    
-
    Set the input for a memory layer. Can only be used for memory layer
    
-
    
-
-
    
-
-
    
-
    
-
    
-
    Data layer
    
-
    
-
    data_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.data_layer(*args, **kwargs)
    
-
    Define DataLayer For NeuralNetwork.
    
-
    The example usage is:
    
-
    data = data_layer(name="input", size=1000)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – Name of this data layer.
      • 
-
    • size (int) – Size of this data layer.
      • 
-
    • height (int|None) – Height of this data layer, used for image
      • 
-
    • width (int|None) – Width of this data layer, used for image
      • 
-
    • layer_attr (ExtraLayerAttribute.) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Fully Connected Layers
    
-
    
-
    fc_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.fc_layer(*args, **kwargs)
    
-
    Helper for declare fully connected layer.
    
-
    The example usage is:
    
-
    fc = fc_layer(input=layer,
-              size=1024,
-              act=LinearActivation(),
-              bias_attr=False)
-
    
-
    
-
    which is equal to:
    
-
    with mixed_layer(size=1024) as fc:
-    fc += full_matrix_projection(input=layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – The Layer Name.
      • 
-
    • input (LayerOutput|list|tuple) – The input layer. Could be a list/tuple of input layer.
      • 
-
    • size (int) – The layer dimension.
      • 
-
    • act (BaseActivation) – Activation Type. Default is tanh.
      • 
-
    • param_attr (ParameterAttribute) – The Parameter Attribute|list.
      • 
-
    • bias_attr (ParameterAttribute|None|Any) – The Bias Attribute. If no bias, then pass False or
-something not type of ParameterAttribute. None will get a
-default Bias.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    selective_fc_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.selective_fc_layer(*args, **kwargs)
    
-
    Selectived fully connected layer. Different from fc_layer, the output
-of this layer maybe sparse. It requires an additional input to indicate
-several selected columns for output. If the selected columns is not
-specified, selective_fc_layer acts exactly like fc_layer.
    
-
    The simple usage is:
    
-
    sel_fc = selective_fc_layer(input=input, size=128, act=TanhActivation())
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – The Layer Name.
      • 
-
    • input (LayerOutput|list|tuple) – The input layer.
      • 
-
    • select (LayerOutput) – The select layer. The output of select layer should be a
-sparse binary matrix, and treat as the mask of selective fc.
-If is None, acts exactly like fc_layer.
      • 
-
    • size (int) – The layer dimension.
      • 
-
    • act (BaseActivation) – Activation Type. Default is tanh.
      • 
-
    • param_attr (ParameterAttribute) – The Parameter Attribute.
      • 
-
    • bias_attr (ParameterAttribute|None|Any) – The Bias Attribute. If no bias, then pass False or
-something not type of ParameterAttribute. None will get a
-default Bias.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Conv Layers
    
-
    
-
    conv_operator
    
-
    
-
    
-paddle.trainer_config_helpers.layers.conv_operator(img, filter, filter_size, num_filters, num_channels=None, stride=1, padding=0, filter_size_y=None, stride_y=None, padding_y=None, trans=False)
    
-
    Different from img_conv_layer, conv_op is an Operator, which can be used
-in mixed_layer. And conv_op takes two inputs to perform convolution.
-The first input is the image and the second is filter kernel. It only
-support GPU mode.
    
-
    The example usage is:
    
-
    op = conv_operator(img=input1,
-                   filter=input2,
-                   filter_size=3,
-                   num_filters=64,
-                   num_channels=64)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • img (LayerOutput) – input image
      • 
-
    • filter (LayerOutput) – input filter
      • 
-
    • filter_size (int) – The x dimension of a filter kernel.
      • 
-
    • filter_size_y (int) – The y dimension of a filter kernel. Since
-PaddlePaddle now supports rectangular filters,
-the filter’s shape can be (filter_size, filter_size_y).
      • 
-
    • num_filters (int) – channel of output data.
      • 
-
    • num_channels (int) – channel of input data.
      • 
-
    • stride (int) – The x dimension of the stride.
      • 
-
    • stride_y (int) – The y dimension of the stride.
      • 
-
    • padding (int) – The x dimension of padding.
      • 
-
    • padding_y (int) – The y dimension of padding.
      • 
-
    
-
    返回:
    A ConvOperator Object.
    
-
    返回类型:
    ConvOperator
    
-
    
-
    
-
-
    
-
    
-
    conv_projection
    
-
    
-
    
-paddle.trainer_config_helpers.layers.conv_projection(*args, **kwargs)
    
-
    Different from img_conv_layer and conv_op, conv_projection is an Projection,
-which can be used in mixed_layer and conat_layer. It use cudnn to implement
-conv and only support GPU mode.
    
-
    The example usage is:
    
-
    proj = conv_projection(input=input1,
-                       filter_size=3,
-                       num_filters=64,
-                       num_channels=64)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – input layer
      • 
-
    • filter_size (int) – The x dimension of a filter kernel.
      • 
-
    • filter_size_y (int) – The y dimension of a filter kernel. Since
-PaddlePaddle now supports rectangular filters,
-the filter’s shape can be (filter_size, filter_size_y).
      • 
-
    • num_filters (int) – channel of output data.
      • 
-
    • num_channels (int) – channel of input data.
      • 
-
    • stride (int) – The x dimension of the stride.
      • 
-
    • stride_y (int) – The y dimension of the stride.
      • 
-
    • padding (int) – The x dimension of padding.
      • 
-
    • padding_y (int) – The y dimension of padding.
      • 
-
    • groups (int) – The group number.
      • 
-
    • param_attr (ParameterAttribute) – Convolution param attribute. None means default attribute
      • 
-
    • trans (boolean) – whether it is convTrans or conv
      • 
-
    
-
    返回:
    A DotMulProjection Object.
    
-
    返回类型:
    DotMulProjection
    
-
    
-
    
-
-
    
-
    
-
    conv_shift_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.conv_shift_layer(*args, **kwargs)
    
-
    
-
    This layer performs cyclic convolution for two input. For example:
    
-
      
-
    • a[in]: contains M elements.
      • 
-
    • b[in]: contains N elements (N should be odd).
      • 
-
    • c[out]: contains M elements.
      • 
-
    
-
    
-
    
-
    
-\[c[i] = \sum_{j=-(N-1)/2}^{(N-1)/2}a_{i+j} * b_{j}\]
    
-
    
-
    In this formular:
    
-
      
-
    • a’s index is computed modulo M. When it is negative, then get item from
-the right side (which is the end of array) to the left.
      • 
-
    • b’s index is computed modulo N. When it is negative, then get item from
-the right size (which is the end of array) to the left.
      • 
-
    
-
    
-
    
-
    The example usage is:
    
-
    conv_shift = conv_shift_layer(a=layer1, b=layer2)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – layer name
      • 
-
    • a (LayerOutput) – Input layer a.
      • 
-
    • b (LayerOutput) – input layer b.
      • 
-
    • layer_attr (ExtraLayerAttribute) – layer’s extra attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    img_conv_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.img_conv_layer(*args, **kwargs)
    
-
    Convolution layer for image. Paddle can support both square and non-square
-input currently.
    
-
    The details of convolution layer, please refer UFLDL’s convolution .
    
-
    Convolution Transpose (deconv) layer for image. Paddle can support both square
-and non-square input currently.
    
-
    The details of convolution transpose layer,
-please refer to the following explanation and references therein
-<http://datascience.stackexchange.com/questions/6107/
-what-are-deconvolutional-layers/>`_ .
-The num_channel means input image’s channel number. It may be 1 or 3 when
-input is raw pixels of image(mono or RGB), or it may be the previous layer’s
-num_filters * num_group.
    
-
    There are several group of filter in PaddlePaddle implementation.
-Each group will process some channel of the inputs. For example, if an input
-num_channel = 256, group = 4, num_filter=32, the PaddlePaddle will create
-32*4 = 128 filters to process inputs. The channels will be split into 4
-pieces. First 256/4 = 64 channels will process by first 32 filters. The
-rest channels will be processed by rest group of filters.
    
-
    The example usage is:
    
-
    conv = img_conv_layer(input=data, filter_size=1, filter_size_y=1,
-                      num_channels=8,
-                      num_filters=16, stride=1,
-                      bias_attr=False,
-                      act=ReluActivation())
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – Layer name.
      • 
-
    • input (LayerOutput) – Layer Input.
      • 
-
    • filter_size (int|tuple|list) – The x dimension of a filter kernel. Or input a tuple for
-two image dimension.
      • 
-
    • filter_size_y (int|None) – The y dimension of a filter kernel. Since PaddlePaddle
-currently supports rectangular filters, the filter’s
-shape will be (filter_size, filter_size_y).
      • 
-
    • num_filters – Each filter group’s number of filter
      • 
-
    • act (BaseActivation) – Activation type. Default is tanh
      • 
-
    • groups (int) – Group size of filters.
      • 
-
    • stride (int|tuple|list) – The x dimension of the stride. Or input a tuple for two image
-dimension.
      • 
-
    • stride_y (int) – The y dimension of the stride.
      • 
-
    • padding (int|tuple|list) – The x dimension of the padding. Or input a tuple for two
-image dimension
      • 
-
    • padding_y (int) – The y dimension of the padding.
      • 
-
    • bias_attr (ParameterAttribute|False) – Convolution bias attribute. None means default bias.
-False means no bias.
      • 
-
    • num_channels (int) – number of input channels. If None will be set
-automatically from previous output.
      • 
-
    • param_attr (ParameterAttribute) – Convolution param attribute. None means default attribute
      • 
-
    • shared_biases (bool) – Is biases will be shared between filters or not.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Layer Extra Attribute.
      • 
-
    • trans (bool) – true if it is a convTransLayer, false if it is a convLayer
      • 
-
    • layer_type (String) – specify the layer_type, default is None. If trans=True,
-layer_type has to be “exconvt” or “cudnn_convt”,
-otherwise layer_type has to be either “exconv” or
-“cudnn_conv”
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    context_projection
    
-
    
-
    
-paddle.trainer_config_helpers.layers.context_projection(*args, **kwargs)
    
-
    Context Projection.
    
-
    It just simply reorganizes input sequence, combines “context_len” sequence
-to one context from context_start. “context_start” will be set to
--(context_len - 1) / 2 by default. If context position out of sequence
-length, padding will be filled as zero if padding_attr = False, otherwise
-it is trainable.
    
-
    For example, origin sequence is [A B C D E F G], context len is 3, then
-after context projection and not set padding_attr, sequence will
-be [ 0AB ABC BCD CDE DEF EFG FG0 ].
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input Sequence.
      • 
-
    • context_len (int) – context length.
      • 
-
    • context_start (int) – context start position. Default is
--(context_len - 1)/2
      • 
-
    • padding_attr (bool|ParameterAttribute) – Padding Parameter Attribute. If false, it means padding
-always be zero. Otherwise Padding is learnable, and
-parameter attribute is set by this parameter.
      • 
-
    
-
    返回:
    Projection
    
-
    返回类型:
    Projection
    
-
    
-
    
-
-
    
-
    
-
    
-
    Image Pooling Layer
    
-
    
-
    img_pool_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.img_pool_layer(*args, **kwargs)
    
-
    Image pooling Layer.
    
-
    The details of pooling layer, please refer ufldl’s pooling .
    
-
      
-
    • ceil_mode=True:
      • 
-
    
-
    
-\[w = 1 + int(ceil(input\_width + 2 * padding - pool\_size) / float(stride))
-h = 1 + int(ceil(input\_height + 2 * padding\_y - pool\_size\_y) / float(stride\_y))\]
    
-
      
-
    • ceil_mode=False:
      • 
-
    
-
    
-\[w = 1 + int(floor(input\_width + 2 * padding - pool\_size) / float(stride))
-h = 1 + int(floor(input\_height + 2 * padding\_y - pool\_size\_y) / float(stride\_y))\]
    
-
    The example usage is:
    
-
    maxpool = img_pool_layer(input=conv,
-                         pool_size=3,
-                         pool_size_y=5,
-                         num_channels=8,
-                         stride=1,
-                         stride_y=2,
-                         padding=1,
-                         padding_y=2,
-                         pool_type=MaxPooling())
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • padding (int) – pooling padding width.
      • 
-
    • padding_y (int|None) – pooling padding height. It’s equal to padding by default.
      • 
-
    • name (basestring.) – name of pooling layer
      • 
-
    • input (LayerOutput) – layer’s input
      • 
-
    • pool_size (int) – pooling window width
      • 
-
    • pool_size_y (int|None) – pooling window height. It’s eaqual to pool_size by default.
      • 
-
    • num_channels (int) – number of input channel.
      • 
-
    • pool_type (BasePoolingType) – pooling type. MaxPooling or AvgPooling. Default is
-MaxPooling.
      • 
-
    • stride (int) – stride width of pooling.
      • 
-
    • stride_y (int|None) – stride height of pooling. It is equal to stride by default.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer attribute.
      • 
-
    • ceil_mode (bool) – Wether to use ceil mode to calculate output height and with.
-Defalut is True. If set false, Otherwise use floor.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    spp_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.spp_layer(*args, **kwargs)
    
-
    Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition.
-The details please refer to
-Kaiming He’s paper.
    
-
    The example usage is:
    
-
    spp = spp_layer(input=data,
-                pyramid_height=2,
-                num_channels=16,
-                pool_type=MaxPooling())
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – layer name.
      • 
-
    • input (LayerOutput) – layer’s input.
      • 
-
    • num_channels (int) – number of input channel.
      • 
-
    • pool_type – Pooling type. MaxPooling or AveragePooling. Default is MaxPooling.
      • 
-
    • pyramid_height (int) – pyramid height.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    maxout_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.maxout_layer(*args, **kwargs)
    
-
    
-
    A layer to do max out on conv layer output.
    
-
      
-
    • Input: output of a conv layer.
      • 
-
    • Output: feature map size same as input. Channel is (input channel) / groups.
      • 
-
    
-
    
-
    
-
    So groups should be larger than 1, and the num of channels should be able
-to devided by groups.
    
-
    
-
    Please refer to Paper:
    
-
    
-
    
-
    
-
    The simple usage is:
    
-
    maxout = maxout_layer(input,
-                      num_channels=128,
-                      groups=4)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – The input layer.
      • 
-
    • num_channels (int|None) – The channel number of input layer. If None will be set
-automatically from previous output.
      • 
-
    • groups (int) – The group number of input layer.
      • 
-
    • name (None|basestring.) – The name of this layer, which can not specify.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Norm Layer
    
-
    
-
    img_cmrnorm_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.img_cmrnorm_layer(*args, **kwargs)
    
-
    Response normalization across feature maps.
-The details please refer to
-Alex’s paper.
    
-
    The example usage is:
    
-
    norm = img_cmrnorm_layer(input=net, size=5)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (None|basestring) – layer name.
      • 
-
    • input (LayerOutput) – layer’s input.
      • 
-
    • size (int) – Normalize in number of \(size\) feature maps.
      • 
-
    • scale (float) – The hyper-parameter.
      • 
-
    • power (float) – The hyper-parameter.
      • 
-
    • num_channels – input layer’s filers number or channels. If
-num_channels is None, it will be set automatically.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    batch_norm_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.batch_norm_layer(*args, **kwargs)
    
-
    Batch Normalization Layer. The notation of this layer as follow.
    
-
    \(x\) is the input features over a mini-batch.
    
-
    
-\[\begin{split}\mu_{\beta} &\gets \frac{1}{m} \sum_{i=1}^{m} x_i \qquad &//\
-\ mini-batch\ mean \\
-\sigma_{\beta}^{2} &\gets \frac{1}{m} \sum_{i=1}^{m}(x_i - \
-\mu_{\beta})^2 \qquad &//\ mini-batch\ variance \\
-\hat{x_i} &\gets \frac{x_i - \mu_\beta} {\sqrt{\
-\sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\
-y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift\end{split}\]
    
-
    The details of batch normalization please refer to this
-paper.
    
-
    The example usage is:
    
-
    norm = batch_norm_layer(input=net, act=ReluActivation())
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – layer name.
      • 
-
    • input (LayerOutput) – batch normalization input. Better be linear activation.
-Because there is an activation inside batch_normalization.
      • 
-
    • batch_norm_type (None|string, None or "batch_norm" or "cudnn_batch_norm") – We have batch_norm and cudnn_batch_norm. batch_norm
-supports both CPU and GPU. cudnn_batch_norm requires
-cuDNN version greater or equal to v4 (>=v4). But
-cudnn_batch_norm is faster and needs less memory
-than batch_norm. By default (None), we will
-automaticly select cudnn_batch_norm for GPU and
-batch_norm for CPU. Otherwise, select batch norm
-type based on the specified type. If you use cudnn_batch_norm,
-we suggested you use latest version, such as v5.1.
      • 
-
    • act (BaseActivation) – Activation Type. Better be relu. Because batch
-normalization will normalize input near zero.
      • 
-
    • num_channels (int) – num of image channels or previous layer’s number of
-filters. None will automatically get from layer’s
-input.
      • 
-
    • bias_attr (ParameterAttribute) – \(\beta\), better be zero when initialize. So the
-initial_std=0, initial_mean=1 is best practice.
      • 
-
    • param_attr (ParameterAttribute) – \(\gamma\), better be one when initialize. So the
-initial_std=0, initial_mean=1 is best practice.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    • use_global_stats (bool|None.) – whether use moving mean/variance statistics
-during testing peroid. If None or True,
-it will use moving mean/variance statistics during
-testing. If False, it will use the mean
-and variance of current batch of test data for
-testing.
      • 
-
    • moving_average_fraction (float.) – Factor used in the moving average
-computation, referred to as facotr,
-\(runningMean = newMean*(1-factor)
-+ runningMean*factor\)
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    sum_to_one_norm_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.sum_to_one_norm_layer(*args, **kwargs)
    
-
    A layer for sum-to-one normalization,
-which is used in NEURAL TURING MACHINE.
    
-
    
-\[out[i] = \frac {in[i]} {\sum_{k=1}^N in[k]}\]
    
-
    where \(in\) is a (batchSize x dataDim) input vector,
-and \(out\) is a (batchSize x dataDim) output vector.
    
-
    The example usage is:
    
-
    sum_to_one_norm = sum_to_one_norm_layer(input=layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input layer.
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Recurrent Layers
    
-
    
-
    recurrent_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.recurrent_layer(*args, **kwargs)
    
-
    Simple recurrent unit layer. It is just a fully connect layer through both
-time and neural network.
    
-
    For each sequence [start, end] it performs the following computation:
    
-
    
-\[\begin{split}out_{i} = act(in_{i})     \      \      \text{for} \ i = start \\
-out_{i} = act(in_{i} + out_{i-1} * W) \ \ \text{for} \ start < i <= end\end{split}\]
    
-
    If reversed is true, the order is reversed:
    
-
    
-\[\begin{split}out_{i} = act(in_{i})           \    \   \text{for} \ i = end  \\
-out_{i} = act(in_{i} + out_{i+1} * W) \ \ \text{for} \ start <= i < end\end{split}\]
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input Layer
      • 
-
    • act (BaseActivation) – activation.
      • 
-
    • bias_attr (ParameterAttribute) – bias attribute.
      • 
-
    • param_attr (ParameterAttribute) – parameter attribute.
      • 
-
    • name (basestring) – name of the layer
      • 
-
    • layer_attr (ExtraLayerAttribute) – Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    lstmemory
    
-
    
-
    
-paddle.trainer_config_helpers.layers.lstmemory(*args, **kwargs)
    
-
    Long Short-term Memory Cell.
    
-
    The memory cell was implemented as follow equations.
    
-
    
-\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    
-
    NOTE: In PaddlePaddle’s implementation, the multiplications
-\(W_{xi}x_{t}\) , \(W_{xf}x_{t}\),
-\(W_{xc}x_t\), \(W_{xo}x_{t}\) are not done in the lstmemory layer,
-so an additional mixed_layer with full_matrix_projection or a fc_layer must
-be included in the configuration file to complete the input-to-hidden
-mappings before lstmemory is called.
    
-
    NOTE: This is a low level user interface. You can use network.simple_lstm
-to config a simple plain lstm layer.
    
-
    Please refer to Generating Sequences With Recurrent Neural Networks for
-more details about LSTM.
    
-
    Link goes as below.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – The lstmemory layer name.
      • 
-
    • input (LayerOutput) – input layer name.
      • 
-
    • reverse (bool) – is sequence process reversed or not.
      • 
-
    • act (BaseActivation) – activation type, TanhActivation by default. \(h_t\)
      • 
-
    • gate_act (BaseActivation) – gate activation type, SigmoidActivation by default.
      • 
-
    • state_act (BaseActivation) – state activation type, TanhActivation by default.
      • 
-
    • bias_attr (ParameterAttribute|None|False) – Bias attribute. None means default bias. False means no
-bias.
      • 
-
    • param_attr (ParameterAttribute|None|False) – Parameter Attribute.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer attribute
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    grumemory
    
-
    
-
    
-paddle.trainer_config_helpers.layers.grumemory(*args, **kwargs)
    
-
    Gate Recurrent Unit Layer.
    
-
    The memory cell was implemented as follow equations.
    
-
    1. update gate \(z\): defines how much of the previous memory to
-keep around or the unit updates its activations. The update gate
-is computed by:
    
-
    
-\[z_t = \sigma(W_{z}x_{t} + U_{z}h_{t-1} + b_z)\]
    
-
    2. reset gate \(r\): determines how to combine the new input with the
-previous memory. The reset gate is computed similarly to the update gate:
    
-
    
-\[r_t = \sigma(W_{r}x_{t} + U_{r}h_{t-1} + b_r)\]
    
-
    3. The candidate activation \(\tilde{h_t}\) is computed similarly to
-that of the traditional recurrent unit:
    
-
    
-\[{\tilde{h_t}} = tanh(W x_{t} + U (r_{t} \odot h_{t-1}) + b)\]
    
-
    4. The hidden activation \(h_t\) of the GRU at time t is a linear
-interpolation between the previous activation \(h_{t-1}\) and the
-candidate activation \(\tilde{h_t}\):
    
-
    
-\[h_t = (1 - z_t) h_{t-1} + z_t {\tilde{h_t}}\]
    
-
    NOTE: In PaddlePaddle’s implementation, the multiplication operations
-\(W_{r}x_{t}\), \(W_{z}x_{t}\) and \(W x_t\) are not computed in
-gate_recurrent layer. Consequently, an additional mixed_layer with
-full_matrix_projection or a fc_layer must be included before grumemory
-is called.
    
-
    More details can be found by referring to Empirical Evaluation of Gated
-Recurrent Neural Networks on Sequence Modeling.
    
-
    The simple usage is:
    
-
    gru = grumemory(input)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (None|basestring) – The gru layer name.
      • 
-
    • input (LayerOutput.) – input layer.
      • 
-
    • reverse (bool) – Whether sequence process is reversed or not.
      • 
-
    • act (BaseActivation) – activation type, TanhActivation by default. This activation
-affects the \({\tilde{h_t}}\).
      • 
-
    • gate_act (BaseActivation) – gate activation type, SigmoidActivation by default.
-This activation affects the \(z_t\) and \(r_t\). It is the
-\(\sigma\) in the above formula.
      • 
-
    • bias_attr (ParameterAttribute|None|False) – Bias attribute. None means default bias. False means no
-bias.
      • 
-
    • param_attr (ParameterAttribute|None|False) – Parameter Attribute.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer attribute
      • 
-
    • size (None) – Stub parameter of size, but actually not used. If set this size
-will get a warning.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Recurrent Layer Group
    
-
    
-
    memory
    
-
    
-
    
-paddle.trainer_config_helpers.layers.memory(*args, **kwargs)
    
-
    The memory layers is a layer cross each time step. Reference this output
-as previous time step layer name ‘s output.
    
-
    The default memory is zero in first time step, previous time step’s
-output in the rest time steps.
    
-
    If boot_bias, the first time step value is this bias and
-with activation.
    
-
    If boot_with_const_id, then the first time stop is a IndexSlot, the
-Arguments.ids()[0] is this cost_id.
    
-
    If boot_layer is not null, the memory is just the boot_layer’s output.
-Set is_seq is true boot layer is sequence.
    
-
    The same name layer in recurrent group will set memory on each time
-step.
    
-
    mem = memory(size=256, name='state')
-state = fc_layer(input=mem, size=256, name='state')
-
    
-
    
-
    If you do not want to specify the name, you can equivalently use set_input()
-to specify the layer needs to be remembered as the following:
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – the name of the layer which this memory remembers.
-If name is None, user should call set_input() to specify the
-name of the layer which this memory remembers.
      • 
-
    • size (int) – size of memory.
      • 
-
    • memory_name (basestring) – the name of the memory.
-It is ignored when name is provided.
      • 
-
    • is_seq (bool) – is sequence for boot_layer
      • 
-
    • boot_layer (LayerOutput|None) – boot layer of memory.
      • 
-
    • boot_bias (ParameterAttribute|None) – boot layer’s bias
      • 
-
    • boot_bias_active_type (BaseActivation) – boot layer’s active type.
      • 
-
    • boot_with_const_id (int) – boot layer’s id.
      • 
-
    
-
    返回:
    LayerOutput object which is a memory.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    recurrent_group
    
-
    
-
    
-paddle.trainer_config_helpers.layers.recurrent_group(*args, **kwargs)
    
-
    Recurrent layer group is an extremely flexible recurrent unit in
-PaddlePaddle. As long as the user defines the calculation done within a
-time step, PaddlePaddle will iterate such a recurrent calculation over
-sequence input. This is extremely usefull for attention based model, or
-Neural Turning Machine like models.
    
-
    The basic usage (time steps) is:
    
-
    def step(input):
-    output = fc_layer(input=layer,
-                      size=1024,
-                      act=LinearActivation(),
-                      bias_attr=False)
-    return output
-
-group = recurrent_group(input=layer,
-                        step=step)
-
    
-
    
-
    You can see following configs for further usages:
    
-
      
-
    • time steps: lstmemory_group, paddle/gserver/tests/sequence_layer_group.conf,                   demo/seqToseq/seqToseq_net.py
      • 
-
    • sequence steps: paddle/gserver/tests/sequence_nest_layer_group.conf
      • 
-
    
-
---
-
-
-
-
    


    参数:
      
-
    • step (callable) – 
      recurrent one time step function.The input of this function is
-input of the group. The return of this function will be
-recurrent group’s return value.
      
-
      The recurrent group scatter a sequence into time steps. And
-for each time step, will invoke step function, and return
-a time step result. Then gather each time step of output into
-layer group’s output.
      
-
      • 
-
    • name (basestring) – recurrent_group’s name.
      • 
-
    • input (LayerOutput|StaticInput|SubsequenceInput|list|tuple) – 
      Input links array.
      
-
      LayerOutput will be scattered into time steps.
-SubsequenceInput will be scattered into sequence steps.
-StaticInput will be imported to each time step, and doesn’t change
-through time. It’s a mechanism to access layer outside step function.
      
-
      • 
-
    • reverse (bool) – If reverse is set true, the recurrent unit will process the
-input sequence in a reverse order.
      • 
-
    • targetInlink (LayerOutput|SubsequenceInput) – 
      the input layer which share info with layer group’s output
      
-
      Param input specifies multiple input layers. For
-SubsequenceInput inputs, config should assign one input
-layer that share info(the number of sentences and the number
-of words in each sentence) with all layer group’s outputs.
-targetInlink should be one of the layer group’s input.
      
-
      • 
-
    • is_generating – If is generating, none of input type should be LayerOutput;
-else, for training or testing, one of the input type must
-be LayerOutput.
      • 
-
    
-
    
-
    : type is_generating: bool
    
-
---
-
-
-
-
-
-
    


    返回:LayerOutput object.
    返回类型:LayerOutput
    
-
    
-
-
    
-
    
-
    lstm_step_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.lstm_step_layer(*args, **kwargs)
    
-
    LSTM Step Layer. It used in recurrent_group. The lstm equations are shown
-as follow.
    
-
    
-\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    
-
    The input of lstm step is \(Wx_t + Wh_{t-1}\), and user should use
-mixed_layer and full_matrix_projection to calculate these
-input vector.
    
-
    The state of lstm step is \(c_{t-1}\). And lstm step layer will do
    
-
    
-\[ \begin{align}\begin{aligned}i_t = \sigma(input + W_{ci}c_{t-1} + b_i)\\...\end{aligned}\end{align} \]
    
-
    This layer contains two outputs. Default output is \(h_t\). The other
-output is \(o_t\), which name is ‘state’ and can use
-get_output_layer to extract this output.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – Layer’s name.
      • 
-
    • size (int) – Layer’s size. NOTE: lstm layer’s size, should be equal as
-input.size/4, and should be equal as
-state.size.
      • 
-
    • input (LayerOutput) – input layer. \(Wx_t + Wh_{t-1}\)
      • 
-
    • state (LayerOutput) – State Layer. \(c_{t-1}\)
      • 
-
    • act (BaseActivation) – Activation type. Default is tanh
      • 
-
    • gate_act (BaseActivation) – Gate Activation Type. Default is sigmoid, and should
-be sigmoid only.
      • 
-
    • state_act (BaseActivation) – State Activation Type. Default is sigmoid, and should
-be sigmoid only.
      • 
-
    • bias_attr (ParameterAttribute) – Bias Attribute.
      • 
-
    • layer_attr (ExtraLayerAttribute) – layer’s extra attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    gru_step_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.gru_step_layer(*args, **kwargs)
    
-
    
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – 
      • 
-
    • output_mem – 
      • 
-
    • size – 
      • 
-
    • act – 
      • 
-
    • name – 
      • 
-
    • gate_act – 
      • 
-
    • bias_attr – 
      • 
-
    • param_attr – the parameter_attribute for transforming the output_mem
-from previous step.
      • 
-
    • layer_attr – 
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
-
    
-
    get_output_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.get_output_layer(*args, **kwargs)
    
-
    Get layer’s output by name. In PaddlePaddle, a layer might return multiple
-values, but returns one layer’s output. If the user wants to use another
-output besides the default one, please use get_output_layer first to get
-the output from input.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – Layer’s name.
      • 
-
    • input (LayerOutput) – get output layer’s input. And this layer should contains
-multiple outputs.
      • 
-
    • arg_name (basestring) – Output name from input.
      • 
-
    • layer_attr – Layer’s extra attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Mixed Layer
    
-
    
-
    mixed_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.mixed_layer(*args, **kwargs)
    
-
    Mixed Layer. A mixed layer will add all inputs together, then activate.
-Each inputs is a projection or operator.
    
-
    There are two styles of usages.
    
-
      
-
    1. When not set inputs parameter, use mixed_layer like this:
      2. 
-
    
-
    with mixed_layer(size=256) as m:
-    m += full_matrix_projection(input=layer1)
-    m += identity_projection(input=layer2)
-
    
-
    
-
      
-
    1. You can also set all inputs when invoke mixed_layer as follows:
      2. 
-
    
-
    m = mixed_layer(size=256,
-                input=[full_matrix_projection(input=layer1),
-                       full_matrix_projection(input=layer2)])
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – mixed layer name. Can be referenced by other layer.
      • 
-
    • size (int) – layer size.
      • 
-
    • input – inputs layer. It is an optional parameter. If set,
-then this function will just return layer’s name.
      • 
-
    • act (BaseActivation) – Activation Type.
      • 
-
    • bias_attr (ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or
-something not type of ParameterAttribute. None will get a
-default Bias.
      • 
-
    • layer_attr (ExtraLayerAttribute) – The extra layer config. Default is None.
      • 
-
    
-
    返回:
    MixedLayerType object can add inputs or layer name.
    
-
    返回类型:
    MixedLayerType
    
-
    
-
    
-
-
    
-
    
-
    embedding_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.embedding_layer(*args, **kwargs)
    
-
    Define a embedding Layer.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – Name of this embedding layer.
      • 
-
    • input (LayerOutput) – The input layer for this embedding. NOTE: must be Index Data.
      • 
-
    • size (int) – The embedding dimension.
      • 
-
    • param_attr (ParameterAttribute|None) – The embedding parameter attribute. See ParameterAttribute
-for details.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra layer Config. Default is None.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    scaling_projection
    
-
    
-
    
-paddle.trainer_config_helpers.layers.scaling_projection(*args, **kwargs)
    
-
    scaling_projection multiplies the input with a scalar parameter and add to
-the output.
    
-
    
-\[out += w * in\]
    
-
    The example usage is:
    
-
    proj = scaling_projection(input=layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input Layer.
      • 
-
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • 
-
    
-
    返回:
    A ScalingProjection object
    
-
    返回类型:
    ScalingProjection
    
-
    
-
    
-
-
    
-
    
-
    dotmul_projection
    
-
    
-
    
-paddle.trainer_config_helpers.layers.dotmul_projection(*args, **kwargs)
    
-
    DotMulProjection with a layer as input.
-It performs element-wise multiplication with weight.
    
-
    
-\[out.row[i] += in.row[i] .* weight\]
    
-
    where \(.*\) means element-wise multiplication.
    
-
    The example usage is:
    
-
    proj = dotmul_projection(input=layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input layer.
      • 
-
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • 
-
    
-
    返回:
    A DotMulProjection Object.
    
-
    返回类型:
    DotMulProjection
    
-
    
-
    
-
-
    
-
    
-
    dotmul_operator
    
-
    
-
    
-paddle.trainer_config_helpers.layers.dotmul_operator(a=None, b=None, scale=1, **kwargs)
    
-
    DotMulOperator takes two inputs and performs element-wise multiplication:
    
-
    
-\[out.row[i] += scale * (a.row[i] .* b.row[i])\]
    
-
    where \(.*\) means element-wise multiplication, and
-scale is a config scalar, its default value is one.
    
-
    The example usage is:
    
-
    op = dotmul_operator(a=layer1, b=layer2, scale=0.5)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • a (LayerOutput) – Input layer1
      • 
-
    • b (LayerOutput) – Input layer2
      • 
-
    • scale (float) – config scalar, default value is one.
      • 
-
    
-
    返回:
    A DotMulOperator Object.
    
-
    返回类型:
    DotMulOperator
    
-
    
-
    
-
-
    
-
    
-
    full_matrix_projection
    
-
    
-
    
-paddle.trainer_config_helpers.layers.full_matrix_projection(*args, **kwargs)
    
-
    Full Matrix Projection. It performs full matrix multiplication.
    
-
    
-\[out.row[i] += in.row[i] * weight\]
    
-
    There are two styles of usage.
    
-
      
-
    1. When used in mixed_layer like this, you can only set the input:
      2. 
-
    
-
    with mixed_layer(size=100) as m:
-    m += full_matrix_projection(input=layer)
-
    
-
    
-
      
-
    1. When used as an independant object like this, you must set the size:
      2. 
-
    
-
    proj = full_matrix_projection(input=layer,
-                              size=100,
-                              param_attr=ParamAttr(name='_proj'))
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – input layer
      • 
-
    • size (int) – The parameter size. Means the width of parameter.
      • 
-
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • 
-
    
-
    返回:
    A FullMatrixProjection Object.
    
-
    返回类型:
    FullMatrixProjection
    
-
    
-
    
-
-
    
-
    
-
    identity_projection
    
-
    
-
    
-paddle.trainer_config_helpers.layers.identity_projection(input, offset=None)
    
-
      
-
    1. IdentityProjection if offset=None. It performs:
      2. 
-
    
-
    
-\[out.row[i] += in.row[i]\]
    
-
    The example usage is:
    
-
    proj = identity_projection(input=layer)
-
    
-
    
-
    2. IdentityOffsetProjection if offset!=None. It likes IdentityProjection,
-but layer size may be smaller than input size.
-It select dimesions [offset, offset+layer_size) from input:
    
-
    
-\[out.row[i] += in.row[i + \textrm{offset}]\]
    
-
    The example usage is:
    
-
    proj = identity_projection(input=layer,
-                           offset=10)
-
    
-
    
-
    Note that both of two projections should not have any parameter.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input Layer.
      • 
-
    • offset (int) – Offset, None if use default.
      • 
-
    
-
    返回:
    A IdentityProjection or IdentityOffsetProjection object
    
-
    返回类型:
    IdentityProjection or IdentityOffsetProjection
    
-
    
-
    
-
-
    
-
    
-
    table_projection
    
-
    
-
    
-paddle.trainer_config_helpers.layers.table_projection(*args, **kwargs)
    
-
    Table Projection. It selects rows from parameter where row_id
-is in input_ids.
    
-
    
-\[out.row[i] += table.row[ids[i]]\]
    
-
    where \(out\) is output, \(table\) is parameter, \(ids\) is input_ids,
-and \(i\) is row_id.
    
-
    There are two styles of usage.
    
-
      
-
    1. When used in mixed_layer like this, you can only set the input:
      2. 
-
    
-
    with mixed_layer(size=100) as m:
-    m += table_projection(input=layer)
-
    
-
    
-
      
-
    1. When used as an independant object like this, you must set the size:
      2. 
-
    
-
    proj = table_projection(input=layer,
-                        size=100,
-                        param_attr=ParamAttr(name='_proj'))
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input layer, which must contains id fields.
      • 
-
    • size (int) – The parameter size. Means the width of parameter.
      • 
-
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • 
-
    
-
    返回:
    A TableProjection Object.
    
-
    返回类型:
    TableProjection
    
-
    
-
    
-
-
    
-
    
-
    trans_full_matrix_projection
    
-
    
-
    
-paddle.trainer_config_helpers.layers.trans_full_matrix_projection(*args, **kwargs)
    
-
    Different from full_matrix_projection, this projection performs matrix
-multiplication, using transpose of weight.
    
-
    
-\[out.row[i] += in.row[i] * w^\mathrm{T}\]
    
-
    \(w^\mathrm{T}\) means transpose of weight.
-The simply usage is:
    
-
    proj = trans_full_matrix_projection(input=layer,
-                                    size=100,
-                                    param_attr=ParamAttr(
-                                         name='_proj',
-                                         initial_mean=0.0,
-                                         initial_std=0.01))
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – input layer
      • 
-
    • size (int) – The parameter size. Means the width of parameter.
      • 
-
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • 
-
    
-
    返回:
    A TransposedFullMatrixProjection Object.
    
-
    返回类型:
    TransposedFullMatrixProjection
    
-
    
-
    
-
-
    
-
    
-
    
-
    Aggregate Layers
    
-
    
-
    pooling_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.pooling_layer(*args, **kwargs)
    
-
    Pooling layer for sequence inputs, not used for Image.
    
-
    The example usage is:
    
-
    seq_pool = pooling_layer(input=layer,
-                         pooling_type=AvgPooling(),
-                         agg_level=AggregateLevel.EACH_SEQUENCE)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • agg_level (AggregateLevel) – AggregateLevel.EACH_TIMESTEP or
-AggregateLevel.EACH_SEQUENCE
      • 
-
    • name (basestring) – layer name.
      • 
-
    • input (LayerOutput) – input layer name.
      • 
-
    • pooling_type (BasePoolingType|None) – Type of pooling, MaxPooling(default), AvgPooling,
-SumPooling, SquareRootNPooling.
      • 
-
    • bias_attr (ParameterAttribute|None|False) – Bias parameter attribute. False if no bias.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – The Extra Attributes for layer, such as dropout.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    last_seq
    
-
    
-
    
-paddle.trainer_config_helpers.layers.last_seq(*args, **kwargs)
    
-
    Get Last Timestamp Activation of a sequence.
    
-
    If stride > 0, this layer slides a window whose size is determined by stride,
-and return the last value of the window as the output. Thus, a long sequence
-will be shorten. Note that for sequence with sub-sequence, the default value
-of stride is -1.
    
-
    The simple usage is:
    
-
    seq = last_seq(input=layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • agg_level – Aggregated level
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • input (LayerOutput) – Input layer name.
      • 
-
    • stride (Int) – window size.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    first_seq
    
-
    
-
    
-paddle.trainer_config_helpers.layers.first_seq(*args, **kwargs)
    
-
    Get First Timestamp Activation of a sequence.
    
-
    If stride > 0, this layer slides a window whose size is determined by stride,
-and return the first value of the window as the output. Thus, a long sequence
-will be shorten. Note that for sequence with sub-sequence, the default value
-of stride is -1.
    
-
    The simple usage is:
    
-
    seq = first_seq(input=layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • agg_level – aggregation level
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • input (LayerOutput) – Input layer name.
      • 
-
    • stride (Int) – window size.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    concat_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.concat_layer(*args, **kwargs)
    
-
    Concat all input vector into one huge vector.
-Inputs can be list of LayerOutput or list of projection.
    
-
    The example usage is:
    
-
    concat = concat_layer(input=[layer1, layer2])
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – Layer name.
      • 
-
    • input (list|tuple|collections.Sequence) – input layers or projections
      • 
-
    • act (BaseActivation) – Activation type.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    seq_concat_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.seq_concat_layer(*args, **kwargs)
    
-
    Concat sequence a with sequence b.
    
-
    
-
    Inputs:
    
-
      
-
    • a = [a1, a2, ..., an]
      • 
-
    • b = [b1, b2, ..., bn]
      • 
-
    • Note that the length of a and b should be the same.
      • 
-
    
-
    
-
    
-
    Output: [a1, b1, a2, b2, ..., an, bn]
    
-
    The example usage is:
    
-
    concat = seq_concat_layer(a=layer1, b=layer2)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – Layer name.
      • 
-
    • a (LayerOutput) – input sequence layer
      • 
-
    • b (LayerOutput) – input sequence layer
      • 
-
    • act (BaseActivation) – Activation type.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    • bias_attr (ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or
-something not type of ParameterAttribute. None will get a
-default Bias.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Reshaping Layers
    
-
    
-
    block_expand_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.block_expand_layer(*args, **kwargs)
    
-
    
-
    Expand feature map to minibatch matrix.
    
-
      
-
    • matrix width is: block_y * block_x * num_channels
      • 
-
    • matirx height is: outputH * outputW
      • 
-
    
-
    
-
    
-
    
-\[ \begin{align}\begin{aligned}outputH = 1 + (2 * padding_y + imgSizeH - block_y + stride_y - 1) / stride_y\\outputW = 1 + (2 * padding_x + imgSizeW - block_x + stride_x - 1) / stride_x\end{aligned}\end{align} \]
    
-
    The expand method is the same with ExpandConvLayer, but saved the transposed
-value. After expanding, output.sequenceStartPositions will store timeline.
-The number of time steps are outputH * outputW and the dimension of each
-time step is block_y * block_x * num_channels. This layer can be used after
-convolution neural network, and before recurrent neural network.
    
-
    The simple usage is:
    
-
    block_expand = block_expand_layer(input=layer,
-                                  num_channels=128,
-                                  stride_x=1,
-                                  stride_y=1,
-                                  block_x=1,
-                                  block_x=3)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – The input layer.
      • 
-
    • num_channels (int|None) – The channel number of input layer.
      • 
-
    • block_x (int) – The width of sub block.
      • 
-
    • block_y (int) – The width of sub block.
      • 
-
    • stride_x (int) – The stride size in horizontal direction.
      • 
-
    • stride_y (int) – The stride size in vertical direction.
      • 
-
    • padding_x (int) – The padding size in horizontal direction.
      • 
-
    • padding_y (int) – The padding size in vertical direction.
      • 
-
    • name (None|basestring.) – The name of this layer, which can not specify.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    expand_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.expand_layer(*args, **kwargs)
    
-
    A layer for “Expand Dense data or (sequence data where the length of each
-sequence is one) to sequence data.”
    
-
    The example usage is:
    
-
    expand = expand_layer(input=layer1,
-                      expand_as=layer2,
-                      expand_level=ExpandLevel.FROM_TIMESTEP)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input layer
      • 
-
    • expand_as (LayerOutput) – Expand as this layer’s sequence info.
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • bias_attr (ParameterAttribute|None|False) – Bias attribute. None means default bias. False means no
-bias.
      • 
-
    • expand_level (ExpandLevel) – whether input layer is timestep(default) or sequence.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    repeat_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.repeat_layer(*args, **kwargs)
    
-
    A layer for repeating the input for num_repeats times. This is equivalent
-to apply concat_layer() with num_repeats same input.
    
-
    
-\[y  = [x, x, \cdots, x]\]
    
-
    The example usage is:
    
-
    expand = repeat_layer(input=layer, num_repeats=4)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input layer
      • 
-
    • num_repeats (int) – Repeat the input so many times
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    rotate_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.rotate_layer(*args, **kwargs)
    
-
    A layer for rotating 90 degrees (clock-wise) for each feature channel,
-usually used when the input sample is some image or feature map.
    
-
    
-\[y(j,i,:) = x(M-i-1,j,:)\]
    
-
    where \(x\) is (M x N x C) input, and \(y\) is (N x M x C) output.
    
-
    The example usage is:
    
-
    rot = rotate_layer(input=layer,
-                   height=100,
-                   width=100)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input layer.
      • 
-
    • height (int) – The height of the sample matrix
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    seq_reshape_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.seq_reshape_layer(*args, **kwargs)
    
-
    A layer for reshaping the sequence. Assume the input sequence has T instances,
-the dimension of each instance is M, and the input reshape_size is N, then the
-output sequence has T*M/N instances, the dimension of each instance is N.
    
-
    Note that T*M/N must be an integer.
    
-
    The example usage is:
    
-
    reshape = seq_reshape_layer(input=layer, reshape_size=4)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input layer.
      • 
-
    • reshape_size (int) – the size of reshaped sequence.
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • act (BaseActivation) – Activation type.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    • bias_attr (ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or
-something not type of ParameterAttribute. None will get a
-default Bias.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Math Layers
    
-
    
-
    addto_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.addto_layer(*args, **kwargs)
    
-
    AddtoLayer.
    
-
    
-\[y = f(\sum_{i} x_i + b)\]
    
-
    where \(y\) is output, \(x\) is input, \(b\) is bias,
-and \(f\) is activation function.
    
-
    The example usage is:
    
-
    addto = addto_layer(input=[layer1, layer2],
-                    act=ReluActivation(),
-                    bias_attr=False)
-
    
-
    
-
    This layer just simply add all input layers together, then activate the sum
-inputs. Each input of this layer should be the same size, which is also the
-output size of this layer.
    
-
    There is no weight matrix for each input, because it just a simple add
-operation. If you want a complicated operation before add, please use
-mixed_layer.
    
-
    It is a very good way to set dropout outside the layers. Since not all
-PaddlePaddle layer support dropout, you can add an add_to layer, set
-dropout here.
-Please refer to dropout_layer for details.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – Layer name.
      • 
-
    • input (LayerOutput|list|tuple) – Input layers. It could be a LayerOutput or list/tuple of
-LayerOutput.
      • 
-
    • act (BaseActivation) – Activation Type, default is tanh.
      • 
-
    • bias_attr (ParameterAttribute|bool) – Bias attribute. If False, means no bias. None is default
-bias.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    linear_comb_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.linear_comb_layer(*args, **kwargs)
    
-
    
-
    A layer for weighted sum of vectors takes two inputs.
    
-
      
-
    • 
-
      Input: size of weights is M
      
-
      size of vectors is M*N
      
-
      
-
      • 
-
    • Output: a vector of size=N
      • 
-
    
-
    
-
    
-
    
-\[z(i) = \sum_{j=0}^{M-1} x(j) y(i+Nj)\]
    
-
    where \(0 \le i \le N-1\)
    
-
    Or in the matrix notation:
    
-
    
-\[z = x^\mathrm{T} Y\]
    
-
    
-
    In this formular:
    
-
      
-
    • \(x\): weights
      • 
-
    • \(y\): vectors.
      • 
-
    • \(z\): the output.
      • 
-
    
-
    
-
    
-
    Note that the above computation is for one sample. Multiple samples are
-processed in one batch.
    
-
    The simple usage is:
    
-
    linear_comb = linear_comb_layer(weights=weight, vectors=vectors,
-                                size=elem_dim)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • weights (LayerOutput) – The weight layer.
      • 
-
    • vectors (LayerOutput) – The vector layer.
      • 
-
    • size (int) – the dimension of this layer.
      • 
-
    • name (basestring) – The Layer Name.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    interpolation_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.interpolation_layer(*args, **kwargs)
    
-
    This layer is for linear interpolation with two inputs,
-which is used in NEURAL TURING MACHINE.
    
-
    
-\[y.row[i] = w[i] * x_1.row[i] + (1 - w[i]) * x_2.row[i]\]
    
-
    where \(x_1\) and \(x_2\) are two (batchSize x dataDim) inputs,
-\(w\) is (batchSize x 1) weight vector, and \(y\) is
-(batchSize x dataDim) output.
    
-
    The example usage is:
    
-
    interpolation = interpolation_layer(input=[layer1, layer2], weight=layer3)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (list|tuple) – Input layer.
      • 
-
    • weight (LayerOutput) – Weight layer.
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    bilinear_interp_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.bilinear_interp_layer(*args, **kwargs)
    
-
    This layer is to implement bilinear interpolation on conv layer output.
    
-
    Please refer to Wikipedia: https://en.wikipedia.org/wiki/Bilinear_interpolation
    
-
    The simple usage is:
    
-
    bilinear = bilinear_interp_layer(input=layer1, out_size_x=64, out_size_y=64)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput.) – A input layer.
      • 
-
    • out_size_x (int|None) – bilinear interpolation output width.
      • 
-
    • out_size_y (int|None) – bilinear interpolation output height.
      • 
-
    • name (None|basestring) – The layer’s name, which cna not be specified.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    power_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.power_layer(*args, **kwargs)
    
-
    This layer applies a power function to a vector element-wise,
-which is used in NEURAL TURING MACHINE.
    
-
    
-\[y = x^w\]
    
-
    where \(x\) is a input vector, \(w\) is scalar weight,
-and \(y\) is a output vector.
    
-
    The example usage is:
    
-
    power = power_layer(input=layer1, weight=layer2)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input layer.
      • 
-
    • weight (LayerOutput) – Weight layer.
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    scaling_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.scaling_layer(*args, **kwargs)
    
-
    A layer for multiplying input vector by weight scalar.
    
-
    
-\[y  = w x\]
    
-
    where \(x\) is size=dataDim input, \(w\) is size=1 weight,
-and \(y\) is size=dataDim output.
    
-
    Note that the above computation is for one sample. Multiple samples are
-processed in one batch.
    
-
    The example usage is:
    
-
    scale = scaling_layer(input=layer1, weight=layer2)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input layer.
      • 
-
    • weight (LayerOutput) – Weight layer.
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    slope_intercept_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.slope_intercept_layer(*args, **kwargs)
    
-
    This layer for applying a slope and an intercept to the input
-element-wise. There is no activation and weight.
    
-
    
-\[y = slope * x + intercept\]
    
-
    The simple usage is:
    
-
    scale = slope_intercept_layer(input=input, slope=-1.0, intercept=1.0)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – The input layer.
      • 
-
    • name (basestring) – The Layer Name.
      • 
-
    • slope (float.) – the scale factor.
      • 
-
    • intercept (float.) – the offset.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    tensor_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.tensor_layer(*args, **kwargs)
    
-
    This layer performs tensor operation for two input.
-For example, each sample:
    
-
    
-\[y_{i} = a * W_{i} * {b^\mathrm{T}}, i=0,1,...,K-1\]
    
-
    
-
    In this formular:
    
-
      
-
    • \(a\): the first input contains M elements.
      • 
-
    • \(b\): the second input contains N elements.
      • 
-
    • \(y_{i}\): the i-th element of y.
      • 
-
    • \(W_{i}\): the i-th learned weight, shape if [M, N]
      • 
-
    • \(b^\mathrm{T}\): the transpose of \(b_{2}\).
      • 
-
    
-
    
-
    
-
    The simple usage is:
    
-
    tensor = tensor_layer(a=layer1, b=layer2, size=1000)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – layer name
      • 
-
    • a (LayerOutput) – Input layer a.
      • 
-
    • b (LayerOutput) – input layer b.
      • 
-
    • size (int.) – the layer dimension.
      • 
-
    • act (BaseActivation) – Activation Type. Default is tanh.
      • 
-
    • param_attr (ParameterAttribute) – The Parameter Attribute.
      • 
-
    • bias_attr (ParameterAttribute|None|Any) – The Bias Attribute. If no bias, then pass False or
-something not type of ParameterAttribute. None will get a
-default Bias.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    cos_sim
    
-
    
-
    
-paddle.trainer_config_helpers.layers.cos_sim(*args, **kwargs)
    
-
    Cosine Similarity Layer. The cosine similarity equation is here.
    
-
    
-\[similarity = cos(\theta) = {\mathbf{a} \cdot \mathbf{b}
-\over \|\mathbf{a}\| \|\mathbf{b}\|}\]
    
-
    The size of a is M, size of b is M*N,
-Similarity will be calculated N times by step M. The output size is
-N. The scale will be multiplied to similarity.
    
-
    Note that the above computation is for one sample. Multiple samples are
-processed in one batch.
    
-
    The example usage is:
    
-
    cos = cos_sim(a=layer1, b=layer2, size=3)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – layer name
      • 
-
    • a (LayerOutput) – input layer a
      • 
-
    • b (LayerOutput) – input layer b
      • 
-
    • scale (float) – scale for cosine value. default is 5.
      • 
-
    • size (int) – layer size. NOTE size_a * size should equal size_b.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    trans_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.trans_layer(*args, **kwargs)
    
-
    A layer for transposing a minibatch matrix.
    
-
    
-\[y = x^\mathrm{T}\]
    
-
    where \(x\) is (M x N) input, and \(y\) is (N x M) output.
    
-
    The example usage is:
    
-
    trans = trans_layer(input=layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input layer.
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Sampling Layers
    
-
    
-
    maxid_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.maxid_layer(*args, **kwargs)
    
-
    A layer for finding the id which has the maximal value for each sample.
-The result is stored in output.ids.
    
-
    The example usage is:
    
-
    maxid = maxid_layer(input=layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Input layer name.
      • 
-
    • name (basestring) – Layer name.
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    sampling_id_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.sampling_id_layer(*args, **kwargs)
    
-
    A layer for sampling id from multinomial distribution from the input layer.
-Sampling one id for one sample.
    
-
    The simple usage is:
    
-
    samping_id = sampling_id_layer(input=input)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – The input layer.
      • 
-
    • name (basestring) – The Layer Name.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Slicing and Joining Layers
    
-
    
-
    pad_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.pad_layer(*args, **kwargs)
    
-
    This operation pads zeros to the input data according to pad_c,pad_h
-and pad_w. pad_c, pad_h, pad_w specifies the which dimension and size
-of padding. And the input data shape is NCHW.
    
-
    For example, pad_c=[2,3] means padding 2 zeros before the
-input data and 3 zeros after the input data in channel dimension.
-pad_h means padding zeros in height dimension. pad_w means padding zeros
-in width dimension.
    
-
    For example,
    
-
    input(2,2,2,3)  = [
-                    [ [[1,2,3], [3,4,5]],
-                      [[2,3,5], [1,6,7]] ],
-                    [ [[4,3,1], [1,8,7]],
-                      [[3,8,9], [2,3,5]] ]
-                  ]
-
-pad_c=[1,1], pad_h=[0,0], pad_w=[0,0]
-
-output(2,4,2,3) = [
-                    [ [[0,0,0], [0,0,0]],
-                      [[1,2,3], [3,4,5]],
-                      [[2,3,5], [1,6,7]],
-                      [[0,0,0], [0,0,0]] ],
-                    [ [[0,0,0], [0,0,0]],
-                      [[4,3,1], [1,8,7]],
-                      [[3,8,9], [2,3,5]],
-                      [[0,0,0], [0,0,0]] ]
-                  ]
-
    
-
    
-
    The simply usage is:
    
-
    pad = pad_layer(input=ipt,
-                pad_c=[4,4],
-                pad_h=[0,0],
-                pad_w=[2,2])
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – layer’s input.
      • 
-
    • pad_c (list|None) – padding size in channel dimension.
      • 
-
    • pad_h (list|None) – padding size in height dimension.
      • 
-
    • pad_w (list|None) – padding size in width dimension.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    • name (basestring) – layer name.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Cost Layers
    
-
    
-
    cross_entropy
    
-
    
-
    
-paddle.trainer_config_helpers.layers.cross_entropy(*args, **kwargs)
    
-
    A loss layer for multi class entropy.
    
-
    cost = cross_entropy(input=input_layer,
-                     label=label_layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput.) – The first input layer.
      • 
-
    • label – The input label.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float.) – The cost is multiplied with coeff.
-The coefficient affects the gradient in the backward.
      • 
-
    • weight (LayerOutout) – The cost of each sample is multiplied with each weight.
-The weight should be a layer with size=1. Note that gradient
-will not be calculated for weight.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput.
    
-
    
-
    
-
-
    
-
    
-
    cross_entropy_with_selfnorm
    
-
    
-
    
-paddle.trainer_config_helpers.layers.cross_entropy_with_selfnorm(*args, **kwargs)
    
-
    A loss layer for multi class entropy with selfnorm.
-Input should be a vector of positive numbers, without normalization.
    
-
    cost = cross_entropy_with_selfnorm(input=input_layer,
-                                   label=label_layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput.) – The first input layer.
      • 
-
    • label – The input label.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float.) – The coefficient affects the gradient in the backward.
      • 
-
    • softmax_selfnorm_alpha (float.) – The scale factor affects the cost.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput.
    
-
    
-
    
-
-
    
-
    
-
    multi_binary_label_cross_entropy
    
-
    
-
    
-paddle.trainer_config_helpers.layers.multi_binary_label_cross_entropy(*args, **kwargs)
    
-
    A loss layer for multi binary label cross entropy.
    
-
    cost = multi_binary_label_cross_entropy(input=input_layer,
-                                        label=label_layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – The first input layer.
      • 
-
    • label – The input label.
      • 
-
    • type (basestring) – The type of cost.
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float) – The coefficient affects the gradient in the backward.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    mse_cost
    
-
    
-
    
-paddle.trainer_config_helpers.layers.mse_cost(*args, **kwargs)
    
-
    
-
    mean squared error cost:
    
-
    
-\[\]
    
-
    
-
    rac{1}{N}sum_{i=1}^N(t_i-y_i)^2
    
-
    
-
    
---
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
    


    param name:layer name.
    type name:basestring
    param input:Network prediction.
    type input:LayerOutput
    param label:Data label.
    type label:LayerOutput
    param weight:The weight affects the cost, namely the scale of cost.
-It is an optional argument.
    type weight:LayerOutput
    param layer_attr:
     layer’s extra attribute.
    type layer_attr:
     ExtraLayerAttribute
    return:LayerOutput object.
    rtype:LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    huber_cost
    
-
    
-
    
-paddle.trainer_config_helpers.layers.huber_cost(*args, **kwargs)
    
-
    A loss layer for huber loss.
    
-
    cost = huber_cost(input=input_layer,
-                  label=label_layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput.) – The first input layer.
      • 
-
    • label – The input label.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float.) – The coefficient affects the gradient in the backward.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput.
    
-
    
-
    
-
-
    
-
    
-
    lambda_cost
    
-
    
-
    
-paddle.trainer_config_helpers.layers.lambda_cost(*args, **kwargs)
    
-
    lambdaCost for lambdaRank LTR approach.
    
-
    The simple usage:
    
-
    cost = lambda_cost(input=input,
-                   score=score,
-                   NDCG_num=8,
-                   max_sort_size=-1)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – Samples of the same query should be loaded as sequence.
      • 
-
    • score – The 2nd input. Score of each sample.
      • 
-
    • NDCG_num (int) – The size of NDCG (Normalized Discounted Cumulative Gain),
-e.g., 5 for NDCG@5. It must be less than for equal to the
-minimum size of lists.
      • 
-
    • max_sort_size (int) – The size of partial sorting in calculating gradient.
-If max_sort_size = -1, then for each list, the
-algorithm will sort the entire list to get gradient.
-In other cases, max_sort_size must be greater than or
-equal to NDCG_num. And if max_sort_size is greater
-than the size of a list, the algorithm will sort the
-entire list of get gradient.
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    rank_cost
    
-
    
-
    
-paddle.trainer_config_helpers.layers.rank_cost(*args, **kwargs)
    
-
    A cost Layer for learning to rank using gradient descent. Details can refer
-to papers.
-This layer contains at least three inputs. The weight is an optional
-argument, which affects the cost.
    
-
    
-\[ \begin{align}\begin{aligned}C_{i,j} & = -\tilde{P_{ij}} * o_{i,j} + log(1 + e^{o_{i,j}})\\o_{i,j} & =  o_i - o_j\\\tilde{P_{i,j}} & = \{0, 0.5, 1\} \ or \ \{0, 1\}\end{aligned}\end{align} \]
    
-
    
-
    In this formula:
    
-
      
-
    • \(C_{i,j}\) is the cross entropy cost.
      • 
-
    • \(\tilde{P_{i,j}}\) is the label. 1 means positive order
-and 0 means reverse order.
      • 
-
    • \(o_i\) and \(o_j\): the left output and right output.
-Their dimension is one.
      • 
-
    
-
    
-
    
-
    The simple usage:
    
-
    cost = rank_cost(left=out_left,
-                 right=out_right,
-                 label=label)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • left (LayerOutput) – The first input, the size of this layer is 1.
      • 
-
    • right (LayerOutput) – The right input, the size of this layer is 1.
      • 
-
    • label (LayerOutput) – Label is 1 or 0, means positive order and reverse order.
      • 
-
    • weight (LayerOutput) – The weight affects the cost, namely the scale of cost.
-It is an optional argument.
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float) – The coefficient affects the gradient in the backward.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    sum_cost
    
-
    
-
    
-paddle.trainer_config_helpers.layers.sum_cost(*args, **kwargs)
    
-
    A loss layer which calculate the sum of the input as loss
    
-
    cost = sum_cost(input=input_layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput.) – The first input layer.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput.
    
-
    
-
    
-
-
    
-
    
-
    crf_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.crf_layer(*args, **kwargs)
    
-
    A layer for calculating the cost of sequential conditional random
-field model.
    
-
    The simple usage:
    
-
    crf = crf_layer(input=input,
-                label=label,
-                size=label_dim)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – The first input layer is the feature.
      • 
-
    • label (LayerOutput) – The second input layer is label.
      • 
-
    • size (int) – The category number.
      • 
-
    • weight (LayerOutput) – The third layer is “weight” of each sample, which is an
-optional argument.
      • 
-
    • param_attr (ParameterAttribute) – Parameter attribute. None means default attribute
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    crf_decoding_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.crf_decoding_layer(*args, **kwargs)
    
-
    A layer for calculating the decoding sequence of sequential conditional
-random field model. The decoding sequence is stored in output.ids.
-If a second input is provided, it is treated as the ground-truth label, and
-this layer will also calculate error. output.value[i] is 1 for incorrect
-decoding or 0 for correct decoding.
    
-
    The simple usage:
    
-
    crf_decoding = crf_decoding_layer(input=input,
-                                  size=label_dim)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – The first input layer.
      • 
-
    • size (int) – size of this layer.
      • 
-
    • label (LayerOutput or None) – None or ground-truth label.
      • 
-
    • param_attr (ParameterAttribute) – Parameter attribute. None means default attribute
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    ctc_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.ctc_layer(*args, **kwargs)
    
-
    Connectionist Temporal Classification (CTC) is designed for temporal
-classication task. That is, for sequence labeling problems where the
-alignment between the inputs and the target labels is unknown.
    
-
    More details can be found by referring to Connectionist Temporal
-Classification: Labelling Unsegmented Sequence Data with Recurrent
-Neural Networks
    
-
    
-
    注解
    
-
    Considering the ‘blank’ label needed by CTC, you need to use
-(num_classes + 1) as the input size. num_classes is the category number.
-And the ‘blank’ is the last category index. So the size of ‘input’ layer, such as
-fc_layer with softmax activation, should be num_classes + 1. The size of ctc_layer
-should also be num_classes + 1.
    
-
    
-
    The simple usage:
    
-
    ctc = ctc_layer(input=input,
-                label=label,
-                size=9055,
-                norm_by_times=True)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – The input layer.
      • 
-
    • label (LayerOutput) – The data layer of label with variable length.
      • 
-
    • size (int) – category numbers + 1.
      • 
-
    • name (basestring|None) – The name of this layer
      • 
-
    • norm_by_times (bool) – Whether to normalization by times. False by default.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    warp_ctc_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.warp_ctc_layer(*args, **kwargs)
    
-
    A layer intergrating the open-source warp-ctc
-<https://github.com/baidu-research/warp-ctc> library, which is used in
-Deep Speech 2: End-toEnd Speech Recognition in English and Mandarin
-<https://arxiv.org/pdf/1512.02595v1.pdf>, to compute Connectionist Temporal
-Classification (CTC) loss.
    
-
    More details of CTC can be found by referring to Connectionist Temporal
-Classification: Labelling Unsegmented Sequence Data with Recurrent
-Neural Networks
    
-
    
-
    注解
    
-
      
-
    • Let num_classes represent the category number. Considering the ‘blank’
-label needed by CTC, you need to use (num_classes + 1) as the input
-size. Thus, the size of both warp_ctc_layer and ‘input’ layer should
-be set to num_classes + 1.
      • 
-
    • You can set ‘blank’ to any value ranged in [0, num_classes], which
-should be consistent as that used in your labels.
      • 
-
    • As a native ‘softmax’ activation is interated to the warp-ctc library,
-‘linear’ activation is expected instead in the ‘input’ layer.
      • 
-
    
-
    
-
    The simple usage:
    
-
    ctc = warp_ctc_layer(input=input,
-                     label=label,
-                     size=1001,
-                     blank=1000,
-                     norm_by_times=False)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – The input layer.
      • 
-
    • label (LayerOutput) – The data layer of label with variable length.
      • 
-
    • size (int) – category numbers + 1.
      • 
-
    • name (basestring|None) – The name of this layer, which can not specify.
      • 
-
    • blank (int) – the ‘blank’ label used in ctc
      • 
-
    • norm_by_times (bool) – Whether to normalization by times. False by default.
      • 
-
    • layer_attr (ExtraLayerAttribute|None) – Extra Layer config.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    nce_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.nce_layer(*args, **kwargs)
    
-
    Noise-contrastive estimation.
-Implements the method in the following paper:
-A fast and simple algorithm for training neural probabilistic language models.
    
-
    The example usage is:
    
-
    cost = nce_layer(input=layer1, label=layer2, weight=layer3,
-                 num_classes=3, neg_distribution=[0.1,0.3,0.6])
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – layer name
      • 
-
    • input (LayerOutput|list|tuple|collections.Sequence) – input layers. It could be a LayerOutput of list/tuple of LayerOutput.
      • 
-
    • label (LayerOutput) – label layer
      • 
-
    • weight (LayerOutput) – weight layer, can be None(default)
      • 
-
    • num_classes (int) – number of classes.
      • 
-
    • act (BaseActivation) – Activation, default is Sigmoid.
      • 
-
    • num_neg_samples (int) – number of negative samples. Default is 10.
      • 
-
    • neg_distribution (list|tuple|collections.Sequence|None) – The distribution for generating the random negative labels.
-A uniform distribution will be used if not provided.
-If not None, its length must be equal to num_classes.
      • 
-
    • bias_attr (ParameterAttribute|None|False) – Bias parameter attribute. True if no bias.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    layer name.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    hsigmoid
    
-
    
-
    
-paddle.trainer_config_helpers.layers.hsigmoid(*args, **kwargs)
    
-
    Organize the classes into a binary tree. At each node, a sigmoid function
-is used to calculate the probability of belonging to the right branch.
-This idea is from “F. Morin, Y. Bengio (AISTATS 05):
-Hierarchical Probabilistic Neural Network Language Model.”
    
-
    The example usage is:
    
-
    cost = hsigmoid(input=[layer1, layer2],
-                label=data_layer)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput|list|tuple) – Input layers. It could be a LayerOutput or list/tuple of
-LayerOutput.
      • 
-
    • label (LayerOutput) – Label layer.
      • 
-
    • num_classes (int|None) – number of classes.
      • 
-
    • name (basestring) – layer name
      • 
-
    • bias_attr (ParameterAttribute|False) – Bias attribute. None means default bias.
-False means no bias.
      • 
-
    • param_attr (ParameterAttribute|None) – Parameter Attribute. None means default parameter.
      • 
-
    • layer_attr (ExtraLayerAttribute) – Extra Layer Attribute.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Check Layer
    
-
    
-
    eos_layer
    
-
    
-
    
-paddle.trainer_config_helpers.layers.eos_layer(*args, **kwargs)
    
-
    A layer for checking EOS for each sample:
-- output_id = (input_id == conf.eos_id)
    
-
    The result is stored in output_.ids.
-It is used by recurrent layer group.
    
-
    The example usage is:
    
-
    eos = eos_layer(input=layer, eos_id=id)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – Layer name.
      • 
-
    • input (LayerOutput) – Input layer name.
      • 
-
    • eos_id (int) – end id of sequence
      • 
-
    • layer_attr (ExtraLayerAttribute.) – extra layer attributes.
      • 
-
    
-
    返回:
    LayerOutput object.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
-
-           
    
-          
    
-          
    
-  
-
-  
    
-
-  
    
-    
    
-        © Copyright 2016, PaddlePaddle developers.
-
-    
    
-  
    
-  Built with Sphinx using a theme provided by Read the Docs. 
-
-
    
-
-        
    
-      
    
-
-    
    
-
-  
    
-  
-
-
-  
-
-    
-      
-      
-      
-      
-      
-       
-  
-
-  
-  
-    
-  
-  
-  
-  
-   
-
-
-
\ No newline at end of file
diff --git a/doc_cn/api/v1/trainer_config_helpers/networks.html b/doc_cn/api/v1/trainer_config_helpers/networks.html
deleted file mode 100644
index 0794b151791a9855bdf4471988b2c9ded7178122..0000000000000000000000000000000000000000
--- a/doc_cn/api/v1/trainer_config_helpers/networks.html
+++ /dev/null
@@ -1,957 +0,0 @@
-
-
-
-
-  
-
-  
-  
-  
-  
-  Networks — PaddlePaddle  文档
-  
-
-  
-  
-
-  
-
-  
-  
-    
-
-  
-
-  
-  
-    
-  
-
-  
-  
-        
-        
-     
-
-  
-  
-  
-
-  
-
-  
-  
-
-
-
-
-
-  
-  
    
-    
-    
    
-      
    
-        Folk me on Github
-        
    
-          
-            English
-            
-            
-          
-          
-        
    
-        
-      
    
-      
    
-        
-        
-
-        
-
    
-  
    
-    
-    
-    
-  
    
-
            
-      
    
-    
    
-  
    
-  
-  
    
-
-    
-    
-    
-    
    
-
-      
-
- 
-
-
-
-
-
-
-
-
    
-  
      
-      
-    
    • Networks
      • 
-  
    
-
    
-      
-      
    
-        
    
-          
    
-           
    
-            
-  
    
-
    Networks
    
-
    The networks module contains pieces of neural network that combine multiple layers.
    
-
    
-
    NLP
    
-
    
-
    sequence_conv_pool
    
-
    
-
    
-paddle.trainer_config_helpers.networks.sequence_conv_pool(*args, **kwargs)
    
-
    Text convolution pooling layers helper.
    
-
    Text input => Context Projection => FC Layer => Pooling => Output.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – name of output layer(pooling layer name)
      • 
-
    • input (LayerOutput) – name of input layer
      • 
-
    • context_len (int) – context projection length. See
-context_projection’s document.
      • 
-
    • hidden_size (int) – FC Layer size.
      • 
-
    • context_start (int or None) – context projection length. See
-context_projection’s context_start.
      • 
-
    • pool_type (BasePoolingType.) – pooling layer type. See pooling_layer’s document.
      • 
-
    • context_proj_layer_name (basestring) – context projection layer name.
-None if user don’t care.
      • 
-
    • context_proj_param_attr (ParameterAttribute or None.) – context projection parameter attribute.
-None if user don’t care.
      • 
-
    • fc_layer_name (basestring) – fc layer name. None if user don’t care.
      • 
-
    • fc_param_attr (ParameterAttribute or None) – fc layer parameter attribute. None if user don’t care.
      • 
-
    • fc_bias_attr (ParameterAttribute or None) – fc bias parameter attribute. False if no bias,
-None if user don’t care.
      • 
-
    • fc_act (BaseActivation) – fc layer activation type. None means tanh
      • 
-
    • pool_bias_attr (ParameterAttribute or None.) – pooling layer bias attr. None if don’t care.
-False if no bias.
      • 
-
    • fc_attr (ExtraLayerAttribute) – fc layer extra attribute.
      • 
-
    • context_attr (ExtraLayerAttribute) – context projection layer extra attribute.
      • 
-
    • pool_attr (ExtraLayerAttribute) – pooling layer extra attribute.
      • 
-
    
-
    返回:
    output layer name.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    text_conv_pool
    
-
    
-
    
-paddle.trainer_config_helpers.networks.text_conv_pool(*args, **kwargs)
    
-
    Text convolution pooling layers helper.
    
-
    Text input => Context Projection => FC Layer => Pooling => Output.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – name of output layer(pooling layer name)
      • 
-
    • input (LayerOutput) – name of input layer
      • 
-
    • context_len (int) – context projection length. See
-context_projection’s document.
      • 
-
    • hidden_size (int) – FC Layer size.
      • 
-
    • context_start (int or None) – context projection length. See
-context_projection’s context_start.
      • 
-
    • pool_type (BasePoolingType.) – pooling layer type. See pooling_layer’s document.
      • 
-
    • context_proj_layer_name (basestring) – context projection layer name.
-None if user don’t care.
      • 
-
    • context_proj_param_attr (ParameterAttribute or None.) – context projection parameter attribute.
-None if user don’t care.
      • 
-
    • fc_layer_name (basestring) – fc layer name. None if user don’t care.
      • 
-
    • fc_param_attr (ParameterAttribute or None) – fc layer parameter attribute. None if user don’t care.
      • 
-
    • fc_bias_attr (ParameterAttribute or None) – fc bias parameter attribute. False if no bias,
-None if user don’t care.
      • 
-
    • fc_act (BaseActivation) – fc layer activation type. None means tanh
      • 
-
    • pool_bias_attr (ParameterAttribute or None.) – pooling layer bias attr. None if don’t care.
-False if no bias.
      • 
-
    • fc_attr (ExtraLayerAttribute) – fc layer extra attribute.
      • 
-
    • context_attr (ExtraLayerAttribute) – context projection layer extra attribute.
      • 
-
    • pool_attr (ExtraLayerAttribute) – pooling layer extra attribute.
      • 
-
    
-
    返回:
    output layer name.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    Images
    
-
    
-
    img_conv_bn_pool
    
-
    
-
    
-paddle.trainer_config_helpers.networks.img_conv_bn_pool(*args, **kwargs)
    
-
    Convolution, batch normalization, pooling group.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – group name
      • 
-
    • input (LayerOutput) – layer’s input
      • 
-
    • filter_size (int) – see img_conv_layer’s document
      • 
-
    • num_filters (int) – see img_conv_layer’s document
      • 
-
    • pool_size (int) – see img_pool_layer’s document.
      • 
-
    • pool_type (BasePoolingType) – see img_pool_layer’s document.
      • 
-
    • act (BaseActivation) – see batch_norm_layer’s document.
      • 
-
    • groups (int) – see img_conv_layer’s document
      • 
-
    • conv_stride (int) – see img_conv_layer’s document.
      • 
-
    • conv_padding (int) – see img_conv_layer’s document.
      • 
-
    • conv_bias_attr (ParameterAttribute) – see img_conv_layer’s document.
      • 
-
    • num_channel (int) – see img_conv_layer’s document.
      • 
-
    • conv_param_attr (ParameterAttribute) – see img_conv_layer’s document.
      • 
-
    • shared_bias (bool) – see img_conv_layer’s document.
      • 
-
    • conv_layer_attr (ExtraLayerOutput) – see img_conv_layer’s document.
      • 
-
    • bn_param_attr (ParameterAttribute.) – see batch_norm_layer’s document.
      • 
-
    • bn_bias_attr – see batch_norm_layer’s document.
      • 
-
    • bn_layer_attr – ParameterAttribute.
      • 
-
    • pool_stride (int) – see img_pool_layer’s document.
      • 
-
    • pool_padding (int) – see img_pool_layer’s document.
      • 
-
    • pool_layer_attr (ExtraLayerAttribute) – see img_pool_layer’s document.
      • 
-
    
-
    返回:
    Layer groups output
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    img_conv_group
    
-
    
-
    
-paddle.trainer_config_helpers.networks.img_conv_group(*args, **kwargs)
    
-
    Image Convolution Group, Used for vgg net.
    
-
    TODO(yuyang18): Complete docs
    
-
---
-
-
-
-
-
-
    


    参数:
      
-
    • conv_batchnorm_drop_rate – 
      • 
-
    • input – 
      • 
-
    • conv_num_filter – 
      • 
-
    • pool_size – 
      • 
-
    • num_channels – 
      • 
-
    • conv_padding – 
      • 
-
    • conv_filter_size – 
      • 
-
    • conv_act – 
      • 
-
    • conv_with_batchnorm – 
      • 
-
    • pool_stride – 
      • 
-
    • pool_type – 
      • 
-
    
-
    返回:
    
-
    
-
    
-
-
    
-
    
-
    simple_img_conv_pool
    
-
    
-
    
-paddle.trainer_config_helpers.networks.simple_img_conv_pool(*args, **kwargs)
    
-
    Simple image convolution and pooling group.
    
-
    Input => conv => pooling
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – group name
      • 
-
    • input (LayerOutput) – input layer name.
      • 
-
    • filter_size (int) – see img_conv_layer for details
      • 
-
    • num_filters (int) – see img_conv_layer for details
      • 
-
    • pool_size (int) – see img_pool_layer for details
      • 
-
    • pool_type (BasePoolingType) – see img_pool_layer for details
      • 
-
    • act (BaseActivation) – see img_conv_layer for details
      • 
-
    • groups (int) – see img_conv_layer for details
      • 
-
    • conv_stride (int) – see img_conv_layer for details
      • 
-
    • conv_padding (int) – see img_conv_layer for details
      • 
-
    • bias_attr (ParameterAttribute) – see img_conv_layer for details
      • 
-
    • num_channel (int) – see img_conv_layer for details
      • 
-
    • param_attr (ParameterAttribute) – see img_conv_layer for details
      • 
-
    • shared_bias (bool) – see img_conv_layer for details
      • 
-
    • conv_layer_attr (ExtraLayerAttribute) – see img_conv_layer for details
      • 
-
    • pool_stride (int) – see img_pool_layer for details
      • 
-
    • pool_padding (int) – see img_pool_layer for details
      • 
-
    • pool_layer_attr (ExtraLayerAttribute) – see img_pool_layer for details
      • 
-
    
-
    返回:
    Layer’s output
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    vgg_16_network
    
-
    
-
    
-paddle.trainer_config_helpers.networks.vgg_16_network(input_image, num_channels, num_classes=1000)
    
-
    Same model from https://gist.github.com/ksimonyan/211839e770f7b538e2d8
    
-
---
-
-
-
-
-
-
    


    参数:
      
-
    • num_classes – 
      • 
-
    • input_image (LayerOutput) – 
      • 
-
    • num_channels (int) – 
      • 
-
    
-
    返回:
    
-
    
-
    
-
-
    
-
    
-
    
-
    Recurrent
    
-
    
-
    LSTM
    
-
    
-
    lstmemory_unit
    
-
    
-
    
-paddle.trainer_config_helpers.networks.lstmemory_unit(*args, **kwargs)
    
-
    Define calculations that a LSTM unit performs in a single time step.
-This function itself is not a recurrent layer, so that it can not be
-directly applied to sequence input. This function is always used in
-recurrent_group (see layers.py for more details) to implement attention
-mechanism.
    
-
    Please refer to  Generating Sequences With Recurrent Neural Networks
-for more details about LSTM. The link goes as follows:
-.. _Link: https://arxiv.org/abs/1308.0850
    
-
    
-\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    
-
    The example usage is:
    
-
    lstm_step = lstmemory_unit(input=[layer1],
-                           size=256,
-                           act=TanhActivation(),
-                           gate_act=SigmoidActivation(),
-                           state_act=TanhActivation())
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – input layer name.
      • 
-
    • name (basestring) – lstmemory unit name.
      • 
-
    • size (int) – lstmemory unit size.
      • 
-
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • 
-
    • act (BaseActivation) – lstm final activiation type
      • 
-
    • gate_act (BaseActivation) – lstm gate activiation type
      • 
-
    • state_act (BaseActivation) – lstm state activiation type.
      • 
-
    • mixed_bias_attr (ParameterAttribute|False) – bias parameter attribute of mixed layer.
-False means no bias, None means default bias.
      • 
-
    • lstm_bias_attr (ParameterAttribute|False) – bias parameter attribute of lstm layer.
-False means no bias, None means default bias.
      • 
-
    • mixed_layer_attr (ExtraLayerAttribute) – mixed layer’s extra attribute.
      • 
-
    • lstm_layer_attr (ExtraLayerAttribute) – lstm layer’s extra attribute.
      • 
-
    • get_output_layer_attr (ExtraLayerAttribute) – get output layer’s extra attribute.
      • 
-
    
-
    返回:
    lstmemory unit name.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    lstmemory_group
    
-
    
-
    
-paddle.trainer_config_helpers.networks.lstmemory_group(*args, **kwargs)
    
-
    lstm_group is a recurrent layer group version of Long Short Term Memory. It
-does exactly the same calculation as the lstmemory layer (see lstmemory in
-layers.py for the maths) does. A promising benefit is that LSTM memory
-cell states, or hidden states in every time step are accessible to the
-user. This is especially useful in attention model. If you do not need to
-access the internal states of the lstm, but merely use its outputs,
-it is recommended to use the lstmemory, which is relatively faster than
-lstmemory_group.
    
-
    NOTE: In PaddlePaddle’s implementation, the following input-to-hidden
-multiplications:
-\(W_{xi}x_{t}\) , \(W_{xf}x_{t}\),
-\(W_{xc}x_t\), \(W_{xo}x_{t}\) are not done in lstmemory_unit to
-speed up the calculations. Consequently, an additional mixed_layer with
-full_matrix_projection must be included before lstmemory_unit is called.
    
-
    The example usage is:
    
-
    lstm_step = lstmemory_group(input=[layer1],
-                            size=256,
-                            act=TanhActivation(),
-                            gate_act=SigmoidActivation(),
-                            state_act=TanhActivation())
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – input layer name.
      • 
-
    • name (basestring) – lstmemory group name.
      • 
-
    • size (int) – lstmemory group size.
      • 
-
    • reverse (bool) – is lstm reversed
      • 
-
    • param_attr (ParameterAttribute) – Parameter config, None if use default.
      • 
-
    • act (BaseActivation) – lstm final activiation type
      • 
-
    • gate_act (BaseActivation) – lstm gate activiation type
      • 
-
    • state_act (BaseActivation) – lstm state activiation type.
      • 
-
    • mixed_bias_attr (ParameterAttribute|False) – bias parameter attribute of mixed layer.
-False means no bias, None means default bias.
      • 
-
    • lstm_bias_attr (ParameterAttribute|False) – bias parameter attribute of lstm layer.
-False means no bias, None means default bias.
      • 
-
    • mixed_layer_attr (ExtraLayerAttribute) – mixed layer’s extra attribute.
      • 
-
    • lstm_layer_attr (ExtraLayerAttribute) – lstm layer’s extra attribute.
      • 
-
    • get_output_layer_attr (ExtraLayerAttribute) – get output layer’s extra attribute.
      • 
-
    
-
    返回:
    the lstmemory group.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    simple_lstm
    
-
    
-
    
-paddle.trainer_config_helpers.networks.simple_lstm(*args, **kwargs)
    
-
    Simple LSTM Cell.
    
-
    It just combine a mixed layer with fully_matrix_projection and a lstmemory
-layer. The simple lstm cell was implemented as follow equations.
    
-
    
-\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    
-
    Please refer Generating Sequences With Recurrent Neural Networks if you
-want to know what lstm is. Link is here.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – lstm layer name.
      • 
-
    • input (LayerOutput) – input layer name.
      • 
-
    • size (int) – lstm layer size.
      • 
-
    • reverse (bool) – whether to process the input data in a reverse order
      • 
-
    • mat_param_attr (ParameterAttribute) – mixed layer’s matrix projection parameter attribute.
      • 
-
    • bias_param_attr (ParameterAttribute|False) – bias parameter attribute. False means no bias, None
-means default bias.
      • 
-
    • inner_param_attr (ParameterAttribute) – lstm cell parameter attribute.
      • 
-
    • act (BaseActivation) – lstm final activiation type
      • 
-
    • gate_act (BaseActivation) – lstm gate activiation type
      • 
-
    • state_act (BaseActivation) – lstm state activiation type.
      • 
-
    • mixed_layer_attr (ExtraLayerAttribute) – mixed layer’s extra attribute.
      • 
-
    • lstm_cell_attr (ExtraLayerAttribute) – lstm layer’s extra attribute.
      • 
-
    
-
    返回:
    lstm layer name.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    bidirectional_lstm
    
-
    
-
    
-paddle.trainer_config_helpers.networks.bidirectional_lstm(*args, **kwargs)
    
-
    A bidirectional_lstm is a recurrent unit that iterates over the input
-sequence both in forward and bardward orders, and then concatenate two
-outputs form a final output. However, concatenation of two outputs
-is not the only way to form the final output, you can also, for example,
-just add them together.
    
-
    Please refer to  Neural Machine Translation by Jointly Learning to Align
-and Translate for more details about the bidirectional lstm.
-The link goes as follows:
-.. _Link: https://arxiv.org/pdf/1409.0473v3.pdf
    
-
    The example usage is:
    
-
    bi_lstm = bidirectional_lstm(input=[input1], size=512)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – bidirectional lstm layer name.
      • 
-
    • input (LayerOutput) – input layer.
      • 
-
    • size (int) – lstm layer size.
      • 
-
    • return_seq (bool) – If set False, outputs of the last time step are
-concatenated and returned.
-If set True, the entire output sequences that are
-processed in forward and backward directions are
-concatenated and returned.
      • 
-
    
-
    返回:
    LayerOutput object accroding to the return_seq.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    GRU
    
-
    
-
    gru_unit
    
-
    
-
    
-paddle.trainer_config_helpers.networks.gru_unit(*args, **kwargs)
    
-
    Define calculations that a gated recurrent unit performs in a single time
-step. This function itself is not a recurrent layer, so that it can not be
-directly applied to sequence input. This function is almost always used in
-the recurrent_group (see layers.py for more details) to implement attention
-mechanism.
    
-
    Please see grumemory in layers.py for the details about the maths.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – input layer name.
      • 
-
    • name (basestring) – name of the gru group.
      • 
-
    • size (int) – hidden size of the gru.
      • 
-
    • act (BaseActivation) – type of the activation
      • 
-
    • gate_act (BaseActivation) – type of the gate activation
      • 
-
    • gru_layer_attr (ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • 
-
    
-
    返回:
    the gru output layer.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    gru_group
    
-
    
-
    
-paddle.trainer_config_helpers.networks.gru_group(*args, **kwargs)
    
-
    gru_group is a recurrent layer group version of Gated Recurrent Unit. It
-does exactly the same calculation as the grumemory layer does. A promising
-benefit is that gru hidden states are accessible to the user. This is
-especially useful in attention model. If you do not need to access
-any internal state, but merely use the outputs of a GRU, it is recommended
-to use the grumemory, which is relatively faster.
    
-
    Please see grumemory in layers.py for more detail about the maths.
    
-
    The example usage is:
    
-
    gru = gur_group(input=[layer1],
-                size=256,
-                act=TanhActivation(),
-                gate_act=SigmoidActivation())
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – input layer name.
      • 
-
    • name (basestring) – name of the gru group.
      • 
-
    • size (int) – hidden size of the gru.
      • 
-
    • reverse (bool) – whether to process the input data in a reverse order
      • 
-
    • act (BaseActivation) – type of the activiation
      • 
-
    • gate_act (BaseActivation) – type of the gate activiation
      • 
-
    • gru_bias_attr (ParameterAttribute|False) – bias. False means no bias, None means default bias.
      • 
-
    • gru_layer_attr (ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • 
-
    
-
    返回:
    the gru group.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    simple_gru
    
-
    
-
    
-paddle.trainer_config_helpers.networks.simple_gru(*args, **kwargs)
    
-
    You maybe see gru_step_layer, grumemory in layers.py, gru_unit, gru_group,
-simple_gru in network.py. The reason why there are so many interfaces is
-that we have two ways to implement recurrent neural network. One way is to
-use one complete layer to implement rnn (including simple rnn, gru and lstm)
-with multiple time steps, such as recurrent_layer, lstmemory, grumemory. But,
-the multiplication operation \(W x_t\) is not computed in these layers.
-See details in their interfaces in layers.py.
-The other implementation is to use an recurrent group which can ensemble a
-series of layers to compute rnn step by step. This way is flexible for
-attenion mechanism or other complex connections.
    
-
      
-
    • gru_step_layer: only compute rnn by one step. It needs an memory as input
-and can be used in recurrent group.
      • 
-
    • gru_unit: a wrapper of gru_step_layer with memory.
      • 
-
    • gru_group: a GRU cell implemented by a combination of multiple layers in
-recurrent group.
-But \(W x_t\) is not done in group.
      • 
-
    • gru_memory: a GRU cell implemented by one layer, which does same calculation
-with gru_group and is faster than gru_group.
      • 
-
    • simple_gru: a complete GRU implementation inlcuding \(W x_t\) and
-gru_group. \(W\) contains \(W_r\), \(W_z\) and \(W\), see
-formula in grumemory.
      • 
-
    
-
    The computational speed is that, grumemory is relatively better than
-gru_group, and gru_group is relatively better than simple_gru.
    
-
    The example usage is:
    
-
    gru = simple_gru(input=[layer1], size=256)
-
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • input (LayerOutput) – input layer name.
      • 
-
    • name (basestring) – name of the gru group.
      • 
-
    • size (int) – hidden size of the gru.
      • 
-
    • reverse (bool) – whether to process the input data in a reverse order
      • 
-
    • act (BaseActivation) – type of the activiation
      • 
-
    • gate_act (BaseActivation) – type of the gate activiation
      • 
-
    • gru_bias_attr (ParameterAttribute|False) – bias. False means no bias, None means default bias.
      • 
-
    • gru_layer_attr (ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • 
-
    
-
    返回:
    the gru group.
    
-
    返回类型:
    LayerOutput
    
-
    
-
    
-
-
    
-
    
-
    
-
    simple_attention
    
-
    
-
    
-paddle.trainer_config_helpers.networks.simple_attention(*args, **kwargs)
    
-
    Calculate and then return a context vector by attention machanism.
-Size of the context vector equals to size of the encoded_sequence.
    
-
    
-\[ \begin{align}\begin{aligned}a(s_{i-1},h_{j}) & = v_{a}f(W_{a}s_{t-1} + U_{a}h_{j})\\e_{i,j} & = a(s_{i-1}, h_{j})\\a_{i,j} & = \frac{exp(e_{i,j})}{\sum_{k=1}^{T_x}{exp(e_{i,k})}}\\c_{i} & = \sum_{j=1}^{T_{x}}a_{i,j}h_{j}\end{aligned}\end{align} \]
    
-
    where \(h_{j}\) is the jth element of encoded_sequence,
-\(U_{a}h_{j}\) is the jth element of encoded_proj
-\(s_{i-1}\) is decoder_state
-\(f\) is weight_act, and is set to tanh by default.
    
-
    Please refer to Neural Machine Translation by Jointly Learning to
-Align and Translate for more details. The link is as follows:
-https://arxiv.org/abs/1409.0473.
    
-
    The example usage is:
    
-
    context = simple_attention(encoded_sequence=enc_seq,
-                           encoded_proj=enc_proj,
-                           decoder_state=decoder_prev,)
-
    
-
    
-
---
-
-
-
-
-
-
    


    参数:
      
-
    • name (basestring) – name of the attention model.
      • 
-
    • softmax_param_attr (ParameterAttribute) – parameter attribute of sequence softmax
-that is used to produce attention weight
      • 
-
    • weight_act (Activation) – activation of the attention model
      • 
-
    • encoded_sequence (LayerOutput) – output of the encoder
      • 
-
    • encoded_proj (LayerOutput) – attention weight is computed by a feed forward neural
-network which has two inputs : decoder’s hidden state
-of previous time step and encoder’s output.
-encoded_proj is output of the feed-forward network for
-encoder’s output. Here we pre-compute it outside
-simple_attention for speed consideration.
      • 
-
    • decoder_state (LayerOutput) – hidden state of decoder in previous time step
      • 
-
    • transform_param_attr (ParameterAttribute) – parameter attribute of the feed-forward
-network that takes decoder_state as inputs to
-compute attention weight.
      • 
-
    
-
    返回:
    a context vector
    
-
    
-
    
-
-
    
-
    
-
    
-
    Miscs
    
-
    
-
    dropout_layer
    
-
    
-
    
-paddle.trainer_config_helpers.networks.dropout_layer(*args, **kwargs)
    
-
    @TODO(yuyang18): Add comments.
    
-
---
-
-
-
-
-
-
    


    参数:
      
-
    • name – 
      • 
-
    • input – 
      • 
-
    • dropout_rate – 
      • 
-
    
-
    返回:
    
-
    
-
    
-
-
    
-
    
-
    outputs
    
-
    
-
    
-paddle.trainer_config_helpers.networks.outputs(layers, *args)
    
-
    Declare the outputs of network. If user have not defined the inputs of
-network, this method will calculate the input order by dfs travel.
    
-
---
-
-
-
-
-
-
    


    参数:layers (list|tuple|LayerOutput) – Output layers.
    返回:
    
-
    
-
-
    
-
    
-
    
-
-
-           
    
-          
    
-          
    
-  
-
-  
    
-
-  
    
-    
    
-        © Copyright 2016, PaddlePaddle developers.
-
-    
    
-  
    
-  Built with Sphinx using a theme provided by Read the Docs. 
-
-
    
-
-        
    
-      
    
-
-    
    
-
-  
    
-  
-
-
-  
-
-    
-      
-      
-      
-      
-      
-       
-  
-
-  
-  
-    
-  
-  
-  
-  
-   
-
-
-
\ No newline at end of file
diff --git a/doc_cn/api/v1/trainer_config_helpers/optimizers.html b/doc_cn/api/v1/trainer_config_helpers/optimizers.html
deleted file mode 100644
index 5a60d30cb7aa304333786e3a9448a8d058673a4c..0000000000000000000000000000000000000000
--- a/doc_cn/api/v1/trainer_config_helpers/optimizers.html
+++ /dev/null
@@ -1,482 +0,0 @@
-
-
-
-
-  
-
-  
-  
-  
-  
-  Optimizers — PaddlePaddle  文档
-  
-
-  
-  
-
-  
-
-  
-  
-    
-
-  
-
-  
-  
-    
-  
-
-  
-  
-        
-        
-     
-
-  
-  
-  
-
-  
-
-  
-  
-
-
-
-
-
-  
-  
    
-    
-    
    
-      
    
-        Folk me on Github
-        
    
-          
-            English
-            
-            
-          
-          
-        
    
-        
-      
    
-      
    
-        
-        
-
-        
-
    
-  
    
-    
-    
-    
-  
    
-
            
-      
    
-    
    
-  
    
-  
-  
    
-
-    
-    
-    
-    
    
-
-      
-
- 
-
-
-
-
-
-
-
-
    
-  
      
-      
-    
    • Optimizers
      • 
-  
    
-
    
-      
-      
    
-        
    
-          
    
-           
    
-            
-  
    
-
    Optimizers
    
-
    
-
    BaseSGDOptimizer
    
-
    
-
    
-class paddle.trainer_config_helpers.optimizers.BaseSGDOptimizer
    
-
    SGD Optimizer.
    
-
    SGD is an optimization method, trying to find a neural network that
-minimize the “cost/error” of it by iteration. In paddle’s implementation
-SGD Optimizer is synchronized, which means all gradients will be wait to
-calculate and reduced into one gradient, then do optimize operation.
    
-
    The neural network consider the learning problem of minimizing an objective
-function, that has the form of a sum
    
-
    
-\[Q(w) = \sum_{i}^{n} Q_i(w)\]
    
-
    The value of function Q sometimes is the cost of neural network (Mean
-Square Error between prediction and label for example). The function Q is
-parametrised by w, the weight/bias of neural network. And weights is what to
-be learned. The i is the i-th observation in (trainning) data.
    
-
    So, the SGD method will optimize the weight by
    
-
    
-\[w = w - \eta \nabla Q(w) = w - \eta \sum_{i}^{n} \nabla Q_i(w)\]
    
-
    where \(\eta\) is learning rate. And \(n\) is batch size.
    
-
    
-
-
    
-
    
-
    MomentumOptimizer
    
-
    
-
    
-class paddle.trainer_config_helpers.optimizers.MomentumOptimizer(momentum=None, sparse=False)
    
-
    MomentumOptimizer.
    
-
    When sparse=True, the update scheme:
    
-
    
-\[\begin{split}\alpha_t &= \alpha_{t-1} / k \\
-\beta_t &= \beta_{t-1} / (1 + \lambda \gamma_t) \\
-u_t &= u_{t-1} - \alpha_t \gamma_t g_t \\
-v_t &= v_{t-1} + \tau_{t-1} \alpha_t \gamma_t g_t \\
-\tau_t &= \tau_{t-1} + \beta_t / \alpha_t\end{split}\]
    
-
    where \(k\) is momentum, \(\lambda\) is decay rate,
-\(\gamma_t\) is learning rate at the t’th step.
    
-
---
-
-
-
-
    


    参数:sparse (bool) – with sparse support or not.
    
-
    
-
-
    
-
    
-
    AdamOptimizer
    
-
    
-
    
-class paddle.trainer_config_helpers.optimizers.AdamOptimizer(beta1=0.9, beta2=0.999, epsilon=1e-08)
    
-
    Adam optimizer.
-The details of please refer Adam: A Method for Stochastic Optimization
    
-
    
-\[\begin{split}m(w, t) & = \beta_1 m(w, t-1) + (1 - \beta_1) \nabla Q_i(w) \\
-v(w, t) & = \beta_2 v(w, t-1) + (1 - \beta_2)(\nabla Q_i(w)) ^2 \\
-w & = w - \frac{\eta}{\sqrt{v(w,t) + \epsilon}}\end{split}\]
    
-
---
-
-
-
-
    


    参数:
      
-
    • beta1 (float) – the \(\beta_1\) in equation.
      • 
-
    • beta2 (float) – the \(\beta_2\) in equation.
      • 
-
    • epsilon (float) – the \(\epsilon\) in equation. It is used to prevent
-divided by zero.
      • 
-
    
-
    
-
    
-
-
    
-
    
-
    AdamaxOptimizer
    
-
    
-
    
-class paddle.trainer_config_helpers.optimizers.AdamaxOptimizer(beta1, beta2)
    
-
    Adamax optimizer.
    
-
    The details of please refer this Adam: A Method for Stochastic Optimization
    
-
    
-\[\begin{split}m_t & = \beta_1 * m_{t-1} + (1-\beta_1)* \nabla Q_i(w) \\
-u_t & = max(\beta_2*u_{t-1}, abs(\nabla Q_i(w))) \\
-w_t & = w_{t-1} - (\eta/(1-\beta_1^t))*m_t/u_t\end{split}\]
    
-
---
-
-
-
-
    


    参数:
      
-
    • beta1 (float) – the \(\beta_1\) in the equation.
      • 
-
    • beta2 (float) – the \(\beta_2\) in the equation.
      • 
-
    
-
    
-
    
-
-
    
-
    
-
    AdaGradOptimizer
    
-
    
-
    
-class paddle.trainer_config_helpers.optimizers.AdaGradOptimizer
    
-
    Adagrad(for ADAptive GRAdient algorithm) optimizer.
    
-
    For details please refer this Adaptive Subgradient Methods for
-Online Learning and Stochastic Optimization.
    
-
    
-\[\begin{split}G &= \sum_{\tau=1}^{t} g_{\tau} g_{\tau}^T \\
-w & = w - \eta diag(G)^{-\frac{1}{2}} \circ g\end{split}\]
    
-
    
-
-
    
-
    
-
    DecayedAdaGradOptimizer
    
-
    
-
    
-class paddle.trainer_config_helpers.optimizers.DecayedAdaGradOptimizer(rho=0.95, epsilon=1e-06)
    
-
    AdaGrad method with decayed sum gradients. The equations of this method
-show as follow.
    
-
    
-\[\begin{split}E(g_t^2) &= \rho * E(g_{t-1}^2) + (1-\rho) * g^2 \\
-learning\_rate &= 1/sqrt( ( E(g_t^2) + \epsilon )\end{split}\]
    
-
---
-
-
-
-
    


    参数:
      
-
    • rho (float) – The \(\rho\) parameter in that equation
      • 
-
    • epsilon (float) – The \(\epsilon\) parameter in that equation.
      • 
-
    
-
    
-
    
-
-
    
-
    
-
    AdaDeltaOptimizer
    
-
    
-
    
-class paddle.trainer_config_helpers.optimizers.AdaDeltaOptimizer(rho=0.95, epsilon=1e-06)
    
-
    AdaDelta method. The details of adadelta please refer to this
-ADADELTA: AN ADAPTIVE LEARNING RATE METHOD.
    
-
    
-\[\begin{split}E(g_t^2) &= \rho * E(g_{t-1}^2) + (1-\rho) * g^2 \\
-learning\_rate &= sqrt( ( E(dx_{t-1}^2) + \epsilon ) / ( \
-                  E(g_t^2) + \epsilon ) ) \\
-E(dx_t^2) &= \rho * E(dx_{t-1}^2) + (1-\rho) * (-g*learning\_rate)^2\end{split}\]
    
-
---
-
-
-
-
    


    参数:
      
-
    • rho (float) – \(\rho\) in equation
      • 
-
    • epsilon (float) – \(\rho\) in equation
      • 
-
    
-
    
-
    
-
-
    
-
    
-
    RMSPropOptimizer
    
-
    
-
    
-class paddle.trainer_config_helpers.optimizers.RMSPropOptimizer(rho=0.95, epsilon=1e-06)
    
-
    RMSProp(for Root Mean Square Propagation) optimizer. For details please
-refer this slide.
    
-
    The equations of this method as follows:
    
-
    
-\[\begin{split}v(w, t) & = \rho v(w, t-1) + (1 - \rho)(\nabla Q_{i}(w))^2 \\
-w & = w - \frac{\eta} {\sqrt{v(w,t) + \epsilon}} \nabla Q_{i}(w)\end{split}\]
    
-
---
-
-
-
-
    


    参数:
      
-
    • rho (float) – the \(\rho\) in the equation. The forgetting factor.
      • 
-
    • epsilon (float) – the \(\epsilon\) in the equation.
      • 
-
    
-
    
-
    
-
-
    
-
    
-
    settings
    
-
    
-
    
-paddle.trainer_config_helpers.optimizers.settings(*args, **kwargs)
    
-
    Set the optimization method, learning rate, batch size, and other training
-settings. The currently supported algorithms are SGD and Async-SGD.
    
-
    
-
    警告
    
-
    Note that the ‘batch_size’ in PaddlePaddle is not equal to global
-training batch size. It represents the single training process’s batch
-size. If you use N processes to train one model, for example use three
-GPU machines, the global batch size is N*’batch_size’.
    
-
    
-
---
-
-
-
-
    


    参数:
      
-
    • batch_size (int) – batch size for one training process.
      • 
-
    • learning_rate (float) – learning rate for SGD
      • 
-
    • learning_method (BaseSGDOptimizer) – The extension optimization algorithms of gradient
-descent, such as momentum, adagrad, rmsprop, etc.
-Note that it should be instance with base type
-BaseSGDOptimizer.
      • 
-
    • regularization (BaseRegularization) – The regularization method.
      • 
-
    • is_async (bool) – Is Async-SGD or not. Default value is False.
      • 
-
    • model_average (ModelAverage) – Model Average Settings.
      • 
-
    • gradient_clipping_threshold (float) – gradient clipping threshold. If gradient
-value larger than some value, will be
-clipped.
      • 
-
    
-
    
-
    
-
-
    
-
    
-
-
-           
    
-          
    
-          
    
-  
-
-  
    
-
-  
    
-    
    
-        © Copyright 2016, PaddlePaddle developers.
-
-    
    
-  
    
-  Built with Sphinx using a theme provided by Read the Docs. 
-
-
    
-
-        
    
-      
    
-
-    
    
-
-  
    
-  
-
-
-  
-
-    
-      
-      
-      
-      
-      
-       
-  
-
-  
-  
-    
-  
-  
-  
-  
-   
-
-
-
\ No newline at end of file
diff --git a/doc_cn/api/v2/config/activation.html b/doc_cn/api/v2/config/activation.html
index db175b22d363176b17191a5ee7f2a84be046fed1..7498e3782d67701fe0ebde290494c35c3baf3cae 100644
--- a/doc_cn/api/v2/config/activation.html
+++ b/doc_cn/api/v2/config/activation.html
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -88,6 +88,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +112,10 @@
           
 
         
@@ -353,6 +375,17 @@
 \[f(z) = 1.7159 * tanh(2/3*z)\]
    +
    +
    +

    SoftSign

    +
    +
    +class paddle.v2.activation.SoftSign
    +

    SoftSign Activation.

    +
    +\[f(z)=\frac{z}{1 + |z|}\]
    +
    +
    diff --git a/doc_cn/api/v2/config/attr.html b/doc_cn/api/v2/config/attr.html index 552b0a506c3f0a1d1a71ef772373a8f7caa9c623..d73fdd656e9d2dce0b5ac1e81311e7f7fd58d922 100644 --- a/doc_cn/api/v2/config/attr.html +++ b/doc_cn/api/v2/config/attr.html @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -203,6 +225,18 @@

    ExtraLayerAttribute 的别名

    +
    +
    +paddle.v2.attr.Hook
    +

    HookAttribute 的别名

    +
    + +
    +
    +paddle.v2.attr.HookAttr
    +

    HookAttribute 的别名

    +
    +
    paddle.v2.attr.ParamAttr
    @@ -217,7 +251,7 @@
    -class paddle.v2.attr.ParameterAttribute(name=None, is_static=False, initial_std=None, initial_mean=None, initial_max=None, initial_min=None, l1_rate=None, l2_rate=None, learning_rate=None, momentum=None, gradient_clipping_threshold=None, sparse_update=False)
    +class paddle.v2.attr.ParameterAttribute(name=None, is_static=False, initial_std=None, initial_mean=None, initial_max=None, initial_min=None, l1_rate=None, l2_rate=None, learning_rate=None, momentum=None, gradient_clipping_threshold=None, sparse_update=False, update_hooks=None, initializer=None)

    Parameter Attributes object. To fine-tuning network training process, user can set attribute to control training details, such as l1,l2 rate / learning rate / how to init param.

    @@ -246,6 +280,10 @@ value larger than some value, will be clipped.
  • sparse_update (bool) – Enable sparse update for this parameter. It will enable both local and remote sparse update.
    • +
  • initializer – If not None, it should be a callable object which accepts +a parameter name and returns numpy array for the initial +value of the parameter
    • +
  • initializer – callable object
    • @@ -283,7 +321,7 @@ layer that not support this attribute, paddle will print an error and core.

  • drop_rate (float) – Dropout rate. Dropout will create a mask on layer output. The dropout rate is the zero rate of this mask. The details of what dropout is please refer to here.
    • -
  • device (int) –

    device ID of layer. device=-1, use CPU. device>0, use GPU. +

  • device (int) –

    device ID of layer. device=-1, use CPU. device>=0, use GPU. The details allocation in parallel_nn please refer to here.

    • diff --git a/doc_cn/api/v1/trainer_config_helpers/evaluators.html b/doc_cn/api/v2/config/evaluators.html similarity index 55% rename from doc_cn/api/v1/trainer_config_helpers/evaluators.html rename to doc_cn/api/v2/config/evaluators.html index 5a3db31df447b59064f1c3e435077bf6b44a55c0..26b8af259c5079ba987b4d5e31ff0d945df210b8 100644 --- a/doc_cn/api/v1/trainer_config_helpers/evaluators.html +++ b/doc_cn/api/v2/config/evaluators.html @@ -32,7 +32,10 @@ - + + + + @@ -62,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -80,11 +83,12 @@
    -
      + @@ -105,12 +109,13 @@

    The input of lstm step is \(Wx_t + Wh_{t-1}\), and user should use mixed and full_matrix_projection to calculate these -input vector.

    +input vectors.

    The state of lstm step is \(c_{t-1}\). And lstm step layer will do

    \[ \begin{align}\begin{aligned}i_t = \sigma(input + W_{ci}c_{t-1} + b_i)\\...\end{aligned}\end{align} \]
    -

    This layer contains two outputs. Default output is \(h_t\). The other -output is \(o_t\), which name is ‘state’ and can use +

    This layer has two outputs. The default output is \(h_t\). The other +output is \(o_t\), whose name is ‘state’ and users can use get_output to extract this output.

    @@ -1279,22 +1502,28 @@ be sigmoid only.

    gru_step

    -class paddle.v2.layer.gru_step(*args, **kwargs)
    +class paddle.v2.layer.gru_step
    参数:
      -
    • name (basestring) – Layer’s name.
      • -
    • size (int) – Layer’s size. NOTE: lstm layer’s size, should be equal as -input.size/4, and should be equal as -state.size.
      • -
    • input (paddle.v2.config_base.Layer) – input layer. \(Wx_t + Wh_{t-1}\)
      • -
    • state (paddle.v2.config_base.Layer) – State Layer. \(c_{t-1}\)
      • -
    • act (paddle.v2.Activation.Base) – Activation type. Default is tanh
      • -
    • gate_act (paddle.v2.Activation.Base) – Gate Activation Type. Default is sigmoid, and should -be sigmoid only.
      • -
    • state_act (paddle.v2.Activation.Base) – State Activation Type. Default is sigmoid, and should -be sigmoid only.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute) – Bias Attribute.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – layer’s extra attribute.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • size (int) – The dimension of this layer’s output, which must be +equal to the dimension of the state.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • state (paddle.v2.config_base.Layer) – The state of the LSTM unit.
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Tanh is the default activation.
      • +
    • gate_act (paddle.v2.activation.Base) – Activation type of the gate. paddle.v2.activation.Sigmoid is the +default activation.
      • +
    • state_act (paddle.v2.activation.Base) – Activation type of the state. paddle.v2.activation.Tanh is the +default activation.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for details.
    @@ -1320,14 +1549,20 @@ to maintain tractability.

    The example usage is:

    def rnn_step(input):
     last_time_step_output = memory(name='rnn', size=512)
-    with mixed_layer(size=512, name='rnn') as simple_rnn:
+    with mixed(size=512, name='rnn') as simple_rnn:
         simple_rnn += full_matrix_projection(input)
         simple_rnn += last_time_step_output
     return simple_rnn
 
+generated_word_embedding = GeneratedInput(
+                       size=target_dictionary_dim,
+                       embedding_name="target_language_embedding",
+                       embedding_size=word_vector_dim)
+
 beam_gen = beam_search(name="decoder",
                        step=rnn_step,
-                       input=[StaticInput(encoder_last)],
+                       input=[StaticInput(encoder_last),
+                              generated_word_embedding],
                        bos_id=0,
                        eos_id=1,
                        beam_size=5)
@@ -1342,14 +1577,17 @@ to maintain tractability.
    - @@ -1387,7 +1625,7 @@ beam size.

    get_output

    -class paddle.v2.layer.get_output(*args, **kwargs)
    +class paddle.v2.layer.get_output

    Get layer’s output by name. In PaddlePaddle, a layer might return multiple values, but returns one layer’s output. If the user wants to use another output besides the default one, please use get_output first to get @@ -1397,11 +1635,12 @@ the output from input.

    @@ -1428,17 +1667,17 @@ multiple outputs. Each inputs is a projection or operator.

    There are two styles of usages.

      -
    1. When not set inputs parameter, use mixed_layer like this:
      2. +
    2. When not set inputs parameter, use mixed like this:
    -
    with mixed_layer(size=256) as m:
+
    with mixed(size=256) as m:
     m += full_matrix_projection(input=layer1)
     m += identity_projection(input=layer2)
 
    
 
    
 
      
-
    1. You can also set all inputs when invoke mixed_layer as follows:
      2. 
+
    2. You can also set all inputs when invoke mixed as follows:
      3. 
 
    
-
    m = mixed_layer(size=256,
+
    m = mixed(size=256,
                 input=[full_matrix_projection(input=layer1),
                        full_matrix_projection(input=layer2)])
 
    
@@ -1450,13 +1689,13 @@ Each inputs is a projection or operator.
    @@ -1475,19 +1714,19 @@ default Bias.

    embedding

    -class paddle.v2.layer.embedding(*args, **kwargs)
    +class paddle.v2.layer.embedding

    Define a embedding Layer.

    参数:
      -
    • input (paddle.v2.config_base.Layer) –
      • -
    • output_mem –
      • -
    • size –
      • -
    • act –
      • -
    • name –
      • -
    • gate_act –
      • -
    • bias_attr –
      • -
    • param_attr – the parameter_attribute for transforming the output_mem -from previous step.
      • -
    • layer_attr –
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer, whose dimension can be divided by 3.
      • +
    • output_mem (paddle.v2.config_base.Layer) – A memory which memorizes the output of this layer at previous +time step.
      • +
    • size (int) – The dimension of this layer’s output. If it is not set or set to None, +it will be set to one-third of the dimension of the input automatically.
      • +
    • act (paddle.v2.activation.Base) – Activation type of this layer’s output. paddle.v2.activation.Tanh +is the default activation.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • gate_act (paddle.v2.activation.Base) – Activation type of this layer’s two gates. paddle.v2.activation.Sigmoid is +the default activation.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The parameter attribute for bias. If this parameter is set to +False or an object whose type is not paddle.v2.attr.ParameterAttribute, no bias +is defined. If this parameter is set to True, +the bias is initialized to zero.
      • +
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for details.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for details.
    参数:
      -
    • name (base string) – Name of the recurrent unit that generates sequences.
      • +
    • name (basestring) – The name of the recurrent unit that is responsible for +generating sequences. It is optional.
    • step (callable) –

      A callable function that defines the calculation in a time step, and it is applied to sequences with arbitrary length by sharing a same set of weights.

      You can refer to the first parameter of recurrent_group, or demo/seqToseq/seqToseq_net.py for more details.

      • -
    • input (list) – Input data for the recurrent unit
      • +
    • input (list) – Input data for the recurrent unit, which should include the +previously generated words as a GeneratedInput object. +In beam_search, none of the input’s type should be paddle.v2.config_base.Layer.
    • bos_id (int) – Index of the start symbol in the dictionary. The start symbol is a special token for NLP task, which indicates the beginning of a sequence. In the generation task, the start @@ -1375,7 +1613,7 @@ beam size.
    返回:

    The generated word index.
    返回类型:

    LayerOutput

    +
    返回类型:

    paddle.v2.config_base.Layer
    参数:
      -
    • name (basestring) – Layer’s name.
      • -
    • input (paddle.v2.config_base.Layer) – get output layer’s input. And this layer should contains +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input layer. And this layer should contain multiple outputs.
      • -
    • arg_name (basestring) – Output name from input.
      • -
    • layer_attr – Layer’s extra attribute.
      • +
    • arg_name (basestring) – The name of the output to be extracted from the input layer.
      • +
    • layer_attr – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    参数:
    • name (basestring) – mixed layer name. Can be referenced by other layer.
    • size (int) – layer size.
      • -
    • input – inputs layer. It is an optional parameter. If set, +
    • input – The input of this layer. It is an optional parameter. If set, then this function will just return layer’s name.
      • -
    • act (BaseActivation) – Activation Type.
      • -
    • bias_attr (ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or -something not type of ParameterAttribute. None will get a -default Bias.
      • -
    • layer_attr (ExtraLayerAttribute) – The extra layer config. Default is None.
      • +
    • act (paddle.v2.activation.Base) – Activation Type. paddle.v2.activation.Linear is the default activation.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer config. Default is None.
    @@ -1506,7 +1745,7 @@ for details.

    scaling_projection

    -class paddle.v2.layer.scaling_projection(**kwargs)
    +class paddle.v2.layer.scaling_projection

    scaling_projection multiplies the input with a scalar parameter and add to the output.

    @@ -1520,7 +1759,7 @@ the output.

    @@ -1540,7 +1779,7 @@ the output.

    dotmul_projection

    -class paddle.v2.layer.dotmul_projection(**kwargs)
    +class paddle.v2.layer.dotmul_projection

    DotMulProjection with a layer as input. It performs element-wise multiplication with weight.

    @@ -1555,7 +1794,7 @@ It performs element-wise multiplication with weight.

    @@ -1575,7 +1814,7 @@ It performs element-wise multiplication with weight.

    dotmul_operator

    -class paddle.v2.layer.dotmul_operator(**kwargs)
    +class paddle.v2.layer.dotmul_operator

    DotMulOperator takes two inputs and performs element-wise multiplication:

    \[out.row[i] += scale * (a.row[i] .* b.row[i])\]
    @@ -1611,7 +1850,7 @@ scale is a config scalar, its default value is one.

    full_matrix_projection

    -class paddle.v2.layer.full_matrix_projection(**kwargs)
    +class paddle.v2.layer.full_matrix_projection

    Full Matrix Projection. It performs full matrix multiplication.

    \[out.row[i] += in.row[i] * weight\]
    @@ -1636,7 +1875,7 @@ scale is a config scalar, its default value is one.

    @@ -1698,12 +1937,49 @@ It select dimesions [offset, offset+layer_size) from input:

    参数:
      -
    • name (basestring) – Name of this embedding layer.
      • -
    • input (paddle.v2.config_base.Layer) – The input layer for this embedding. NOTE: must be Index Data.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer, which must be Index Data.
    • size (int) – The embedding dimension.
      • -
    • param_attr (paddle.v2.attr.ParameterAttribute|None) – The embedding parameter attribute. See paddle.v2.attr.ParameterAttribute +
    • param_attr (paddle.v2.attr.ParameterAttribute | None) – The embedding parameter attribute. See paddle.v2.attr.ParameterAttribute for details.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra layer Config. Default is None.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute | None) – Extra layer Config. Default is None.
    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input Layer.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
    参数:
      -
    • input (paddle.v2.config_base.Layer) – input layer
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • size (int) – The parameter size. Means the width of parameter.
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
    @@ -1657,7 +1896,7 @@ scale is a config scalar, its default value is one.

    identity_projection

    -class paddle.v2.layer.identity_projection(**kwargs)
    +class paddle.v2.layer.identity_projection
    1. IdentityProjection if offset=None. It performs:
    @@ -1683,7 +1922,7 @@ It select dimesions [offset, offset+layer_size) from input:
    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input Layer.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • offset (int) – Offset, None if use default.
    +
    +
    +

    slice_projection

    +
    +
    +class paddle.v2.layer.slice_projection
    +

    slice_projection can slice the input value into multiple parts, +and then select some of them to merge into a new output.

    +
    +\[output = [input.slices()]\]
    +

    The example usage is:

    +
    proj = slice_projection(input=layer, slices=[(0, 10), (20, 30)])
+
    +
    +

    Note that slice_projection should not have any parameter.

    + +++ + + + + + + + +
    参数:
      +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • slices (pair of int) – An array of slice parameters. +Each slice contains the start and end offsets based +on the input.
      • +
    +
    返回:

    A SliceProjection object

    +
    返回类型:

    SliceProjection

    +
    +
    +

    table_projection

    -class paddle.v2.layer.table_projection(**kwargs)
    +class paddle.v2.layer.table_projection

    Table Projection. It selects rows from parameter where row_id is in input_ids.

    @@ -1731,7 +2007,7 @@ and \(i\) is row_id.

    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input layer, which must contains id fields.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer, which must contains id fields.
    • size (int) – The parameter size. Means the width of parameter.
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
    @@ -1752,7 +2028,7 @@ and \(i\) is row_id.

    trans_full_matrix_projection

    -class paddle.v2.layer.trans_full_matrix_projection(**kwargs)
    +class paddle.v2.layer.trans_full_matrix_projection

    Different from full_matrix_projection, this projection performs matrix multiplication, using transpose of weight.

    @@ -1772,7 +2048,7 @@ The simply usage is:

    参数:
      -
    • input (paddle.v2.config_base.Layer) – input layer
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • size (int) – The parameter size. Means the width of parameter.
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
    @@ -1792,16 +2068,46 @@ The simply usage is:

    Aggregate Layers

    +
    +

    AggregateLevel

    +
    +
    +class paddle.v2.layer.AggregateLevel
    +

    PaddlePaddle supports three sequence types:

    +
      +
    • SequenceType.NO_SEQUENCE means the sample is not a sequence.
      • +
    • SequenceType.SEQUENCE means the sample is a sequence.
      • +
    • SequenceType.SUB_SEQUENCE means the sample is a nested sequence, +each timestep of which is also a sequence.
      • +
    +

    Accordingly, AggregateLevel supports two modes:

    +
      +
    • AggregateLevel.TO_NO_SEQUENCE means the aggregation acts on each +timestep of a sequence, both SUB_SEQUENCE and SEQUENCE will +be aggregated to NO_SEQUENCE.
      • +
    • AggregateLevel.TO_SEQUENCE means the aggregation acts on each +sequence of a nested sequence, SUB_SEQUENCE will be aggregated to +SEQUENCE.
      • +
    +
    + +

    pooling

    -class paddle.v2.layer.pooling(*args, **kwargs)
    +class paddle.v2.layer.pooling

    Pooling layer for sequence inputs, not used for Image.

    +

    If stride > 0, this layer slides a window whose size is determined by stride, +and return the pooling value of the window as the output. Thus, a long sequence +will be shorten.

    +

    The parameter stride specifies the intervals at which to apply the pooling +operation. Note that for sequence with sub-sequence, the default value +of stride is -1.

    The example usage is:

    seq_pool = pooling(input=layer,
                          pooling_type=AvgPooling(),
-                         agg_level=AggregateLevel.EACH_SEQUENCE)
+                         agg_level=AggregateLevel.TO_NO_SEQUENCE)
    @@ -1809,14 +2115,17 @@ The simply usage is:

    @@ -1835,7 +2144,7 @@ SumPooling, SquareRootNPooling.

    last_seq

    -class paddle.v2.layer.last_seq(*args, **kwargs)
    +class paddle.v2.layer.last_seq

    Get Last Timestamp Activation of a sequence.

    If stride > 0, this layer slides a window whose size is determined by stride, and return the last value of the window as the output. Thus, a long sequence @@ -1851,9 +2160,9 @@ of stride is -1.

    @@ -1873,7 +2182,7 @@ of stride is -1.

    first_seq

    -class paddle.v2.layer.first_seq(*args, **kwargs)
    +class paddle.v2.layer.first_seq

    Get First Timestamp Activation of a sequence.

    If stride > 0, this layer slides a window whose size is determined by stride, and return the first value of the window as the output. Thus, a long sequence @@ -1889,9 +2198,9 @@ of stride is -1.

    @@ -1911,9 +2220,9 @@ of stride is -1.

    concat

    -class paddle.v2.layer.concat(*args, **kwargs)
    -

    Concat all input vector into one huge vector. -Inputs can be list of paddle.v2.config_base.Layer or list of projection.

    +class paddle.v2.layer.concat +

    Concatenate all input vectors to one vector. +Inputs can be a list of paddle.v2.config_base.Layer or a list of projection.

    The example usage is:

    concat = concat(input=[layer1, layer2])
    @@ -1923,10 +2232,11 @@ Inputs can be list of paddle.v2.config_base.Layer or list of projection.

    @@ -1945,18 +2255,19 @@ Inputs can be list of paddle.v2.config_base.Layer or list of projection.

    seq_concat

    -class paddle.v2.layer.seq_concat(*args, **kwargs)
    -

    Concat sequence a with sequence b.

    +class paddle.v2.layer.seq_concat +

    Concatenate sequence a and sequence b.

    Inputs:
      -
    • a = [a1, a2, ..., an]
      • +
    • a = [a1, a2, ..., am]
    • b = [b1, b2, ..., bn]
      • -
    • Note that the length of a and b should be the same.
    -

    Output: [a1, b1, a2, b2, ..., an, bn]

    +

    Output: [a1, ..., am, b1, ..., bn]

    +

    Note that the above computation is for one sample. Multiple samples are +processed in one batch.

    The example usage is:

    concat = seq_concat(a=layer1, b=layer2)
    @@ -1966,14 +2277,102 @@ Inputs can be list of paddle.v2.config_base.Layer or list of projection.

    + + + + + + +
    参数:
      -
    • agg_level (AggregateLevel) – AggregateLevel.EACH_TIMESTEP or -AggregateLevel.EACH_SEQUENCE
      • -
    • name (basestring) – layer name.
      • -
    • input (paddle.v2.config_base.Layer) – input layer name.
      • -
    • pooling_type (BasePoolingType|None) – Type of pooling, MaxPooling(default), AvgPooling, +
    • agg_level (AggregateLevel) – AggregateLevel.TO_NO_SEQUENCE or +AggregateLevel.TO_SEQUENCE
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • pooling_type (BasePoolingType | None) – Type of pooling, MaxPooling(default), AvgPooling, SumPooling, SquareRootNPooling.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute|None|False) – Bias parameter attribute. False if no bias.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – The Extra Attributes for layer, such as dropout.
      • +
    • stride (Int) – The step size between successive pooling regions.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute | None) – The Extra Attributes for layer, such as dropout.
    参数:
    • agg_level – Aggregated level
      • -
    • name (basestring) – Layer name.
      • -
    • input (paddle.v2.config_base.Layer) – Input layer name.
      • -
    • stride (Int) – window size.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • stride (Int) – The step size between successive pooling regions.
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
    参数:
    • agg_level – aggregation level
      • -
    • name (basestring) – Layer name.
      • -
    • input (paddle.v2.config_base.Layer) – Input layer name.
      • -
    • stride (Int) – window size.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • stride (Int) – The step size between successive pooling regions.
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
    参数:
      -
    • name (basestring) – Layer name.
      • -
    • input (list|tuple|collections.Sequence) – input layers or projections
      • -
    • act (paddle.v2.Activation.Base) – Activation type.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (list | tuple | collections.Sequence) – The input layers or projections
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Identity is the default activation.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    参数:
      -
    • name (basestring) – Layer name.
      • -
    • a (paddle.v2.config_base.Layer) – input sequence layer
      • -
    • b (paddle.v2.config_base.Layer) – input sequence layer
      • -
    • act (paddle.v2.Activation.Base) – Activation type.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or -something not type of paddle.v2.attr.ParameterAttribute. None will get a -default Bias.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • a (paddle.v2.config_base.Layer) – The first input sequence layer
      • +
    • b (paddle.v2.config_base.Layer) – The second input sequence layer
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Identity is the default activation.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
      • +
    +
    返回:

    paddle.v2.config_base.Layer object.

    +
    返回类型:

    paddle.v2.config_base.Layer

    +
    +
    + +
    +
    +

    seq_slice

    +
    +
    +class paddle.v2.layer.seq_slice
    +

    seq_slice will return one or several sub-sequences from the +input sequence layer given start and end indices.

    +
    +
      +
    • If only start indices are given, and end indices are set to None, +this layer slices the input sequence from the given start indices +to its end.
      • +
    • If only end indices are given, and start indices are set to None, +this layer slices the input sequence from its beginning to the +given end indices.
      • +
    • If start and end indices are both given, they should have the same +number of elements.
      • +
    +
    +

    If start or end indices contains more than one elements, the input sequence +will be sliced for multiple times.

    +
    seq_silce = seq_slice(input=input_seq,
+                            starts=start_pos, ends=end_pos)
+
    +
    + +++ + + + + + + + +
    参数:
      +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer, which should be a sequence.
      • +
    • starts (paddle.v2.config_base.Layer | None) – The start indices to slice the input sequence.
      • +
    • ends (paddle.v2.config_base.Layer | None) – The end indices to slice the input sequence.
      • +
    +
    返回:

    paddle.v2.config_base.Layer object.

    +
    返回类型:

    paddle.v2.config_base.Layer

    +
    +
    + +
    +
    +

    kmax_sequence_score

    +
    +
    +

    sub_nested_seq

    +
    +
    +class paddle.v2.layer.sub_nested_seq
    +

    The sub_nested_seq accepts two inputs: the first one is a nested +sequence; the second one is a set of selceted indices in the nested sequence.

    +

    Then sub_nest_seq trims the first nested sequence input according +to the selected indices to form a new output. This layer is useful in +beam training.

    +

    The example usage is:

    +
    sub_nest_seq = sub_nested_seq(input=data, selected_indices=selected_ids)
+
    +
    + +++ + @@ -1995,7 +2394,7 @@ default Bias.

    block_expand

    -class paddle.v2.layer.block_expand(*args, **kwargs)
    +class paddle.v2.layer.block_expand
    Expand feature map to minibatch matrix.
      @@ -2006,11 +2405,11 @@ default Bias.
    \[ \begin{align}\begin{aligned}outputH = 1 + (2 * padding_y + imgSizeH - block_y + stride_y - 1) / stride_y\\outputW = 1 + (2 * padding_x + imgSizeW - block_x + stride_x - 1) / stride_x\end{aligned}\end{align} \]
    -

    The expand method is the same with ExpandConvLayer, but saved the transposed +

    The expanding method is the same with ExpandConvLayer, but saved the transposed value. After expanding, output.sequenceStartPositions will store timeline. -The number of time steps are outputH * outputW and the dimension of each +The number of time steps is outputH * outputW and the dimension of each time step is block_y * block_x * num_channels. This layer can be used after -convolution neural network, and before recurrent neural network.

    +convolutional neural network, and before recurrent neural network.

    The simple usage is:

    block_expand = block_expand(input=layer,
                                   num_channels=128,
@@ -2025,16 +2424,19 @@ convolution neural network, and before recurrent neural network.
    @@ -2048,18 +2450,36 @@ convolution neural network, and before recurrent neural network.

    参数:
      +
    • input (paddle.v2.config_base.Layer) – The input of this layer. It is a nested sequence.
      • +
    • selected_indices – A set of sequence indices in the nested sequence.
      • +
    • name (basestring) – The name of this layer. It is optional.
    参数:
      -
    • input (paddle.v2.config_base.Layer) – The input layer.
      • -
    • num_channels (int|None) – The channel number of input layer.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • num_channels (int) – The number of input channels. If the parameter is not set or +set to None, its actual value will be automatically set to +the channels number of the input.
    • block_x (int) – The width of sub block.
    • block_y (int) – The width of sub block.
    • stride_x (int) – The stride size in horizontal direction.
    • stride_y (int) – The stride size in vertical direction.
    • padding_x (int) – The padding size in horizontal direction.
    • padding_y (int) – The padding size in vertical direction.
      • -
    • name (None|basestring.) – The name of this layer, which can not specify.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • +
    • name (basestring.) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    +
    +
    +

    ExpandLevel

    +
    +
    +class paddle.v2.layer.ExpandLevel
    +

    Please refer to AggregateLevel first.

    +

    ExpandLevel supports two modes:

    +
      +
    • ExpandLevel.FROM_NO_SEQUENCE means the expansion acts on +NO_SEQUENCE, which will be expanded to +SEQUENCE or SUB_SEQUENCE.
      • +
    • ExpandLevel.FROM_SEQUENCE means the expansion acts on +SEQUENCE, which will be expanded to +SUB_SEQUENCE.
      • +
    +
    +
    -

    expand

    +

    expand

    -class paddle.v2.layer.expand(*args, **kwargs)
    +class paddle.v2.layer.expand

    A layer for “Expand Dense data or (sequence data where the length of each sequence is one) to sequence data.”

    The example usage is:

    expand = expand(input=layer1,
                       expand_as=layer2,
-                      expand_level=ExpandLevel.FROM_TIMESTEP)
+                      expand_level=ExpandLevel.FROM_NO_SEQUENCE)
    @@ -2067,11 +2487,12 @@ sequence is one) to sequence data.”

    @@ -2128,7 +2558,7 @@ to apply concat() with num_repeats same input.

    rotate

    -class paddle.v2.layer.rotate(*args, **kwargs)
    +class paddle.v2.layer.rotate

    A layer for rotating 90 degrees (clock-wise) for each feature channel, usually used when the input sample is some image or feature map.

    @@ -2145,10 +2575,12 @@ usually used when the input sample is some image or feature map.

    @@ -2167,7 +2599,7 @@ usually used when the input sample is some image or feature map.

    seq_reshape

    -class paddle.v2.layer.seq_reshape(*args, **kwargs)
    +class paddle.v2.layer.seq_reshape

    A layer for reshaping the sequence. Assume the input sequence has T instances, the dimension of each instance is M, and the input reshape_size is N, then the output sequence has T*M/N instances, the dimension of each instance is N.

    @@ -2181,14 +2613,15 @@ output sequence has T*M/N instances, the dimension of each instance is N.

    @@ -2210,7 +2643,7 @@ default Bias.

    addto

    -class paddle.v2.layer.addto(*args, **kwargs)
    +class paddle.v2.layer.addto

    AddtoLayer.

    \[y = f(\sum_{i} x_i + b)\]
    @@ -2218,32 +2651,30 @@ default Bias. and \(f\) is activation function.

    The example usage is:

    addto = addto(input=[layer1, layer2],
-                    act=paddle.v2.Activation.Relu(),
+                    act=paddle.v2.activation.Relu(),
                     bias_attr=False)
    -

    This layer just simply add all input layers together, then activate the sum -inputs. Each input of this layer should be the same size, which is also the -output size of this layer.

    +

    This layer just simply adds all input layers together, then activates the +sum. All inputs should share the same dimension, which is also the dimension +of this layer’s output.

    There is no weight matrix for each input, because it just a simple add operation. If you want a complicated operation before add, please use mixed.

    -

    It is a very good way to set dropout outside the layers. Since not all -PaddlePaddle layer support dropout, you can add an add_to layer, set -dropout here. -Please refer to dropout for details.

    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input layer
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
    • expand_as (paddle.v2.config_base.Layer) – Expand as this layer’s sequence info.
      • -
    • name (basestring) – Layer name.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute|None|False) – Bias attribute. None means default bias. False means no -bias.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
    • expand_level (ExpandLevel) – whether input layer is timestep(default) or sequence.
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
    @@ -2092,11 +2513,14 @@ bias.

    repeat

    -class paddle.v2.layer.repeat(*args, **kwargs)
    -

    A layer for repeating the input for num_repeats times. This is equivalent -to apply concat() with num_repeats same input.

    +class paddle.v2.layer.repeat +

    A layer for repeating the input for num_repeats times.

    +

    If as_row_vector:

    -\[y = [x, x, \cdots, x]\]
    +\[y = [x_1,\cdots, x_n, \cdots, x_1, \cdots, x_n]\] +

    If not as_row_vector:

    +
    +\[y = [x_1,\cdots, x_1, \cdots, x_n, \cdots, x_n]\]

    The example usage is:

    expand = repeat(input=layer, num_repeats=4)
    @@ -2106,10 +2530,16 @@ to apply concat() with num_repeats same input.
    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input layer
      • -
    • num_repeats (int) – Repeat the input so many times
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • num_repeats (int) – The times of repeating the input.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • as_row_vector (bool) – Whether to treat the input as row vectors or not. If +the parameter is set to True, the repeating operation +will be performed in the column direction. Otherwise, +it will be performed in the row direction.
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Identity is the default activation.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • -
    • height (int) – The height of the sample matrix
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • height (int) – The height of the sample matrix.
      • +
    • width (int) – The width of the sample matrix.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • -
    • reshape_size (int) – the size of reshaped sequence.
      • -
    • name (basestring) – Layer name.
      • -
    • act (paddle.v2.Activation.Base) – Activation type.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute or None or bool) – The Bias Attribute. If no bias, then pass False or -something not type of paddle.v2.attr.ParameterAttribute. None will get a -default Bias.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • reshape_size (int) – The dimension of the reshaped sequence.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Identity is the default activation.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
    @@ -2262,7 +2693,7 @@ bias.

    linear_comb

    -class paddle.v2.layer.linear_comb(*args, **kwargs)
    +class paddle.v2.layer.linear_comb
    A layer for weighted sum of vectors takes two inputs.
      @@ -2304,9 +2735,10 @@ processed in one batch.

    @@ -2325,8 +2757,8 @@ processed in one batch.

    interpolation

    -class paddle.v2.layer.interpolation(*args, **kwargs)
    -

    This layer is for linear interpolation with two inputs, +class paddle.v2.layer.interpolation +

    This layer performs linear interpolation on two inputs, which is used in NEURAL TURING MACHINE.

    \[y.row[i] = w[i] * x_1.row[i] + (1 - w[i]) * x_2.row[i]\]
    @@ -2342,10 +2774,11 @@ which is used in NEURAL TURING MACHINE.

    @@ -2364,8 +2797,8 @@ which is used in NEURAL TURING MACHINE.

    bilinear_interp

    -class paddle.v2.layer.bilinear_interp(*args, **kwargs)
    -

    This layer is to implement bilinear interpolation on conv layer output.

    +class paddle.v2.layer.bilinear_interp +

    This layer implements bilinear interpolation on convolutional layer’s output.

    Please refer to Wikipedia: https://en.wikipedia.org/wiki/Bilinear_interpolation

    The simple usage is:

    bilinear = bilinear_interp(input=layer1, out_size_x=64, out_size_y=64)
@@ -2376,11 +2809,12 @@ which is used in NEURAL TURING MACHINE.
    @@ -2395,17 +2829,86 @@ which is used in NEURAL TURING MACHINE.

    -
    -

    power

    +
    +

    dot_prod

    -class paddle.v2.layer.power(*args, **kwargs)
    +class paddle.v2.layer.dot_prod +

    A layer for computing the dot product of two vectors.

    +

    The example usage is:

    +
    dot_prod = dot_prod(input1=vec1, input2=vec2)
+
    +
    +
    参数:
      -
    • name (basestring) – Layer name.
      • -
    • input (paddle.v2.config_base.Layer|list|tuple) – Input layers. It could be a paddle.v2.config_base.Layer or list/tuple of +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer | list | tuple) – The input layers. It could be a paddle.v2.config_base.Layer or list/tuple of paddle.v2.config_base.Layer.
      • -
    • act (paddle.v2.Activation.Base) – Activation Type, default is tanh.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute|bool) – Bias attribute. If False, means no bias. None is default -bias.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer attribute.
      • +
    • act (paddle.v2.activation.Base) – Activation Type. paddle.v2.activation.Linear is the default activation.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    参数:
    • weights (paddle.v2.config_base.Layer) – The weight layer.
    • vectors (paddle.v2.config_base.Layer) – The vector layer.
      • -
    • size (int) – the dimension of this layer.
      • -
    • name (basestring) – The Layer Name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • +
    • size (int) – The dimension of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    参数:
      -
    • input (list|tuple) – Input layer.
      • +
    • input (list | tuple) – The input of this layer.
    • weight (paddle.v2.config_base.Layer) – Weight layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    参数:
      -
    • input (paddle.v2.config_base.Layer.) – A input layer.
      • -
    • out_size_x (int|None) – bilinear interpolation output width.
      • -
    • out_size_y (int|None) – bilinear interpolation output height.
      • -
    • name (None|basestring) – The layer’s name, which cna not be specified.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer attribute.
      • +
    • input (paddle.v2.config_base.Layer.) – The input of this layer.
      • +
    • out_size_x (int) – The width of the output.
      • +
    • out_size_y (int) – The height of the output.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    +++ + + + + + + + +
    参数:
      +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input1 (paddle.v2.config_base.Layer) – The first input layer.
      • +
    • input2 (paddle.v2.config_base.Layer) – The second input layer.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    +
    返回:

    paddle.v2.config_base.Layer object.

    +
    返回类型:

    paddle.v2.config_base.Layer

    +
    +
    + +
    +
    +

    out_prod

    +
    +
    +class paddle.v2.layer.out_prod
    +

    A layer for computing the outer product of two vectors +The result is a matrix of size(input1) x size(input2)

    +

    The example usage is:

    +
    out_prod = out_prod(input1=vec1, input2=vec2)
+
    +
    + +++ + + + + + + + +
    参数:
      +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input1 – The first input layer.
      • +
    • input2 (paddle.v2.config_base.Layer) – The second input layer.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    +
    返回:

    paddle.v2.config_base.Layer object.

    +
    返回类型:

    paddle.v2.config_base.Layer

    +
    +
    + +
    +
    +

    power

    +
    +
    +class paddle.v2.layer.power

    This layer applies a power function to a vector element-wise, which is used in NEURAL TURING MACHINE.

    \[y = x^w\]
    -

    where \(x\) is a input vector, \(w\) is scalar weight, -and \(y\) is a output vector.

    +

    where \(x\) is an input vector, \(w\) is a scalar exponent, +and \(y\) is an output vector.

    The example usage is:

    power = power(input=layer1, weight=layer2)
    @@ -2415,10 +2918,11 @@ and \(y\) is a output vector.

    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • -
    • weight (paddle.v2.config_base.Layer) – Weight layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • weight (paddle.v2.config_base.Layer) – The exponent of the power.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    @@ -2437,7 +2941,7 @@ and \(y\) is a output vector.

    scaling

    -class paddle.v2.layer.scaling(*args, **kwargs)
    +class paddle.v2.layer.scaling

    A layer for multiplying input vector by weight scalar.

    \[y = w x\]
    @@ -2454,10 +2958,11 @@ processed in one batch.

    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • -
    • weight (paddle.v2.config_base.Layer) – Weight layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • weight (paddle.v2.config_base.Layer) – The weight of each sample.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    @@ -2471,14 +2976,89 @@ processed in one batch.

    +
    +
    +

    clip

    +
    +
    +class paddle.v2.layer.clip
    +
    +

    A layer for clipping the input value by the threshold.

    +
    +\[out[i] = \min\left(\max\left(in[i],p_{1}\]
    +
    +

    ight),p_{2} +ight)

    +
    +
    clip = clip(input=input, min=-10, max=10)
+
    +
    + +++ + + + + + + + + + + + + + + + + + + + + + +
    param name:The name of this layer. It is optional.
    type name:basestring
    param input:The input of this layer.
    type input:paddle.v2.config_base.Layer.
    param min:The lower threshold for clipping.
    type min:float
    param max:The upper threshold for clipping.
    type max:float
    return:paddle.v2.config_base.Layer object.
    rtype:paddle.v2.config_base.Layer
    +
    +
    + +
    +
    +

    resize

    +
    +
    +class paddle.v2.layer.resize
    +

    The resize layer resizes the input matrix with a shape of [Height, Width] +into the output matrix with a shape of [Height x Width / size, size], +where size is the parameter of this layer indicating the output dimension.

    + +++ + + + + + + + +
    参数:
      +
    • input (paddle.v2.config_base.Layer.) – The input of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • size (int) – The resized output dimension of this layer.
      • +
    +
    返回:

    A paddle.v2.config_base.Layer object.

    +
    返回类型:

    paddle.v2.config_base.Layer

    +
    +
    +

    slope_intercept

    -class paddle.v2.layer.slope_intercept(*args, **kwargs)
    -

    This layer for applying a slope and an intercept to the input -element-wise. There is no activation and weight.

    +class paddle.v2.layer.slope_intercept +

    This layer for applying a slope and an intercept to the input.

    \[y = slope * x + intercept\]

    The simple usage is:

    @@ -2490,11 +3070,12 @@ element-wise. There is no activation and weight.

    参数:
      -
    • input (paddle.v2.config_base.Layer) – The input layer.
      • -
    • name (basestring) – The Layer Name.
      • -
    • slope (float.) – the scale factor.
      • -
    • intercept (float.) – the offset.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • slope (float) – The scale factor.
      • +
    • intercept (float) – The offset.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    @@ -2513,9 +3094,9 @@ element-wise. There is no activation and weight.

    tensor

    -class paddle.v2.layer.tensor(*args, **kwargs)
    -

    This layer performs tensor operation for two input. -For example, each sample:

    +class paddle.v2.layer.tensor +

    This layer performs tensor operation on two inputs. +For example:

    \[y_{i} = a * W_{i} * {b^\mathrm{T}}, i=0,1,...,K-1\]
    @@ -2538,16 +3119,19 @@ For example, each sample:

    参数:
      -
    • name (basestring) – layer name
      • -
    • a (paddle.v2.config_base.Layer) – Input layer a.
      • -
    • b (paddle.v2.config_base.Layer) – input layer b.
      • -
    • size (int.) – the layer dimension.
      • -
    • act (paddle.v2.Activation.Base) – Activation Type. Default is tanh.
      • -
    • param_attr (paddle.v2.attr.ParameterAttribute) – The Parameter Attribute.
      • -
    • bias_attr (paddle.v2.attr.ParameterAttribute|None|Any) – The Bias Attribute. If no bias, then pass False or -something not type of paddle.v2.attr.ParameterAttribute. None will get a -default Bias.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • a (paddle.v2.config_base.Layer) – The first input of this layer.
      • +
    • b (paddle.v2.config_base.Layer) – The second input of this layer.
      • +
    • size (int) – The dimension of this layer.
      • +
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Linear is the default activation.
      • +
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for +details.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The parameter attribute for bias. If this parameter is set to +False or an object whose type is not paddle.v2.attr.ParameterAttribute, +no bias is defined. If this parameter is set to True, +the bias is initialized to zero.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute | None) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    @@ -2566,7 +3150,7 @@ default Bias.

    cos_sim

    -class paddle.v2.layer.cos_sim(*args, **kwargs)
    +class paddle.v2.layer.cos_sim

    Cosine Similarity Layer. The cosine similarity equation is here.

    \[similarity = cos(\theta) = {\mathbf{a} \cdot \mathbf{b} @@ -2585,12 +3169,12 @@ processed in one batch.

    参数:
      -
    • name (basestring) – layer name
      • -
    • a (paddle.v2.config_base.Layer) – input layer a
      • -
    • b (paddle.v2.config_base.Layer) – input layer b
      • -
    • scale (float) – scale for cosine value. default is 5.
      • -
    • size (int) – layer size. NOTE size_a * size should equal size_b.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • a (paddle.v2.config_base.Layer) – The first input of this layer.
      • +
    • b (paddle.v2.config_base.Layer) – The second input of this layer.
      • +
    • scale (float) – The scale of the cosine similarity. 1 is the default value.
      • +
    • size (int) – The dimension of this layer. NOTE size_a * size should equal size_b.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for details.
    @@ -2604,12 +3188,55 @@ processed in one batch.

    +
    +
    +

    l2_distance

    +
    +
    +class paddle.v2.layer.l2_distance
    +

    This layer calculates and returns the Euclidean distance between two input +vectors x and y. The equation is as follows:

    +
    +\[l2_distance(\mathbf{x}, \mathbf{y}) = \sqrt{\sum_{i=1}^D(x_i - y_i)}\]
    +

    The output size of this layer is fixed to be 1. Note that the above +computation is for one sample. Multiple samples are processed in one batch.

    +

    The example usage is:

    +
    l2_sim = l2_distance(x=layer1, y=layer2)
+
    +
    + +++ + + + + + + + +
    参数:
      +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • x (paddle.v2.config_base.Layer) – The first input x for this layer, whose output is a matrix with +dimensionality N x D. N is the sample number in a mini-batch. +D is the dimensionality of x’s output.
      • +
    • y (paddle.v2.config_base.Layer) – The second input y for this layer, whose output is a matrix with +dimensionality N x D. N is the sample number in a mini-batch. +D is the dimensionality of y’s output.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attributes, for example, drop rate. +See paddle.v2.attr.ExtraAttribute for more details.
      • +
    +
    返回:

    The returned paddle.v2.config_base.Layer object.

    +
    返回类型:

    paddle.v2.config_base.Layer

    +
    +
    +

    trans

    -class paddle.v2.layer.trans(*args, **kwargs)
    +class paddle.v2.layer.trans

    A layer for transposing a minibatch matrix.

    \[y = x^\mathrm{T}\]
    @@ -2623,9 +3250,51 @@ processed in one batch.

    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input layer.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    + + +返回:

    paddle.v2.config_base.Layer object.

    + + +返回类型:

    paddle.v2.config_base.Layer

    + + + + +
    + +
    +
    +

    scale_shift

    +
    +
    +class paddle.v2.layer.scale_shift
    +

    A layer applies a linear transformation to each element in each row of +the input matrix. For each element, the layer first re-scales it and then +adds a bias to it.

    +

    This layer is very like the SlopeInterceptLayer, except the scale and +bias are trainable.

    +
    +\[y = w * x + b\]
    +
    scale_shift = scale_shift(input=input, bias_attr=False)
+
    +
    + +++ + @@ -2647,7 +3316,7 @@ processed in one batch.

    maxid

    -class paddle.v2.layer.max_id(*args, **kwargs)
    +class paddle.v2.layer.max_id

    A layer for finding the id which has the maximal value for each sample. The result is stored in output.ids.

    The example usage is:

    @@ -2659,9 +3328,10 @@ The result is stored in output.ids.

    @@ -2680,8 +3350,8 @@ The result is stored in output.ids.

    sampling_id

    -class paddle.v2.layer.sampling_id(*args, **kwargs)
    -

    A layer for sampling id from multinomial distribution from the input layer. +class paddle.v2.layer.sampling_id +

    A layer for sampling id from a multinomial distribution from the input layer. Sampling one id for one sample.

    The simple usage is:

    samping_id = sampling_id(input=input)
@@ -2692,9 +3362,10 @@ Sampling one id for one sample.
    @@ -2708,6 +3379,122 @@ Sampling one id for one sample.

    参数:
      +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute of scaling. See paddle.v2.attr.ParameterAttribute for +details.
      • +
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object +whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the +parameter is set to True, the bias is initialized to zero.
    参数:
      -
    • input (paddle.v2.config_base.Layer) – Input layer name.
      • -
    • name (basestring) – Layer name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    参数:
      -
    • input (paddle.v2.config_base.Layer) – The input layer.
      • -
    • name (basestring) – The Layer Name.
      • -
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • +
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
    +
    +
    +

    multiplex

    +
    +
    +class paddle.v2.layer.multiplex
    +

    This layer multiplex multiple layers according to the indexes, +which are provided by the first input layer. +inputs[0]: the indexes of the layers to form the output of size batchSize. +inputs[1:N]; the candidate output data. +For each index i from 0 to batchSize - 1, the i-th row of the output is the +the same to the i-th row of the (index[i] + 1)-th layer.

    +

    For each i-th row of output: +.. math:

    +
    y[i][j] = x_{x_{0}[i] + 1}[i][j], j = 0,1, ... , (x_{1}.width - 1)
+
    +
    +

    where, y is output. \(x_{k}\) is the k-th input layer and +\(k = x_{0}[i] + 1\).

    +

    The example usage is:

    +
    maxid = multiplex(input=layers)
+
    +
    + +++ + + + + + + + +
    参数:
      +
    • input (list of paddle.v2.config_base.Layer) – Input layers.
      • +
    • name (basestring) – The name of this layer. It is optional.
      • +
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for +details.
      • +
    +
    返回:

    paddle.v2.config_base.Layer object.

    +
    返回类型:

    paddle.v2.config_base.Layer

    +
    +
    + +
    +
    +
    +

    Factorization Machine Layer

    +
    +

    factorization_machine

    +
    +
    +class paddle.v2.layer.factorization_machine
    +
    +

    The Factorization Machine models pairwise feature interactions as inner +product of the learned latent vectors corresponding to each input feature. +The Factorization Machine can effectively capture feature interactions +especially when the input is sparse.

    +

    This implementation only consider the 2-order feature interactions using +Factorization Machine with the formula:

    +
    +\[y = \sum_{i=1}^{n-1}\sum_{j=i+1}^n\langle v_i, v_j\]
    +
    +

    angle x_i x_j

    +
    +
    +
    Note:
    +
    X is the input vector with size n. V is the factor matrix. Each row of V +is the latent vector corresponding to each input dimesion. The size of +each latent vector is k.
    +
    +

    For details of Factorization Machine, please refer to the paper: +Factorization machines.

    + +++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    param input:The input layer. Supported input types: all input data types +on CPU, and only dense input types on GPU.
    type input:paddle.v2.config_base.Layer
    param factor_size:
     The hyperparameter that defines the dimensionality of +the latent vector size.
    type context_len:
     int
    param act:Activation Type. Default is linear activation.
    type act:paddle.v2.activation.Base
    param param_attr:
     The parameter attribute. See paddle.v2.attr.ParameterAttribute for +details.
    type param_attr:
     paddle.v2.attr.ParameterAttribute
    param layer_attr:
     Extra Layer config.
    type layer_attr:
     paddle.v2.attr.ExtraAttributeNone
    return:paddle.v2.config_base.Layer object.
    rtype:paddle.v2.config_base.Layer
    +
    +
    +
    @@ -2716,14 +3503,14 @@ Sampling one id for one sample.

    pad

    -class paddle.v2.layer.pad(*args, **kwargs)
    +class paddle.v2.layer.pad

    This operation pads zeros to the input data according to pad_c,pad_h -and pad_w. pad_c, pad_h, pad_w specifies the which dimension and size -of padding. And the input data shape is NCHW.

    -

    For example, pad_c=[2,3] means padding 2 zeros before the -input data and 3 zeros after the input data in channel dimension. -pad_h means padding zeros in height dimension. pad_w means padding zeros -in width dimension.

    +and pad_w. pad_c, pad_h, pad_w specify the size in the corresponding +dimension. And the input data shape is NCHW.

    +

    For example, pad_c=[2,3] means padding 2 zeros before the input data +and 3 zeros after the input data in the channel dimension. pad_h means +padding zeros in the height dimension. pad_w means padding zeros in the +width dimension.

    For example,

    input(2,2,2,3)  = [
                     [ [[1,2,3], [3,4,5]],
@@ -2758,12 +3545,13 @@ in width dimension.
    
 
 
 参数:
      
-
    • input (paddle.v2.config_base.Layer) – layer’s input.
      • 
-
    • pad_c (list|None) – padding size in channel dimension.
      • 
-
    • pad_h (list|None) – padding size in height dimension.
      • 
-
    • pad_w (list|None) – padding size in width dimension.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
-
    • name (basestring) – layer name.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • pad_c (list | None) – The padding size in the channel dimension.
      • 
+
    • pad_h (list | None) – The padding size in the height dimension.
      • 
+
    • pad_w (list | None) – The padding size in the width dimension.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
 
    
 
 
@@ -2785,8 +3573,9 @@ in width dimension.
    
 
    cross_entropy_cost
    
 
    
 
    
-class paddle.v2.layer.cross_entropy_cost(*args, **kwargs)
    
+class paddle.v2.layer.cross_entropy_cost
 
    A loss layer for multi class entropy.
    
+
    The example usage is:
    
 
    cost = cross_entropy(input=input,
                      label=label)
 
    
@@ -2796,22 +3585,22 @@ in width dimension.
    
 
 
 参数:
      
-
    • input (paddle.v2.config_base.Layer.) – The first input layer.
      • 
+
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
 
    • label – The input label.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float.) – The cost is multiplied with coeff.
-The coefficient affects the gradient in the backward.
      • 
-
    • weight (LayerOutout) – The cost of each sample is multiplied with each weight.
-The weight should be a layer with size=1. Note that gradient
-will not be calculated for weight.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • weight (LayerOutout) – The weight layer defines a weight for each sample in the
+mini-batch. It is optional.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
 返回:
    paddle.v2.config_base.Layer object.
    
 
 
-返回类型:
    paddle.v2.config_base.Layer.
    
+返回类型:
    paddle.v2.config_base.Layer
    
 
 
 
@@ -2823,9 +3612,10 @@ will not be calculated for weight.
 
    cross_entropy_with_selfnorm_cost
    
 
    
 
    
-class paddle.v2.layer.cross_entropy_with_selfnorm_cost(*args, **kwargs)
    
+class paddle.v2.layer.cross_entropy_with_selfnorm_cost
 
    A loss layer for multi class entropy with selfnorm.
 Input should be a vector of positive numbers, without normalization.
    
+
    The example usage is:
    
 
    cost = cross_entropy_with_selfnorm(input=input,
                                    label=label)
 
    
@@ -2835,19 +3625,21 @@ Input should be a vector of positive numbers, without normalization.
    
 
 
 参数:
      
-
    • input (paddle.v2.config_base.Layer.) – The first input layer.
      • 
+
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
 
    • label – The input label.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float.) – The coefficient affects the gradient in the backward.
      • 
-
    • softmax_selfnorm_alpha (float.) – The scale factor affects the cost.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • softmax_selfnorm_alpha (float) – The scale factor affects the cost.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
 返回:
    paddle.v2.config_base.Layer object.
    
 
 
-返回类型:
    paddle.v2.config_base.Layer.
    
+返回类型:
    paddle.v2.config_base.Layer
    
 
 
 
@@ -2859,8 +3651,9 @@ Input should be a vector of positive numbers, without normalization.
    
 
    multi_binary_label_cross_entropy_cost
    
 
    
 
    
-class paddle.v2.layer.multi_binary_label_cross_entropy_cost(*args, **kwargs)
    
+class paddle.v2.layer.multi_binary_label_cross_entropy_cost
 
    A loss layer for multi binary label cross entropy.
    
+
    The example usage is:
    
 
    cost = multi_binary_label_cross_entropy(input=input,
                                         label=label)
 
    
@@ -2872,10 +3665,11 @@ Input should be a vector of positive numbers, without normalization.
    
 参数:
      
 
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
 
    • label – The input label.
      • 
-
    • type (basestring) – The type of cost.
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float) – The coefficient affects the gradient in the backward.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -2890,37 +3684,123 @@ Input should be a vector of positive numbers, without normalization.
    
 
    
 
 
    
-
    
-
    huber_cost
    
+
    
+
    huber_regression_cost
    
 
    
 
    
-class paddle.v2.layer.huber_cost(*args, **kwargs)
    
-
    A loss layer for huber loss.
    
-
    cost = huber_cost(input=input,
-                  label=label)
+class paddle.v2.layer.huber_regression_cost
+
    
+
    In statistics, the Huber loss is a loss function used in robust regression,
+that is less sensitive to outliers in data than the squared error loss.
+Given a prediction f(x), a label y and \(\delta\), the loss function
+is defined as:
    
+
    ight )^2, left | y-f(x)
+ight |leq delta
    
+
    
+
    loss = delta left | y-f(x)
    
+
    ight |-0.5delta ^2, otherwise
    
+
    
+
    The example usage is:
    
+
    cost = huber_regression_cost(input=input, label=label)
 
    
 
    
 
-
+
-
+
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
    
 
 
 
    参数:
      
-
    • input (paddle.v2.config_base.Layer.) – The first input layer.
      • 
-
    • label – The input label.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • coeff (float.) – The coefficient affects the gradient in the backward.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
-
    
-
    param input:The first input layer.
 
    返回:
    paddle.v2.config_base.Layer object.
    
-
    type input:paddle.v2.config_base.Layer
 
    返回类型:
    paddle.v2.config_base.Layer.
    
-
    param label:The input label.
    type input:paddle.v2.config_base.Layer
    param name:The name of this layer. It is optional.
    type name:basestring
    param delta:The difference between the observed and predicted values.
    type delta:float
    param coeff:The weight of the gradient in the back propagation.
+1.0 is the default value.
    type coeff:float
    param layer_attr:
     The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
    type layer_attr:
     paddle.v2.attr.ExtraAttribute
    return:paddle.v2.config_base.Layer object.
    rtype:paddle.v2.config_base.Layer.
 
    
     
 
    
+
    
+
    
+
+
    
+
    
+
    huber_classification_cost
    
+
    
+
    
+class paddle.v2.layer.huber_classification_cost
    
+
    
+
    For classification purposes, a variant of the Huber loss called modified Huber
+is sometimes used. Given a prediction f(x) (a real-valued classifier score) and
+a true binary class label :math:`yin left {-1, 1
    
+
    
+
    ight }`, the modified Huber
    
+
    loss is defined as:
    
+
    ight )^2, yf(x)geq 1
    
+
    
+
    loss = -4yf(x),  ext{otherwise}
    
+
    The example usage is:
    
+
    cost = huber_classification_cost(input=input, label=label)
+
    
+
    
+
+++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
    


    param input:The first input layer.
    type input:paddle.v2.config_base.Layer
    param label:The input label.
    type input:paddle.v2.config_base.Layer
    param name:The name of this layer. It is optional.
    type name:basestring
    param coeff:The weight of the gradient in the back propagation.
+1.0 is the default value.
    type coeff:float
    param layer_attr:
     The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
    type layer_attr:
     paddle.v2.attr.ExtraAttribute
    return:paddle.v2.config_base.Layer object.
    rtype:paddle.v2.config_base.Layer
    
+
    
+
    
 
    
 
 
    
@@ -2928,9 +3808,9 @@ Input should be a vector of positive numbers, without normalization.
    
 
    lambda_cost
    
 
    
 
    
-class paddle.v2.layer.lambda_cost(*args, **kwargs)
    
+class paddle.v2.layer.lambda_cost
 
    lambdaCost for lambdaRank LTR approach.
    
-
    The simple usage:
    
+
    The example usage is:
    
 
    cost = lambda_cost(input=input,
                    score=score,
                    NDCG_num=8,
@@ -2942,20 +3822,20 @@ Input should be a vector of positive numbers, without normalization.
    
 
 
 参数:
      
-
    • input (paddle.v2.config_base.Layer) – Samples of the same query should be loaded as sequence.
      • 
-
    • score – The 2nd input. Score of each sample.
      • 
+
    • input (paddle.v2.config_base.Layer) – The first input of this layer, which is often a document
+samples list of the same query and whose type must be sequence.
      • 
+
    • score – The scores of the samples.
      • 
 
    • NDCG_num (int) – The size of NDCG (Normalized Discounted Cumulative Gain),
-e.g., 5 for NDCG@5. It must be less than for equal to the
-minimum size of lists.
      • 
-
    • max_sort_size (int) – The size of partial sorting in calculating gradient.
-If max_sort_size = -1, then for each list, the
-algorithm will sort the entire list to get gradient.
-In other cases, max_sort_size must be greater than or
-equal to NDCG_num. And if max_sort_size is greater
-than the size of a list, the algorithm will sort the
-entire list of get gradient.
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+e.g., 5 for NDCG@5. It must be less than or equal to the
+minimum size of the list.
+
    • max_sort_size (int) – The size of partial sorting in calculating gradient. If
+max_sort_size is equal to -1 or greater than the number
+of the samples in the list, then the algorithm will sort
+the entire list to compute the gradient. In other cases,
+max_sort_size must be greater than or equal to NDCG_num.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -2970,52 +3850,39 @@ entire list of get gradient.
 
    
 
 
    
-
    
-
    mse_cost
    
+
    
+
    square_error_cost
    
 
    
 
    
-class paddle.v2.layer.mse_cost(*args, **kwargs)
    
-
    
-
    mean squared error cost:
    
+class paddle.v2.layer.square_error_cost
+
    sum of square error cost:
    
 
    
-\[\]
    
-
    
-
    rac{1}{N}sum_{i=1}^N(t_i-y_i)^2
    
-
    
-
    
+\[cost = \sum_{i=1}^N(t_i-y_i)^2\]
+
    
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
+
-
+
    
 
 
 
    param name:layer name.
    type name:basestring
    param input:Network prediction.
    type input:paddle.v2.config_base.Layer
    param label:Data label.
    type label:paddle.v2.config_base.Layer
    param weight:The weight affects the cost, namely the scale of cost.
-It is an optional argument.
    type weight:paddle.v2.config_base.Layer
    param layer_attr:
     layer’s extra attribute.
    type layer_attr:
     paddle.v2.attr.ExtraAttribute
    参数:
      
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • weight (paddle.v2.config_base.Layer) – The weight layer defines a weight for each sample in the
+mini-batch. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    
+    		
 
    return:paddle.v2.config_base.Layer object.
    返回:
    paddle.v2.config_base.Layer object.
    
+    		
 
    rtype:paddle.v2.config_base.Layer
    返回类型:
    paddle.v2.config_base.Layer
    
+    		
 
    
     
 
    
-
    
 
    
 
 
    
@@ -3023,11 +3890,13 @@ It is an optional argument.
 
    rank_cost
    
 
    
 
    
-class paddle.v2.layer.rank_cost(*args, **kwargs)
    
-
    A cost Layer for learning to rank using gradient descent. Details can refer
-to papers.
-This layer contains at least three inputs. The weight is an optional
-argument, which affects the cost.
    
+class paddle.v2.layer.rank_cost
+
    A cost Layer for learning to rank using gradient descent.
    
+
    
+
    Reference:
    
+
    `Learning to Rank using Gradient Descent
+http://research.microsoft.com/en-us/um/people/cburges/papers/ICML_ranking.pdf`_
    
+
    
 
    
 \[ \begin{align}\begin{aligned}C_{i,j} & = -\tilde{P_{ij}} * o_{i,j} + log(1 + e^{o_{i,j}})\\o_{i,j} & =  o_i - o_j\\\tilde{P_{i,j}} & = \{0, 0.5, 1\} \ or \ \{0, 1\}\end{aligned}\end{align} \]
    
 
    
@@ -3041,7 +3910,7 @@ Their dimension is one.
 
 
    
 
    
-
    The simple usage:
    
+
    The example usage is:
    
 
    cost = rank_cost(left=out_left,
                  right=out_right,
                  label=label)
@@ -3055,11 +3924,13 @@ Their dimension is one.
 
  • left (paddle.v2.config_base.Layer) – The first input, the size of this layer is 1.
    • 
 
  • right (paddle.v2.config_base.Layer) – The right input, the size of this layer is 1.
    • 
 
  • label (paddle.v2.config_base.Layer) – Label is 1 or 0, means positive order and reverse order.
    • 
-
  • weight (paddle.v2.config_base.Layer) – The weight affects the cost, namely the scale of cost.
-It is an optional argument.
    • 
-
  • name (None|basestring) – The name of this layers. It is not necessary.
    • 
-
  • coeff (float) – The coefficient affects the gradient in the backward.
    • 
-
  • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
    • 
+
  • weight (paddle.v2.config_base.Layer) – The weight layer defines a weight for each sample in the
+mini-batch. It is optional.
    • 
+
  • name (basestring) – The name of this layer. It is optional.
    • 
+
  • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
    • 
+
  • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
    • 
 
 
 
@@ -3078,8 +3949,9 @@ It is an optional argument.
 
    sum_cost
    
 
    
 
    
-class paddle.v2.layer.sum_cost(*args, **kwargs)
    
-
    A loss layer which calculate the sum of the input as loss
    
+class paddle.v2.layer.sum_cost
+
    A loss layer which calculates the sum of the input as loss.
    
+
    The example usage is:
    
 
    cost = sum_cost(input=input)
 
    
 
    
@@ -3088,9 +3960,10 @@ It is an optional argument.
 
 
 参数:
      
-
    • input (paddle.v2.config_base.Layer.) – The first input layer.
      • 
-
    • name (None|basestring.) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -3109,10 +3982,10 @@ It is an optional argument.
 
    crf
    
 
    
 
    
-class paddle.v2.layer.crf(*args, **kwargs)
    
+class paddle.v2.layer.crf
 
    A layer for calculating the cost of sequential conditional random
 field model.
    
-
    The simple usage:
    
+
    The example usage is:
    
 
    crf = crf(input=input,
                 label=label,
                 size=label_dim)
@@ -3123,14 +3996,18 @@ field model.
    
 
 
 参数:
      
-
    • input (paddle.v2.config_base.Layer) – The first input layer is the feature.
      • 
-
    • label (paddle.v2.config_base.Layer) – The second input layer is label.
      • 
+
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
 
    • size (int) – The category number.
      • 
-
    • weight (paddle.v2.config_base.Layer) – The third layer is “weight” of each sample, which is an
-optional argument.
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter attribute. None means default attribute
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • 
+
    • weight (paddle.v2.config_base.Layer) – The weight layer defines a weight for each sample in the
+mini-batch. It is optional.
      • 
+
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for
+details.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -3149,13 +4026,13 @@ optional argument.
 
    crf_decoding
    
 
    
 
    
-class paddle.v2.layer.crf_decoding(*args, **kwargs)
    
+class paddle.v2.layer.crf_decoding
 
    A layer for calculating the decoding sequence of sequential conditional
 random field model. The decoding sequence is stored in output.ids.
-If a second input is provided, it is treated as the ground-truth label, and
-this layer will also calculate error. output.value[i] is 1 for incorrect
-decoding or 0 for correct decoding.
    
-
    The simple usage:
    
+If the input ‘label’ is provided, it is treated as the ground-truth label, and
+this layer will also calculate error. output.value[i] is 1 for an incorrect
+decoding and 0 for the correct.
    
+
    The example usage is:
    
 
    crf_decoding = crf_decoding(input=input,
                                   size=label_dim)
 
    
@@ -3166,11 +4043,13 @@ decoding or 0 for correct decoding.
    
 
 参数:
      
 
    • input (paddle.v2.config_base.Layer) – The first input layer.
      • 
-
    • size (int) – size of this layer.
      • 
-
    • label (paddle.v2.config_base.Layer or None) – None or ground-truth label.
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter attribute. None means default attribute
      • 
-
    • name (None|basestring) – The name of this layers. It is not necessary.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • 
+
    • size (int) – The dimension of this layer.
      • 
+
    • label (paddle.v2.config_base.Layer | None) – The input label.
      • 
+
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for
+details.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -3189,22 +4068,25 @@ decoding or 0 for correct decoding.
    
 
    ctc
    
 
    
 
    
-class paddle.v2.layer.ctc(*args, **kwargs)
    
+class paddle.v2.layer.ctc
 
    Connectionist Temporal Classification (CTC) is designed for temporal
-classication task. That is, for sequence labeling problems where the
+classication task. e.g. sequence labeling problems where the
 alignment between the inputs and the target labels is unknown.
    
-
    More details can be found by referring to Connectionist Temporal
-Classification: Labelling Unsegmented Sequence Data with Recurrent
-Neural Networks
    
+
    
+
    Reference:
    
+
    `Connectionist Temporal Classification: Labelling Unsegmented Sequence Data
+with Recurrent Neural Networks
+http://machinelearning.wustl.edu/mlpapers/paper_files/icml2006_GravesFGS06.pdf`_
    
+
    
 
    
 
    注解
    
-
    Considering the ‘blank’ label needed by CTC, you need to use
-(num_classes + 1) as the input size. num_classes is the category number.
-And the ‘blank’ is the last category index. So the size of ‘input’ layer, such as
-fc with softmax activation, should be num_classes + 1. The size of ctc
-should also be num_classes + 1.
    
+
    Considering the ‘blank’ label needed by CTC, you need to use (num_classes + 1)
+as the size of the input, where num_classes is the category number.
+And the ‘blank’ is the last category index. So the size of ‘input’ layer (e.g.
+fc with softmax activation) should be (num_classes + 1). The size of
+ctc should also be (num_classes + 1).
    
 
    
-
    The simple usage:
    
+
    The example usage is:
    
 
    ctc = ctc(input=input,
                 label=label,
                 size=9055,
@@ -3216,12 +4098,13 @@ should also be num_classes + 1.
    
 
 
 参数:
      
-
    • input (paddle.v2.config_base.Layer) – The input layer.
      • 
-
    • label (paddle.v2.config_base.Layer) – The data layer of label with variable length.
      • 
-
    • size (int) – category numbers + 1.
      • 
-
    • name (basestring|None) – The name of this layer
      • 
-
    • norm_by_times (bool) – Whether to normalization by times. False by default.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • size (int) – The dimension of this layer, which must be equal to (category number + 1).
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • norm_by_times (bool) – Whether to do normalization by times. False is the default.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -3240,29 +4123,32 @@ should also be num_classes + 1.
    
 
    warp_ctc
    
 
    
 
    
-class paddle.v2.layer.warp_ctc(*args, **kwargs)
    
-
    A layer intergrating the open-source warp-ctc
-<https://github.com/baidu-research/warp-ctc> library, which is used in
-Deep Speech 2: End-toEnd Speech Recognition in English and Mandarin
-<https://arxiv.org/pdf/1512.02595v1.pdf>, to compute Connectionist Temporal
-Classification (CTC) loss.
    
-
    More details of CTC can be found by referring to Connectionist Temporal
-Classification: Labelling Unsegmented Sequence Data with Recurrent
-Neural Networks
    
+class paddle.v2.layer.warp_ctc
+
    A layer intergrating the open-source warp-ctc library, which is used in
+Deep Speech 2: End-toEnd Speech Recognition in English and Mandarin, to compute Connectionist Temporal
+Classification (CTC) loss. Besides, another warp-ctc repository, which is forked from
+the official one, is maintained to enable more compiling options. During the
+building process, PaddlePaddle will clone the source codes, build and
+install it to third_party/install/warpctc directory.
    
+
    
+
    Reference:
    
+
    `Connectionist Temporal Classification: Labelling Unsegmented Sequence Data
+with Recurrent Neural Networks
+http://machinelearning.wustl.edu/mlpapers/paper_files/icml2006_GravesFGS06.pdf`_
    
+
    
 
    
 
    注解
    
 
      
-
    • Let num_classes represent the category number. Considering the ‘blank’
-label needed by CTC, you need to use (num_classes + 1) as the input
-size. Thus, the size of both warp_ctc and ‘input’ layer should
-be set to num_classes + 1.
      • 
+
    • Let num_classes represents the category number. Considering the ‘blank’
+label needed by CTC, you need to use (num_classes + 1) as the size of
+warp_ctc layer.
      • 
 
    • You can set ‘blank’ to any value ranged in [0, num_classes], which
-should be consistent as that used in your labels.
      • 
+should be consistent with those used in your labels.
 
    • As a native ‘softmax’ activation is interated to the warp-ctc library,
-‘linear’ activation is expected instead in the ‘input’ layer.
      • 
+‘linear’ activation is expected to be used instead in the ‘input’ layer.
 
    
 
    
-
    The simple usage:
    
+
    The example usage is:
    
 
    ctc = warp_ctc(input=input,
                      label=label,
                      size=1001,
@@ -3275,13 +4161,14 @@ should be consistent as that used in your labels.
 
 
 参数:
      
-
    • input (paddle.v2.config_base.Layer) – The input layer.
      • 
-
    • label (paddle.v2.config_base.Layer) – The data layer of label with variable length.
      • 
-
    • size (int) – category numbers + 1.
      • 
-
    • name (basestring|None) – The name of this layer, which can not specify.
      • 
-
    • blank (int) – the ‘blank’ label used in ctc
      • 
-
    • norm_by_times (bool) – Whether to normalization by times. False by default.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttributeNone) – Extra Layer config.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • size (int) – The dimension of this layer, which must be equal to (category number + 1).
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • blank (int) – The ‘blank’ label used in ctc.
      • 
+
    • norm_by_times (bool) – Whether to do normalization by times. False is the default.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
@@ -3300,12 +4187,16 @@ should be consistent as that used in your labels.
 
    nce
    
 
    
 
    
-class paddle.v2.layer.nce(*args, **kwargs)
    
-
    Noise-contrastive estimation.
-Implements the method in the following paper:
-A fast and simple algorithm for training neural probabilistic language models.
    
+class paddle.v2.layer.nce
+
    Noise-contrastive estimation.
    
+
    
+
    Reference:
    
+
    `A fast and simple algorithm for training neural probabilistic language
+models. https://www.cs.toronto.edu/~amnih/papers/ncelm.pdf`_
    
+
    
 
    The example usage is:
    
-
    cost = nce(input=layer1, label=layer2, weight=layer3,
+
    cost = nce(input=[layer1, layer2], label=layer2,
+                 param_attr=[attr1, attr2], weight=layer3,
                  num_classes=3, neg_distribution=[0.1,0.3,0.6])
 
    
 
    
@@ -3314,22 +4205,34 @@ A fast and simple algorithm for training neural probabilistic language models.
 
 参数:
      
-
    • name (basestring) – layer name
      • 
-
    • input (paddle.v2.config_base.Layer|list|tuple|collections.Sequence) – input layers. It could be a paddle.v2.config_base.Layer of list/tuple of paddle.v2.config_base.Layer.
      • 
-
    • label (paddle.v2.config_base.Layer) – label layer
      • 
-
    • weight (paddle.v2.config_base.Layer) – weight layer, can be None(default)
      • 
-
    • num_classes (int) – number of classes.
      • 
-
    • act (paddle.v2.Activation.Base) – Activation, default is Sigmoid.
      • 
-
    • num_neg_samples (int) – number of negative samples. Default is 10.
      • 
-
    • neg_distribution (list|tuple|collections.Sequence|None) – The distribution for generating the random negative labels.
-A uniform distribution will be used if not provided.
-If not None, its length must be equal to num_classes.
      • 
-
    • bias_attr (paddle.v2.attr.ParameterAttribute|None|False) – Bias parameter attribute. True if no bias.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input (paddle.v2.config_base.Layer | list | tuple | collections.Sequence) – The first input of this layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • weight (paddle.v2.config_base.Layer) – The weight layer defines a weight for each sample in the
+mini-batch. It is optional.
      • 
+
    • num_classes (int) – The number of classes.
      • 
+
    • act (paddle.v2.activation.Base) – Activation type. paddle.v2.activation.Sigmoid is the default activation.
      • 
+
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for
+details.
      • 
+
    • num_neg_samples (int) – The number of sampled negative labels. 10 is the
+default value.
      • 
+
    • neg_distribution (list | tuple | collections.Sequence | None) – The discrete noisy distribution over the output
+space from which num_neg_samples negative labels
+are sampled. If this parameter is not set, a
+uniform distribution will be used. A user-defined
+distribution is a list whose length must be equal
+to the num_classes. Each member of the list defines
+the probability of a class given input x.
      • 
+
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The parameter attribute for bias. If this parameter is set to
+False or an object whose type is not paddle.v2.attr.ParameterAttribute,
+no bias is defined. If this parameter is set to True,
+the bias is initialized to zero.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
 
 
-返回:
    layer name.
    
+返回:
    paddle.v2.config_base.Layer object.
    
 
 
 返回类型:
    paddle.v2.config_base.Layer
    
@@ -3344,11 +4247,13 @@ If not None, its length must be equal to num_classes.
 
    hsigmoid
    
 
    
 
    
-class paddle.v2.layer.hsigmoid(*args, **kwargs)
    
+class paddle.v2.layer.hsigmoid
 
    Organize the classes into a binary tree. At each node, a sigmoid function
-is used to calculate the probability of belonging to the right branch.
-This idea is from “F. Morin, Y. Bengio (AISTATS 05):
-Hierarchical Probabilistic Neural Network Language Model.”
    
+is used to calculate the probability of belonging to the right branch.
    
+
    
+
    Reference:
    
+
    Hierarchical Probabilistic Neural Network Language Model
    
+
    
 
    The example usage is:
    
 
    cost = hsigmoid(input=[layer1, layer2],
                 label=data)
@@ -3359,15 +4264,17 @@ Hierarchical Probabilistic Neural Network Language Model.”
    
 
 
 参数:
      
-
    • input (paddle.v2.config_base.Layer|list|tuple) – Input layers. It could be a paddle.v2.config_base.Layer or list/tuple of
-paddle.v2.config_base.Layer.
      • 
-
    • label (paddle.v2.config_base.Layer) – Label layer.
      • 
-
    • num_classes (int|None) – number of classes.
      • 
-
    • name (basestring) – layer name
      • 
-
    • bias_attr (paddle.v2.attr.ParameterAttribute|False) – Bias attribute. None means default bias.
-False means no bias.
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute|None) – Parameter Attribute. None means default parameter.
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – Extra Layer Attribute.
      • 
+
    • input (paddle.v2.config_base.Layer | list | tuple) – The input of this layer.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • num_classes (int) – The number of classes. And it should be larger than 2. If the parameter
+is not set or set to None, its actual value will be automatically set to
+the number of labels.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • bias_attr (paddle.v2.attr.ParameterAttribute | None | bool | Any) – The bias attribute. If the parameter is set to False or an object
+whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined. If the
+parameter is set to True, the bias is initialized to zero.
      • 
+
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for details.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for details.
      • 
 
    
 
 
@@ -3381,6 +4288,86 @@ False means no bias.
 
 
    
 
+
    
+
    
+
    smooth_l1_cost
    
+
    
+
    
+class paddle.v2.layer.smooth_l1_cost
    
+
    This is a L1 loss but more smooth. It requires that the
+sizes of input and label are equal. The formula is as follows,
    
+
    
+\[L = \sum_{i} smooth_{L1}(input_i - label_i)\]
    
+
    in which
    
+
    
+\[\begin{split}smooth_{L1}(x) = \begin{cases} 0.5x^2& \text{if}  \ |x| < 1 \\ |x|-0.5& \text{otherwise} \end{cases}\end{split}\]
    
+
    
+
    Reference:
    
+
    `Fast R-CNN
+https://arxiv.org/pdf/1504.08083v2.pdf`_
    
+
    
+
    The example usage is:
    
+
    cost = smooth_l1_cost(input=input,
+                      label=label)
+
    
+
    
+
+++
+
+
+
+
+
+
+
+
    


    参数:
      
+
    • input (paddle.v2.config_base.Layer) – The input layer.
      • 
+
    • label – The input label.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • coeff (float) – The weight of the gradient in the back propagation.
+1.0 is the default value.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    
+
    返回:
    paddle.v2.config_base.Layer object.
    
+
    返回类型:
    paddle.v2.config_base.Layer
    
+
    
+
    
+
+
    
+
    
+
    multibox_loss
    
+
    
+
    
+class paddle.v2.layer.multibox_loss
    
+
    Compute the location loss and the confidence loss for ssd.
    
+
+++
+
+
+
+
+
+
    


    参数:
      
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input_loc (paddle.v2.config_base.Layer | List of paddle.v2.config_base.Layer) – The input predict locations.
      • 
+
    • input_conf (paddle.v2.config_base.Layer | List of paddle.v2.config_base.Layer) – The input priorbox confidence.
      • 
+
    • priorbox (paddle.v2.config_base.Layer) – The input priorbox location and the variance.
      • 
+
    • label (paddle.v2.config_base.Layer) – The input label.
      • 
+
    • num_classes (int) – The number of the classification.
      • 
+
    • overlap_threshold (float) – The threshold of the overlap.
      • 
+
    • neg_pos_ratio (float) – The ratio of the negative bbox to the positive bbox.
      • 
+
    • neg_overlap (float) – The negative bbox overlap threshold.
      • 
+
    • background_id (int) – The background class index.
      • 
+
    
+
    返回:
    paddle.v2.config_base.Layer
    
+
    
+
    
+
 
    
 
    
 
    
@@ -3389,7 +4376,7 @@ False means no bias.
 
    eos
    
 
    
 
    
-class paddle.v2.layer.eos(*args, **kwargs)
    
+class paddle.v2.layer.eos
 
    A layer for checking EOS for each sample:
 - output_id = (input_id == conf.eos_id)
    
 
    The result is stored in output_.ids.
@@ -3403,10 +4390,161 @@ It is used by recurrent layer group.
    
 
 
 参数:
      
-
    • name (basestring) – Layer name.
      • 
-
    • input (paddle.v2.config_base.Layer) – Input layer name.
      • 
-
    • eos_id (int) – end id of sequence
      • 
-
    • layer_attr (paddle.v2.attr.ExtraAttribute) – extra layer attributes.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • eos_id (int) – End id of sequence
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    
+
+
+返回:
    paddle.v2.config_base.Layer object.
    
+
+
+返回类型:
    paddle.v2.config_base.Layer
    
+
+
+
+
+
    
+
+
    
+
    
+
    
+
    Miscs
    
+
    
+
    dropout
    
+
    
+
    
+class paddle.v2.layer.dropout
    
+
    The example usage is:
    
+
    dropout = dropout(input=input, dropout_rate=0.5)
+
    
+
    
+
+++
+
+
+
+
+
+
+
+
    


    参数:
      
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • dropout_rate (float) – The probability of dropout.
      • 
+
    
+
    返回:
    paddle.v2.config_base.Layer object.
    
+
    返回类型:
    paddle.v2.config_base.Layer
    
+
    
+
    
+
+
    
+
    
+
    
+
    Activation with learnable parameter
    
+
    
+
    prelu
    
+
    
+
    
+class paddle.v2.layer.prelu
    
+
    The Parametric Relu activation that actives outputs with a learnable weight.
    
+
    
+
    Reference:
    
+
    `Delving Deep into Rectifiers: Surpassing Human-Level Performance on
+ImageNet Classification http://arxiv.org/pdf/1502.01852v1.pdf`_
    
+
    
+
    
+\[\begin{split}z_i &\quad if \quad z_i > 0 \\
+a_i * z_i  &\quad \mathrm{otherwise}\end{split}\]
    
+
    The example usage is:
    
+
    prelu = prelu(input=layers, partial_sum=1)
+
    
+
    
+
+++
+
+
+
+
+
+
+
+
    


    参数:
      
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • partial_sum (int) – 
      this parameter makes a group of inputs share the same weight.
      
+
        
+
      • partial_sum = 1, indicates the element-wise activation: each element has a weight.
        • 
+
      • partial_sum = number of elements in one channel, indicates the channel-wise activation, elements in a channel share the same weight.
        • 
+
      • partial_sum = number of outputs, indicates all elements share the same weight.
        • 
+
      
+
      • 
+
    • channel_shared (bool) – 
      whether or not the parameter are shared across channels.
      
+
        
+
      • channel_shared = True, we set the partial_sum to the number of outputs.
        • 
+
      • channel_shared = False, we set the partial_sum to the number of elements in one channel.
        • 
+
      
+
      • 
+
    • num_channels (int) – number of input channel.
      • 
+
    • param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute. See paddle.v2.attr.ParameterAttribute for details.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute | None) – The extra layer attribute. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    
+
    返回:
    paddle.v2.config_base.Layer object.
    
+
    返回类型:
    paddle.v2.config_base.Layer
    
+
    
+
    
+
+
    
+
    
+
    gated_unit
    
+
    
+
    
+class paddle.v2.layer.gated_unit
    
+
    The gated unit layer implements a simple gating mechanism over the input.
+The input \(X\) is first projected into a new space \(X'\), and
+it is also used to produce a gate weight \(\sigma\). Element-wise
+product between :match:`X’` and \(\sigma\) is finally returned.
    
+
    
+
    Reference:
    
+
    `Language Modeling with Gated Convolutional Networks
+https://arxiv.org/abs/1612.08083`_
    
+
    
+
    
+\[y=\text{act}(X \cdot W + b)\otimes \sigma(X \cdot V + c)\]
    
+
    The example usage is:
    
+
+++
+
@@ -3420,6 +4558,43 @@ It is used by recurrent layer group.
    


    参数:
      
+
    • input (paddle.v2.config_base.Layer) – The input of this layer.
      • 
+
    • size (int) – The dimension of this layer’s output.
      • 
+
    • act (paddle.v2.activation.Base) – Activation type of the projection. paddle.v2.activation.Linear is the default
+activation.
      • 
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • gate_attr (paddle.v2.attr.ExtraAttribute | None) – The extra layer attribute of the gate. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    • gate_param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute of the gate. See paddle.v2.attr.ParameterAttribute
+for details.
      • 
+
    • gate_bias_attr (paddle.v2.attr.ParameterAttribute | bool | None | Any) – The bias attribute of the gate. If this parameter is set to False or
+an object whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined.
+If this parameter is set to True, the bias is initialized to zero.
      • 
+
    • inproj_attr (paddle.v2.attr.ExtraAttribute | None) – Extra layer attributes of the projection. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
+
    • inproj_param_attr (paddle.v2.attr.ParameterAttribute) – The parameter attribute of the projection. See paddle.v2.attr.ParameterAttribute
+for details.
      • 
+
    • inproj_bias_attr (paddle.v2.attr.ParameterAttribute | bool | None | Any) – The bias attribute of the projection. If this parameter is set to False
+or an object whose type is not paddle.v2.attr.ParameterAttribute, no bias is defined.
+If this parameter is set to True, the bias is initialized to zero.
      • 
+
    • layer_attr (paddle.v2.attr.ExtraAttribute | None) – Extra layer attribute of the product. See paddle.v2.attr.ExtraAttribute for
+details.
      • 
 
    
     		
 
    
 
    
 
    
 
+
    
+
    
+
    
+
    Detection output Layer
    
+
    
+
    detection_output
    
+
    
+
    
+class paddle.v2.layer.detection_output
    
+
    Apply the NMS to the output of network and compute the predict bounding
+box location. The output’s shape of this layer could be zero if there is
+no valid bounding box.
    
+
+++
+
+
+
+
+
+
    


    参数:
      
+
    • name (basestring) – The name of this layer. It is optional.
      • 
+
    • input_loc (paddle.v2.config_base.Layer | List of paddle.v2.config_base.Layer.) – The input predict locations.
      • 
+
    • input_conf (paddle.v2.config_base.Layer | List of paddle.v2.config_base.Layer.) – The input priorbox confidence.
      • 
+
    • priorbox (paddle.v2.config_base.Layer) – The input priorbox location and the variance.
      • 
+
    • num_classes (int) – The number of the classification.
      • 
+
    • nms_threshold (float) – The Non-maximum suppression threshold.
      • 
+
    • nms_top_k (int) – The bbox number kept of the NMS’s output
      • 
+
    • keep_top_k (int) – The bbox number kept of the layer’s output
      • 
+
    • confidence_threshold (float) – The classification confidence threshold
      • 
+
    • background_id (int) – The background class index.
      • 
+
    
+
    返回:
    paddle.v2.config_base.Layer
    
+
    
+
    
+
 
    
 
    
 
    
@@ -3431,7 +4606,7 @@ It is used by recurrent layer group.
    
   
     
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -88,6 +88,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +112,10 @@
           
 
         
@@ -196,45 +218,45 @@
 
    NLP
    
 
    
 
    sequence_conv_pool
    
-
    
+
    
 
    
-class paddle.v2.networks.sequence_conv_pool(*args, **kwargs)
    
-
    Text convolution pooling layers helper.
    
+paddle.v2.networks.sequence_conv_pool(*args, **kwargs)
+
    Text convolution pooling group.
    
 
    Text input => Context Projection => FC Layer => Pooling => Output.
    
 
-
-
@@ -244,45 +266,45 @@ False if no bias.
    
 
    text_conv_pool
    
-
    
+
    
 
    
-class paddle.v2.networks.text_conv_pool(*args, **kwargs)
    
-
    Text convolution pooling layers helper.
    
+paddle.v2.networks.text_conv_pool(*args, **kwargs)
+
    Text convolution pooling group.
    
 
    Text input => Context Projection => FC Layer => Pooling => Output.
    
 
    
 
 
 
 
    参数:
      
-
    • name (basestring) – name of output layer(pooling layer name)
      • 
-
    • input (paddle.v2.config_base.Layer) – name of input layer
      • 
+
    • name (basestring) – group name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
 
    • context_len (int) – context projection length. See
 context_projection’s document.
      • 
 
    • hidden_size (int) – FC Layer size.
      • 
-
    • context_start (int or None) – context projection length. See
+
    • context_start (int|None) – context start position. See
 context_projection’s context_start.
      • 
-
    • pool_type (BasePoolingType.) – pooling layer type. See pooling’s document.
      • 
-
    • context_proj_name (basestring) – context projection layer name.
+
    • pool_type (BasePoolingType) – pooling layer type. See pooling_layer’s document.
      • 
+
    • context_proj_layer_name (basestring) – context projection layer name.
 None if user don’t care.
      • 
-
    • context_proj_param_attr (paddle.v2.attr.ParameterAttribute or None.) – context projection parameter attribute.
+
    • context_proj_param_attr (ParameterAttribute|None) – padding parameter attribute of context projection layer.
+If false, it means padding always be zero.
      • 
+
    • fc_layer_name (basestring) – fc layer name. None if user don’t care.
      • 
+
    • fc_param_attr (ParameterAttribute|None) – fc layer parameter attribute. None if user don’t care.
      • 
+
    • fc_bias_attr (ParameterAttribute|False|None) – fc bias parameter attribute. False if no bias,
 None if user don’t care.
      • 
-
    • fc_name (basestring) – fc layer name. None if user don’t care.
      • 
-
    • fc_param_attr (paddle.v2.attr.ParameterAttribute or None) – fc layer parameter attribute. None if user don’t care.
      • 
-
    • fc_bias_attr (paddle.v2.attr.ParameterAttribute or None) – fc bias parameter attribute. False if no bias,
+
    • fc_act (BaseActivation) – fc layer activation type. None means tanh.
      • 
+
    • pool_bias_attr (ParameterAttribute|False|None) – pooling layer bias attr. False if no bias.
 None if user don’t care.
      • 
-
    • fc_act (paddle.v2.Activation.Base) – fc layer activation type. None means tanh
      • 
-
    • pool_bias_attr (paddle.v2.attr.ParameterAttribute or None.) – pooling layer bias attr. None if don’t care.
-False if no bias.
      • 
-
    • fc_attr (paddle.v2.attr.ExtraAttribute) – fc layer extra attribute.
      • 
-
    • context_attr (paddle.v2.attr.ExtraAttribute) – context projection layer extra attribute.
      • 
-
    • pool_attr (paddle.v2.attr.ExtraAttribute) – pooling layer extra attribute.
      • 
+
    • fc_attr (ExtraLayerAttribute) – fc layer extra attribute.
      • 
+
    • context_attr (ExtraLayerAttribute) – context projection layer extra attribute.
      • 
+
    • pool_attr (ExtraLayerAttribute) – pooling layer extra attribute.
      • 
 
    
     		
 
    返回:
    output layer name.
    
+
    返回:
    layer’s output.
    
     		
 
    返回类型:
    paddle.v2.config_base.Layer
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
    
-
-
@@ -295,43 +317,44 @@ False if no bias.
    Images
    
 
    img_conv_bn_pool
    
-
    
+
    
 
    
-class paddle.v2.networks.img_conv_bn_pool(*args, **kwargs)
    
+paddle.v2.networks.img_conv_bn_pool(*args, **kwargs)
 
    Convolution, batch normalization, pooling group.
    
+
    Img input => Conv => BN => Pooling => Output.
    
 
    
 
 
 
 
    参数:
      
-
    • name (basestring) – name of output layer(pooling layer name)
      • 
-
    • input (paddle.v2.config_base.Layer) – name of input layer
      • 
+
    • name (basestring) – group name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
 
    • context_len (int) – context projection length. See
 context_projection’s document.
      • 
 
    • hidden_size (int) – FC Layer size.
      • 
-
    • context_start (int or None) – context projection length. See
+
    • context_start (int|None) – context start position. See
 context_projection’s context_start.
      • 
-
    • pool_type (BasePoolingType.) – pooling layer type. See pooling’s document.
      • 
-
    • context_proj_name (basestring) – context projection layer name.
+
    • pool_type (BasePoolingType) – pooling layer type. See pooling_layer’s document.
      • 
+
    • context_proj_layer_name (basestring) – context projection layer name.
 None if user don’t care.
      • 
-
    • context_proj_param_attr (paddle.v2.attr.ParameterAttribute or None.) – context projection parameter attribute.
+
    • context_proj_param_attr (ParameterAttribute|None) – padding parameter attribute of context projection layer.
+If false, it means padding always be zero.
      • 
+
    • fc_layer_name (basestring) – fc layer name. None if user don’t care.
      • 
+
    • fc_param_attr (ParameterAttribute|None) – fc layer parameter attribute. None if user don’t care.
      • 
+
    • fc_bias_attr (ParameterAttribute|False|None) – fc bias parameter attribute. False if no bias,
 None if user don’t care.
      • 
-
    • fc_name (basestring) – fc layer name. None if user don’t care.
      • 
-
    • fc_param_attr (paddle.v2.attr.ParameterAttribute or None) – fc layer parameter attribute. None if user don’t care.
      • 
-
    • fc_bias_attr (paddle.v2.attr.ParameterAttribute or None) – fc bias parameter attribute. False if no bias,
+
    • fc_act (BaseActivation) – fc layer activation type. None means tanh.
      • 
+
    • pool_bias_attr (ParameterAttribute|False|None) – pooling layer bias attr. False if no bias.
 None if user don’t care.
      • 
-
    • fc_act (paddle.v2.Activation.Base) – fc layer activation type. None means tanh
      • 
-
    • pool_bias_attr (paddle.v2.attr.ParameterAttribute or None.) – pooling layer bias attr. None if don’t care.
-False if no bias.
      • 
-
    • fc_attr (paddle.v2.attr.ExtraAttribute) – fc layer extra attribute.
      • 
-
    • context_attr (paddle.v2.attr.ExtraAttribute) – context projection layer extra attribute.
      • 
-
    • pool_attr (paddle.v2.attr.ExtraAttribute) – pooling layer extra attribute.
      • 
+
    • fc_attr (ExtraLayerAttribute) – fc layer extra attribute.
      • 
+
    • context_attr (ExtraLayerAttribute) – context projection layer extra attribute.
      • 
+
    • pool_attr (ExtraLayerAttribute) – pooling layer extra attribute.
      • 
 
    
     		
 
    返回:
    output layer name.
    
+
    返回:
    layer’s output.
    
     		
 
    返回类型:
    paddle.v2.config_base.Layer
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
    
-
-
@@ -341,31 +364,37 @@ False if no bias.
    
 
    img_conv_group
    
-
    
+
    
 
    
-class paddle.v2.networks.img_conv_group(**kwargs)
    
+paddle.v2.networks.img_conv_group(*args, **kwargs)
 
    Image Convolution Group, Used for vgg net.
    
-
    TODO(yuyang18): Complete docs
    
 
    
 
 
 
 
    参数:
      
-
    • name (basestring) – group name
      • 
-
    • input (paddle.v2.config_base.Layer) – layer’s input
      • 
-
    • filter_size (int) – see img_conv’s document
      • 
-
    • num_filters (int) – see img_conv’s document
      • 
-
    • pool_size (int) – see img_pool’s document.
      • 
-
    • pool_type (BasePoolingType) – see img_pool’s document.
      • 
-
    • act (paddle.v2.Activation.Base) – see batch_norm’s document.
      • 
-
    • groups (int) – see img_conv’s document
      • 
-
    • conv_stride (int) – see img_conv’s document.
      • 
-
    • conv_padding (int) – see img_conv’s document.
      • 
-
    • conv_bias_attr (paddle.v2.attr.ParameterAttribute) – see img_conv’s document.
      • 
-
    • num_channel (int) – see img_conv’s document.
      • 
-
    • conv_param_attr (paddle.v2.attr.ParameterAttribute) – see img_conv’s document.
      • 
-
    • shared_bias (bool) – see img_conv’s document.
      • 
-
    • conv_attr (Extrapaddle.v2.config_base.Layer) – see img_conv’s document.
      • 
-
    • bn_param_attr (paddle.v2.attr.ParameterAttribute.) – see batch_norm’s document.
      • 
-
    • bn_bias_attr – see batch_norm’s document.
      • 
-
    • bn_attr – paddle.v2.attr.ParameterAttribute.
      • 
-
    • pool_stride (int) – see img_pool’s document.
      • 
-
    • pool_padding (int) – see img_pool’s document.
      • 
-
    • pool_attr (paddle.v2.attr.ExtraAttribute) – see img_pool’s document.
      • 
+
    • name (basestring) – group name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • filter_size (int) – see img_conv_layer for details.
      • 
+
    • num_filters (int) – see img_conv_layer for details.
      • 
+
    • pool_size (int) – see img_pool_layer for details.
      • 
+
    • pool_type (BasePoolingType) – see img_pool_layer for details.
      • 
+
    • act (BaseActivation) – see batch_norm_layer for details.
      • 
+
    • groups (int) – see img_conv_layer for details.
      • 
+
    • conv_stride (int) – see img_conv_layer for details.
      • 
+
    • conv_padding (int) – see img_conv_layer for details.
      • 
+
    • conv_bias_attr (ParameterAttribute) – see img_conv_layer for details.
      • 
+
    • num_channel (int) – see img_conv_layer for details.
      • 
+
    • conv_param_attr (ParameterAttribute) – see img_conv_layer for details.
      • 
+
    • shared_bias (bool) – see img_conv_layer for details.
      • 
+
    • conv_layer_attr (ExtraLayerOutput) – see img_conv_layer for details.
      • 
+
    • bn_param_attr (ParameterAttribute) – see batch_norm_layer for details.
      • 
+
    • bn_bias_attr (ParameterAttribute) – see batch_norm_layer for details.
      • 
+
    • bn_layer_attr (ExtraLayerAttribute) – see batch_norm_layer for details.
      • 
+
    • pool_stride (int) – see img_pool_layer for details.
      • 
+
    • pool_padding (int) – see img_pool_layer for details.
      • 
+
    • pool_layer_attr (ExtraLayerAttribute) – see img_pool_layer for details.
      • 
 
    
     		
 
    返回:
    Layer groups output
    
+
    返回:
    layer’s output
    
     		
 
    返回类型:
    paddle.v2.config_base.Layer
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
    
-
+
+
@@ -375,66 +404,72 @@ False if no bias.
    
 
    simple_img_conv_pool
    
-
    
+
    
 
    
-class paddle.v2.networks.simple_img_conv_pool(*args, **kwargs)
    
+paddle.v2.networks.simple_img_conv_pool(*args, **kwargs)
 
    Simple image convolution and pooling group.
    
-
    Input => conv => pooling
    
+
    Img input => Conv => Pooling => Output.
    
 
    
 
 
 
 
    参数:
      
-
    • conv_batchnorm_drop_rate – 
      • 
-
    • input – 
      • 
-
    • conv_num_filter – 
      • 
-
    • pool_size – 
      • 
-
    • num_channels – 
      • 
-
    • conv_padding – 
      • 
-
    • conv_filter_size – 
      • 
-
    • conv_act – 
      • 
-
    • conv_with_batchnorm – 
      • 
-
    • pool_stride – 
      • 
-
    • pool_type – 
      • 
+
    • conv_batchnorm_drop_rate (list) – if conv_with_batchnorm[i] is true,
+conv_batchnorm_drop_rate[i] represents the drop rate of each batch norm.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • conv_num_filter (list|tuple) – list of output channels num.
      • 
+
    • pool_size (int) – pooling filter size.
      • 
+
    • num_channels (int) – input channels num.
      • 
+
    • conv_padding (int) – convolution padding size.
      • 
+
    • conv_filter_size (int) – convolution filter size.
      • 
+
    • conv_act (BaseActivation) – activation funciton after convolution.
      • 
+
    • conv_with_batchnorm (list) – if conv_with_batchnorm[i] is true,
+there is a batch normalization operation after each convolution.
      • 
+
    • pool_stride (int) – pooling stride size.
      • 
+
    • pool_type (BasePoolingType) – pooling type.
      • 
+
    • param_attr (ParameterAttribute) – param attribute of convolution layer,
+None means default attribute.
      • 
 
    
     		
 
    返回:
    
+
    返回:
    layer’s output
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
    
-
-
    
 
 
 
 
    参数:
      
-
    • name (basestring) – group name
      • 
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
-
    • filter_size (int) – see img_conv for details
      • 
-
    • num_filters (int) – see img_conv for details
      • 
-
    • pool_size (int) – see img_pool for details
      • 
-
    • pool_type (BasePoolingType) – see img_pool for details
      • 
-
    • act (paddle.v2.Activation.Base) – see img_conv for details
      • 
-
    • groups (int) – see img_conv for details
      • 
-
    • conv_stride (int) – see img_conv for details
      • 
-
    • conv_padding (int) – see img_conv for details
      • 
-
    • bias_attr (paddle.v2.attr.ParameterAttribute) – see img_conv for details
      • 
-
    • num_channel (int) – see img_conv for details
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute) – see img_conv for details
      • 
-
    • shared_bias (bool) – see img_conv for details
      • 
-
    • conv_attr (paddle.v2.attr.ExtraAttribute) – see img_conv for details
      • 
-
    • pool_stride (int) – see img_pool for details
      • 
-
    • pool_padding (int) – see img_pool for details
      • 
-
    • pool_attr (paddle.v2.attr.ExtraAttribute) – see img_pool for details
      • 
+
    • name (basestring) – group name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • filter_size (int) – see img_conv_layer for details.
      • 
+
    • num_filters (int) – see img_conv_layer for details.
      • 
+
    • pool_size (int) – see img_pool_layer for details.
      • 
+
    • pool_type (BasePoolingType) – see img_pool_layer for details.
      • 
+
    • act (BaseActivation) – see img_conv_layer for details.
      • 
+
    • groups (int) – see img_conv_layer for details.
      • 
+
    • conv_stride (int) – see img_conv_layer for details.
      • 
+
    • conv_padding (int) – see img_conv_layer for details.
      • 
+
    • bias_attr (ParameterAttribute) – see img_conv_layer for details.
      • 
+
    • num_channel (int) – see img_conv_layer for details.
      • 
+
    • param_attr (ParameterAttribute) – see img_conv_layer for details.
      • 
+
    • shared_bias (bool) – see img_conv_layer for details.
      • 
+
    • conv_layer_attr (ExtraLayerAttribute) – see img_conv_layer for details.
      • 
+
    • pool_stride (int) – see img_pool_layer for details.
      • 
+
    • pool_padding (int) – see img_pool_layer for details.
      • 
+
    • pool_layer_attr (ExtraLayerAttribute) – see img_pool_layer for details.
      • 
 
    
     		
 
    返回:
    Layer’s output
    
+
    返回:
    layer’s output
    
     		
 
    返回类型:
    paddle.v2.config_base.Layer
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
    
 
    
 
+
    
+
    
+
    small_vgg
    
 
    
 
    
 
    vgg_16_network
    
-
    
+
    
 
    
-class paddle.v2.networks.vgg_16_network(**kwargs)
    
+paddle.v2.networks.vgg_16_network(input_image, num_channels, num_classes=1000)
 
    Same model from https://gist.github.com/ksimonyan/211839e770f7b538e2d8
    
 
-
+
+
@@ -449,25 +484,25 @@ False if no bias.
    LSTM
    
 
    lstmemory_unit
    
-
    
+
    
 
    
-class paddle.v2.networks.lstmemory_unit(*args, **kwargs)
    
-
    Define calculations that a LSTM unit performs in a single time step.
-This function itself is not a recurrent layer, so that it can not be
-directly applied to sequence input. This function is always used in
+paddle.v2.networks.lstmemory_unit(*args, **kwargs)
+
    lstmemory_unit defines the caculation process of a LSTM unit during a
+single time step. This function is not a recurrent layer, so it can not be
+directly used to process sequence input. This function is always used in
 recurrent_group (see layers.py for more details) to implement attention
 mechanism.
    
 
    Please refer to  Generating Sequences With Recurrent Neural Networks
 for more details about LSTM. The link goes as follows:
 .. _Link: https://arxiv.org/abs/1308.0850
    
 
    
-\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    
+\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{x_i}x_{t} + W_{h_i}h_{t-1} + W_{c_i}c_{t-1} + b_i)\\f_t & = \sigma(W_{x_f}x_{t} + W_{h_f}h_{t-1} + W_{c_f}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{x_c}x_t+W_{h_c}h_{t-1} + b_c)\\o_t & = \sigma(W_{x_o}x_{t} + W_{h_o}h_{t-1} + W_{c_o}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    The example usage is:
    lstm_step = lstmemory_unit(input=[layer1],
                            size=256,
-                           act=paddle.v2.Activation.Tanh(),
-                           gate_act=paddle.v2.Activation.Sigmoid(),
-                           state_act=paddle.v2.Activation.Tanh())
+                           act=TanhActivation(),
+                           gate_act=SigmoidActivation(),
+                           state_act=TanhActivation())
 
    
 
    
 
 
 
 
    参数:
      
-
    • num_classes – 
      • 
-
    • input_image (paddle.v2.config_base.Layer) – 
      • 
-
    • num_channels (int) – 
      • 
+
    • num_classes (int) – number of class.
      • 
+
    • input_image (LayerOutput) – input layer.
      • 
+
    • num_channels (int) – input channels num.
      • 
 
    
     		
 
    返回:
    
+
    返回:
    layer’s output
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
 
 
 
    
@@ -475,27 +510,36 @@ for more details about LSTM. The link goes as follows:
 
-
-
@@ -505,29 +549,29 @@ False means no bias, None means default bias.
    
 
    lstmemory_group
    
-
    
+
    
 
    
-class paddle.v2.networks.lstmemory_group(*args, **kwargs)
    
-
    lstm_group is a recurrent layer group version of Long Short Term Memory. It
+paddle.v2.networks.lstmemory_group(*args, **kwargs)
+
    lstm_group is a recurrent_group version of Long Short Term Memory. It
 does exactly the same calculation as the lstmemory layer (see lstmemory in
 layers.py for the maths) does. A promising benefit is that LSTM memory
-cell states, or hidden states in every time step are accessible to the
+cell states(or hidden states) in every time step are accessible to the
 user. This is especially useful in attention model. If you do not need to
-access the internal states of the lstm, but merely use its outputs,
+access the internal states of the lstm and merely use its outputs,
 it is recommended to use the lstmemory, which is relatively faster than
 lstmemory_group.
    
 
    NOTE: In PaddlePaddle’s implementation, the following input-to-hidden
 multiplications:
-\(W_{xi}x_{t}\) , \(W_{xf}x_{t}\),
-\(W_{xc}x_t\), \(W_{xo}x_{t}\) are not done in lstmemory_unit to
-speed up the calculations. Consequently, an additional mixed with
+\(W_{x_i}x_{t}\) , \(W_{x_f}x_{t}\),
+\(W_{x_c}x_t\), \(W_{x_o}x_{t}\) are not done in lstmemory_unit to
+speed up the calculations. Consequently, an additional mixed_layer with
 full_matrix_projection must be included before lstmemory_unit is called.
    
 
    The example usage is:
    
 
    lstm_step = lstmemory_group(input=[layer1],
                             size=256,
-                            act=paddle.v2.Activation.Tanh(),
-                            gate_act=paddle.v2.Activation.Sigmoid(),
-                            state_act=paddle.v2.Activation.Tanh())
+                            act=TanhActivation(),
+                            gate_act=SigmoidActivation(),
+                            state_act=TanhActivation())
 
    
 
    
 
    
 
 
    参数:
      
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
-
    • name (basestring) – lstmemory unit name.
      • 
-
    • size (int) – lstmemory unit size.
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
      • 
-
    • act (paddle.v2.Activation.Base) – lstm final activiation type
      • 
-
    • gate_act (paddle.v2.Activation.Base) – lstm gate activiation type
      • 
-
    • state_act (paddle.v2.Activation.Base) – lstm state activiation type.
      • 
-
    • mixed_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias parameter attribute of mixed layer.
-False means no bias, None means default bias.
      • 
-
    • lstm_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias parameter attribute of lstm layer.
-False means no bias, None means default bias.
      • 
-
    • mixed_attr (paddle.v2.attr.ExtraAttribute) – mixed layer’s extra attribute.
      • 
-
    • lstm_attr (paddle.v2.attr.ExtraAttribute) – lstm layer’s extra attribute.
      • 
-
    • get_output_attr (paddle.v2.attr.ExtraAttribute) – get output layer’s extra attribute.
      • 
+
    • input (LayerOutput) – Input layer.
      • 
+
    • out_memory (LayerOutput | None) – The output of previous time step.
      • 
+
    • name (basestring) – The lstmemory unit name.
      • 
+
    • size (int) – The lstmemory unit size.
      • 
+
    • param_attr (ParameterAttribute) – The parameter attribute for the weights in
+input to hidden projection.
+None means default attribute.
      • 
+
    • act (BaseActivation) – The last activiation type of lstm.
      • 
+
    • gate_act (BaseActivation) – The gate activiation type of lstm.
      • 
+
    • state_act (BaseActivation) – The state activiation type of lstm.
      • 
+
    • input_proj_bias_attr (ParameterAttribute|bool|None) – The parameter attribute for the bias in
+input to hidden projection.
+False or None means no bias.
+If this parameter is set to True,
+the bias is initialized to zero.
      • 
+
    • input_proj_layer_attr (ExtraLayerAttribute) – The extra layer attribute for
+input to hidden projection of the LSTM unit,
+such as dropout, error clipping.
      • 
+
    • lstm_bias_attr (ParameterAttribute|True|None) – The parameter attribute for the bias in lstm layer.
+False or None means no bias.
+If this parameter is set to True,
+the bias is initialized to zero.
      • 
+
    • lstm_layer_attr (ExtraLayerAttribute) – The extra attribute of lstm layer.
      • 
 
    
     		
 
    返回:
    lstmemory unit name.
    
+
    返回:
    The lstmemory unit name.
    
     		
 
    返回类型:
    paddle.v2.config_base.Layer
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
    
@@ -535,28 +579,37 @@ full_matrix_projection must be included before lstmemory_unit is called.
    
-
@@ -566,41 +619,41 @@ False means no bias, None means default bias.
    
 
    simple_lstm
    
-
    
+
    
 
    
-class paddle.v2.networks.simple_lstm(*args, **kwargs)
    
+paddle.v2.networks.simple_lstm(*args, **kwargs)
 
    Simple LSTM Cell.
    
-
    It just combine a mixed layer with fully_matrix_projection and a lstmemory
-layer. The simple lstm cell was implemented as follow equations.
    
+
    It just combines a mixed layer with fully_matrix_projection and a lstmemory
+layer. The simple lstm cell was implemented with follow equations.
    
 
    
 \[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]
    
-
    Please refer Generating Sequences With Recurrent Neural Networks if you
-want to know what lstm is. Link is here.
    
+
    Please refer to Generating Sequences With Recurrent Neural Networks for more
+details about lstm. Link is here.
    
 
    
 
 
 
    参数:
      
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
-
    • name (basestring) – lstmemory group name.
      • 
-
    • size (int) – lstmemory group size.
      • 
-
    • reverse (bool) – is lstm reversed
      • 
-
    • param_attr (paddle.v2.attr.ParameterAttribute) – Parameter config, None if use default.
      • 
-
    • act (paddle.v2.Activation.Base) – lstm final activiation type
      • 
-
    • gate_act (paddle.v2.Activation.Base) – lstm gate activiation type
      • 
-
    • state_act (paddle.v2.Activation.Base) – lstm state activiation type.
      • 
-
    • mixed_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias parameter attribute of mixed layer.
-False means no bias, None means default bias.
      • 
-
    • lstm_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias parameter attribute of lstm layer.
-False means no bias, None means default bias.
      • 
-
    • mixed_attr (paddle.v2.attr.ExtraAttribute) – mixed layer’s extra attribute.
      • 
-
    • lstm_attr (paddle.v2.attr.ExtraAttribute) – lstm layer’s extra attribute.
      • 
-
    • get_output_attr (paddle.v2.attr.ExtraAttribute) – get output layer’s extra attribute.
      • 
+
    • input (LayerOutput) – Input layer.
      • 
+
    • size (int) – The lstmemory group size.
      • 
+
    • name (basestring) – The name of lstmemory group.
      • 
+
    • out_memory (LayerOutput | None) – The output of previous time step.
      • 
+
    • reverse (bool) – Process the input in a reverse order or not.
      • 
+
    • param_attr (ParameterAttribute) – The parameter attribute for the weights in
+input to hidden projection.
+None means default attribute.
      • 
+
    • act (BaseActivation) – The last activiation type of lstm.
      • 
+
    • gate_act (BaseActivation) – The gate activiation type of lstm.
      • 
+
    • state_act (BaseActivation) – The state activiation type of lstm.
      • 
+
    • input_proj_bias_attr (ParameterAttribute|bool|None) – The parameter attribute for the bias in
+input to hidden projection.
+False or None means no bias.
+If this parameter is set to True,
+the bias is initialized to zero.
      • 
+
    • input_proj_layer_attr (ExtraLayerAttribute) – The extra layer attribute for
+input to hidden projection of the LSTM unit,
+such as dropout, error clipping.
      • 
+
    • lstm_bias_attr (ParameterAttribute|True|None) – The parameter attribute for the bias in lstm layer.
+False or None means no bias.
+If this parameter is set to True,
+the bias is initialized to zero.
      • 
+
    • lstm_layer_attr (ExtraLayerAttribute) – The extra attribute of lstm layer.
      • 
 
    
     		
 
    
 
    返回:
    the lstmemory group.
    
     		
 
    返回类型:
    paddle.v2.config_base.Layer
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
    
-
-
@@ -610,12 +663,12 @@ means default bias.
    
 
    bidirectional_lstm
    
-
    
+
    
 
    
-class paddle.v2.networks.bidirectional_lstm(*args, **kwargs)
    
+paddle.v2.networks.bidirectional_lstm(*args, **kwargs)
 
    A bidirectional_lstm is a recurrent unit that iterates over the input
-sequence both in forward and bardward orders, and then concatenate two
-outputs form a final output. However, concatenation of two outputs
+sequence both in forward and backward orders, and then concatenate two
+outputs to form a final output. However, concatenation of two outputs
 is not the only way to form the final output, you can also, for example,
 just add them together.
    
 
    Please refer to  Neural Machine Translation by Jointly Learning to Align
@@ -632,20 +685,19 @@ The link goes as follows:
 
    
-
-
@@ -658,12 +710,12 @@ concatenated and returned.
 
    GRU
    
 
    
 
    gru_unit
    
-
    
+
    
 
    
-class paddle.v2.networks.gru_unit(*args, **kwargs)
    
-
    Define calculations that a gated recurrent unit performs in a single time
-step. This function itself is not a recurrent layer, so that it can not be
-directly applied to sequence input. This function is almost always used in
+paddle.v2.networks.gru_unit(*args, **kwargs)
+
    gru_unit defines the calculation process of a gated recurrent unit during a single
+time step. This function is not a recurrent layer, so it can not be
+directly used to process sequence input. This function is always used in
 the recurrent_group (see layers.py for more details) to implement attention
 mechanism.
    
 
    Please see grumemory in layers.py for the details about the maths.
    
@@ -672,19 +724,20 @@ mechanism.
    
 
    
-
@@ -694,21 +747,21 @@ mechanism.
    
 
 
    
 
    gru_group
    
-
    
+
    
 
    
-class paddle.v2.networks.gru_group(*args, **kwargs)
    
-
    gru_group is a recurrent layer group version of Gated Recurrent Unit. It
+paddle.v2.networks.gru_group(*args, **kwargs)
+
    gru_group is a recurrent_group version of Gated Recurrent Unit. It
 does exactly the same calculation as the grumemory layer does. A promising
 benefit is that gru hidden states are accessible to the user. This is
 especially useful in attention model. If you do not need to access
-any internal state, but merely use the outputs of a GRU, it is recommended
+any internal state and merely use the outputs of a GRU, it is recommended
 to use the grumemory, which is relatively faster.
    
 
    Please see grumemory in layers.py for more detail about the maths.
    
 
    The example usage is:
    
-
    gru = gur_group(input=[layer1],
+
    gru = gru_group(input=[layer1],
                 size=256,
-                act=paddle.v2.Activation.Tanh(),
-                gate_act=paddle.v2.Activation.Sigmoid())
+                act=TanhActivation(),
+                gate_act=SigmoidActivation())
 
    
 
    
 
    
 
 
 
 
    参数:
      
 
    • name (basestring) – lstm layer name.
      • 
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
+
    • input (LayerOutput) – layer’s input.
      • 
 
    • size (int) – lstm layer size.
      • 
-
    • reverse (bool) – whether to process the input data in a reverse order
      • 
-
    • mat_param_attr (paddle.v2.attr.ParameterAttribute) – mixed layer’s matrix projection parameter attribute.
      • 
-
    • bias_param_attr (paddle.v2.attr.ParameterAttribute|False) – bias parameter attribute. False means no bias, None
+
    • reverse (bool) – process the input in a reverse order or not.
      • 
+
    • mat_param_attr (ParameterAttribute) – parameter attribute of matrix projection in mixed layer.
      • 
+
    • bias_param_attr (ParameterAttribute|False) – bias parameter attribute. False means no bias, None
 means default bias.
      • 
-
    • inner_param_attr (paddle.v2.attr.ParameterAttribute) – lstm cell parameter attribute.
      • 
-
    • act (paddle.v2.Activation.Base) – lstm final activiation type
      • 
-
    • gate_act (paddle.v2.Activation.Base) – lstm gate activiation type
      • 
-
    • state_act (paddle.v2.Activation.Base) – lstm state activiation type.
      • 
-
    • mixed_attr (paddle.v2.attr.ExtraAttribute) – mixed layer’s extra attribute.
      • 
-
    • lstm_cell_attr (paddle.v2.attr.ExtraAttribute) – lstm layer’s extra attribute.
      • 
+
    • inner_param_attr (ParameterAttribute) – parameter attribute of lstm cell.
      • 
+
    • act (BaseActivation) – last activiation type of lstm.
      • 
+
    • gate_act (BaseActivation) – gate activiation type of lstm.
      • 
+
    • state_act (BaseActivation) – state activiation type of lstm.
      • 
+
    • mixed_layer_attr (ExtraLayerAttribute) – extra attribute of mixed layer.
      • 
+
    • lstm_cell_attr (ExtraLayerAttribute) – extra attribute of lstm.
      • 
 
    
     		
 
    返回:
    lstm layer name.
    
+
    返回:
    layer’s output.
    
     		
 
    返回类型:
    paddle.v2.config_base.Layer
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
 
    参数:
      
 
    • name (basestring) – bidirectional lstm layer name.
      • 
-
    • input (paddle.v2.config_base.Layer) – input layer.
      • 
+
    • input (LayerOutput) – input layer.
      • 
 
    • size (int) – lstm layer size.
      • 
-
    • return_seq (bool) – If set False, outputs of the last time step are
+
    • return_seq (bool) – If set False, the last time step of output are
 concatenated and returned.
-If set True, the entire output sequences that are
-processed in forward and backward directions are
-concatenated and returned.
      • 
+If set True, the entire output sequences in forward
+and backward directions are concatenated and returned.
 
    
     		
 
    返回:
    paddle.v2.config_base.Layer object accroding to the return_seq.
    
+
    返回:
    LayerOutput object.
    
     		
 
    返回类型:
    paddle.v2.config_base.Layer
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
    参数:
      
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • memory_boot (LayerOutput | None) – the initialization state of the LSTM cell.
      • 
 
    • name (basestring) – name of the gru group.
      • 
 
    • size (int) – hidden size of the gru.
      • 
-
    • act (paddle.v2.Activation.Base) – type of the activation
      • 
-
    • gate_act (paddle.v2.Activation.Base) – type of the gate activation
      • 
-
    • gru_attr (paddle.v2.attr.ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • 
+
    • act (BaseActivation) – activation type of gru
      • 
+
    • gate_act (BaseActivation) – gate activation type or gru
      • 
+
    • gru_layer_attr (ExtraLayerAttribute) – Extra attribute of the gru layer.
      • 
 
    
     		
 
    
 
    返回:
    the gru output layer.
    
     		
 
    返回类型:
    paddle.v2.config_base.Layer
    
+
    返回类型:
    LayerOutput
    
     		
 
    
 
    
@@ -716,21 +769,23 @@ to use the grumemory, which is relatively faster.
    
     
-
@@ -740,23 +795,23 @@ to use the grumemory, which is relatively faster.
    
 
 
    
 
    simple_gru
    
-
    
+
    
 
    
-class paddle.v2.networks.simple_gru(*args, **kwargs)
    
-
    You maybe see gru_step, grumemory in layers.py, gru_unit, gru_group,
+paddle.v2.networks.simple_gru(*args, **kwargs)
+
    You may see gru_step_layer, grumemory in layers.py, gru_unit, gru_group,
 simple_gru in network.py. The reason why there are so many interfaces is
 that we have two ways to implement recurrent neural network. One way is to
 use one complete layer to implement rnn (including simple rnn, gru and lstm)
-with multiple time steps, such as recurrent, lstmemory, grumemory. But,
+with multiple time steps, such as recurrent_layer, lstmemory, grumemory. But
 the multiplication operation \(W x_t\) is not computed in these layers.
 See details in their interfaces in layers.py.
 The other implementation is to use an recurrent group which can ensemble a
 series of layers to compute rnn step by step. This way is flexible for
 attenion mechanism or other complex connections.
    
 
      
-
    • gru_step: only compute rnn by one step. It needs an memory as input
+
    • gru_step_layer: only compute rnn by one step. It needs an memory as input
 and can be used in recurrent group.
      • 
-
    • gru_unit: a wrapper of gru_step with memory.
      • 
+
    • gru_unit: a wrapper of gru_step_layer with memory.
      • 
 
    • gru_group: a GRU cell implemented by a combination of multiple layers in
 recurrent group.
 But \(W x_t\) is not done in group.
      • 
@@ -777,21 +832,102 @@ gru_group, and gru_group is relatively better than simple_gru.
      
 
    
+
+
+
+
+
+
+
    
 
 
    参数:
      
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • memory_boot (LayerOutput | None) – the initialization state of the LSTM cell.
      • 
 
    • name (basestring) – name of the gru group.
      • 
 
    • size (int) – hidden size of the gru.
      • 
-
    • reverse (bool) – whether to process the input data in a reverse order
      • 
-
    • act (paddle.v2.Activation.Base) – type of the activiation
      • 
-
    • gate_act (paddle.v2.Activation.Base) – type of the gate activiation
      • 
-
    • gru_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias. False means no bias, None means default bias.
      • 
-
    • gru_attr (paddle.v2.attr.ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • 
+
    • reverse (bool) – process the input in a reverse order or not.
      • 
+
    • act (BaseActivation) – activiation type of gru
      • 
+
    • gate_act (BaseActivation) – gate activiation type of gru
      • 
+
    • gru_bias_attr (ParameterAttribute|False|None) – bias parameter attribute of gru layer,
+False means no bias, None means default bias.
      • 
+
    • gru_layer_attr (ExtraLayerAttribute) – Extra attribute of the gru layer.
      • 
 
    
     		
 
    
 
    返回:
    the gru group.
    
     		
 
    返回类型:
    paddle.v2.config_base.Layer
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
    参数:
      
-
    • input (paddle.v2.config_base.Layer) – input layer name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • name (basestring) – name of the gru group.
      • 
+
    • size (int) – hidden size of the gru.
      • 
+
    • reverse (bool) – process the input in a reverse order or not.
      • 
+
    • act (BaseActivation) – activiation type of gru
      • 
+
    • gate_act (BaseActivation) – gate activiation type of gru
      • 
+
    • gru_bias_attr (ParameterAttribute|False|None) – bias parameter attribute of gru layer,
+False means no bias, None means default bias.
      • 
+
    • gru_layer_attr (ExtraLayerAttribute) – Extra attribute of the gru layer.
      • 
+
    
+
    返回:
    the gru group.
    
+
    返回类型:
    LayerOutput
    
+
    
+
    
+
+
    
+
    
+
    simple_gru2
    
+
    
+
    
+paddle.v2.networks.simple_gru2(*args, **kwargs)
    
+
    simple_gru2 is the same with simple_gru, but using grumemory instead.
+Please refer to grumemory in layers.py for more detail about the math.
+simple_gru2 is faster than simple_gru.
    
+
    The example usage is:
    
+
    gru = simple_gru2(input=[layer1], size=256)
+
    
+
    
+
+++
+
-
+
+
+
    


    参数:
      
+
    • input (LayerOutput) – input layer.
      • 
 
    • name (basestring) – name of the gru group.
      • 
 
    • size (int) – hidden size of the gru.
      • 
-
    • reverse (bool) – whether to process the input data in a reverse order
      • 
-
    • act (paddle.v2.Activation.Base) – type of the activiation
      • 
-
    • gate_act (paddle.v2.Activation.Base) – type of the gate activiation
      • 
-
    • gru_bias_attr (paddle.v2.attr.ParameterAttribute|False) – bias. False means no bias, None means default bias.
      • 
-
    • gru_attr (paddle.v2.attr.ParameterAttribute|False) – Extra parameter attribute of the gru layer.
      • 
+
    • reverse (bool) – process the input in a reverse order or not.
      • 
+
    • act (BaseActivation) – activiation type of gru
      • 
+
    • gate_act (BaseActivation) – gate activiation type of gru
      • 
+
    • gru_bias_attr (ParameterAttribute|False|None) – bias parameter attribute of gru layer,
+False means no bias, None means default bias.
      • 
+
    • gru_layer_attr (ExtraLayerAttribute) – Extra attribute of the gru layer.
      • 
 
    
     		
 
    
 
    返回:
    the gru group.
    
     		
 
    返回类型:
    paddle.v2.config_base.Layer
    
+
    返回类型:
    LayerOutput
    
+
    
+
    
+
+
    
+
    
+
    bidirectional_gru
    
+
    
+
    
+paddle.v2.networks.bidirectional_gru(*args, **kwargs)
    
+
    A bidirectional_gru is a recurrent unit that iterates over the input
+sequence both in forward and backward orders, and then concatenate two
+outputs to form a final output. However, concatenation of two outputs
+is not the only way to form the final output, you can also, for example,
+just add them together.
    
+
    The example usage is:
    
+
    bi_gru = bidirectional_gru(input=[input1], size=512)
+
    
+
    
+
+++
+
+
+
+
+
@@ -802,10 +938,10 @@ gru_group, and gru_group is relatively better than simple_gru.
    
 
    simple_attention
    
-
    
+
    
 
    
-class paddle.v2.networks.simple_attention(*args, **kwargs)
    
-
    Calculate and then return a context vector by attention machanism.
+paddle.v2.networks.simple_attention(*args, **kwargs)
+
    Calculate and return a context vector with attention mechanism.
 Size of the context vector equals to size of the encoded_sequence.
    
 
    
 \[ \begin{align}\begin{aligned}a(s_{i-1},h_{j}) & = v_{a}f(W_{a}s_{t-1} + U_{a}h_{j})\\e_{i,j} & = a(s_{i-1}, h_{j})\\a_{i,j} & = \frac{exp(e_{i,j})}{\sum_{k=1}^{T_x}{exp(e_{i,k})}}\\c_{i} & = \sum_{j=1}^{T_{x}}a_{i,j}h_{j}\end{aligned}\end{align} \]
    
@@ -828,24 +964,27 @@ Align and Translate for more details. The link is as follows:
 
    
-
+
+
@@ -853,27 +992,51 @@ compute attention weight.
-
-
    
-
    Miscs
    
-
    
-
    dropout_layer
    
-
    
+
    
+
    dot_product_attention
    
+
    
 
    
-class paddle.v2.networks.dropout_layer(*args, **kwargs)
    
-
    @TODO(yuyang18): Add comments.
    
+paddle.v2.networks.dot_product_attention(*args, **kwargs)
+
    Calculate and return a context vector with dot-product attention mechanism.
+The dimension of the context vector equals to that of the attended_sequence.
    
+
    
+\[ \begin{align}\begin{aligned}a(s_{i-1},h_{j}) & = s_{i-1}^\mathrm{T} h_{j}\\e_{i,j} & = a(s_{i-1}, h_{j})\\a_{i,j} & = \frac{exp(e_{i,j})}{\sum_{k=1}^{T_x}{exp(e_{i,k})}}\\c_{i} & = \sum_{j=1}^{T_{x}}a_{i,j}z_{j}\end{aligned}\end{align} \]
    
+
    where \(h_{j}\) is the jth element of encoded_sequence,
+\(z_{j}\) is the jth element of attended_sequence,
+\(s_{i-1}\) is transformed_state.
    
+
    The example usage is:
    
+
    context = dot_product_attention(encoded_sequence=enc_seq,
+                                attended_sequence=att_seq,
+                                transformed_state=state,)
+
    
+
    
 
    


    参数:
      
+
    • name (basestring) – bidirectional gru layer name.
      • 
+
    • input (LayerOutput) – input layer.
      • 
+
    • size (int) – gru layer size.
      • 
+
    • return_seq (bool) – If set False, the last time step of output are
+concatenated and returned.
+If set True, the entire output sequences in forward
+and backward directions are concatenated and returned.
      • 
+
    
+
    返回:
    LayerOutput object.
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
 
    参数:
      
 
    • name (basestring) – name of the attention model.
      • 
-
    • softmax_param_attr (paddle.v2.attr.ParameterAttribute) – parameter attribute of sequence softmax
-that is used to produce attention weight
      • 
-
    • weight_act (Activation) – activation of the attention model
      • 
-
    • encoded_sequence (paddle.v2.config_base.Layer) – output of the encoder
      • 
-
    • encoded_proj (paddle.v2.config_base.Layer) – attention weight is computed by a feed forward neural
+
    • softmax_param_attr (ParameterAttribute) – parameter attribute of sequence softmax
+that is used to produce attention weight.
      • 
+
    • weight_act (BaseActivation) – activation of the attention model.
      • 
+
    • encoded_sequence (LayerOutput) – output of the encoder
      • 
+
    • encoded_proj (LayerOutput) – attention weight is computed by a feed forward neural
 network which has two inputs : decoder’s hidden state
 of previous time step and encoder’s output.
 encoded_proj is output of the feed-forward network for
 encoder’s output. Here we pre-compute it outside
 simple_attention for speed consideration.
      • 
-
    • decoder_state (paddle.v2.config_base.Layer) – hidden state of decoder in previous time step
      • 
-
    • transform_param_attr (paddle.v2.attr.ParameterAttribute) – parameter attribute of the feed-forward
+
    • decoder_state (LayerOutput) – hidden state of decoder in previous time step
      • 
+
    • transform_param_attr (ParameterAttribute) – parameter attribute of the feed-forward
 network that takes decoder_state as inputs to
 compute attention weight.
      • 
 
    
     		
 
    返回:
    a context vector
    
+
    返回:
    a context vector
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
 
 
 
    
-
+
+
diff --git a/doc_cn/api/v2/config/optimizer.html b/doc_cn/api/v2/config/optimizer.html
index 3b1cf8da435a5627dcf602ce96380bce5dd485ea..5c92d2ee9c9c5d6104ee469acc0c9d32b764536b 100644
--- a/doc_cn/api/v2/config/optimizer.html
+++ b/doc_cn/api/v2/config/optimizer.html
@@ -35,7 +35,7 @@
     
-         
+        
@@ -65,7 +65,7 @@
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -88,6 +88,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +112,10 @@
           
 
         
@@ -196,23 +218,36 @@
 
    
 
    
 class paddle.v2.optimizer.Momentum(momentum=None, sparse=False, **kwargs)
    
-
    SGD Optimizer.
    
-
    SGD is an optimization method, trying to find a neural network that
-minimize the “cost/error” of it by iteration. In paddle’s implementation
-SGD Optimizer is synchronized, which means all gradients will be wait to
-calculate and reduced into one gradient, then do optimize operation.
    
-
    The neural network consider the learning problem of minimizing an objective
-function, that has the form of a sum
    
+
    Momentum Optimizer.
    
+
    When sparse=False, the momentum update formula is as follows:
    
 
    
-\[Q(w) = \sum_{i}^{n} Q_i(w)\]
    
-
    The value of function Q sometimes is the cost of neural network (Mean
-Square Error between prediction and label for example). The function Q is
-parametrised by w, the weight/bias of neural network. And weights is what to
-be learned. The i is the i-th observation in (trainning) data.
    
-
    So, the SGD method will optimize the weight by
    
+\[\begin{split}v_{t} &= k * v_{t-1} - \gamma_t (g_{t} + \lambda w_{t-1}) \\
+w_{t} &= w_{t-1} + v_{t} \\\end{split}\]
    
+
    where, \(k\) is momentum, \(\lambda\) is decay rate,
+\(\gamma_t\) is learning rate at the t’th iteration.
+\(w_{t}\) is the weight as the t’th iteration.
+And the \(v_{t}\) is the history momentum variable.
    
+
    When sparse=True, the update scheme:
    
 
    
-\[w = w - \eta \nabla Q(w) = w - \eta \sum_{i}^{n} \nabla Q_i(w)\]
    
-
    where \(\eta\) is learning rate. And \(n\) is batch size.
    
+\[\begin{split}\alpha_t &= \alpha_{t-1} / k \\
+\beta_t &= \beta_{t-1} / (1 + \lambda \gamma_t) \\
+u_t &= u_{t-1} - \alpha_t \gamma_t g_t \\
+v_t &= v_{t-1} + \tau_{t-1} \alpha_t \gamma_t g_t \\
+\tau_t &= \tau_{t-1} + \beta_t / \alpha_t\end{split}\]
    
+
    where \(k\) is momentum, \(\lambda\) is decay rate,
+\(\gamma_t\) is learning rate at the t’th iteration.
    
+
    
 
 
 
 
    参数:
      
-
    • name – 
      • 
-
    • input – 
      • 
-
    • dropout_rate – 
      • 
+
    • name (basestring) – A prefix attached to the name of each layer that defined inside
+the dot_product_attention.
      • 
+
    • softmax_param_attr (ParameterAttribute) – The parameter attribute of sequence softmax
+that is used to produce attention weight.
      • 
+
    • encoded_sequence (LayerOutput) – The output hidden vectors of the encoder.
      • 
+
    • attended_sequence (LayerOutput) – The attention weight is computed by a feed forward neural
+network which has two inputs : decoder’s transformed hidden
+state of previous time step and encoder’s output.
+attended_sequence is the sequence to be attended.
      • 
+
    • transformed_state (LayerOutput) – The transformed hidden state of decoder in previous time step.
+Since the dot-product operation will be performed on it and the
+encoded_sequence, their dimensions must be equal. For flexibility,
+we suppose transformations of the decoder’s hidden state have been
+done outside dot_product_attention and no more will be performed
+inside. Then users can use either the original or transformed one.
      • 
 
    
     		
 
    返回:
    
+
    返回:
    The context vector.
    
+
    返回类型:
    LayerOutput
    
     		
 
    
     
         
          
 
   
   
     
    
+++
+
+
+
+
    


    参数:
      
+
    • momentum (float) – the momentum factor.
      • 
+
    • sparse (bool) – with sparse support or not, False by default.
      • 
+
    
+
    
 
    
 
 
    
@@ -226,7 +261,7 @@ The details of please refer 
 \[\begin{split}m(w, t) & = \beta_1 m(w, t-1) + (1 - \beta_1) \nabla Q_i(w) \\
 v(w, t) & = \beta_2 v(w, t-1) + (1 - \beta_2)(\nabla Q_i(w)) ^2 \\
-w & = w - \frac{\eta}{\sqrt{v(w,t) + \epsilon}}\end{split}\]
    
+w & = w - \frac{\eta m(w, t)}{\sqrt{v(w,t) + \epsilon}}\end{split}\]
    
 @@ -374,7 +409,7 @@ w & = w - \frac{\eta} {\sqrt{v(w,t) + \epsilon}} \nabla Q_{i}(w)\end{split}\
         Next 
       
       
-         Previous
+         Previous
   
diff --git a/doc_cn/api/v2/config/pooling.html b/doc_cn/api/v2/config/pooling.html
index 92a04d013ee5924685c2b649028d4ae539530c9a..2780404d75cc1abacd3b79f0f59659f3082f69a6 100644
--- a/doc_cn/api/v2/config/pooling.html
+++ b/doc_cn/api/v2/config/pooling.html
@@ -65,7 +65,7 @@
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -88,6 +88,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +112,10 @@
           
 
         
diff --git a/doc_cn/api/v2/data.html b/doc_cn/api/v2/data.html
index a75ee12d388b60f52aa5813383c7e76dbaa88f0d..887fbafdcb8bd8dcb7c1c527e1e582b224547eae 100644
--- a/doc_cn/api/v2/data.html
+++ b/doc_cn/api/v2/data.html
@@ -34,7 +34,7 @@
         
     
         
-        
+        
          
 
   
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -88,6 +88,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +112,10 @@
           
 
         
@@ -189,1076 +211,12 @@
             
   
    
 
    Data Reader Interface and DataSets
    
-
    
-
    DataTypes
    
-
    
-
    
-paddle.v2.data_type.dense_vector(dim, seq_type=0)
    
-
    Dense Vector. It means the input feature is dense float vector. For example,
-if the input is an image with 28*28 pixels, the input of Paddle neural
-network should be a dense vector with dimension 784.
    
-
    
 
 

       
     
     
    
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • dim (int) – dimension of this vector.
      • 
-
    • seq_type (int) – sequence type of input.
      • 
-
    
-
    返回:
    An input type object.
    
-
    返回类型:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.dense_vector_sequence(dim)
    
-
    Data type of a sequence of dense vector.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:dim (int) – dimension of dense vector.
    返回:An input type object
    返回类型:InputType
    
-
    
-
-
    
-
    
-paddle.v2.data_type.integer_value(value_range, seq_type=0)
    
-
    Data type of integer.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • seq_type (int) – sequence type of this input.
      • 
-
    • value_range (int) – range of this integer.
      • 
-
    
-
    返回:
    An input type object
    
-
    返回类型:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.integer_value_sequence(value_range)
    
-
    Data type of a sequence of integer.
    
-
---
-
-
-
-
    


    参数:value_range (int) – range of each element.
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_binary_vector(dim, seq_type=0)
    
-
    Sparse binary vector. It means the input feature is a sparse vector and the
-every element in this vector is either zero or one.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • dim (int) – dimension of this vector.
      • 
-
    • seq_type (int) – sequence type of this input.
      • 
-
    
-
    返回:
    An input type object.
    
-
    返回类型:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_binary_vector_sequence(dim)
    
-
    
-
    Data type of a sequence of sparse vector, which every element is either zero
    
-
    or one.
    
-
    
-
---
-
-
-
-
-
-
-
-
    


    参数:dim (int) – dimension of sparse vector.
    返回:An input type object
    返回类型:InputType
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_non_value_slot(dim, seq_type=0)
    
-
    Sparse binary vector. It means the input feature is a sparse vector and the
-every element in this vector is either zero or one.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • dim (int) – dimension of this vector.
      • 
-
    • seq_type (int) – sequence type of this input.
      • 
-
    
-
    返回:
    An input type object.
    
-
    返回类型:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_value_slot(dim, seq_type=0)
    
-
    Sparse vector. It means the input feature is a sparse vector. Most of the
-elements in this vector are zero, others could be any float value.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • dim (int) – dimension of this vector.
      • 
-
    • seq_type (int) – sequence type of this input.
      • 
-
    
-
    返回:
    An input type object.
    
-
    返回类型:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_vector(dim, seq_type=0)
    
-
    Sparse vector. It means the input feature is a sparse vector. Most of the
-elements in this vector are zero, others could be any float value.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • dim (int) – dimension of this vector.
      • 
-
    • seq_type (int) – sequence type of this input.
      • 
-
    
-
    返回:
    An input type object.
    
-
    返回类型:
    InputType
    
-
    
-
    
-
-
    
-
    
-paddle.v2.data_type.sparse_vector_sequence(dim)
    
-
    Data type of a sequence of sparse vector, which most elements are zero,
-others could be any float value.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:dim (int) – dimension of sparse vector.
    返回:An input type object
    返回类型:InputType
    
-
    
-
-
    
-
    
-class paddle.v2.data_type.InputType(dim, seq_type, tp)
    
-
    InputType is the base class for paddle input types.
    
-
    
-
    注解
    
-
    this is a base class, and should never be used by user.
    
-
    
-
---
-
-
-
-
    


    参数:
      
-
    • dim (int) – dimension of input. If the input is an integer, it means the
-value range. Otherwise, it means the size of layer.
      • 
-
    • seq_type (int) – sequence type of input. 0 means it is not a sequence. 1
-means it is a variable length sequence. 2 means it is a
-nested sequence.
      • 
-
    • type (int) – data type of input.
      • 
-
    
-
    
-
    
-
-
    
-
    
-
    DataFeeder
    
-
    
-
    
-class paddle.v2.data_feeder.DataFeeder(data_types, feeding=None)
    
-
    DataFeeder converts the data returned by paddle.reader into a data structure
-of Arguments which is defined in the API. The paddle.reader usually returns
-a list of mini-batch data entries. Each data entry in the list is one sample.
-Each sample is a list or a tuple with one feature or multiple features.
-DataFeeder converts this mini-batch data entries into Arguments in order
-to feed it to C++ interface.
    
-
    The simple usage shows below
    
-
    feeding = ['image', 'label']
-data_types = enumerate_data_types_of_data_layers(topology)
-feeder = DataFeeder(data_types=data_types, feeding=feeding)
-
-minibatch_data = [([1.0, 2.0, 3.0, ...], 5)]
-
-arg = feeder(minibatch_data)
-
    
-
    
-
    If mini-batch data and data layers are not one to one mapping, we
-could pass a dictionary to feeding parameter to represent the mapping
-relationship.
    
-
    data_types = [('image', paddle.data_type.dense_vector(784)),
-              ('label', paddle.data_type.integer_value(10))]
-feeding = {'image':0, 'label':1}
-feeder = DataFeeder(data_types=data_types, feeding=feeding)
-minibatch_data = [
-                   ( [1.0,2.0,3.0,4.0], 5, [6,7,8] ),  # first sample
-                   ( [1.0,2.0,3.0,4.0], 5, [6,7,8] )   # second sample
-                 ]
-# or minibatch_data = [
-#                       [ [1.0,2.0,3.0,4.0], 5, [6,7,8] ],  # first sample
-#                       [ [1.0,2.0,3.0,4.0], 5, [6,7,8] ]   # second sample
-#                     ]
-arg = feeder.convert(minibatch_data)
-
    
-
    
-
    
-
    注解
    
-
    This module is for internal use only. Users should use the reader
-interface.
    
-
    
-
---
-
-
-
-
    


    参数:
      
-
    • data_types (list) – A list to specify data name and type. Each item is
-a tuple of (data_name, data_type).
      • 
-
    • feeding (dict|collections.Sequence|None) – A dictionary or a sequence to specify the position of each
-data in the input data.
      • 
-
    
-
    
-
    
-
    
-convert(dat, argument=None)
    
-
    
---
-
-
-
-
    


    参数:
      
-
    • dat (list) – A list of mini-batch data. Each sample is a list or tuple
-one feature or multiple features.
      • 
-
    • argument (py_paddle.swig_paddle.Arguments) – An Arguments object contains this mini-batch data with
-one or multiple features. The Arguments definition is
-in the API.
      • 
-
    
-
    
-
    
-
-
    
-
-
    
-
    
-
    Reader
    
-
    At training and testing time, PaddlePaddle programs need to read data. To ease
-the users’ work to write data reading code, we define that
    
-
      
-
    • A reader is a function that reads data (from file, network, random number
-generator, etc) and yields data items.
      • 
-
    • A reader creator is a function that returns a reader function.
      • 
-
    • A reader decorator is a function, which accepts one or more readers, and
-returns a reader.
      • 
-
    • A batch reader is a function that reads data (from reader, file, network,
-random number generator, etc) and yields a batch of data items.
      • 
-
    
-
    
-
    Data Reader Interface
    
-
    Indeed, data reader doesn’t have to be a function that reads and yields data
-items. It can be any function with no parameter that creates a iterable
-(anything can be used in for x in iterable):
    
-
    iterable = data_reader()
-
    
-
    
-
    Element produced from the iterable should be a single entry of data,
-not a mini batch. That entry of data could be a single item, or a tuple of
-items.
-Item should be of supported type (e.g., numpy 1d
-array of float32, int, list of int)
    
-
    An example implementation for single item data reader creator:
    
-
    def reader_creator_random_image(width, height):
-    def reader():
-        while True:
-            yield numpy.random.uniform(-1, 1, size=width*height)
-return reader
-
    
-
    
-
    An example implementation for multiple item data reader creator:
    
-
    def reader_creator_random_image_and_label(width, height, label):
-    def reader():
-        while True:
-            yield numpy.random.uniform(-1, 1, size=width*height), label
-return reader
-
    
-
    
-
    TODO(yuyang18): Should we add whole design doc here?
    
-
    
-
    
-paddle.v2.reader.map_readers(func, *readers)
    
-
    Creates a data reader that outputs return value of function using
-output of each data readers as arguments.
    
-
---
-
-
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • func – function to use. The type of func should be (Sample) => Sample
      • 
-
    • readers – readers whose outputs will be used as arguments of func.
      • 
-
    
-
    Type:
    callable
    
-
    返回:
    the created data reader.
    
-
    返回类型:
    callable
    
-
    
-
    
-
-
    
-
    
-paddle.v2.reader.buffered(reader, size)
    
-
    Creates a buffered data reader.
    
-
    The buffered data reader will read and save data entries into a
-buffer. Reading from the buffered data reader will proceed as long
-as the buffer is not empty.
    
-
---
-
-
-
-
-
-
    


    参数:
      
-
    • reader (callable) – the data reader to read from.
      • 
-
    • size (int) – max buffer size.
      • 
-
    
-
    返回:
    the buffered data reader.
    
-
    
-
    
-
-
    
-
    
-paddle.v2.reader.compose(*readers, **kwargs)
    
-
    Creates a data reader whose output is the combination of input readers.
    
-
    If input readers output following data entries:
-(1, 2)    3    (4, 5)
-The composed reader will output:
-(1, 2, 3, 4, 5)
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • readers – readers that will be composed together.
      • 
-
    • check_alignment (bool) – if True, will check if input readers are aligned
-correctly. If False, will not check alignment and trailing outputs
-will be discarded. Defaults to True.
      • 
-
    
-
    返回:
    the new data reader.
    
-
    引发:
    ComposeNotAligned – outputs of readers are not aligned.
-Will not raise when check_alignment is set to False.
    
-
    
-
    
-
-
    
-
    
-paddle.v2.reader.chain(*readers)
    
-
    Creates a data reader whose output is the outputs of input data
-readers chained together.
    
-
    If input readers output following data entries:
-[0, 0, 0]
-[1, 1, 1]
-[2, 2, 2]
-The chained reader will output:
-[0, 0, 0, 1, 1, 1, 2, 2, 2]
    
-
---
-
-
-
-
-
-
-
-
    


    参数:readers – input readers.
    返回:the new data reader.
    返回类型:callable
    
-
    
-
-
    
-
    
-paddle.v2.reader.shuffle(reader, buf_size)
    
-
    Creates a data reader whose data output is shuffled.
    
-
    Output from the iterator that created by original reader will be
-buffered into shuffle buffer, and then shuffled. The size of shuffle buffer
-is determined by argument buf_size.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • reader (callable) – the original reader whose output will be shuffled.
      • 
-
    • buf_size (int) – shuffle buffer size.
      • 
-
    
-
    返回:
    the new reader whose output is shuffled.
    
-
    返回类型:
    callable
    
-
    
-
    
-
-
    
-
    
-paddle.v2.reader.firstn(reader, n)
    
-
    Limit the max number of samples that reader could return.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • reader (callable) – the data reader to read from.
      • 
-
    • n (int) – the max number of samples that return.
      • 
-
    
-
    返回:
    the decorated reader.
    
-
    返回类型:
    callable
    
-
    
-
    
-
-
    
-
    Creator package contains some simple reader creator, which could be used in user
-program.
    
-
    
-
    
-paddle.v2.reader.creator.np_array(x)
    
-
    Creates a reader that yields elements of x, if it is a
-numpy vector. Or rows of x, if it is a numpy matrix.
-Or any sub-hyperplane indexed by the highest dimension.
    
-
---
-
-
-
-
-
-
    


    参数:x – the numpy array to create reader from.
    返回:data reader created from x.
    
-
    
-
-
    
-
    
-paddle.v2.reader.creator.text_file(path)
    
-
    Creates a data reader that outputs text line by line from given text file.
-Trailing new line (‘\n’) of each line will be removed.
    
-
---
-
-
-
-
-
-
    


    Path:path of the text file.
    返回:data reader of text file
    
-
    
-
-
    
-
    
-
    minibatch
    
-
    
-
    
-paddle.v2.minibatch.batch(reader, batch_size)
    
-
    Create a batched reader.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
    返回:
    the batched reader.
    
-
    返回类型:
    callable
    
-
    
-
    
-
-
    
-
    
-
    Dataset
    
-
    Dataset package.
    
-
    
-
    mnist
    
-
    MNIST dataset.
    
-
    This module will download dataset from http://yann.lecun.com/exdb/mnist/ and
-parse training set and test set into paddle reader creators.
    
-
    
-
    
-paddle.v2.dataset.mnist.train()
    
-
    MNIST training set creator.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 9].
    
-
---
-
-
-
-
-
-
    


    返回:Training reader creator
    返回类型:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.mnist.test()
    
-
    MNIST test set cretor.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 9].
    
-
---
-
-
-
-
-
-
    


    返回:Test reader creator.
    返回类型:callable
    
-
    
-
-
    
-
    
-
    cifar
    
-
    CIFAR dataset.
    
-
    This module will download dataset from
-https://www.cs.toronto.edu/~kriz/cifar.html and parse train/test set into
-paddle reader creators.
    
-
    The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes,
-with 6000 images per class. There are 50000 training images and 10000 test
-images.
    
-
    The CIFAR-100 dataset is just like the CIFAR-10, except it has 100 classes
-containing 600 images each. There are 500 training images and 100 testing
-images per class.
    
-
    
-
    
-paddle.v2.dataset.cifar.train100()
    
-
    CIFAR-100 training set creator.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 99].
    
-
---
-
-
-
-
-
-
    


    返回:Training reader creator
    返回类型:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.cifar.test100()
    
-
    CIFAR-100 test set cretor.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 9].
    
-
---
-
-
-
-
-
-
    


    返回:Test reader creator.
    返回类型:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.cifar.train10()
    
-
    CIFAR-10 training set creator.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 9].
    
-
---
-
-
-
-
-
-
    


    返回:Training reader creator
    返回类型:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.cifar.test10()
    
-
    CIFAR-10 test set cretor.
    
-
    It returns a reader creator, each sample in the reader is image pixels in
-[0, 1] and label in [0, 9].
    
-
---
-
-
-
-
-
-
    


    返回:Test reader creator.
    返回类型:callable
    
-
    
-
-
    
-
    
-
    conll05
    
-
    Conll05 dataset.
-Paddle semantic role labeling Book and demo use this dataset as an example.
-Because Conll05 is not free in public, the default downloaded URL is test set
-of Conll05 (which is public). Users can change URL and MD5 to their Conll
-dataset. And a pre-trained word vector model based on Wikipedia corpus is used
-to initialize SRL model.
    
-
    
-
    
-paddle.v2.dataset.conll05.get_dict()
    
-
    Get the word, verb and label dictionary of Wikipedia corpus.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.conll05.get_embedding()
    
-
    Get the trained word vector based on Wikipedia corpus.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.conll05.test()
    
-
    Conll05 test set creator.
    
-
    Because the training dataset is not free, the test dataset is used for
-training. It returns a reader creator, each sample in the reader is nine
-features, including sentence sequence, predicate, predicate context,
-predicate context flag and tagged sequence.
    
-
---
-
-
-
-
-
-
    


    返回:Training reader creator
    返回类型:callable
    
-
    
-
-
    
-
    
-
    imdb
    
-
    IMDB dataset.
    
-
    This module downloads IMDB dataset from
-http://ai.stanford.edu/%7Eamaas/data/sentiment/. This dataset contains a set
-of 25,000 highly polar movie reviews for training, and 25,000 for testing.
-Besides, this module also provides API for building dictionary.
    
-
    
-
    
-paddle.v2.dataset.imdb.build_dict(pattern, cutoff)
    
-
    Build a word dictionary from the corpus. Keys of the dictionary are words,
-and values are zero-based IDs of these words.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.imdb.train(word_idx)
    
-
    IMDB training set creator.
    
-
    It returns a reader creator, each sample in the reader is an zero-based ID
-sequence and label in [0, 1].
    
-
---
-
-
-
-
-
-
-
-
    


    参数:word_idx (dict) – word dictionary
    返回:Training reader creator
    返回类型:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.imdb.test(word_idx)
    
-
    IMDB test set creator.
    
-
    It returns a reader creator, each sample in the reader is an zero-based ID
-sequence and label in [0, 1].
    
-
---
-
-
-
-
-
-
-
-
    


    参数:word_idx (dict) – word dictionary
    返回:Test reader creator
    返回类型:callable
    
-
    
-
-
    
-
    
-
    imikolov
    
-
    imikolov’s simple dataset.
    
-
    This module will download dataset from
-http://www.fit.vutbr.cz/~imikolov/rnnlm/ and parse training set and test set
-into paddle reader creators.
    
-
    
-
    
-paddle.v2.dataset.imikolov.build_dict()
    
-
    Build a word dictionary from the corpus,  Keys of the dictionary are words,
-and values are zero-based IDs of these words.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.imikolov.train(word_idx, n)
    
-
    imikolov training set creator.
    
-
    It returns a reader creator, each sample in the reader is a word ID
-tuple.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • word_idx (dict) – word dictionary
      • 
-
    • n (int) – sliding window size
      • 
-
    
-
    返回:
    Training reader creator
    
-
    返回类型:
    callable
    
-
    
-
    
-
-
    
-
    
-paddle.v2.dataset.imikolov.test(word_idx, n)
    
-
    imikolov test set creator.
    
-
    It returns a reader creator, each sample in the reader is a word ID
-tuple.
    
-
---
-
-
-
-
-
-
-
-
    


    参数:
      
-
    • word_idx (dict) – word dictionary
      • 
-
    • n (int) – sliding window size
      • 
-
    
-
    返回:
    Test reader creator
    
-
    返回类型:
    callable
    
-
    
-
    
-
-
    
-
    
-
    movielens
    
-
    Movielens 1-M dataset.
    
-
    Movielens 1-M dataset contains 1 million ratings from 6000 users on 4000
-movies, which was collected by GroupLens Research. This module will download
-Movielens 1-M dataset from
-http://files.grouplens.org/datasets/movielens/ml-1m.zip and parse training
-set and test set into paddle reader creators.
    
-
    
-
    
-paddle.v2.dataset.movielens.get_movie_title_dict()
    
-
    Get movie title dictionary.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.max_movie_id()
    
-
    Get the maximum value of movie id.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.max_user_id()
    
-
    Get the maximum value of user id.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.max_job_id()
    
-
    Get the maximum value of job id.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.movie_categories()
    
-
    Get movie categoriges dictionary.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.user_info()
    
-
    Get user info dictionary.
    
-
    
-
-
    
-
    
-paddle.v2.dataset.movielens.movie_info()
    
-
    Get movie info dictionary.
    
-
    
-
-
    
-
    
-class paddle.v2.dataset.movielens.MovieInfo(index, categories, title)
    
-
    Movie id, title and categories information are stored in MovieInfo.
    
-
    
-
-
    
-
    
-class paddle.v2.dataset.movielens.UserInfo(index, gender, age, job_id)
    
-
    User id, gender, age, and job information are stored in UserInfo.
    
-
    
-
-
    
-
    
-
    sentiment
    
-
    The script fetch and preprocess movie_reviews data set that provided by NLTK
    
-
    TODO(yuyang18): Complete dataset.
    
-
    
-
    
-paddle.v2.dataset.sentiment.get_word_dict()
    
-
    Sorted the words by the frequency of words which occur in sample
-:return:
    
-
    
-
    words_freq_sorted
    
-
    
-
-
    
-
    
-paddle.v2.dataset.sentiment.train()
    
-
    Default training set reader creator
    
-
    
-
-
    
-
    
-paddle.v2.dataset.sentiment.test()
    
-
    Default test set reader creator
    
-
    
-
-
    
-
    
-
    uci_housing
    
-
    UCI Housing dataset.
    
-
    This module will download dataset from
-https://archive.ics.uci.edu/ml/machine-learning-databases/housing/ and
-parse training set and test set into paddle reader creators.
    
-
    
-
    
-paddle.v2.dataset.uci_housing.train()
    
-
    UCI_HOUSING training set creator.
    
-
    It returns a reader creator, each sample in the reader is features after
-normalization and price number.
    
-
---
-
-
-
-
-
-
    


    返回:Training reader creator
    返回类型:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.uci_housing.test()
    
-
    UCI_HOUSING test set creator.
    
-
    It returns a reader creator, each sample in the reader is features after
-normalization and price number.
    
-
---
-
-
-
-
-
-
    


    返回:Test reader creator
    返回类型:callable
    
-
    
-
-
    
-
    
-
    wmt14
    
-
    WMT14 dataset.
-The original WMT14 dataset is too large and a small set of data for set is
-provided. This module will download dataset from
-http://paddlepaddle.cdn.bcebos.com/demo/wmt_shrinked_data/wmt14.tgz and
-parse training set and test set into paddle reader creators.
    
-
    
-
    
-paddle.v2.dataset.wmt14.train(dict_size)
    
-
    WMT14 training set creator.
    
-
    It returns a reader creator, each sample in the reader is source language
-word ID sequence, target language word ID sequence and next word ID
-sequence.
    
-
---
-
-
-
-
-
-
    


    返回:Training reader creator
    返回类型:callable
    
-
    
-
-
    
-
    
-paddle.v2.dataset.wmt14.test(dict_size)
    
-
    WMT14 test set creator.
    
-
    It returns a reader creator, each sample in the reader is source language
-word ID sequence, target language word ID sequence and next word ID
-sequence.
    
-
---
-
-
-
-
-
-
    


    返回:Test reader creator
    返回类型:callable
    
-
    
-
-
    
 
    
 
    
 
@@ -1269,7 +227,7 @@ sequence.
    
   
     
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -85,6 +85,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +109,10 @@
           
      
 
    • 新手入门
 
      • 
-
    • 运行分布式训练
      • 
+
    • PaddlePaddle分布式训练
      • 
 
    • Kubernetes 简介
      • 
 
    • Kubernetes单机训练
      • 
 
    • Kubernetes分布式训练
      • 
-
    • 如何贡献/修改文档
      • 
 
    • 如何贡献代码
      • 
+
    • 如何贡献/修改文档
      • 
 
    • RNN相关模型
 
      • 
 
    • API
 
         
@@ -173,7 +195,7 @@
 
      
   
        
       
-    
      • Parameter Attributes
        • 
+    
      • Evaluator
        • 
   
      
 
      
       
@@ -182,98 +204,80 @@
           
      
            
      
             
-  
      
-
      Parameter Attributes
      
+  
      
+
      Evaluator
      
+
      
+
      Evaluator
      
 
      
-
      
-class paddle.trainer_config_helpers.attrs.ParameterAttribute(name=None, is_static=False, initial_std=None, initial_mean=None, initial_max=None, initial_min=None, l1_rate=None, l2_rate=None, learning_rate=None, momentum=None, gradient_clipping_threshold=None, sparse_update=False)
      
-
      Parameter Attributes object. To fine-tuning network training process, user
-can set attribute to control training details, such as l1,l2 rate / learning
-rate / how to init param.
      
-
      NOTE: IT IS A HIGH LEVEL USER INTERFACE.
      
+
      
+class paddle.v2.fluid.evaluator.Evaluator(name, **kwargs)
      
+
      Base Class for all evaluators
      
 
      
 
 
 
 
      参数:
        
-
      • is_static (bool) – True if this parameter will be fixed while training.
        • 
-
      • initial_std (float or None) – Gauss Random initialization standard deviation.
-None if not using Gauss Random initialize parameter.
        • 
-
      • initial_mean (float or None) – Gauss Random initialization mean.
-None if not using Gauss Random initialize parameter.
        • 
-
      • initial_max (float or None) – Uniform initialization max value.
        • 
-
      • initial_min (float or None) – Uniform initialization min value.
        • 
-
      • l1_rate (float or None) – the l1 regularization factor
        • 
-
      • l2_rate (float or None) – the l2 regularization factor
        • 
-
      • learning_rate (float or None) – The parameter learning rate. None means 1.
-The learning rate when optimize is LEARNING_RATE =
-GLOBAL_LEARNING_RATE * PARAMETER_LEARNING_RATE
-* SCHEDULER_FACTOR.
        • 
-
      • momentum (float or None) – The parameter momentum. None means use global value.
        • 
-
      • gradient_clipping_threshold (float) – gradient clipping threshold. If gradient
-value larger than some value, will be
-clipped.
        • 
-
      • sparse_update (bool) – Enable sparse update for this parameter. It will
-enable both local and remote sparse update.
        • 
+
      • name (str) – The name of evaluator. such as, “accuracy”. Used for generate
+temporary variable name.
        • 
+
      • main_program (Program, optional) – The evaluator should be added to this
+main_program. Default default_main_program()
        • 
+
      • startup_program (Program, optional) – The parameter should be added to this
+startup_program. Default default_startup_program()
        • 
 
      
       		
 
      
       
 
      
+
      
+
      
+states
      
+
      list – The list of state variables. states will be reset to zero
+when reset is invoked.
      
+
      
+
+
      
+
      
+metrics
      
+
      list – The list of metrics variables. They will be calculate
+every mini-batch
      
+
      
+
 
      
-
      
-set_default_parameter_name(name)
      
-
      Set default parameter name. If parameter not set, then will use default
-parameter name.
      
-
---
-
-
-
-
      


      参数:name (basestring) – default parameter name.
      
+
      
+reset(executor, reset_program=None)
      
+
      reset metric states at the begin of each pass/user specified batch
      
 
      
 
+
      
+
      
+eval(executor, eval_program=None)
      
+
      Evaluate the statistics merged by multiple mini-batches.
      
 
      
 
-
      
-
      
-class paddle.trainer_config_helpers.attrs.ExtraLayerAttribute(error_clipping_threshold=None, drop_rate=None, device=None)
      
-
      Some high level layer attributes config. You can set all attributes here,
-but some layer doesn’t support all attributes. If you set an attribute to a
-layer that not support this attribute, paddle will print an error and core.
      
+
      
+
      
+create_state(suffix, dtype, shape)
      
+
      Create state variable.
      
+
      NOTE: It is not a public API.
      
 
      
 
 
 
 
      参数:
        
-
      • error_clipping_threshold (float) – Error clipping threshold.
        • 
-
      • drop_rate (float) – Dropout rate. Dropout will create a mask on layer output.
-The dropout rate is the zero rate of this mask. The
-details of what dropout is please refer to here.
        • 
-
      • device (int) – 
        device ID of layer. device=-1, use CPU. device>0, use GPU.
-The details allocation in parallel_nn please refer to here.
        
-
        • 
+
      • suffix (str) – the state suffix.
        • 
+
      • dtype (str|core.DataType) – the state data type
        • 
+
      • shape (tuple|list) – the shape of state
        • 
 
      
       		
 
      
       
 
      
+
      Returns: State variable
      
 
      
 
-
      
-
      
-paddle.trainer_config_helpers.attrs.ParamAttr
      
-
      ParameterAttribute 的别名
      
-
      
-
-
      
-
      
-paddle.trainer_config_helpers.attrs.ExtraAttr
      
-
      ExtraLayerAttribute 的别名
      
 
      
 
+
      
 
      
 
 
diff --git a/doc_cn/api/v1/trainer_config_helpers/poolings.html b/doc_cn/api/v2/fluid/executor.html
similarity index 71%
rename from doc_cn/api/v1/trainer_config_helpers/poolings.html
rename to doc_cn/api/v2/fluid/executor.html
index 5bc182609871e5911070cd69fd1bb5f8fb7d88f8..78e5ce24943434ab25a2b974a65d240e1f1cb320 100644
--- a/doc_cn/api/v1/trainer_config_helpers/poolings.html
+++ b/doc_cn/api/v2/fluid/executor.html
@@ -8,7 +8,7 @@
   
   
   
-  Poolings — PaddlePaddle  文档
+  Executor — PaddlePaddle  文档
   
 
   
@@ -62,7 +62,7 @@
     
      
     
      
       
      
-        Folk me on Github
+        Fork me on Github
         
      
           
             English
@@ -85,6 +85,7 @@
 
    • 进阶指南
      • 
 
    • API
      • 
 
    • FAQ
      • 
+
    • MOBILE
      • 
 
    
 
         
@@ -108,9 +109,10 @@
           
 
         
@@ -108,9 +109,10 @@
           
      
 
    • 新手入门
 
      • 
-
    • 运行分布式训练
      • 
+
    • PaddlePaddle分布式训练
      • 
 
    • Kubernetes 简介
      • 
 
    • Kubernetes单机训练
      • 
 
    • Kubernetes分布式训练
      • 
-
    • 如何贡献/修改文档
      • 
 
    • 如何贡献代码
      • 
+
    • 如何贡献/修改文档
      • 
 
    • RNN相关模型
 
      • 
 
    • API
 
         
@@ -173,7 +195,7 @@
 
      
   
        
       
-    
      • Activations
        • 
+    
      • Initializer
        • 
   
      
 
      
       
@@ -182,140 +204,92 @@
           
      
            
      
             
-  
      
-
      Activations
      
-
      
-
      BaseActivation
      
-
      
-
      
-paddle.trainer_config_helpers.activations.BaseActivation
      
-
      Base 的别名
      
-
      
-
-
      
-
      
-
      AbsActivation
      
-
      
-
      
-paddle.trainer_config_helpers.activations.AbsActivation
      
-
      Abs 的别名
      
-
      
-
-
      
-
      
-
      ExpActivation
      
-
      
-
      
-paddle.trainer_config_helpers.activations.ExpActivation
      
-
      Exp 的别名
      
-
      
-
-
      
-
      
-
      IdentityActivation
      
-
      
-
      
-paddle.trainer_config_helpers.activations.IdentityActivation
      
-
      Linear 的别名
      
-
      
-
-
      
-
      
-
      LinearActivation
      
-
      
-
      
-paddle.trainer_config_helpers.activations.LinearActivation
      
-
      Linear 的别名
      
-
      
-
-
      
-
      
-
      LogActivation
      
-
      
-
      
-paddle.trainer_config_helpers.activations.LogActivation
      
-
      Log 的别名
      
-
      
-
-
      
-
      
-
      SquareActivation
      
-
      
-
      
-paddle.trainer_config_helpers.activations.SquareActivation
      
-
      Square 的别名
      
-
      
-
-
      
-
      
-
      SigmoidActivation
      
-
      
-
      
-paddle.trainer_config_helpers.activations.SigmoidActivation
      
-
      Sigmoid 的别名
      
-
      
-
-
      
-
      
-
      SoftmaxActivation
      
-
      
-
      
-paddle.trainer_config_helpers.activations.SoftmaxActivation
      
-
      Softmax 的别名
      
-
      
-
-
      
-
      
-
      SequenceSoftmaxActivation
      
-
      
+  
      
+
      Initializer
      
+
      
+
      Initializer
      
+
      
 
      
-paddle.trainer_config_helpers.activations.SequenceSoftmaxActivation
      
-
      SequenceSoftmax 的别名
      
+class paddle.v2.fluid.initializer.Initializer
+
      Base class for variable initializers
      
+
      Defines the common interface of variable initializers.
+They add operations to the init program that are used
+to initialize variables. Users should not use this class
+directly, but need to use one of its implementations.
      
 
      
 
 
      
-
      
-
      ReluActivation
      
-
      
+
      
+
      ConstantInitializer
      
+
      
 
      
-paddle.trainer_config_helpers.activations.ReluActivation
      
-
      Relu 的别名
      
+class paddle.v2.fluid.initializer.ConstantInitializer(value=0.0)
+
      Implements the constant initializer
      
 
      
 
 
      
-
      
-
      BReluActivation
      
-
      
+
      
+
      UniformInitializer
      
+
      
 
      
-paddle.trainer_config_helpers.activations.BReluActivation
      
-
      BRelu 的别名
      
+class paddle.v2.fluid.initializer.UniformInitializer(low=-1.0, high=1.0, seed=0)
+
      Implements the random uniform distribution initializer
      
 
      
 
 
      
-
      
-
      SoftReluActivation
      
-
      
+
      
+
      NormalInitializer
      
+
      
 
      
-paddle.trainer_config_helpers.activations.SoftReluActivation
      
-
      SoftRelu 的别名
      
+class paddle.v2.fluid.initializer.NormalInitializer(loc=0.0, scale=1.0, seed=0)
+
      Implements the  random Normal(Gaussian) distribution initializer
      
 
      
 
 
      
-
      
-
      TanhActivation
      
-
      
+
      
+
      XavierInitializer
      
+
      
 
      
-paddle.trainer_config_helpers.activations.TanhActivation
      
-
      Tanh 的别名
      
+class paddle.v2.fluid.initializer.XavierInitializer(uniform=True, fan_in=None, fan_out=None, seed=0)
+
      Implements the Xavier initializer
      
+
      This class implements the Xavier weight initializer from the paper
+Understanding the difficulty of training deep feedforward neural
+networks[1] by Xavier Glorot and Yoshua Bengio.
      
+
      This initializer is designed to keep the scale of the gradients
+approximately same in all the layers. In case of Uniform distribution,
+the range is [-x, x], where x = sqrt(6 / (fan_in + fan_out)).
+In case of Normal distribution, the mean is 0 and the standard deviation
+is sqrt(2/ (fan_in + fan_out)).
      
+
      References
      
+
      
+
      [1] Understanding the difficulty of training deep feedforward neural
      
+
      networks. International conference on artificial intelligence and
+statistics.
+(http://proceedings.mlr.press/v9/glorot10a.html)
      
+
      
 
      
 
 
      
-
      
-
      STanhActivation
      
-
      
+
      
+
      MSRAInitializer
      
+
      
 
      
-paddle.trainer_config_helpers.activations.STanhActivation
      
-
      STanh 的别名
      
+class paddle.v2.fluid.initializer.MSRAInitializer(uniform=True, fan_in=None, seed=0)
+
      Implements the MSRA initializer a.k.a. Kaiming Initializer
      
+
      This class implements the weight initialization from the paper
+Delving Deep into Rectifiers: Surpassing Human-Level Performance on
+ImageNet Classification[1] by Kaiming He, Xiangyu Zhang, Shaoqing Ren
+and Jian Sun. This is a robust initialization method that particularly
+considers the rectifier nonlinearities. In case of Uniform distribution,
+the range is [-x, x], where x = sqrt(6 / fan_in). In case of Normal
+distribution, the mean is 0 and the standard deviation
+is sqrt(2/ fan_in).
      
+
      References
      
+
      
+
      [1] Delving Deep into Rectifiers: Surpassing Human-Level Performance
      
+
      on ImageNet Classification
+(https://arxiv.org/abs/1502.01852)
      
+
      
 
      
 
 
      
diff --git a/doc_cn/api/v2/fluid/layers.html b/doc_cn/api/v2/fluid/layers.html
new file mode 100644
index 0000000000000000000000000000000000000000..3971fe6ac65412efa305880b64c80495952a6175
--- /dev/null
+++ b/doc_cn/api/v2/fluid/layers.html
@@ -0,0 +1,1124 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Layers — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • Layers
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      Layers
      
+
      
+
      fc
      
+
      
+
      
+paddle.v2.fluid.layers.fc(input, size, num_flatten_dims=1, param_attr=None, bias_attr=None, act=None, name=None, main_program=None, startup_program=None)
      
+
      Fully Connected Layer.
      
+
+++
+
+
+
+
      


      参数:
        
+
      • input – The input tensor to the function
        • 
+
      • size – The size of the layer
        • 
+
      • num_flatten_dims – Number of columns in input
        • 
+
      • param_attr – The parameters/weights to the FC Layer
        • 
+
      • param_initializer – Initializer used for the weight/parameter. If None, XavierInitializer() is used
        • 
+
      • bias_attr – The bias parameter for the FC layer
        • 
+
      • bias_initializer – Initializer used for the bias. If None, then ConstantInitializer() is used
        • 
+
      • act – Activation to be applied to the output of FC layer
        • 
+
      • name – Name/alias of the function
        • 
+
      • main_program – Name of the main program that calls this
        • 
+
      • startup_program – Name of the startup program
        • 
+
      
+
      
+
      This function can take in multiple inputs and performs the Fully Connected
+function (linear transformation) on top of each of them.
+So for input x, the output will be : Wx + b. Where W is the parameter,
+b the bias and x is the input.
      
+
      The function also applies an activation (non-linearity) on top of the
+output, if activation is passed in the input.
      
+
      All the input variables of this function are passed in as local variables
+to the LayerHelper constructor.
      
+
      
+
+
      
+
      
+
      embedding
      
+
      
+
      
+paddle.v2.fluid.layers.embedding(input, size, is_sparse=False, param_attr=None, dtype='float32', main_program=None, startup_program=None)
      
+
      Embedding Layer.
      
+
+++
+
+
+
+
      


      参数:
        
+
      • param_initializer – 
        • 
+
      • input – The input to the function
        • 
+
      • size – The size of the layer
        • 
+
      • is_sparse – A flag that decleares whether the input is sparse
        • 
+
      • param_attr – Parameters for this layer
        • 
+
      • dtype – The type of data : float32, float_16, int etc
        • 
+
      • main_program – Name of the main program that calls this
        • 
+
      • startup_program – Name of the startup program
        • 
+
      
+
      
+
      This function can take in the input (which is a vector of IDs) and
+performs a lookup in the lookup_table using these IDs, to result into
+the embedding of each ID in the input.
      
+
      All the input variables of this function are passed in as local variables
+to the LayerHelper constructor.
      
+
      
+
+
      
+
      
+
      dynamic_lstm
      
+
      
+
      
+paddle.v2.fluid.layers.dynamic_lstm(input, size, param_attr=None, bias_attr=None, use_peepholes=True, is_reverse=False, gate_activation='sigmoid', cell_activation='tanh', candidate_activation='tanh', dtype='float32', main_program=None, startup_program=None)
      
+
      
+
+
      
+
      
+
      data
      
+
      
+
      
+paddle.v2.fluid.layers.data(name, shape, append_batch_size=True, dtype='float32', lod_level=0, type=VarType.LOD_TENSOR, main_program=None, startup_program=None, stop_gradient=True)
      
+
      Data Layer.
      
+
+++
+
+
+
+
      


      参数:
        
+
      • name – The name/alias of the function
        • 
+
      • shape – Tuple declaring the shape.
        • 
+
      • append_batch_size – Whether or not to append the data as a batch.
        • 
+
      • dtype – The type of data : float32, float_16, int etc
        • 
+
      • type – The output type. By default it is LOD_TENSOR.
        • 
+
      • lod_level (int) – The LoD Level. 0 means the input data is not a sequence.
        • 
+
      • main_program – Name of the main program that calls this
        • 
+
      • startup_program – Name of the startup program
        • 
+
      • stop_gradient – A boolean that mentions whether gradient should flow.
        • 
+
      
+
      
+
      This function takes in input and based on whether data has
+to be returned back as a minibatch, it creates the global variable using
+the helper functions. The global variables can be accessed by all the
+following operations and layers in the graph.
      
+
      All the input variables of this function are passed in as local variables
+to the LayerHelper constructor.
      
+
      
+
+
      
+
      
+
      mean
      
+
      
+
      
+paddle.v2.fluid.layers.mean(**kwargs)
      
+
      Mean Operator.
      
+
      Out is a scalar which is the mean of all elements in X.
      
+
+++
+
+
+
+
+
+
      


      参数:x – The input of mean op
+Duplicable: False  Optional: False
      返回:The output of mean op
      
+
      
+
+
      
+
      
+
      mul
      
+
      
+
      
+paddle.v2.fluid.layers.mul(**kwargs)
      
+
      Mul Operator.
      
+
      This operator is used to perform matrix multiplication for input X and Y.
      
+
      The equation is:
      
+
      
+
      $$Out = X * Y$$
      
+
      Both the input X and Y can carry the LoD (Level of Details) information,
+or not. But the output only shares the LoD information with input X.
      
+
+++
+
+
+
+
+
+
      


      参数:
        
+
      • x – The first input of mul op
+Duplicable: False  Optional: False
        • 
+
      • y – The second input of mul op
+Duplicable: False  Optional: False
        • 
+
      • x_num_col_dims (INT) – (int, default 1) mul_op can take tensors with more than two dimensions as input X,
+in that case, tensors will be reshaped to a matrix. The matrix’s first
+dimension(column length) will be the product of tensor’s last
+num_col_dims dimensions, and the matrix’s second dimension(row length)
+will be the product of tensor’s first rank - num_col_dims dimensions.
        • 
+
      • y_num_col_dims (INT) – (int, default 1) mul_op can take tensors with more than two dimensions as input Y,
+in that case, tensors will be reshaped to a matrix. Just like input X.
        • 
+
      
+
      返回:
      The output of mul op
      
+
      
+
      
+
+
      
+
      
+
      elementwise_add
      
+
      
+
      
+paddle.v2.fluid.layers.elementwise_add(**kwargs)
      
+
      Limited Elementwise Add Operator.
      
+
      The equation is:
      
+
      $Out = X + Y$
      
+
      X is a tensor of any dimension and the dimensions of tensor Y must be smaller than
+or equal to the dimensions of X.
      
+
      There are two cases for this operator:
+1. The shape of Y is same with X;
+2. The shape of Y is a subset of X.
      
+
      For case 2:
+Y will be broadcasted to match the shape of X and axis should be
+the starting dimension index for broadcasting Y onto X.
      
+
      example
      
+
      shape(X) = (2, 3, 4, 5), shape(Y) = (,)
+shape(X) = (2, 3, 4, 5), shape(Y) = (5,)
+shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5)
+shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1
+shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0
      
+
      Both the input X and Y can carry the LoD (Level of Details) information,
+or not. But the output only shares the LoD information with input X.
      
+
+++
+
+
+
+
+
+
      


      参数:
        
+
      • x – (Tensor) The first input tensor of elementwise op
+Duplicable: False  Optional: False
        • 
+
      • y – (Tensor) The second input tensor of elementwise op
+Duplicable: False  Optional: False
        • 
+
      • axis (INT) – (int, default -1) The starting dimension index for broadcasting Y onto X
        • 
+
      
+
      返回:
      The output of elementwise op
      
+
      
+
      
+
+
      
+
      
+
      elementwise_div
      
+
      
+
      
+paddle.v2.fluid.layers.elementwise_div(**kwargs)
      
+
      Limited Elementwise Div Operator.
      
+
      The equation is:
      
+
      $Out = X / Y$
      
+
      X is a tensor of any dimension and the dimensions of tensor Y must be smaller than
+or equal to the dimensions of X.
      
+
      There are two cases for this operator:
+1. The shape of Y is same with X;
+2. The shape of Y is a subset of X.
      
+
      For case 2:
+Y will be broadcasted to match the shape of X and axis should be
+the starting dimension index for broadcasting Y onto X.
      
+
      example
      
+
      shape(X) = (2, 3, 4, 5), shape(Y) = (,)
+shape(X) = (2, 3, 4, 5), shape(Y) = (5,)
+shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5)
+shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1
+shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0
      
+
      Both the input X and Y can carry the LoD (Level of Details) information,
+or not. But the output only shares the LoD information with input X.
      
+
+++
+
+
+
+
+
+
      


      参数:
        
+
      • x – (Tensor) The first input tensor of elementwise op
+Duplicable: False  Optional: False
        • 
+
      • y – (Tensor) The second input tensor of elementwise op
+Duplicable: False  Optional: False
        • 
+
      • axis (INT) – (int, default -1) The starting dimension index for broadcasting Y onto X
        • 
+
      
+
      返回:
      The output of elementwise op
      
+
      
+
      
+
+
      
+
      
+
      dropout
      
+
      
+
      
+paddle.v2.fluid.layers.dropout(**kwargs)
      
+
      Dropout Operator.
      
+
      Dropout refers to randomly dropping out units in a nerual network. It is a
+regularization technique for reducing overfitting by preventing neuron
+co-adaption during training. The dropout operator randomly set (according to
+the given dropout probability) the outputs of some units to zero, while others
+are set equal to their corresponding inputs.
      
+
+++
+
+
+
+
+
+
      


      参数:
        
+
      • x – The input of dropout op.
+Duplicable: False  Optional: False
        • 
+
      • dropout_prob (FLOAT) – Probability of setting units to zero.
        • 
+
      • is_test (BOOLEAN) – True if in test phase.
        • 
+
      • seed (INT) – Dropout random seed.
        • 
+
      
+
      返回:
      The output of dropout op.
      
+
      
+
      
+
+
      
+
      
+
      reshape
      
+
      
+
      
+paddle.v2.fluid.layers.reshape(**kwargs)
      
+
      Reshape Operator.
      
+
      Reshape Input(X) into the shape specified by Attr(shape).
      
+
      An example:
+Given a 2-D tensor X with 2 rows and 2 columns
      
+
      
+
      [[1, 2], [3, 4]]
      
+
      and target shape = [1, 4], the reshape operator will transform
+the tensor X into a 1-D tensor:
      
+
      
+
      [1, 2, 3, 4]
      
+
+++
+
+
+
+
+
+
      


      参数:
        
+
      • x – The input tensor of reshape operator.
+Duplicable: False  Optional: False
        • 
+
      • shape (INTS) – (vector<int>) Target shape of reshape operator.
        • 
+
      
+
      返回:
      The output tensor of reshape operator.
      
+
      
+
      
+
+
      
+
      
+
      sigmoid
      
+
      
+
      
+paddle.v2.fluid.layers.sigmoid(**kwargs)
      
+
      Sigmoid Activation Operator
      
+
      $$y = frac{1}{1 + e^{-x}}$$
      
+
+++
+
+
+
+
+
+
      


      参数:x – Input of Sigmoid operator
+Duplicable: False  Optional: False
      返回:Output of Sigmoid operator
      
+
      
+
+
      
+
      
+
      scale
      
+
      
+
      
+paddle.v2.fluid.layers.scale(**kwargs)
      
+
      Scale operator
      
+
      $$Out = scale*X$$
      
+
+++
+
+
+
+
+
+
      


      参数:
        
+
      • x – (Tensor) Input tensor of scale operator.
+Duplicable: False  Optional: False
        • 
+
      • scale (FLOAT) – (float, default 0)The scaling factor of the scale operator.
        • 
+
      
+
      返回:
      (Tensor) Output tensor of scale operator.
      
+
      
+
      
+
+
      
+
      
+
      reshape
      
+
      
+
      
+paddle.v2.fluid.layers.reshape(**kwargs)
      
+
      Reshape Operator.
      
+
      Reshape Input(X) into the shape specified by Attr(shape).
      
+
      An example:
+Given a 2-D tensor X with 2 rows and 2 columns
      
+
      
+
      [[1, 2], [3, 4]]
      
+
      and target shape = [1, 4], the reshape operator will transform
+the tensor X into a 1-D tensor:
      
+
      
+
      [1, 2, 3, 4]
      
+
+++
+
+
+
+
+
+
      


      参数:
        
+
      • x – The input tensor of reshape operator.
+Duplicable: False  Optional: False
        • 
+
      • shape (INTS) – (vector<int>) Target shape of reshape operator.
        • 
+
      
+
      返回:
      The output tensor of reshape operator.
      
+
      
+
      
+
+
      
+
      
+
      transpose
      
+
      
+
      
+paddle.v2.fluid.layers.transpose(**kwargs)
      
+
      Transpose Operator.
      
+
      The input tensor will be permuted according to the axis values given.
+The op functions similar to how numpy.transpose works in python.
+For example:
      
+
      
+
      >> input = numpy.arange(6).reshape((2,3))
+>> input
+array([[0, 1, 2],
      
+
      
+
      [3, 4, 5]])
      
+
      >> axis = [1, 0]
+>> output = input.transpose(axis)
+>> output
+array([[0, 3],
      
+
      
+
      
+
      [1, 4],
      
+
      [2, 5]])
      
+
      
+
      
+
      
+
      So, given a input tensor of shape(N, C, H, W) and the axis is {0, 2, 3, 1},
+the output tensor shape will be (N, H, W, C)
      
+
+++
+
+
+
+
+
+
      


      参数:
        
+
      • x – (Tensor)The input tensor, tensors with rank at most 6 are supported
+Duplicable: False  Optional: False
        • 
+
      • axis (INTS) – (vector<int>)A list of values, and the size of the list should be the same with the input tensor rank, the tensor will permute the axes according the the values given
        • 
+
      
+
      返回:
      (Tensor)The output tensor
      
+
      
+
      
+
+
      
+
      
+
      sigmoid_cross_entropy_with_logits
      
+
      
+
      
+
      cast
      
+
      
+
      
+paddle.v2.fluid.layers.cast(x, dtype, main_program=None)
      
+
      This function takes in the input with input_dtype
+and casts it to the output_dtype as the output.
      
+
      
+
+
      
+
      
+
      concat
      
+
      
+
      
+paddle.v2.fluid.layers.concat(input, axis, main_program=None, startup_program=None)
      
+
      This function concats the input along the axis mentioned
+and returns that as the output.
      
+
      
+
+
      
+
      
+
      sums
      
+
      
+
      
+paddle.v2.fluid.layers.sums(input, out=None, main_program=None, startup_program=None)
      
+
      This function takes in the input and performs the sum operation on it
+and returns that as the output.
      
+
      
+
+
      
+
      
+
      linear_chain_crf
      
+
      
+
      
+paddle.v2.fluid.layers.linear_chain_crf(input, label, param_attr=None, main_program=None, startup_program=None)
      
+
      
+
+
      
+
      
+
      assign
      
+
      
+
      
+paddle.v2.fluid.layers.embedding(input, size, is_sparse=False, param_attr=None, dtype='float32', main_program=None, startup_program=None)
      
+
      Embedding Layer.
      
+
+++
+
+
+
+
      


      参数:
        
+
      • param_initializer – 
        • 
+
      • input – The input to the function
        • 
+
      • size – The size of the layer
        • 
+
      • is_sparse – A flag that decleares whether the input is sparse
        • 
+
      • param_attr – Parameters for this layer
        • 
+
      • dtype – The type of data : float32, float_16, int etc
        • 
+
      • main_program – Name of the main program that calls this
        • 
+
      • startup_program – Name of the startup program
        • 
+
      
+
      
+
      This function can take in the input (which is a vector of IDs) and
+performs a lookup in the lookup_table using these IDs, to result into
+the embedding of each ID in the input.
      
+
      All the input variables of this function are passed in as local variables
+to the LayerHelper constructor.
      
+
      
+
+
      
+
      
+
      split_lod_tensor
      
+
      
+
      
+paddle.v2.fluid.layers.split_lod_tensor(input, mask, level=0, main_program=None, startup_program=None)
      
+
      
+
+
      
+
      
+
      merge_lod_tensor
      
+
      
+
      
+paddle.v2.fluid.layers.merge_lod_tensor(in_true, in_false, x, mask, level=0, main_program=None, startup_program=None)
      
+
      
+
+
      
+
      
+
      cos_sim
      
+
      
+
      
+paddle.v2.fluid.layers.cos_sim(X, Y, **kwargs)
      
+
      This function performs the cosine similarity between two tensors
+X and Y and returns that as the output.
      
+
      
+
+
      
+
      
+
      cross_entropy
      
+
      
+
      
+paddle.v2.fluid.layers.cross_entropy(input, label, **kwargs)
      
+
      This function computes cross_entropy using the input and label.
      
+
      
+
+
      
+
      
+
      square_error_cost
      
+
      
+
      
+paddle.v2.fluid.layers.square_error_cost(input, label, **kwargs)
      
+
      This functions returns the squared error cost using the input and label.
+The output is appending the op to do the above.
      
+
      
+
+
      
+
      
+
      accuracy
      
+
      
+
      
+paddle.v2.fluid.layers.accuracy(input, label, k=1, correct=None, total=None, **kwargs)
      
+
      This function computes the accuracy using the input and label.
+The output is the top_k inputs and their indices.
      
+
      
+
+
      
+
      
+
      sequence_conv
      
+
      
+
      
+paddle.v2.fluid.layers.sequence_conv(input, num_filters, filter_size=3, filter_stride=1, padding=None, bias_attr=None, param_attr=None, act=None, main_program=None, startup_program=None)
      
+
      This function creates the op for sequence_conv, using the inputs and
+other convolutional configurations for the filters and stride as given
+in the input parameters to the function.
      
+
      
+
+
      
+
      
+
      conv2d
      
+
      
+
      
+paddle.v2.fluid.layers.conv2d(input, num_filters, filter_size, stride=[1, 1], padding=None, groups=None, param_attr=None, bias_attr=None, act=None, name=None, main_program=None, startup_program=None)
      
+
      This function creates the op for a 2-dimensional Convolution.
+This is performed using the parameters of filters(size, dimensionality etc)
+, stride and other configurations for a Convolution operation.
+This funciton can also append an activation on top of the
+conv-2d output, if mentioned in the input parameters.
      
+
      
+
+
      
+
      
+
      sequence_pool
      
+
      
+
      
+paddle.v2.fluid.layers.sequence_pool(input, pool_type, **kwargs)
      
+
      This function add the operator for sequence pooling.
+This is applied on top of the input using pool_type mentioned
+in the parameters.
      
+
      
+
+
      
+
      
+
      pool2d
      
+
      
+
      
+paddle.v2.fluid.layers.pool2d(input, pool_size, pool_type, pool_stride=[1, 1], pool_padding=[0, 0], global_pooling=False, main_program=None, startup_program=None)
      
+
      This function adds the operator for pooling in 2 dimensions, using the
+pooling configurations mentioned in input parameters.
      
+
      
+
+
      
+
      
+
      batch_norm
      
+
      
+
      
+paddle.v2.fluid.layers.batch_norm(input, act=None, is_test=False, momentum=0.9, epsilon=1e-05, param_attr=None, bias_attr=None, data_layout='NCHW', main_program=None, startup_program=None)
      
+
      This function helps create an operator to implement
+the BatchNorm layer using the configurations from the input parameters.
      
+
      
+
+
      
+
      
+
      beam_search_decode
      
+
      
+
      
+paddle.v2.fluid.layers.beam_search_decode(ids, scores, main_program=None, startup_program=None)
      
+
      
+
+
      
+
      
+
      lstm
      
+
      
+
      
+paddle.v2.fluid.layers.lstm(x, c_pre_init, hidden_dim, forget_bias=None, main_program=None, startup_program=None)
      
+
      This function helps create an operator for the LSTM (Long Short Term
+Memory) cell that can be used inside an RNN.
      
+
      
+
+
      
+
      
+
      lod_rank_table
      
+
      
+
      
+paddle.v2.fluid.layers.lod_rank_table(x, level=0, main_program=None)
      
+
      This function creates an operator for creating a LOD_RANK_TABLE
+using the input x.
      
+
      
+
+
      
+
      
+
      max_sequence_len
      
+
      
+
      
+paddle.v2.fluid.layers.max_sequence_len(rank_table, main_program=None)
      
+
      This function creates an operator to calculate the length of
+max seqence through input rank_table(should be a lod_rank_table)
      
+
      
+
+
      
+
      
+
      topk
      
+
      
+
      
+paddle.v2.fluid.layers.topk(input, k, main_program=None, startup_program=None)
      
+
      
+
+
      
+
      
+
      lod_tensor_to_array
      
+
      
+
      
+paddle.v2.fluid.layers.lod_tensor_to_array(x, table, main_program=None)
      
+
      This function creates an operator to convert an LOD_Tensor to
+an array.
      
+
      
+
+
      
+
      
+
      array_to_lod_tensor
      
+
      
+
      
+paddle.v2.fluid.layers.array_to_lod_tensor(x, table, main_program=None, startup_program=None)
      
+
      This function creates an operator to convert an array to a
+LOD_Tensor.
      
+
      
+
+
      
+
      
+
      fill_constant
      
+
      
+
      
+paddle.v2.fluid.layers.fill_constant(shape, dtype, value, out=None, main_program=None, startup_program=None)
      
+
      This function creates a tensor , with shape as mentioned in the input and
+specified dtype and fills this up with a constant value that
+comes in the input. It also sets the stop_gradient to be True.
      
+
      
+
+
      
+
      
+
      fill_constant_batch_size_like
      
+
      
+
      
+paddle.v2.fluid.layers.fill_constant_batch_size_like(input, shape, dtype, value, input_dim_idx=0, output_dim_idx=0, main_program=None, startup_program=None)
      
+
      
+
+
      
+
      
+
      ones
      
+
      
+
      
+paddle.v2.fluid.layers.ones(shape, dtype, main_program=None)
      
+
      This function performs the same function as fill_constant() declared above
+with the constant value being 1.0.
      
+
      
+
+
      
+
      
+
      zeros
      
+
      
+
      
+paddle.v2.fluid.layers.zeros(shape, dtype, main_program=None)
      
+
      This function performs the same function as fill_constant() declared above
+with the constant value being 0.0.
      
+
      
+
+
      
+
      
+
      increment
      
+
      
+
      
+paddle.v2.fluid.layers.increment(x, value=1.0, in_place=True, main_program=None, startup_program=None)
      
+
      This function creates an operator to increment each value in the input
+x by an amount: value as mentioned in the input parameter. This
+operation is performed in-place by default.
      
+
      
+
+
      
+
      
+
      array_write
      
+
      
+
      
+paddle.v2.fluid.layers.array_write(x, i, array=None, main_program=None, startup_program=None)
      
+
      This function creates an operator to write the data out as a
+LOD_TENSOR_ARRAY.
      
+
      
+
+
      
+
      
+
      create_array
      
+
      
+
      
+paddle.v2.fluid.layers.create_array(dtype, main_program=None)
      
+
      
+
+
      
+
      
+
      less_than
      
+
      
+
      
+paddle.v2.fluid.layers.less_than(x, y, cond=None, main_program=None, **ignored)
      
+
      
+
+
      
+
      
+
      array_read
      
+
      
+
      
+paddle.v2.fluid.layers.array_read(array, i, main_program=None, startup_program=None)
      
+
      This function creates an operator to read the data in as a
+LOD_TENSOR_ARRAY.
      
+
      
+
+
      
+
      
+
      shrink_memory
      
+
      
+
      
+paddle.v2.fluid.layers.shrink_memory(x, i, table, main_program=None, startup_program=None)
      
+
      This function creates an operator to shrink_rnn_memory using the RankTable
+as mentioned in the input parameter.
      
+
      
+
+
      
+
      
+
      array_length
      
+
      
+
      
+paddle.v2.fluid.layers.array_length(array, main_program=None)
      
+
      This function creates an operator to find the length of the
+LOD_TENSOR_ARRAY.
      
+
      
+
+
      
+
      
+
      conv2d_transpose
      
+
      
+
      
+paddle.v2.fluid.layers.conv2d_transpose(input, num_filters, output_size=None, filter_size=None, padding=None, stride=None, param_attr=None, main_program=None, startup_program=None)
      
+
      The transpose of conv2d layer.
      
+
      This layer is also known as deconvolution layer.
      
+
+++
+
+
+
+
+
+
+
+
      


      参数:
        
+
      • input (Variable) – The input image with [N, C, H, W] format.
        • 
+
      • num_filters (int) – The number of filter. It is as same as the output
+image channel.
        • 
+
      • output_size (int|tuple|None) – The output image size. If output size is a
+tuple, it must contain two integers, (image_H, image_W). This
+parameter only works when filter_size is None.
        • 
+
      • filter_size (int|tuple|None) – The filter size. If filter_size is a tuple,
+it must contain two integers, (filter_size_H, filter_size_W).
+Otherwise, the filter will be a square.  None if use output size to
+calculate filter_size
        • 
+
      • padding (int|tuple) – The padding size. If padding is a tuple, it must
+contain two integers, (padding_H, padding_W). Otherwise, the
+padding_H = padding_W = padding.
        • 
+
      • stride (int|tuple) – The stride size. If stride is a tuple, it must
+contain two integers, (stride_H, stride_W). Otherwise, the
+stride_H = stride_W = stride.
        • 
+
      • param_attr – Parameter Attribute.
        • 
+
      • main_program (Program) – the main program
        • 
+
      • startup_program (Program) – the startup program
        • 
+
      
+
      返回:
      Output image.
      
+
      返回类型:
      Variable
      
+
      
+
      
+
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/api/v2/fluid/nets.html b/doc_cn/api/v2/fluid/nets.html
new file mode 100644
index 0000000000000000000000000000000000000000..8fe779d65d481c5c8183eeefd4ca4310f286edd3
--- /dev/null
+++ b/doc_cn/api/v2/fluid/nets.html
@@ -0,0 +1,293 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Nets — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • Nets
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      Nets
      
+
      
+
      simple_img_conv_pool
      
+
      
+
      
+paddle.v2.fluid.nets.simple_img_conv_pool(input, num_filters, filter_size, pool_size, pool_stride, act, pool_type='max', main_program=None, startup_program=None)
      
+
      
+
+
      
+
      
+
      img_conv_group
      
+
      
+
      
+paddle.v2.fluid.nets.img_conv_group(input, conv_num_filter, pool_size, conv_padding=1, conv_filter_size=3, conv_act=None, conv_with_batchnorm=False, conv_batchnorm_drop_rate=None, pool_stride=1, pool_type=None, main_program=None, startup_program=None)
      
+
      Image Convolution Group, Used for vgg net.
      
+
      
+
+
      
+
      
+
      sequence_conv_pool
      
+
      
+
      
+paddle.v2.fluid.nets.sequence_conv_pool(input, num_filters, filter_size, act='sigmoid', pool_type='max', main_program=None, startup_program=None)
      
+
      
+
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/api/v2/fluid/optimizer.html b/doc_cn/api/v2/fluid/optimizer.html
new file mode 100644
index 0000000000000000000000000000000000000000..5aea41ac8e5e70f58dd3391a58b2e4cb7b0b78a0
--- /dev/null
+++ b/doc_cn/api/v2/fluid/optimizer.html
@@ -0,0 +1,370 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Optimizer — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • Optimizer
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      Optimizer
      
+
      
+
      Optimizer
      
+
      
+
      
+class paddle.v2.fluid.optimizer.Optimizer(global_step=None)
      
+
      Optimizer Base class.
      
+
      Define the common interface of an optimizer.
+User should not use this class directly,
+but need to use one of it’s implementation.
      
+
      
+
      
+create_optimization_pass(parameters_and_grads, loss, startup_program=None)
      
+
      Add optimization operators to update gradients to variables.
      
+
+++
+
+
+
+
+
+
+
+
      


      参数:
        
+
      • loss – the target that this optimization is for.
        • 
+
      • parameters_and_grads – a list of (variable, gradient) pair to update.
        • 
+
      
+
      返回:
      a list of operators that will complete one step of
+optimization. This will include parameter update ops, global step
+update ops and any other custom ops required by subclasses to manage
+their internal state.
+:param startup_program:
      
+
      返回类型:
      return_op_list
      
+
      
+
      
+
+
      
+
      
+minimize(loss, startup_program=None, parameter_list=None, no_grad_set=None)
      
+
      Add operations to minimize loss by updating parameter_list.
      
+
      This method combines interface append_backward_ops() and
+create_optimization_pass() into one.
      
+
      
+
+
      
+
+
      
+
      
+
      SGDOptimizer
      
+
      
+
      
+class paddle.v2.fluid.optimizer.SGDOptimizer(learning_rate, global_step=None)
      
+
      Simple SGD optimizer without any state.
      
+
      
+
+
      
+
      
+
      MomentumOptimizer
      
+
      
+
      
+class paddle.v2.fluid.optimizer.MomentumOptimizer(learning_rate, momentum, use_nesterov=False, global_step=None)
      
+
      Simple Momentum optimizer with velocity state
      
+
      
+
+
      
+
      
+
      AdagradOptimizer
      
+
      
+
      
+class paddle.v2.fluid.optimizer.AdagradOptimizer(learning_rate, epsilon=1e-06, global_step=None)
      
+
      Simple Adagrad optimizer with moment state
      
+
      
+
+
      
+
      
+
      AdamOptimizer
      
+
      
+
      
+class paddle.v2.fluid.optimizer.AdamOptimizer(learning_rate=0.001, beta1=0.9, beta2=0.999, epsilon=1e-08, global_step=None)
      
+
      Implements the Adam Optimizer
      
+
      
+
+
      
+
      
+
      AdamaxOptimizer
      
+
      
+
      
+class paddle.v2.fluid.optimizer.AdamaxOptimizer(learning_rate=0.001, beta1=0.9, beta2=0.999, epsilon=1e-08, global_step=None)
      
+
      Implements the Adamax Optimizer
      
+
      
+
+
      
+
      
+
      DecayedAdagradOptimizer
      
+
      
+
      
+class paddle.v2.fluid.optimizer.DecayedAdagradOptimizer(learning_rate, decay=0.95, epsilon=1e-06, global_step=None)
      
+
      Simple Decayed Adagrad optimizer with moment state
      
+
      
+
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/api/v2/fluid/param_attr.html b/doc_cn/api/v2/fluid/param_attr.html
new file mode 100644
index 0000000000000000000000000000000000000000..6b5ce57a1748e26703763c133a2eb852837ebe7e
--- /dev/null
+++ b/doc_cn/api/v2/fluid/param_attr.html
@@ -0,0 +1,271 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  ParamAttr — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • ParamAttr
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      ParamAttr
      
+
      
+
      ParamAttr
      
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/api/v1/trainer_config_helpers/data_sources.html b/doc_cn/api/v2/fluid/profiler.html
similarity index 64%
rename from doc_cn/api/v1/trainer_config_helpers/data_sources.html
rename to doc_cn/api/v2/fluid/profiler.html
index c4bdaaac86a35cb475a68393305d75fe22d9cbd1..a1a2291b62848314a4b4613495f216bbb9ff09b4 100644
--- a/doc_cn/api/v1/trainer_config_helpers/data_sources.html
+++ b/doc_cn/api/v2/fluid/profiler.html
@@ -8,7 +8,7 @@
   
   
   
-  DataSources — PaddlePaddle  文档
+  Profiler — PaddlePaddle  文档
   
 
   
@@ -62,7 +62,7 @@
     
      
     
      
       
      
-        Folk me on Github
+        Fork me on Github
         
      
           
             English
@@ -85,6 +85,7 @@
 
    • 进阶指南
      • 
 
    • API
      • 
 
    • FAQ
      • 
+
    • MOBILE
      • 
 
    
 
         
@@ -108,9 +109,10 @@
           
 
         
@@ -111,9 +112,10 @@
           
 
         
@@ -193,6 +215,7 @@
 
      
 
    • Activation
      • 
 
    • Layers
      • 
+
    • Evaluators
      • 
 
    • Optimizer
      • 
 
    • Pooling
      • 
 
    • Networks
      • 
diff --git a/doc_cn/api/v2/run_logic.html b/doc_cn/api/v2/run_logic.html
index 41d93ba73652b890ce5f7442241e1186f1f74038..00c82bbb6596b87d5e7274a855cfc7df2bcdc3f7 100644
--- a/doc_cn/api/v2/run_logic.html
+++ b/doc_cn/api/v2/run_logic.html
@@ -35,7 +35,7 @@
     
         
         
-         
+         
 
   
   
@@ -65,7 +65,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -88,6 +88,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -111,9 +112,10 @@
           
 
         
@@ -194,9 +216,31 @@
 
    
 
    
 class paddle.v2.parameters.Parameters
    
-
    Parameters is a dictionary contains Paddle’s parameter. The key of
-Parameters is the name of parameter. The value of Parameters is a plain
-numpy.ndarry .
    
+
    Parameters manages all the learnable parameters in a neural network.
+It stores parameters’ information in an OrderedDict. The key is
+the name of a parameter, and value is a parameter’s configuration(in
+protobuf format), such as initialization mean and std, its size, whether it
+is a static parameter, and so on.
    
+
+++
+
+
+
+
    


    参数:
      
+
    • __param_conf__ (OrderedDict) – store the configurations of learnable parameters in
+the network in an OrderedDict. Parameter is added one by one into the
+dict by following their created order in the network: parameters of
+the previous layers in a network are careted first. You can visit the
+parameters from bottom to top by iterating over this dict.
      • 
+
    • __gradient_machines__ (list) – all of the parameters in a neural network are
+appended to a PaddlePaddle gradient machine, which is used internally to
+copy parameter values between C++ and Python end.
      • 
+
    • __tmp_params__ (dict) – a dict to store dummy parameters if no
+__gradient_machines__ is appended to Parameters.
      • 
+
    
+
    
 
    Basically usage is
    
 
    data = paddle.layers.data(...)
 ...
@@ -295,6 +339,26 @@ Parameters is the name of parameter. The value of Parameters is a plain
 
 
    
 
+
    
+
    
+get_grad(key)
    
+
    Get grandient by parameter name.
    
+
+++
+
+
+
+
+
+
+
+
+
+
    


    Note:It will always copy the parameter from C++ side.
    参数:key (basestring) – parameter name
    返回:The grandient matrix.
    返回类型:np.ndarray
    
+
    
+
 
    
 
    
 set(parameter_name, value)
    
@@ -325,7 +389,7 @@ Trainer.train.
    
 
 
 
-参数:gradient_machine (api.GradientMachine) – Paddle C++ GradientMachine object.
+参数:gradient_machine (api.GradientMachine) – PaddlePaddle C++ GradientMachine object.
 
 返回:
 
@@ -373,6 +437,66 @@ Trainer.train.
    
 
 
    
 
+
    
+
    
+to_tar(f)
    
+
    Save parameters to a tar file.
    
+
    
+
    WARNING: You should use paddle.v2.trainer.SGD.save_parameter_to_tar(f)
    
+
    to save parameters most of the time. Otherwise, some settings such
+as model average will not take effect.
    
+
    
+
+++
+
+
+
+
+
+
    


    参数:f (file) – 
    返回:
    
+
    
+
+
    
+
    
+static from_tar(f)
    
+
    Create a Parameters object from the given file. And
+the Parameters only contains the parameters in this
+file. It is adapted the parameters are same in the
+defined network and the given file. For example, it
+can be used in the inference.
    
+
+++
+
+
+
+
+
+
+
+
    


    参数:f (tar file) – the initialized model file.
    返回:A Parameters object.
    返回类型:Parameters.
    
+
    
+
+
    
+
    
+init_from_tar(f)
    
+
    Different from from_tar, this interface can be used to
+init partial network parameters from another saved model.
    
+
+++
+
+
+
+
+
+
    


    参数:f (tar file) – the initialized model file.
    返回:Nothing.
    
+
    
+
 
    
 
 
    
@@ -381,7 +505,7 @@ Trainer.train.
    
 
    Module Trainer
    
 
    
 
    
-class paddle.v2.trainer.SGD(cost, parameters, update_equation, extra_layers=None)
    
+class paddle.v2.trainer.SGD(cost, parameters, update_equation, extra_layers=None, is_local=True, pserver_spec=None, use_etcd=True)
 
    Simple SGD Trainer.
 SGD Trainer combines data reader, network topolopy and update_equation together
 to train/test a neural network.
    
@@ -390,11 +514,18 @@ to train/test a neural network.
    
 
 
 参数:
      
-
    • update_equation (paddle.v2.optimizer.Optimizer) – The optimizer object.
      • 
 
    • cost (paddle.v2.config_base.Layer) – Target cost that neural network should be optimized.
      • 
 
    • parameters (paddle.v2.parameters.Parameters) – The parameters dictionary.
      • 
+
    • update_equation (paddle.v2.optimizer.Optimizer) – The optimizer object.
      • 
 
    • extra_layers (paddle.v2.config_base.Layer) – Some layers in the neural network graph are not
 in the path of cost layer.
      • 
+
    • is_local (bool) – Whether trainning locally
      • 
+
    • pserver_spec (string) – comma string for pserver location,
+eg:127.10.0.10:3000,127.10.0.11:3000,
+and this parameter is only used for fault
+tolerant mode cluster training.
      • 
+
    • use_etcd – Whether using etcd pserver.
      • 
+
    • use_etcd – bool
      • 
 
    
 
 
@@ -412,7 +543,7 @@ in the path of cost layer.
 
  • reader (collections.Iterable) – A reader that reads and yeilds data items. Usually we use a
 batched reader to do mini-batch training.
    • 
 
  • num_passes – The total train passes.
    • 
-
  • event_handler ((BaseEvent) => None) – Event handler. A method will be invoked when event
+
  • event_handler ((BaseEvent) => None) – Event handler. A method will be invoked when event
 occurred.
    • 
 
  • feeding (dict|list) – Feeding is a map of neural network input name and array
 index that reader returns.
    • 
@@ -435,7 +566,8 @@ index that reader returns.
 
 
 参数:
      
-
    • reader (collections.Iterable) – A reader that reads and yeilds data items.
      • 
+
    • reader (collections.Iterable) – A batch reader that reads and yeilds data items,
+it should be a paddle.v2.batch.
      • 
 
    • feeding (dict) – Feeding is a map of neural network input name and array
 index that reader returns.
      • 
 
    
@@ -476,8 +608,10 @@ index that reader returns.
 
 
    
 
    
-class paddle.v2.event.EndPass(pass_id, evaluator)
    
-
    Event On One Pass Training Complete.
    
+class paddle.v2.event.EndPass(pass_id, evaluator, gm)
+
    Event On One Pass Training Complete.
+To get the output of a specific layer, add “event.gm.getLayerOutputs(‘predict_layer’)”
+in your event_handler call back
    
 
    
 
 
    
@@ -488,8 +622,16 @@ index that reader returns.
 
 
    
 
    
-class paddle.v2.event.EndIteration(pass_id, batch_id, cost, evaluator)
    
-
    Event On One Batch Training Complete.
    
+class paddle.v2.event.EndForwardBackward(pass_id, batch_id, gm)
+
    Event On One Batch ForwardBackward Complete.
    
+
    
+
+
    
+
    
+class paddle.v2.event.EndIteration(pass_id, batch_id, cost, evaluator, gm)
    
+
    Event On One Batch Training Complete.
+To get the output of a specific layer, add “event.gm.getLayerOutputs(‘predict_layer’)”
+in your event_handler call back
    
 
    
 
 
    
@@ -500,9 +642,18 @@ index that reader returns.
 paddle.v2.infer(output_layer, parameters, input, feeding=None, field='value')
 
    Infer a neural network by given neural network output and parameters.  The
 user should pass either a batch of input data or reader method.
    
-
    Example usages:
    
-
    result = paddle.infer(prediction, parameters, input=SomeData,
-                      batch_size=32)
+
    Example usage for sinlge output_layer:
    
+
    result = paddle.infer(output_layer=prediction,
+                      parameters=parameters,
+                      input=SomeData)
+print result
+
    
+
    
+
    Example usage for multiple outout_layers and fields:
    
+
    result = paddle.infer(output_layer=[prediction1, prediction2],
+                      parameters=parameters,
+                      input=SomeData,
+                      field=[id, value]])
 print result
 
    
 
    
@@ -511,7 +662,8 @@ user should pass either a batch of input data or reader method.
    
 
 
 参数:
      
-
    • output_layer (paddle.v2.config_base.Layer) – output of the neural network that would be inferred
      • 
+
    • output_layer (paddle.v2.config_base.Layer or a list of
+paddle.v2.config_base.Layer) – output of the neural network that would be inferred
      • 
 
    • parameters (paddle.v2.parameters.Parameters) – parameters of the neural network.
      • 
 
    • input (collections.Iterable) – input data batch. Should be a python iterable object, and each
 element is the data batch.
      • 
@@ -525,7 +677,9 @@ or max_id.
 
    
 
 
-返回:
    a numpy array
    
+返回:
    The prediction result. If there are multiple outout_layers and fields,
+the return order is outout_layer1.field1, outout_layer2.field1, ...,
+outout_layer1.field2, outout_layer2.field2 ...
    
 
 
 返回类型:
    numpy.ndarray
    
@@ -548,7 +702,7 @@ or max_id.
         Next 
       
       
-         Previous
+         Previous
       
     
    
   
diff --git a/doc_cn/design/api.html b/doc_cn/design/api.html
index 04a4275820085753aa6232ee17f585308a7b8f81..083dee56c8b597192a0f12d1daba6358bc91d163 100644
--- a/doc_cn/design/api.html
+++ b/doc_cn/design/api.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -85,6 +85,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +109,10 @@
           
 
         
@@ -187,7 +209,7 @@
 
    
 
    Ingredients
    
 
    As our design principle is starting from the essence: how could we
-allow users to express and solve their problems at neural networks.
+allow users to express and solve their problems as neural networks.
 Some essential concepts that our API have to provide include:
    
 
      
 
    1. A topology is an expression of layers.
      2. 
@@ -390,7 +412,7 @@ access a Kubernetes cluster, s/he should be able to call
      
                   num_parameter_servers=15)
 
    
 
    
-
    The pseudo code if paddle.dist_train is as follows:
    
+
    The pseudo code of paddle.dist_train is as follows:
    
 
    def dist_train(topology, parameters, trainer, reader, ...):
     if os.getenv("KUBERNETES_SERVICE_HOST") == None:
         image_name = k8s_user + '/' + k8s_job
diff --git a/doc_cn/design/auto_gradient_check.html b/doc_cn/design/auto_gradient_check.html
new file mode 100644
index 0000000000000000000000000000000000000000..ff20871aa589c11d5f070374f63e2e4723b8aced
--- /dev/null
+++ b/doc_cn/design/auto_gradient_check.html
@@ -0,0 +1,432 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Auto Gradient Checker Design — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Auto Gradient Checker Design
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Auto Gradient Checker Design
    
+
    
+
    
+
    Backgraound：
    
+
      
+
    • Generally, it is easy to check whether the forward computation of an Operator is correct or not. However, backpropagation is a notoriously difficult algorithm to debug and get right:
        
+
      1. you should get the right backpropagation formula according to the forward computation.
        2. 
+
      2. you should implement it right in CPP.
        3. 
+
      3. it’s difficult to prepare test data.
        4. 
+
      
+
      • 
+
    • Auto gradient checking gets a numerical gradient by forward Operator and use it as a reference of the backward Operator’s result. It has several advantages:
        
+
      1. numerical gradient checker only need forward operator.
        2. 
+
      2. user only need to prepare the input data for forward Operator.
        3. 
+
      
+
      • 
+
    
+
    
+
    
+
    Mathematical Theory
    
+
    The following two document from Stanford has a detailed explanation of how to get numerical gradient and why it’s useful.
    
+
    
+
+
    
+
    
+
    
+
    Numeric Gradient Implementation
    
+
    
+
    Python Interface
    
+
    def get_numerical_gradient(op,
+                         input_values,
+                         output_name,
+                         input_to_check,
+                         delta=0.005,
+                         local_scope=None):
+    """
+    Get Numeric Gradient for an operator's input.
+
+    :param op: C++ operator instance, could be an network
+    :param input_values: The input variables. Should be an dictionary, whose key is
+    variable name, and value is numpy array.
+    :param output_name: The final output variable name.
+    :param input_to_check: The input variable with respect to which to compute the gradient.
+    :param delta: The perturbation value for numeric gradient method. The
+    smaller delta is, the more accurate result will get. But if that delta is
+     too small, it will suffer from numerical stability problem.
+    :param local_scope: The local scope used for get_numeric_gradient.
+    :return: The gradient array in numpy format.
+    """
+
    
+
    
+
    
+
    
+
    Explaination:
    
+
      
+
    • Why need output_name
        
+
      • An Operator may have multiple Output, one can get independent gradient from each Output. So caller should specify the name of the output variable.
        • 
+
      
+
      • 
+
    • Why need input_to_check
        
+
      • One operator may have multiple inputs. Gradient Op can calculate the gradient of these inputs at the same time. But Numeric Gradient needs to calculate them one by one. So get_numeric_gradient is designed to calculate the gradient for one input. If you need to compute multiple inputs, you can call get_numeric_gradient multiple times.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    Core Algorithm Implementation
    
+
        # we only compute gradient of one element a time.
+    # we use a for loop to compute the gradient of each element.
+    for i in xrange(tensor_size):
+        # get one input element by its index i.
+        origin = tensor_to_check.get_float_element(i)
+
+        # add delta to it, run op and then get the new value of the result tensor.
+        x_pos = origin + delta
+        tensor_to_check.set_float_element(i, x_pos)
+        y_pos = get_output()
+
+        # plus delta to this element, run op and get the new value of the result tensor.
+        x_neg = origin - delta
+        tensor_to_check.set_float_element(i, x_neg)
+        y_neg = get_output()
+
+        # restore old value
+        tensor_to_check.set_float_element(i, origin)
+
+        # compute the gradient of this element and store it into a numpy array.
+        gradient_flat[i] = (y_pos - y_neg) / delta / 2
+
+    # reshape the gradient result to the shape of the source tensor.
+    return gradient_flat.reshape(tensor_to_check.get_dims())
+
    
+
    
+
    
+
    
+
    
+
    Auto Graident Checker Framework
    
+
    Each Operator Kernel has three kinds of Gradient:
    
+
      
+
    1. Numerical gradient
      2. 
+
    2. CPU kernel gradient
      3. 
+
    3. GPU kernel gradient (if supported)
      4. 
+
    
+
    The numerical gradient only relies on forward Operator. So we use the numerical gradient as the reference value. And the gradient checking is performed in the following three steps:
    
+
      
+
    1. calculate the numerical gradient
      2. 
+
    2. calculate CPU kernel gradient with the backward Operator and compare it with the numerical gradient
      3. 
+
    3. calculate GPU kernel gradient with the backward Operator and compare it with the numeric gradient (if supported)
      4. 
+
    
+
    
+
    Python Interface
    
+
        def check_grad(self,
+                   forward_op,
+                   input_vars,
+                   inputs_to_check,
+                   output_name,
+                   no_grad_set=None,
+                   only_cpu=False,
+                   max_relative_error=0.005):
+        """
+        :param forward_op: used to create backward_op
+        :param input_vars: numpy value of input variable. The following
+            computation will use these variables.
+        :param inputs_to_check: the input variable with respect to which to compute the gradient.
+        :param output_name: The final output variable name.
+        :param max_relative_error: The relative tolerance parameter.
+        :param no_grad_set: used when create backward ops
+        :param only_cpu: only compute and check gradient on cpu kernel.
+        :return:
+        """
+
    
+
    
+
    
+
    
+
    How to check if two numpy array is close enough?
    
+
    if abs_numerical_grad is nearly zero, then use abs error for numerical_grad
    
+
    numerical_grad = ...
+operator_grad = numpy.array(scope.find_var(grad_var_name(name)).get_tensor())
+
+abs_numerical_grad = numpy.abs(numerical_grad)
+# if abs_numerical_grad is nearly zero, then use abs error for numeric_grad, not relative
+# error.
+abs_numerical_grad[abs_numerical_grad < 1e-3] = 1
+
+diff_mat = numpy.abs(abs_numerical_grad - operator_grad) / abs_numerical_grad
+max_diff = numpy.max(diff_mat)
+
    
+
    
+
    
+
    Notes：
    
+
    The Input data for auto gradient checker should be reasonable to avoid numerical  stability problem.
    
+
    
+
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/block.html b/doc_cn/design/block.html
new file mode 100644
index 0000000000000000000000000000000000000000..4c14c3fc23c56cf250481ac092c9a40557b7e744
--- /dev/null
+++ b/doc_cn/design/block.html
@@ -0,0 +1,572 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Block and Scope — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Block and Scope
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Block and Scope
    
+
    
+
    The Representation of Computation
    
+
    Both deep learning systems and programming languages help users describe computation procedures.  These systems use various representations of computation:
    
+
      
+
    • Caffe, Torch, and Paddle: sequences of layers.
      • 
+
    • TensorFlow, Caffe2, Mxnet: graph of operators.
      • 
+
    • PaddlePaddle: nested blocks, like C++ and Java programs.
      • 
+
    
+
    
+
    
+
    Block in Programming Languages and Deep Learning
    
+
    In programming languages, a block is a pair of curly braces that includes local variables definitions and a sequence of instructions or operators.
    
+
    Blocks work with control flow structures like if, else, and for, which have equivalents in deep learning:
    
+
    | programming languages | PaddlePaddle          |
+|———————–|———————–|
+| for, while loop       | RNN, WhileOp          |
+| if, if-else, switch   | IfElseOp, SwitchOp    |
+| sequential execution  | a sequence of layers  |
    
+
    A key difference is that a C++ program describes a one pass computation, whereas a deep learning program describes both the forward and backward passes.
    
+
    
+
    
+
    Stack Frames and the Scope Hierarchy
    
+
    The existence of the backward pass makes the execution of a block of PaddlePaddle different from traditional programs:
    
+
    | programming languages | PaddlePaddle                    |
+|———————–|———————————|
+| stack                 | scope hierarchy                 |
+| stack frame           | scope                           |
+| push at entering block| push at entering block          |
+| pop at leaving block  | destroy when minibatch completes|
    
+
      
+
    1. In traditional programs:
        
+
      • When the execution enters the left curly brace of a block, the runtime pushes a frame into the stack, where it realizes local variables.
        • 
+
      • After the execution leaves the right curly brace, the runtime pops the frame.
        • 
+
      • The maximum number of frames in the stack is the maximum depth of nested blocks.
        • 
+
      
+
      2. 
+
    2. In PaddlePaddle
        
+
      • When the execution enters a block, PaddlePaddle adds a new scope, where it realizes variables.
        • 
+
      • PaddlePaddle doesn’t pop a scope after the execution of the block because variables therein are used by the backward pass.  So it has a stack forest known as a scope hierarchy.
        • 
+
      • The height of the highest tree is the maximum depth of nested blocks.
        • 
+
      • After the processing of a minibatch, PaddlePaddle destroys the scope hierarchy.
        • 
+
      
+
      3. 
+
    
+
    
+
    
+
    Use Blocks in C++ and PaddlePaddle Programs
    
+
    Let us consolidate the discussion by presenting some examples.
    
+
    
+
    Blocks with if-else and IfElseOp
    
+
    The following C++ programs shows how blocks are used with the if-else structure:
    
+
    namespace pd = paddle;
+
+int x = 10;
+int y = 1;
+int z = 10;
+bool cond = false;
+int o1, o2;
+if (cond) {
+  int z = x + y;
+  o1 = z;
+  o2 = pd::layer::softmax(z);
+} else {
+  int d = pd::layer::fc(z);
+  o1 = d;
+  o2 = d+1;
+}
+
    
+
    
+
    An equivalent PaddlePaddle program from the design doc of the IfElseOp operator is as follows:
    
+
    import paddle as pd
+
+x = minibatch([10, 20, 30]) # shape=[None, 1]
+y = var(1) # shape=[1], value=1
+z = minibatch([10, 20, 30]) # shape=[None, 1]
+cond = larger_than(x, 15) # [false, true, true]
+
+ie = pd.ifelse()
+with ie.true_block():
+    d = pd.layer.add_scalar(x, y)
+    ie.output(d, pd.layer.softmax(d))
+with ie.false_block():
+    d = pd.layer.fc(z)
+    ie.output(d, d+1)
+o1, o2 = ie(cond)
+
    
+
    
+
    In both examples, the left branch computes x+y and softmax(x+y), the right branch computes fc(x) and x+1 .
    
+
    The difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances.
    
+
    
+
    
+
    Blocks with for and RNNOp
    
+
    The following RNN model in PaddlePaddle from the RNN design doc :
    
+
    x = sequence([10, 20, 30]) # shape=[None, 1]
+m = var(0) # shape=[1]
+W = var(0.314, param=true) # shape=[1]
+U = var(0.375, param=true) # shape=[1]
+
+rnn = pd.rnn()
+with rnn.step():
+  h = rnn.memory(init = m)
+  h_prev = rnn.previous_memory(h)
+  a = layer.fc(W, x)
+  b = layer.fc(U, h_prev)  
+  s = pd.add(a, b)
+  act = pd.sigmoid(s)
+  rnn.update_memory(h, act)
+  rnn.output(a, b)
+o1, o2 = rnn()
+
    
+
    
+
    has its equivalent C++ program as follows
    
+
    int* x = {10, 20, 30};
+int* m = {0};
+int* W = {0.314};
+int* U = {0.375};
+
+int mem[sizeof(x) / sizeof(x[0]) + 1];
+int o1[sizeof(x) / sizeof(x[0]) + 1];
+int o2[sizeof(x) / sizeof(x[0]) + 1];
+for (int i = 1; i <= sizeof(x)/sizeof(x[0]); ++i) {
+  int x = x[i-1];
+  if (i == 1) mem[0] = m;
+  int a = W * x;
+  int b = Y * mem[i-1];
+  int s = fc_out + hidden_out;
+  int act = sigmoid(sum);
+  mem[i] = act;
+  o1[i] = act;
+  o2[i] = hidden_out;
+}
+
    
+
    
+
    
+
    
+
    
+
    Compilation and Execution
    
+
    Like TensorFlow, a PaddlePaddle program is written in Python. The first part describes a neural network as a protobuf message, and the rest executes the message for training or inference.
    
+
    The generation of this protobuf message is similar to how a compiler generates a binary executable file. The execution of the message is similar to how the OS executes the binary file.
    
+
    
+
    
+
    The “Binary Executable File Format”
    
+
    The definition of the protobuf message is as follows:
    
+
    message BlockDesc {
+  repeated VarDesc vars = 1;
+  repeated OpDesc ops = 2;
+}
+
    
+
    
+
    The step net in above RNN example would look like
    
+
    BlockDesc {
+  vars = {
+    VarDesc {...} // x
+    VarDesc {...} // h
+    VarDesc {...} // fc_out
+    VarDesc {...} // hidden_out
+    VarDesc {...} // sum
+    VarDesc {...} // act
+  }
+  ops = {
+    OpDesc {...} // matmul
+    OpDesc {...} // add_two
+    OpDesc {...} // sigmoid
+  }
+};
+
    
+
    
+
    Also, the RNN operator in above example is serialized into a protobuf message of type OpDesc and would look like:
    
+
    OpDesc {
+  inputs = {0} // the index of x in vars of BlockDesc above
+  outputs = {5, 3} // indices of act and hidden_out in vars of BlockDesc above
+  attrs {
+    "states" : {1} // the index of h
+    "step_net" : <above step net>
+  }
+};
+
    
+
    
+
    This OpDesc value is in the ops field of the BlockDesc value representing the global block.
    
+
    
+
    
+
    The Compilation of Blocks
    
+
    During the generation of the Protobuf message, the Block should store VarDesc (the Protobuf message which describes Variable) and OpDesc (the Protobuf message which describes Operator).
    
+
    VarDesc in a block should have its name scope to avoid local variables affect parent block’s name scope.
+Child block’s name scopes should inherit the parent’s so that OpDesc in child block can reference a VarDesc that stored in parent block. For example:
    
+
    a = pd.Variable(shape=[20, 20])
+b = pd.fc(a, params=["fc.w", "fc.b"])
+
+rnn = pd.create_rnn()
+with rnn.stepnet():
+    x = a.as_step_input()
+    # reuse fc's parameter
+    fc_without_b = pd.get_variable("fc.w")
+    rnn.output(fc_without_b)
+
+out = rnn()
+
    
+
    
+
    The method pd.get_variable can help retrieve a Variable by the name. The Variable may be stored in a parent block, but might be retrieved in a child block, so block should have a variable scope that supports inheritance.
    
+
    In compiler design, the symbol table is a data structure created and maintained by compilers to store information about the occurrence of various entities such as variable names, function names, classes, etc.
    
+
    To store the definition of variables and operators, we define a C++ class SymbolTable, like the one used in compilers.
    
+
    SymbolTable can do the following:
    
+
      
+
    • store the definitions (some names and attributes) of variables and operators,
      • 
+
    • verify if a variable was declared,
      • 
+
    • make it possible to implement type checking (offer Protobuf message pointers to InferShape handlers).
      • 
+
    
+
    // Information in SymbolTable is enough to trace the dependency graph. So maybe
+// the Eval() interface takes a SymbolTable is enough.
+class SymbolTable {
+ public:
+  SymbolTable(SymbolTable* parent) : parent_(parent) {}
+
+  OpDesc* NewOp(const string& name="");
+
+  // TODO determine whether name is generated by python or C++.
+  // Currently assume that a unique name will be generated by C++ if the
+  // argument name is left default.
+  VarDesc* Var(const string& name="");
+
+  // find a VarDesc by name, if recursive is true, find parent's SymbolTable
+  // recursively.
+  // this interface is introduced to support InferShape, find protobuf messages
+  // of variables and operators, pass pointers into InferShape.
+  //
+  // NOTE maybe some C++ classes such as VarDescBuilder and OpDescBuilder should
+  // be proposed and embedded into pybind to enable python operation on C++ pointers.
+  VarDesc* FindVar(const string& name, bool recursive=true);
+
+  OpDesc* FindOp(const string& name);
+
+  BlockDesc Compile() const;
+
+ private:
+  SymbolTable* parent_;
+
+  map<string, OpDesc> ops_;
+  map<string, VarDesc> vars_;
+};
+
    
+
    
+
    After all the description of variables and operators is added into SymbolTable,
+the block has enough information to run.
    
+
    The Block class takes a BlockDesc as input, and provides Run and InferShape functions.
    
+
    namespace {
+
+class Block : OperatorBase {
+public:
+  Block(const BlockDesc& desc) desc_(desc) {}
+
+  void InferShape(const framework::Scope& scope) const override {
+    if (!symbols_ready_) {
+      CreateVariables(scope);
+      CreateOperators();
+    }
+    // should run InferShape first.
+    for (auto& op : runtime_table_.ops()) {
+      op->InferShape(scope);
+    }
+  }
+
+  void Run(const framework::Scope& scope,
+           const platform::DeviceContext& dev_ctx) const override {
+    PADDLE_ENFORCE(symbols_ready_, "operators and variables should be created first.");
+    for (auto& op : runtime_table_.ops()) {
+      op->Run(scope, dev_ctx);
+    }
+  }
+
+  void CreateVariables(const framework::Scope& scope);
+  void CreateOperators();
+
+  // some other necessary interfaces of NetOp are listed below
+  // ...
+
+private:
+  BlockDesc desc_;
+  bool symbols_ready_{false};
+};
+
    
+
    
+
    
+
    
+
    The Execution of Blocks
    
+
    Block inherits from OperatorBase, which has a Run method.
+Block’s Run method will run its operators sequentially.
    
+
    There is another important interface called Eval, which takes some arguments called targets and generates a minimal graph which treats targets as the end points and creates a new Block. After Run, Eval will get the latest value and return the targets.
    
+
    The definition of Eval is as follows:
    
+
    // clean a block description by targets using the corresponding dependency graph.
+// return a new BlockDesc with minimal number of operators.
+// NOTE: The return type is not a Block but the block's description so that this can be distributed
+// to a cluster.
+BlockDesc Prune(const BlockDesc& desc, vector<string> targets);
+
+void Block::Eval(const vector<string>& targets,
+                 const framework::Scope& scope,
+                 const platform::DeviceContext& dev_ctx) {
+  BlockDesc min_desc = Prune(desc_, targets);
+  Block min_block(min_desc);
+  min_block.Run(scope, dev_ctx);
+}
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/build_system/README.html b/doc_cn/design/build_system/README.html
new file mode 100644
index 0000000000000000000000000000000000000000..8fa529386cde2005843e1416c96cd2596a1b40d4
--- /dev/null
+++ b/doc_cn/design/build_system/README.html
@@ -0,0 +1,414 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Required CMake Function — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Required CMake Function
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    A few months ago when we were trying to replace CMake with Bazel, @emailweixu suggested that we rewrite those handy Bazel functions using CMake. Now it seems that it’s the right time to get this done, as we are facing problems from the porting of Majel and the development of new the parameter server using Go and C++.
    
+
    Here are some initial thoughts. Your comments are welcome!
    
+
    
+
    Required CMake Function
    
+
    I think we need only the following few CMake functions to make a project description mean and clean:
    
+
    | C++ | CUDA C++ | Go |
+|—|—|—|
+| cc_library | nv_library | go_library |
+| cc_binary | nv_binary | go_binary |
+| cc_test | nv_test | go_test |
    
+
      
+
    • The _library functions generate  .a files from source code.
      • 
+
    • The _binary functions generate executable binary files.
      • 
+
    • The _test functions generate executable unit test files. They work like _binary but links -lgtest and -lgtest_main.
      • 
+
    
+
    The difference between nv_ functions and cc_ functions is that the former use nvcc instead of the system-default C++ compiler.
    
+
    Both nv_ and cc_ functions enables C++11 (-std=c++11).
    
+
    Also,
    
+
      
+
    • to describe external dependencies, we need external_library.
      • 
+
    • to build shared libraries, we need shared_library.
      • 
+
    
+
    
+
    
+
    An Example Project
    
+
    Suppose that we have aforementioned functions defined in our /cmake directory.  The following example CMakeLists.txt describes a project including the following source files:
    
+
      
+
    • tensor.h
      • 
+
    • tensor.cc
      • 
+
    • tensor_test.cc
      • 
+
    • ops.h
      • 
+
    • ops.cu
      • 
+
    • ops_test.cu
      • 
+
    • api.go
      • 
+
    • api_test.go
      • 
+
    
+
    Suppose that ops.cu depends on CUDNN.
    
+
    # cc_binary parses tensor.cc and figures out that target also depend
+# on tensor.h.
+cc_binary(tensor
+  SRCS
+  tensor.cc)
+
+# The dependency to target tensor implies that if any of
+# tensor{.h,.cc,_test.cc} is changed, tensor_test need to be re-built.
+cc_test(tensor_test
+  SRCS
+  tensor_test.cc
+  DEPS
+  tensor)
+
+# I don't have a clear idea what parameters external_library need to
+# have.  @gangliao as a CMake expert would have better ideas.
+external_library(cudnn
+  ....)
+
+# Suppose that ops.cu depends on external target CUDNN.  Also, ops.cu
+# include global functions that take Tensor as their parameters, so
+# ops depend on tensor.  This implies that if any of tensor.{h.cc},
+# ops.{h,cu} is changed, ops need to be re-built.
+nv_library(ops
+  SRCS
+  ops.cu
+  DEPS
+  tensor
+  cudnn)  # cudnn is defined later.
+
+nv_test(ops_test
+  SRCS
+  ops_test.cu
+  DEPS
+  ops)
+
+# Because api.go defines a GO wrapper to ops and tensor, it depends on
+# both.  This implies that if any of tensor.{h,cc}, ops.{h,cu}, or
+# api.go is changed, api need to be re-built.
+go_library(api
+  SRCS
+  api.go
+  DEPS
+  tensor # Because ops depend on tensor, this line is optional.
+  ops)
+
+go_test(api_test
+  SRCS
+  api_test.go
+  DEPS
+  api)
+
+
+# This builds libapi.so.  shared_library might use CMake target
+# api_shared so to distinguish it from above target api.
+shared_library(api
+  DEPS
+  api)
+
    
+
    
+
    
+
    
+
    Implementation
    
+
    As above example CMakeLists.txt executes, each function invocation adds “nodes” to a dependency graph.  It also use this graph to generate CMake commands including add_executable, add_dependencies, target_link_libraries, and add_test.
    
+
    
+
    
+
    Using Package Manager For Go
    
+
    Building Go binaries and libraries need to satisfy their dependencies, generally
+we can do go get ./... to download and compile all external dependencies. The
+problems are:
    
+
      
+
    1. go get will always get the latest code from the default branch of the
+remote repo, so changes of dependents might break the build. This is very
+different with what we already have in cmake/external which download a
+specific version or commit id of the dependency.
      2. 
+
    2. Some locations can not access external dependencies through the internet, as mentioned
+in https://github.com/PaddlePaddle/Paddle/issues/2605. Using package management
+tools can package the dependencies as a “vendor” package, which can be mirrored
+at many cloud file hosting, so users what to compile paddle by themselves can
+download this “vendor” package from a mirror site.
      3. 
+
    
+
    
+
    Choose A Suitable Tool
    
+
    As mentioned by @wangkuiyi, Here
+list dozens of Go package managers. We choose the tool using following principles:
    
+
      
+
    • Most “active” projects with more stars, more pull requests or commits
      • 
+
    • Widely used project
      • 
+
    
+
    After comparing all these projects, we shall choose between the most popular
+tools: Godep and Glide.
    
+
    Here’s a brief comparison between Godep and Glide
+: https://github.com/Masterminds/glide/wiki/Go-Package-Manager-Comparison. There are
+also many complaints about using Godep. There’s also a new “official” pakcage
+management tool has been started at: https://github.com/golang/dep to resolve
+such problems, but it’s currently at Alpha stage. So the best choice now is
+glide obviously.
    
+
    
+
    
+
    Manage Go Packages
    
+
      
+
    • Dependencies: go/glide.yaml will store the dependencies and their versions which
+is directly imported by paddle. go/glide.lock will store all dependencies recursively
+with their commit id. Builds will “lock” to these packages if we don’t glide up
+them
      • 
+
    • Vendor package: go/vendor directory will generated when running cmake command. cmake
+will download the code corresponding to go/glide.lock. If we put a vendor folder
+under go/, cmake will just check the commit id to the packages under the folder,
+if commit id matches, there will be no download at all.
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/dist/README.html b/doc_cn/design/cluster_train/README.html
similarity index 72%
rename from doc_cn/design/dist/README.html
rename to doc_cn/design/cluster_train/README.html
index bc95880400a19ca7b2ba111eec7e8c6efbc85766..a83fea28807cab924b2890b22dbadc6d65b19729 100644
--- a/doc_cn/design/dist/README.html
+++ b/doc_cn/design/cluster_train/README.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -85,6 +85,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +109,10 @@
           
 
         
@@ -198,15 +220,17 @@
 
    Training Job
    
 
    A training job will be created once user asks Paddle cloud to train a model. The training job is made up of different processes that collaboratively consume data and produce a trained model. There are three kinds of processes:
    
 
      
-
    1. the master process, which dispatches tasks to
      2. 
+
    2. the master server process, which dispatches tasks to
      3. 
 
    3. one or more trainer processes, which run distributed training and synchronize gradients/models via
      4. 
-
    4. one or more parameter server processes, where each holds a shard of the global model.
      5. 
+
    5. one or more parameter server processes, where each holds a shard of the global model, and receive the uploaded gradients from every trainer process, so they can run the optimize functions to update their parameters.
      6. 
 
    
 
    Their relation is illustrated in the following graph:
    
 
    
-
    
-
    Master Process
    
-
    The master process will:
    
+
    By coordinating these processes, PaddlePaddle supports use both Synchronize Stochastic Gradient Descent (sync SGD) and Asynchronous Stochastic Gradient Descent (async SGD) to train user-defined neural network topologies.
    
+
    When training with sync SGD, parameter servers wait for all trainers to finish gradients update and then send the updated parameters to trainers, training can not proceed until the trainer received the updated parameters. This creates a synchronization point between trainers. When training with async SGD, each trainer upload gradient and download new parameters individually, without the synchronization with other trainers. Using asyc SGD will be faster in terms of time per pass, but have more noise in gradient since trainers are likely to have a stale model.
    
+
    
+
    Master Server Process
    
+
    The master server process will:
    
 
      
 
    • Partition a dataset into tasks and dispatch tasks to trainers.
      • 
 
    • Keep track of training progress on the dataset with task queue. A training job will iterate on the dataset for a full pass until it goes into next pass.
      • 
@@ -217,10 +241,10 @@
 
    
 
    
 
    Task Queue
    
-
    The master process has three task queues to track training progress. As illustrated in the graph below, Job A and Job B both have one master process. Each master process has three task queues.
    
+
    The master server has three task queues to track training progress. As illustrated in the graph below, Job A and Job B both have one master server. Each master server process has three task queues.
    
 
    
 
      
-
    • The todo queue holds tasks to be dispatched. When a job starts, the master process fills in the todo queue with all tasks.
      • 
+
    • The todo queue holds tasks to be dispatched. When a job starts, the master server fills in the todo queue with all tasks.
      • 
 
    • The pending queue holds tasks that are currently training by trainers.
      • 
 
    • the done queue holds tasks that are already trained.
      • 
 
    
@@ -228,10 +252,10 @@
 
    
 
      
 
    1. When a new pass of training starts, all tasks will be placed in the todo queue.
      2. 
-
    2. The master process will dispatch few tasks to each trainer at a time, puts them in the pending queue and waits for completion.
      3. 
-
    3. The trainer will work on its tasks and tell the master process once a task is completed. The master process will dispatch a new task to that trainer.
      4. 
-
    4. If a task timeout. the master process will move it back to the todo queue. The timeout count will increase by one. If the timeout count is above a threshold, the task is likely to cause a trainer to crash, so it will be discarded.
      5. 
-
    5. The master process will move completed task to the done queue. When the todo queue is empty, the master process will start a new pass by moving all tasks in the done queue to todo queue and reset the timeout counter of all tasks to zero.
      6. 
+
    6. Upon trainer requests for new task, the master server will dispatch a task from todo queue to it, put the task in the pending queue and wait for completion.
      7. 
+
    7. The trainer will work on its task and tell the master server once the task is completed and ask for new task. The master server will dispatch a new task to that trainer.
      8. 
+
    8. If a task fails for any reason in trainer, or takes longer than a specific period of time,  the master server will move the task back to the todo queue. The timeout count for that task will increase by one. If the timeout count is above a threshold, the task is likely to cause a trainer to crash, then it will be discarded.
      9. 
+
    9. The master server will move completed task to the done queue. When the todo queue is empty, the master server will start a new pass by moving all tasks in the done queue to todo queue and reset the timeout counter of all tasks to zero.
      10. 
 
    
 
    
 
    
@@ -239,8 +263,9 @@
 
    Trainer Process
    
 
    The trainer process will:
    
 
      
-
    • Receive tasks from the master.
      • 
-
    • Work on the tasks: calculate and upload gradient to parameter servers, and update local model by downloading new parameters from parameter servers.
      • 
+
    • Request tasks from the master.
      • 
+
    • Work on the tasks
      • 
+
    • Upload gradient to parameter servers, and update local model by downloading new parameters from parameter servers.
      • 
 
    
 
    
 
    
@@ -272,7 +297,7 @@
 
    
 
    
 
    Fault Tolerant
    
-
    The training job will pause if the master processes is dead, or any of the parameter server process is dead. They will be started by Kubernetes and recover in few minutes. Please refer to fault recovery.
    
+
    The training job will pause if the master server processes is dead, or any of the parameter server process is dead. They will be started by Kubernetes and recover in few minutes. Please refer to fault recovery.
    
 
    The training job will continue to make progress if there is at least one training process running. The strategy depends on the type of optimization algorithm:
    
 
      
 
    • sync-SGD
      
@@ -288,27 +313,26 @@
 
      PaddlePaddle uses etcd to keep track of the states of processes. Because etcd is a distributed reliable key-value store, the restarted process can recover its states from etcd. The model parameters are periodically saved into distributed file system, so a restarted parameter server can recover its parameters from the saved file.
      
 
      Now we will introduce how each process recovers from a failure, the graph below shows how etcd is used:
      
 
      
-
      
-
      Master Process
      
+
      
+
      Master Server Process
      
 
      When the master is started by the Kubernetes, it executes the following steps at startup:
      
 
        
 
      1. Grabs a unique master lock in etcd, which prevents concurrent master instantiations.
        2. 
 
      2. Recovers the task queues from etcd if they already exist, otherwise, the master will create them.
        3. 
-
      3. Watches the trainer prefix keys /trainer/ on etcd to find the live trainers.
        4. 
-
      4. Starts dispatching the tasks to the trainers, and updates task queue using an etcd transaction to ensure lock is held during the update.
        5. 
+
      5. Write its ip address to /master/addr so that trainers can discover it.
        6. 
+
      6. Listens to trainers’ request of task, dispatch one upon request, and updates task queue using an etcd transaction to ensure lock is held during the update.
        7. 
 
      
-
      The master process will kill itself if its etcd lease expires.
      
-
      When the master process is dead for any reason, Kubernetes will restart it. It will be online again with all states recovered from etcd in few minutes.
      
+
      When the master server process is dead for any reason, Kubernetes will restart it. It will be online again with all states recovered from etcd in few minutes.
      
 
      
 
      
 
      Trainer Process
      
 
      When the trainer is started by the Kubernetes, it executes the following steps at startup:
      
 
        
-
      1. Watches the available parameter server prefix keys /ps/ on etcd and waits until the count of parameter servers reaches the desired count.
        2. 
-
      2. Generates a unique ID, and sets key /trainer/<unique ID> with its contact address as value. The key will be deleted when the lease expires, so the master will be aware of the trainer being online and offline.
        3. 
-
      3. Waits for tasks from the master to start training.
        4. 
+
      4. Watches the available parameter server prefix keys /ps/ on etcd and waits until the count of parameter servers reaches the desired count /ps_desired.
        5. 
+
      5. Finds and watches /master/addr to get master’s address.
        6. 
+
      6. Requests for tasks from the master to start training.
        7. 
 
      
-
      If trainer’s etcd lease expires, it will try set key /trainer/<unique ID> again so that the master process can discover the trainer again.
      
+
      When a trainer fails, Kuberentes would try to restart it. The recovered trainer would fetch tasks from master and go on training.
      
 
      
 
      
 
      Parameter Server Process
      
@@ -330,6 +354,14 @@
 
      If the parameter server’s etcd lease expires, the parameter server will kill itself.
      
 
      
 
      
+
      
+
      Parameter Server Checkpointing
      
+
      See here
      
+
      
+
      
+
      Store and dispatching trainning data
      
+
      See here
      
+
      
 
      
 
      Dynamic Scaling
      
 
      
diff --git a/doc_cn/design/cluster_train/checkpointing.html b/doc_cn/design/cluster_train/checkpointing.html
new file mode 100644
index 0000000000000000000000000000000000000000..6f5b5d0ba290d5b1647f00954eebc759f36c6bc0
--- /dev/null
+++ b/doc_cn/design/cluster_train/checkpointing.html
@@ -0,0 +1,318 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  模型参数检查点（Checkpointing） — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • 模型参数检查点（Checkpointing）
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      模型参数检查点（Checkpointing）
      
+
      模型数据检查点的实现，可以有效的避免parameter server的单点或多点同时故障。模型参数检查点通过定期向磁盘上保存一份存储在parameter server内存中的模型数据的完整镜像，来保证训练过程可以从中间状态重新启动。在一个不可中断并缺少备份的训练任务中，可以通过阶段性的保存每个parameter server的数据快照（snapshot）到 分布式存储服务 达到容灾的目的，比如每隔10分钟最新的快照，并删除更早的快照。在出现单点故障时，只需要恢复这台节点，或者将这台节点迁移到另一个节点并启动即可恢复训练任务。
      
+
      
+
      
+
      快照保存的设计如下：
      
+
      说明：
      
+
        
+
      • parameter server在集群中启动后，自动挂载分布式存储目录，并把快照保存到这个目录下。
        • 
+
      • 注：每个parameter server的检查点各自独立保存，暂时不考虑多个parameter server同步的保存一个特定时间点的全局检查点，因为这样做也没法保证消除随机性。
        • 
+
      
+
      检查点保存程序流程：
      
+
        
+
      1. 如果满足条件”每隔10分钟”时，parameter server会获取parameters内存的read_lock，启动一个新的线程开始保存检查点。如果已经正在执行保存检查点的线程，则忽略。由于对parameters的更新需要获取parameters内存的write_lock，所以在写入快照的过程中，parameter server会暂停参数更新并等待。
        2. 
+
      2. parameter server生成一个UUID，向指定的目录中一个新的文件（文件名为此UUID）写入快照数据。在快照写入完成后，计算这个文件的MD5 sum。然后在etcd的/checkpoints/[pserver_id]中写入json内容：{"uuid": [UUID], "md5", "MD5 sum", "timestamp": xxxx}。
        3. 
+
      3. 删除磁盘目录中不是当前uuid的快照文件。
        4. 
+
      4. 释放对paramters内存的锁定，停止保存检查点的线程。
        5. 
+
      
+
      这里需要用户额外注意，在您的实际环境中，训练任务的运行可能会占满trainer和parameter server之间的网络带宽，如果parameter server此时还需要通过网络访问分布式存储以保存快照，可能会造成网络拥塞，而出现阶段性的运行停滞。
      
+
      
+
      
+
      从快照恢复
      
+
      在parameter server第一次启动或任意时间parameter server故障后被Kubernetes重新启动，则需要回滚到上一个检查点：
      
+
        
+
      1. 从etcd中读取节点：/checkpoints/[pserver_id]获取最新的检查点的文件uuid
        2. 
+
      2. 从磁盘文件中加载uuid文件名的检查点快照文件，并加载其中的参数
        3. 
+
      3. 如果上面两步出现错误，则使用启动参数定义的初始化方法初始化参数
        4. 
+
      4. 开始提供服务
        5. 
+
      
+
      
+
      
+
      
+
      TODO List
      
+
      
+
      推测执行/加速执行（TODO）
      
+
      在异构集群中，如果存在某些trainer执行速度过慢会影响整体集群的速度（如图中Trainer 1），此时master将负责启动一个新的Trainer（Accelerate Trainer 2），使用同样的训练数据block。哪个trainer先完成block的训练，则把另一个慢速的kill掉。
      
+
      
+
      
+
      动态扩容/缩容
      
+
      目前只考虑动态扩容trainer数量，可以减小系统复杂性。
      
+
      
+
      
+
      
+
      术语
      
+
        
+
      • model: 指深度学习训练之后得到的所有参数，使用这个神经网络可以完成对新数据的预测
        • 
+
      • parameters: 神经网络中的参数，包括权重w和偏置b。一个神经网络的模型由大量的参数组成
        • 
+
      • shard: 分片，通常指将一个整体拆分成多份的其中的一份。
        • 
+
      • model shard: 将一个神经网络参数拆分成多份，每个shard分别存储在其中一台parameter server之上
        • 
+
      • parameter block: 多个parameter block构成一个model shard
        • 
+
      • 单点故障: 任意时刻只可能同时有一台服务器故障。由于集群中同时存在两台机器故障的概率极低（（平均故障率*平均故障修复时间）^2）只对特殊在线系统考虑两台以上同时故障的容灾。
        • 
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/cluster_train/data_dispatch.html b/doc_cn/design/cluster_train/data_dispatch.html
new file mode 100644
index 0000000000000000000000000000000000000000..0f3cfcd7cb13af9c747c7488e2345e3d034b27c7
--- /dev/null
+++ b/doc_cn/design/cluster_train/data_dispatch.html
@@ -0,0 +1,419 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  训练数据的存储和分发 — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • 训练数据的存储和分发
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      训练数据的存储和分发
      
+
      
+
      概念解释
      
+
      
+
      
+
      流程介绍
      
+
      生产环境中的训练数据集通常体积很大，并被存储在诸如Hadoop HDFS，Ceph，AWS S3之类的分布式存储之上。这些分布式存储服务通常会把数据切割成多个分片分布式的存储在多个节点之上。这样就可以在云端执行多种数据类计算任务，包括：
      
+
        
+
      • 数据预处理任务
        • 
+
      • Paddle训练任务
        • 
+
      • 在线模型预测服务
        • 
+
      
+
      
+
+
      在上图中显示了在一个实际生产环境中的应用（人脸识别）的数据流图。生产环境的日志数据会通过实时流的方式（Kafka）和离线数据的方式（HDFS）存储，并在集群中运行多个分布式数据处理任务，比如流式数据处理（online data process），离线批处理（offline data process）完成数据的预处理，提供给paddle作为训练数据。用户也可以上传labeled data到分布式存储补充训练数据。在paddle之上运行的深度学习训练输出的模型会提供给在线人脸识别的应用使用。
      
+
      
+
      
+
      训练数据存储
      
+
      我们选择CephFS作为存储系统。
      
+
        
+
      • 无论是从PFSClient的角度，还是从Pod中运行任务的角度，统一用/pfs/$DATACENTER/home/$USER来访问用户自己的数据。
        • 
+
      • /pfs/$DATACENTER/common下存放公共数据集合
          
+
        • 做只读挂载
          • 
+
        
+
        • 
+
      
+
      
+
+
      
+
      
+
      文件预处理
      
+
      在开始训练之前, 数据集需要预先被转换成PaddlePaddle分布式训练使用的存储格RecordIO。我们提供两个转换方式：
      
+
        
+
      1. 用户在本地转换好再上传
        2. 
+
      2. 用户上传数据后，在机群上运行转换程序
        3. 
+
      
+
      转换生成的文件名会是以下格式：
      
+
      name_prefix-aaaaa-of-bbbbb
+
      
+
      
+
      “aaaaa”和”bbbbb”都是五位的数字，每一个文件是数据集的一个shard，”aaaaa”代表shard的index，”bbbbb”代表这个shard的最大index。
      
+
      比如ImageNet这个数据集可能被分成1000个shard，它们的文件名是：
      
+
      imagenet-00000-of-00999
+imagenet-00001-of-00999
+...
+imagenet-00999-of-00999
+
      
+
      
+
      
+
      转换库
      
+
      无论是在本地或是云端转换，我们都提供Python的转换库，接口是：
      
+
      def convert(output_path, reader, num_shards, name_prefix)
+
      
+
      
+
        
+
      • output_path: directory in which output files will be saved.
        • 
+
      • reader: a data reader, from which the convert program will read data instances.
        • 
+
      • num_shards: the number of shards that the dataset will be partitioned into.
        • 
+
      • name_prefix: the name prefix of generated files.
        • 
+
      
+
      reader每次输出一个data instance，这个instance可以是单个值，或者用tuple表示的多个值：
      
+
      yield 1 # 单个值
+yield numpy.random.uniform(-1, 1, size=28*28) # 单个值
+yield numpy.random.uniform(-1, 1, size=28*28), 0 # 多个值
+
      
+
      
+
      每个值的类型可以是整形、浮点型数据、字符串，或者由它们组成的list，以及numpy.ndarray。如果是其它类型，会被Pickle序列化成字符串。
      
+
      
+
      
+
      
+
      示例程序
      
+
      
+
      使用转换库
      
+
      以下reader_creator生成的reader每次输出一个data instance，每个data instance包涵两个值：numpy.ndarray类型的值和整型的值：
      
+
      def reader_creator():
+    def reader():
+        for i in range(1000):
+            yield numpy.random.uniform(-1, 1, size=28*28), 0 # 多个值
+    return reader
+
      
+
      
+
      把reader_creator生成的reader传入convert函数即可完成转换：
      
+
      convert("./", reader_creator(), 100, random_images)
+
      
+
      
+
      以上命令会在当前目录下生成100个文件：
      
+
      random_images-00000-of-00099
+random_images-00001-of-00099
+...
+random_images-00099-of-00099
+
      
+
      
+
      
+
      
+
      进行训练
      
+
      PaddlePaddle提供专用的data reader creator，生成给定RecordIO文件对应的data reader。无论在本地还是在云端，reader的使用方式都是一致的：
      
+
      # ...
+reader = paddle.reader.creator.RecordIO("/pfs/datacenter_name/home/user_name/random_images-*-of-*")
+batch_reader = paddle.batch(paddle.dataset.mnist.train(), 128)
+trainer.train(batch_reader, ...)
+
      
+
      
+
      以上代码的reader输出的data instance与生成数据集时，reader输出的data instance是一模一样的。
      
+
      
+
      
+
      
+
      上传训练文件
      
+
      使用下面命令，可以把本地的数据上传到存储集群中。
      
+
      paddle pfs cp filename /pfs/$DATACENTER/home/$USER/folder/
+
      
+
      
+
      比如，把之前示例中转换完毕的random_images数据集上传到云端的/home/可以用以下指令：
      
+
      paddle pfs cp random_images-*-of-* /pfs/$DATACENTER/home/$USER/folder/
+
      
+
      
+
      需要$DATACENTER的配置写到配置文件中，例如
      
+
      # config file
+[datacenter_1]
+username=user
+usercert=user.pem
+userkey=user-key.pem
+endpoint=datacenter1.paddlepaddle.org
+
+[datacenter_2]
+username=user
+usercert=user.pem
+userkey=user-key.pem
+endpoint=datacenter2.paddlepaddle.org
+
      
+
      
+
      
+
      
+
      
+
      TODO
      
+
      
+
      文件访问的权限
      
+
      控制用户权限
      
+
        
+
      • 用户可以把自己的数据分享给别人
        • 
+
      
+
      
+
      
+
      文件访问方式
      
+
      不用mount的方式来访问数据，而是直接用API的接口远程访问
      
+
      例如：
      
+
      f = open('/pfs/datacenter_name/home/user_name/test1.dat')
+
      
+
      
+
      
+
      
+
      支持用户自定义的数据预处理job
      
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/cluster_train/large_model_dist_train.html b/doc_cn/design/cluster_train/large_model_dist_train.html
new file mode 100644
index 0000000000000000000000000000000000000000..43957d7b416794bc3144910e2b589118a64a0fb6
--- /dev/null
+++ b/doc_cn/design/cluster_train/large_model_dist_train.html
@@ -0,0 +1,356 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Alalysis of large model distributed training in Paddle — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • Alalysis of large model distributed training in Paddle
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      Alalysis of large model distributed training in Paddle
      
+
      NOTE: This is only some note for how we implemeted this scheme in V1, not a new design.
      
+
      
+
      What is it
      
+
      We often encounter cases that the embedding layer parameters(sparse) are so large that we can not store it in the trainer’s memory when training. So we need to put them to several servers, and fetch them row by row instead of fetch all of the parameters.
      
+
      
+
      
+
      How to use
      
+
      Specify command-line argument like  --loadsave_parameters_in_pserver=true --ports_num_for_sparse=1 --use_old_updater=1 when starting the paddle trainer. And also add something like --ports_num_for_sparse=1 --pserver_num_threads=5 when starting pserver processes.
      
+
      Accrodingly, configure your embedding layers like:
      
+
      SPARSE_REMOTE=True
+
+w1 = data_layer(name="w1", size=dict_size)
+emb1 = embedding_layer(input=w1, size=32, param_attr=ParameterAttribute(sparse_update=SPARSE_REMOTE))
+w2 = data_layer(name="w2", size=dict_size)
+emb2 = embedding_layer(input=w2, size=32, param_attr=ParameterAttribute(sparse_update=SPARSE_REMOTE))
+...
+
      
+
      
+
      
+
      
+
      Implementation details
      
+
      enum MatType {
+  MAT_NORMAL,
+  MAT_NORMAL_SHARED,
+  MAT_VALUE_SHARED,
+  MAT_SPARSE_ROW_IDS,
+  MAT_SPARSE_ROW_AUTO_GROW,
+  MAT_CACHE_ROW,
+  MAT_SPARSE_ROW,
+  MAT_SPARSE_ROW_PREFETCH,
+  MAT_SPARSE_ROW_PREFETCH_FULL_SIZE,
+};
+
      
+
      
+
      MAT_SPARSE_ROW_PREFETCH is what we use when configured to fetch only row of matrix when training.
      
+
      In trainer_internal.cpp:L93 trainOneBatch:
      
+
        if (config_->getOptConfig().use_sparse_remote_updater()) {
+    REGISTER_TIMER("prefetch");
+    gradientMachine_->prefetch(inArgs);
+    parameterUpdater_->getParametersRemote();
+  }
+
      
+
      
+
      When doing actual network forward and backward, at the beginning of each batch, the trainer will try to download one row of data from pserver.
      
+
      In trainer/RemoteParameterUpdater.cpp: parameterUpdater_->getParametersRemote();:
      
+
      if (fullSize) {
+    ...
+} else {
+getParams = [&] {
+    parameterClient_->getParameterSparse(
+        /* recvParameterType= */ PARAMETER_VALUE, sendBackParameterType);
+};
+applyL1 = [](Parameter& para, real decayRate) {
+    para.getMat(PARAMETER_VALUE)->applyL1(/*lr=*/1.0f, decayRate);
+};
+}
+
      
+
      
+
      Calling parameterClient_->getParameterSparse will do remote call to pserver’s getParameterSparse:
      
+
      void ParameterServer2::getParameterSparse(const SendParameterRequest& request,
+                                          std::vector<Buffer>& inputBuffers,
+                                          SendParameterResponse* response,
+                                          std::vector<Buffer>* outputBuffers) {
+  (void)inputBuffers;
+  auto& buffer = *readWriteBuffer_;
+  size_t numReals = 0;
+  for (const auto& block : request.blocks()) {
+    numReals += getParameterConfig(block).dims(1);
+  }
+  buffer.resize(numReals);
+
+  VLOG(3) << "pserver: getParameterSparse, numReals=" << numReals;
+
+  ReadLockGuard guard(parameterMutex_);
+  size_t offset = 0;
+  for (const auto& block : request.blocks()) {
+    size_t width = getParameterConfig(block).dims(1);
+    Buffer buf = {buffer.data() + offset, width};
+    int type = request.send_back_parameter_type();
+    sendBackParameterSparse(block, type, response, &buf, width, outputBuffers);
+    offset += width;
+  }
+}
+
      
+
      
+
      getParameterConfig(block).dims(1) returns the width of the current “parameter block”(a shard of parameter object),
+then getParameterSparse remote call returns only one row of data to the client.
      
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/cluster_train/master_server.html b/doc_cn/design/cluster_train/master_server.html
new file mode 100644
index 0000000000000000000000000000000000000000..82812487652db98653fd03081492074b42e108ee
--- /dev/null
+++ b/doc_cn/design/cluster_train/master_server.html
@@ -0,0 +1,354 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Master Server — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • Design Doc: Master Server
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      Design Doc: Master Server
      
+
      For an overview of master server’s role, please refer to distributed training design doc. In this design doc we will discuss the master server in more details. The master will be implemented in Go.
      
+
      
+
      Dataset
      
+
      
+
      A dataset is a list of files in RecordIO format. A RecordIO file consists of chunks, whereas each chunk consists some records.
      
+
      
+
      
+
      Task Queue
      
+
      As mentioned in distributed training design doc, a task is a data shard that the master server assigns to the trainer process to train on. A task consists of one or multiple chunks from one or multiple files. The master server maintains task queues to track the training progress.
      
+
      
+
      Task Queue Creation
      
+
        
+
      1. Each trainer will make an RPC call (using Go’s rpc package) to the master server, telling it the RecordIO files representing the dataset specified by the user. Since every trainer will tell the master server the same dataset, only the first RPC call will be honored.
        
+
        The RPC interface is:
        
+
        func (m *RPCServer) ReportDataset(Paths []string, dummy *int) error {
+}
+
        
+
        
+
        2. 
+
      2. The master server will scan through each RecordIO file to generate the chunk index and know how many chunks does each file have. A chunk can be referenced by the file path and the index of the chunk within the file. The chunk index is in memory data structure that enables fast access to each chunk, and the index of the chunk with the file is an integer start from 0, representing the n-th chunk within the file.
        
+
        The definition of the chunk is:
        
+
        type Chunk struct {
+    Idx   int // index of the chunk within the file
+    Path  string
+    Index recordio.Index // chunk index
+}
+
        
+
        
+
        3. 
+
      3. Chunks are grouped into tasks, and tasks are filled into the todo queue. The pending queue and the done queue are initialized with no element.
        
+
        The definition of the task is:
        
+
        type Task struct {
+    Index  int
+    Chunks []Chunk
+}
+
        
+
        
+
        The elements in the tasks queues is of type TaskEntry, containing a timeout counter (described in task retry logic), and a task:
        
+
        type TaskEntry struct {
+    NumTimeout int
+    Task       Task
+}
+
        
+
        
+
        The definition of task queues is:
        
+
        type TaskQueues struct {
+    Todo    []TaskEntry
+    Pending map[int]TaskEntry // map from task index to task entry
+    Done    []TaskEntry
+}
+
        
+
        
+
        4. 
+
      
+
      
+
      
+
      Task Queue Persistence
      
+
      The task queues need to be persisted on etcd for fault recovery. Since the task queues only change once a task is completed or timed out, which is not very frequent, we can afford to synchronize with etcd every time the task queues change.
      
+
      We will serialize the task queues data structure with gob encoding, compress with gzip, and save into etcd synchronously under key /task_queues.
      
+
      
+
      
+
      Task Dispatch
      
+
      The trainer will make an RPC call to master to get a new task when:
      
+
        
+
      • the trainer first started, or
        • 
+
      • the trainer finishes a task.
        • 
+
      
+
      The RPC interface is:
      
+
      func (m *RPCServer) GetTask(finished *Task, result *Task) error {
+}
+
      
+
      
+
      Argument finished will be nil when the trainer is just started.
      
+
      During the RPC call the master will do the following:
      
+
        
+
      • Make a copy of the task queues, and update the copy reflecting the finished tasks and the new pending tasks.
        • 
+
      • Synchronize the copy of task queues with etcd using a transaction conditioned on holding the master lock.
        • 
+
      • Replace the task queues with the copy and report to the trainer with the new tasks if succeeded, or discard the copy and report the error to the trainer if failed.
        • 
+
      
+
      
+
      
+
      Task Retry Logic
      
+
      When a task is dispatched to the trainer, the master will schedule a function for execution after the timeout duration (based on the moving average of task completion time). If the task entry in still in the pending queue, its timeout counter will increase by one, and the task will be moved to todo queue. If the timeout counter is above the threshold, the master will log the error and discard the task.
      
+
      Please note that since a timed out task could be completed after it has been dispatched for retry, so it is possible for a task to be processed multiple times. We do not try to prevent it from happening since it’s fine to train on the same task multiple times due to the stochastic nature of the stochastic gradient decent algorithm.
      
+
      
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/cluster_train/pserver_client.html b/doc_cn/design/cluster_train/pserver_client.html
new file mode 100644
index 0000000000000000000000000000000000000000..ac6e881489584c75365c17ba50c0eb2d0a308a01
--- /dev/null
+++ b/doc_cn/design/cluster_train/pserver_client.html
@@ -0,0 +1,431 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: The Client Library of Parameter Server — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
      
+    
+    
      
+      
      
+        Fork me on Github
+        
      
+          
+            English
+            
+            
+          
+          
+        
      
+        
+      
      
+      
      
+        
+        
+
+        
+
      
+  
      
+    
+    
+    
+  
      
+
              
+      
      
+    
      
+  
      
+  
+  
      
+
+    
+    
+    
+    
      
+
+      
+
+ 
+
+
+
+
+
+
+
+
      
+  
        
+      
+    
      • Design Doc: The Client Library of Parameter Server
        • 
+  
      
+
      
+      
+      
      
+        
      
+          
      
+           
      
+            
+  
      
+
      Design Doc: The Client Library of Parameter Server
      
+
      For an overview of trainer’s role, please refer to distributed training design doc. In this design doc, we will discuss the parameter server’s client library, which will manage communication with parameter servers. The library will be implemented in Go and made available as a static or dynamic library with a C header file.
      
+
      
+
      Parameter Partition
      
+
      Each parameter will be partitioned into parameter blocks to make the parameters evenly distributed on parameter servers. The partition is done automatically by the client library. The sparse parameter require a little different treatment:
      
+
      
+
      Sparse Parameter
      
+
      The sparse parameter is a parameter that is updated sparsely. The name is somewhat misleading, it does not have a sparse representation, it has the same representation as a dense vector.
      
+
      Because a sparse parameter is updated sparsely, the trainer will have to partition the sparse parameter. Because the parameter server will merge all sparse parameter shard into the same file when saving the parameter. It needs special naming convention:
      
+
      If a sparse parameter is partitioned into n shards, they should be named as:
      
+
      name:sparse-0
+name:sparse-1
+...
+name:sparse-n-1
+
      
+
      
+
      The library is unaware of the partition, and treat each parameter independently. Only when saving parameters, the parameter servers will merge the sparse parameters according to the naming convention.
      
+
      
+
      
+
      
+
      Model Optimization Using Gradients
      
+
      There are two ways to perform model optimization using gradients:
      
+
        
+
      • On Client
        
+
        The client does multiple steps of forward and backward update. In each step, the gradients are calculated and a new model is generated. After some steps, the client will calculate the difference between the newest model and the old model at step 0. The difference will be updated to parameter servers. Parameter servers will just update parameters using the difference without any optimization using gradients (such as Adam and L1 regularization).
        
+
        • 
+
      • On Parameter Server
        
+
        The client will send accumulated gradients to parameter servers, the parameter server will do the optimization using gradients.
        
+
        • 
+
      
+
      
+
      
+
      L1 and L2 Regularization
      
+
      PaddlePaddle allows L1 or L2 regularizations to be specified per parameter, so when the trainer initializes the parameter it needs include a parameter configuration when L1 or L2 regularization is necessary.
      
+
      
+
      
+
      Parameter Initialization
      
+
      The parameters on parameter servers need to be initialized. To provide maximum flexibility, the trainer will initialize the parameters. Only one trainer will do the initialization, the other trainers will wait for the completion of initialization and get the parameters from the parameter servers.
      
+
      
+
      Trainer Selection
      
+
      To select the trainer for initialization, every trainer will try to get a distributed lock, whoever owns the lock will do the initialization. As illustrated below:
      
+
      
+
      
+
      
+
      Trainer Selection Process
      
+
      The trainer select process is encapsulated in the C API function:
      
+
      int paddle_begin_init_params(paddle_pserver_client* client, const char* config_proto);
+
      
+
      
+
      The selected trainer’s call to paddle_begin_init_params will return with 1, and the other trainers’ call to paddle_begin_init_params will return 0. paddle_get_params will be blocked until initialization is completed. As illustrated below:
      
+
      
+
      
+
      
+
      
+
      C Interface
      
+
      typedef enum {
+  PADDLE_ELEMENT_TYPE_INT32   = 0,
+  PADDLE_ELEMENT_TYPE_UINT32  = 1,
+  PADDLE_ELEMENT_TYPE_INT64   = 2,
+  PADDLE_ELEMENT_TYPE_UINT64  = 3,
+  PADDLE_ELEMENT_TYPE_FLOAT32 = 4,
+  PADDLE_ELEMENT_TYPE_FLOAT64 = 5,
+} paddle_element_type;
+
+typedef struct {
+  char*               name;
+  paddle_element_type element_type;
+  unsigned char*      content;
+  int                 content_len;
+} paddle_parameter, paddle_gradient;
+
+typedef int paddle_pserver_client;
+
+/**
+ * @brief creates a pserver client that talks to etcd for coordination.
+ */
+paddle_pserver_client paddle_new_etcd_pserver_client(char* etcd_addr);
+
+/**
+ * @brief creates a pserver client given pserver addresses.
+ *
+ * @param pserver_addrs comma-separated pserver addresses.
+ * @param selected if current pserver client is selected to initialize all parameter servers.
+ */
+paddle_pserver_client paddle_new_pserver_client(char* pserver_addrs, int selected);
+void paddle_pserver_client_release(paddle_pserver_client c);
+
+/**
+ * @brief paddle_begin_init_params begins to initialize parameters on
+ * parameter servers.
+ *
+ * paddle_begin_init_params will be called from multiple trainers,
+ * only one trainer will be selected to initialize the parameters on
+ * parameter servers. Other trainers need to get the initialized
+ * parameters from parameter servers using @paddle_get_params.
+ *
+ * @return 1 if the trainer is selected to initialize parameter
+ * servers, otherwise 0.
+ */
+int paddle_begin_init_params(paddle_pserver_client client);
+
+/**
+ * @brief paddle_init_param initializes the parameter on parameter
+ * servers.
+ *
+ * @param param the parameter to initialize.
+ * @param param_config_proto the configuration for the parameter.
+ * @param config_len the length of param_config_proto
+ * @return 0 if successful, otherwise -1. On failure, the trainer
+ * needs to restart the entire initialization process (starting from
+ * @paddle_begin_init_param). Or simply exit the program and wait for
+ * the cluster management system to restart the trainer.
+ */
+int paddle_init_param(paddle_pserver_client client, paddle_parameter param, const unsigned char* param_config_proto, int config_len);
+
+/**
+ * @brief paddle_finish_init_params tells parameter servers client has
+ * sent all parameters to parameter servers as initialization.
+ *
+ * @return 0 if successful, otherwise -1. On failure, the trainer
+ * needs to restart the entire initialization process (starting from
+ * @paddle_begin_init_param). Or simply exit the program and wait for
+ * the cluster management system to restart the trainer.
+ */
+int paddle_finish_init_params(paddle_pserver_client client);
+
+/**
+ * @brief paddle_send_grads sends gradients to parameter servers for
+ * updating parameters.
+ *
+ * @param grads the array of gradients to send.
+ * @param len the length of the gradient array.
+ * @param learning_rate the learning rate for the gradients.
+ * @return 0 if successful, otherwise -1.
+ */
+int paddle_send_grads(paddle_pserver_client client, const paddle_gradient* grads, int len);
+
+/**
+ * @brief paddle_get_params gets parameters from parameter servers.
+ *
+ * paddle_get_params will block until parameters are initialized on
+ * the parameter servers.
+ *
+ * @param dst the destination array of parameter pointers to save to.
+ * The parameter pointer must be pre-popullated with required parameter name,
+ * and the content of parameter must be pre-allocated of the size of required
+ * parameter on pserver.
+ * @param len the length of the names array and the paddle_parameter
+ * array.
+ * @return 0 if successful, otherwise -1.
+ */
+int paddle_get_params(paddle_pserver_client client, paddle_parameter** dst, int len);
+
+/**
+ * @brief paddle_save_model indicates parameters to save the parameter
+ * to the given path
+ *
+ * @param path the path to save parameters.
+ * @return 0 if successful, otherwise -1.
+ */
+int paddle_save_model(paddle_pserver_client client, const char* path);
+
      
+
      
+
      
+
      
+
+
+           
      
+          
      
+          
      
+  
+
+  
      
+
+  
      
+    
      
+        © Copyright 2016, PaddlePaddle developers.
+
+    
      
+  
      
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
      
+
+        
      
+      
      
+
+    
      
+
+  
      
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/tutorials/rec/ml_dataset_cn.html b/doc_cn/design/cluster_train/remote_parameter_updater.html
similarity index 62%
rename from doc_cn/tutorials/rec/ml_dataset_cn.html
rename to doc_cn/design/cluster_train/remote_parameter_updater.html
index 745874280a83374001cb141a94c9f19baded3384..a649ced2ca4be747627683b5d70a65d06a81b1fb 100644
--- a/doc_cn/tutorials/rec/ml_dataset_cn.html
+++ b/doc_cn/design/cluster_train/remote_parameter_updater.html
@@ -8,7 +8,7 @@
   
   
   
-  MovieLens数据集 — PaddlePaddle  文档
+  Design Doc: Remote Parameter Updater for Cluster Train — PaddlePaddle  文档
   
 
   
@@ -62,7 +62,7 @@
     
      
     
      
       
      
-        Folk me on Github
+        Fork me on Github
         
      
           
             English
@@ -85,6 +85,7 @@
 
    • 进阶指南
      • 
 
    • API
      • 
 
    • FAQ
      • 
+
    • MOBILE
      • 
 
    
 
         
@@ -108,9 +109,10 @@
           
 
         
@@ -173,7 +195,7 @@
 
    
   
      
       
-    
    • MovieLens数据集
      • 
+    
    • Design Doc: Remote Parameter Updater for Cluster Train
      • 
   
    
 
    
       
@@ -182,103 +204,24 @@
           
    
            
    
             
-  
    
-
    MovieLens数据集
    
-
    MovieLens 数据集由GroupLens Research实验室搜集整理。
-该数据集包含一些用户信息、电影信息以及电影评分[1-5]。根据数据量规模，该数据及有很多不同的版本。
-我们用MovieLens 百万数据集作为示例数据
-集，其中包含6,000位用户对4,000部电影的1,000,000条评价。该数据集于2003年2月发布。
    
-
    
-
    数据集特征
    
-
    在ml-1m 数据集中有许多的特征。在[ml-1m 数据集]
-(http://files.grouplens.org/datasets/movielens/ml-1m.zip)中的这些数据文件(含有”.dat”的后缀)实际上是CSV文件，
-分隔符为”::”。以下我们翻译数据集网站中README文件的描述:
    
-
    
-
    评分文件描述(ratings.dat)
    
-
    所有的评分数据都包含在”ratings.dat”文件中，遵循如下的格式:
    
-
    用户ID::电影ID::评分::时间戳
    
-
      
-
    • 用户ID范围从1到6040
      • 
-
    • 电影ID范围从1到3952
      • 
-
    • 评分被调整为5星的规模(只允许整数的星级)
      • 
-
    • 时间戳表示为从1970-01-01(UTC)来的秒数，与time(2)的返回值一致
      • 
-
    • 每位用户至少有20条评分
      • 
-
    
+  
    
+
    Design Doc: Remote Parameter Updater for Cluster Train
    
+
    For an overview of distribute training, please refer to distributed training design doc. In this design doc, we will discuss the parameter updater that will use parameter server cclient The Client Library of Parameter Server Design Doc to manage and update parameters.
    
+
    
+
    Parameter Updater
    
+
    Parameter Updater is used by trainer to manage and update parameter, there are mainly two kind of parameter updater: local and remote, since this design is for cluster train, we will only discuss remote parameter updater here.
    
+
    
+
    Remote Parameter Updater
    
+
    Remote Parameter Updater manage parameters through remote parameter server with the client that communicate with pserver(The Client Library of Parameter Server Design Doc)
    
+
    In PaddlePaddle Python V2 API, trainer is implemented in python, and the trainer will hold a instance of parameter updater and call it’s functions directly. In this design, we will also expose the api of RemoteParameterUpdater to python with swig.
    
+
    
+
    Sparse Remote Parameter Updater
    
+
    Since we will only implement dense parameter management new, the mechanism for sparse parameter will be discussed in next stage.
    
 
    
-
    
-
    用户文件描述(users.dat)
    
-
    所有的用户信息都包含在”users.dat”文件中，遵循如下的格式:
    
-
    用户ID::性别::年龄::职业::邮编
    
-
    所有的人口统计学信息由用户自愿提供，没有进行正确性的检查。只有含有人
-口统计学信息的用户才被包含在数据集中。
    
-
      
-
    • 性别，用”M”表示男性，”F”表示女性
      • 
-
    • 年龄从下列列表范围中选取:
        
-
      • 1:  “18岁以下”
        • 
-
      • 18:  “18-24岁”
        • 
-
      • 25:  “25-34岁”
        • 
-
      • 35:  “35-44岁”
        • 
-
      • 45:  “45-49岁”
        • 
-
      • 50:  “50-55岁”
        • 
-
      • 56:  “56+”
        • 
-
      
-
      • 
-
    • 职业从下面所列中选择:
        
-
      • 0:  “其他”或不确定
        • 
-
      • 1:  “学术/教育工作者”
        • 
-
      • 2:  “艺术家”
        • 
-
      • 3:  “文书工作/管理员”
        • 
-
      • 4:  “大学生/研究生”
        • 
-
      • 5:  “客户服务”
        • 
-
      • 6:  “医生/医疗保健”
        • 
-
      • 7:  “行政工作/管理人员”
        • 
-
      • 8:  “农民”
        • 
-
      • 9:  “操持家务者”
        • 
-
      • 10:  “高中毕业生”
        • 
-
      • 11:  “律师”
        • 
-
      • 12:  “程序员”
        • 
-
      • 13:  “退休人员”
        • 
-
      • 14:  “销售/市场”
        • 
-
      • 15:  “科学家”
        • 
-
      • 16:  “自由职业者”
        • 
-
      • 17:  “技术员/工程师”
        • 
-
      • 18:  “推销员/手工艺者”
        • 
-
      • 19:  “无业人士”
        • 
-
      • 20:  “作家”
        • 
-
      
-
      • 
-
    
 
    
-
    
-
    电影文件描述(movies.dat)
    
-
    所有的电影信息都包含在”movies.dat”文件中，遵循如下的格式:
    
-
    电影ID::电影名称::电影类型
    
-
      
-
    • 电影名称（包括发行时间）与IMDB网站提供的一致
      • 
-
    • 电影类型如符合多种用管道符号|分割，选自下列类型:
        
-
      • 动作片
        • 
-
      • 冒险片
        • 
-
      • 动画片
        • 
-
      • 儿童片
        • 
-
      • 喜剧片
        • 
-
      • 犯罪片
        • 
-
      • 纪录片
        • 
-
      • 戏剧
        • 
-
      • 奇幻片
        • 
-
      • 黑色电影
        • 
-
      • 恐怖片
        • 
-
      • 音乐剧
        • 
-
      • 悬疑片
        • 
-
      • 浪漫片
        • 
-
      • 科幻片
        • 
-
      • 惊险电影
        • 
-
      • 战争片
        • 
-
      • 西部片
        • 
-
      
-
      • 
-
    • 由于意外的副本记录和测试记录，有些电影ID可能与实际电影不相符合
      • 
-
    • 电影大部分是手工输入数据，因此可能会有一些错误和不一致发生
      • 
-
    
+
    
+
    Interface Design
    
+
    TBD
    
 
    
 
    
 
    
diff --git a/doc_cn/design/cluster_train/save_model.html b/doc_cn/design/cluster_train/save_model.html
new file mode 100644
index 0000000000000000000000000000000000000000..c8c3ad90162c4e1c469022adb72cccb983e4643f
--- /dev/null
+++ b/doc_cn/design/cluster_train/save_model.html
@@ -0,0 +1,373 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Save Model — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Save Model
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Save Model
    
+
    
+
    Overview
    
+
    The model is the output of the training process. There are two
+ways from which user can obtain a model:
    
+
      
+
    • Save model triggered by user code: user code asks PaddlePaddle to
+save a model.
      • 
+
    • Convert model from the checkpoint: model being converted from
+pservers’ periodic checkpoint. In this way, the user can cancel a
+job at any time, and still have a relatively fresh model (we
+checkpoint around every 5 minutes).
      • 
+
    
+
    
+
    Trainer Saving Model vs. Pservers Saving Model
    
+
    Both trainers and pservers have access to the model. So the model can
+be saved from a trainer or pservers. We need to decide where the model
+is saved from.
    
+
    
+
    Dense Update vs. Sparse Update
    
+
    There are two types of model update methods: dense update and sparse
+update (when the model parameter is configured to be sparse).
    
+
      
+
    • Dense update
      
+
      Every trainer has it’s own full copy of the model. Every model
+update will update the entire model.
      
+
      • 
+
    • Sparse update
      
+
      The training input is sparse, and the trainer does not have the
+entire model. It will only download the sub-model necessary related
+to the input. When updating the model, only the sub-model related to
+the training input is updated.
      
+
      • 
+
    
+
    
+
    
+
    Pservers Saving Model
    
+
    The benefit of letting pservers save model is they have the entire
+model all the time. However, since pservers are on different nodes, it
+requires a merging process to merge model shards into the same
+model. Thus requires the pservers to write models to a distributed
+filesystem, making the checkpoint shards visible to the merge program.
    
+
    
+
    
+
    Trainer Saving Model
    
+
    The benefit of letting one trainer to save the model is it does not
+require a distributed filesystem. And it’s reusing the same save model
+logic when training locally - except when doing sparse update, the
+trainer needs to download the entire model during the saving process.
    
+
    
+
    
+
    Conclusion
    
+
    Given trainer saving model does not require a distributed filesystem,
+and is an intuitive extension to trainer saving model when training
+locally, we decide to let the trainer save the model when doing
+distributed training.
    
+
    
+
    
+
    
+
    Convert Model from Checkpoint
    
+
    TODO
    
+
    
+
    
+
    
+
    Timeline
    
+
    We first implement trainer save the model. Converting the latest
+snapshot to a model will be a TODO for future.
    
+
    
+
    
+
    Trainer Save Model
    
+
    
+
    Trainer Election
    
+
    One trainer will be elected as the one to save the model. When using
+etcd, trainer ID is a randomly generated UUID, the trainer will
+contact the master server requesting to save the model, and find out
+if itself is elected. When the master server is not used, unique
+trainer IDs will be given by the administrator, the trainer whose ID
+is “0” is elected to save the model.
    
+
    
+
    
+
    Model Save Path
    
+
    Each trainer will be given the directory to save the model. The
+elected trainer will save the model to
+given-directory/trainerID. Since the trainer ID is unique, this
+would prevent concurrent save to the same file when multiple trainers
+are elected to save the model when split-brain problem happens.
    
+
    
+
    
+
    What Happens When Model Is Saving
    
+
    It takes some time to save model, we need to define what will happen
+when save model is taking place.
    
+
    When doing dense update, the trainer uses the local model. Pservers
+does not need to pause model update.
    
+
    When doing sparse update. The trainer needs to download the entire
+model while saving. To get the most accurate model, the model update
+needs to be paused before the download starts and resumed after the
+download finishes. Otherwise, the trainer gets a model that is
+“polluted”: some part of the model is old, some part of the model is
+new.
    
+
    It’s unclear that the “polluted” model will be inferior due to the
+stochastic nature of deep learning, and pausing the model update will
+add more complexity to the system. Since supporting sparse update is a
+TODO item. We defer the evaluation of pause the model update or not
+during saving model to the future.
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/cluster_train/submit-job.html b/doc_cn/design/cluster_train/submit-job.html
new file mode 100644
index 0000000000000000000000000000000000000000..fca642ff3abbe61093116b2bd9333d1c6c1b7153
--- /dev/null
+++ b/doc_cn/design/cluster_train/submit-job.html
@@ -0,0 +1,402 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Submit a Distributed Training Job — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Submit a Distributed Training Job
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Submit a Distributed Training Job
    
+
    The user can submit a distributed training job with Python code, rather than with a command-line interface.
    
+
    
+
    Runtime Environment On Kubernetes
    
+
    For a distributed training job, there is two Docker image called runtime Docker image and base Docker image. The runtime Docker image is the Docker image that gets scheduled by Kubernetes to run during training. The base Docker image is for building the runtime Docker image.
    
+
    
+
    Base Docker Image
    
+
    Usually, the base Docker image is PaddlePaddle product Docker image including paddle binary files and python package. And of course, users can specify any image name hosted on any docker registry which users have the access right.
    
+
    
+
    
+
    Runtime Docker Image
    
+
    The trainer package which user upload and some Python dependencies are packaged into a runtime Docker image based on base Docker image.
    
+
      
+
    • Handle Python Dependencies
      
+
      You need to provide requirements.txt file in your trainer-package folder. Example:
      
+
      pillow
+protobuf==3.1.0
+
      
+
      
+
      More details about requirements, an example project looks like:
      
+
        paddle_example
+    |-quick_start
+      |-trainer.py
+      |-dataset.py
+      |-requirements.txt
+
      
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    Submit Distributed Training Job With Python Code
    
+
    
+
      
+
    • paddle.job.dist_train() will call the Job Server API /v1/packages to upload the trainer package and save them on CephFS, and then call /v1/trainer/job to submit the PaddlePaddle distributed job.
      • 
+
    • /v1/trainer/job will start a building job for preparing the runtime Docker image. When the building job is finished, Job Server will submit the PaddlePaddle distributed job to Kubernetes.
      • 
+
    • NOTE: For the first version, we will not prepare the runtime Docker image, instead, the package is uploaded to Paddle Cloud, and Paddle Cloud will mount the package in a temporary folder into the base Docker image. We will not support custom Python dependencies in the first version as well.
      • 
+
    
+
    You can call paddle.job.dist_train and provide distributed training configuration as the parameters:
    
+
    paddle.job.dist_train(
+  trainer=dist_trainer(),
+  paddle_job=PaddleJob(
+    job_name = "paddle-cloud",
+    entry_point = "python %s"%__file__,
+    trainer_package = "/example/word2vec",
+    image = "yancey1989/paddle-job",
+    trainers = 10,
+    pservers = 3,
+    trainer_cpu = 1,
+    trainer_gpu = 1,
+    trainer_mem = "10G",
+    pserver_cpu = 1,
+    pserver_mem = "2G"
+  ))
+
    
+
    
+
    The parameter trainer of paddle.job.dist_train is a function and you can implement it as follows:
    
+
    def dist_trainer():
+  def trainer_creator():
+    trainer = paddle.v2.trainer.SGD(...)
+    trainer.train(...)
+  return trainer_creator
+
    
+
    
+
    The pseudo code of paddle.job.dist_train is as follows:
    
+
    def dist_train(trainer, paddle_job):
+  # if the code is running on cloud, set PADDLE_ON_CLOUD=YES
+  if os.getenv("RUNNING_ON_CLOUD", "NO") == "NO":
+    #submit the paddle job
+    paddle_job.submit()
+  else:
+    #start the training
+    trainer()
+
    
+
    
+
    
+
    PaddleJob Parameters
    
+
    parameter | type | explanation
+— | — | —
+job_name | str | the unique name for the training job
+entry_point | str | entry point for startup trainer process
+trainer_package | str | trainer package file path which user have the access right
+image|str|the base image for building the runtime image
+pservers|int| Parameter Server process count
+trainers|int| Trainer process count
+pserver_cpu|int| CPU count for each Parameter Server process
+pserver_mem|str| memory allocated for each Parameter Server process, a plain integer using one of these suffixes: E, P, T, G, M, K
+trainer_cpu|int| CPU count for each Trainer process
+trainer_mem|str| memory allocated for each Trainer process, a plain integer using one of these suffixes: E, P, T, G, M, K
+trainer_gpu|int| GPU count for each Trainer process, if you only want CPU, do not set this parameter
    
+
    
+
    
+
    Deploy Parameter Server, Trainer and Master Process
    
+
      
+
    • Deploy PaddlePaddle Parameter Server processes, it’s a Kubernetes ReplicaSet.
      • 
+
    • Deploy PaddlePaddle Trainer processes, it’s a Kubernetes Job.
      • 
+
    • Deploy PaddlePaddle Master processes, it’s a Kubernetes ReplicaSet.
      • 
+
    
+
    
+
    
+
    
+
    Job Server
    
+
      
+
    • RESTful API
      
+
      Job server provides RESTful HTTP API for receiving the trainer package and displaying
+PaddlePaddle job related informations.
      
+
        
+
      • POST /v1/package receive the trainer package and save them on CephFS
        • 
+
      • POST /v1/trainer/job submit a trainer job
        • 
+
      • GET /v1/jobs/ list all jobs
        • 
+
      • GET /v1/jobs/<job-name> the status of a job
        • 
+
      • DELETE /v1/jobs/<job-name> delete a job
        • 
+
      • GET /v1/version job server version
        • 
+
      
+
      • 
+
    • Build Runtime Docker Image on Kubernetes
      
+
      paddle.job.dist_train will upload the trainer package to Job Server, save them on the distributed filesystem, and then start up a job for building the runtime Docker image that gets scheduled by Kubernetes to run during training.
      
+
      There are some benefits for building runtime Docker image on JobServer:
      
+
        
+
      • On Paddle Cloud, users will run the trainer code in a Jupyter Notebook which is a Kubernetes Pod, if we want to execute docker build in the Pod, we should mount the host’s docker.sock to the Pod, user’s code will connect the host’s Docker Engine directly, it’s not safe.
        • 
+
      • Users only need to upload the training package files, does not need to install docker engine, docker registry as dependencies.
        • 
+
      • If we want to change another image type, such as RKT, users do not need to care about it.
        • 
+
      
+
      • 
+
    • Deploy Parameter Server, Trainer and Master Processes
      
+
      POST /v1/trainer/job receives the distributed training parameters, and deploy the job as follows:
      
+
        
+
      • Deploy PaddlePaddle Parameter Server processes, it’s a Kubernetes ReplicaSet.
        • 
+
      • Deploy PaddlePaddle Trainer processes, it’s a Kubernetes Job.
        • 
+
      • Deploy PaddlePaddle Master processes, it’s a Kubernetes ReplicaSet.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/evaluator.html b/doc_cn/design/evaluator.html
new file mode 100644
index 0000000000000000000000000000000000000000..5f61dfd422f90ec8e878eade3d7705ad3a33c126
--- /dev/null
+++ b/doc_cn/design/evaluator.html
@@ -0,0 +1,325 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Evaluator Design — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Evaluator Design
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Evaluator Design
    
+
    
+
    Problem Statement
    
+
    During training or inference, we provide an evaluation function to measure the model performance, for example, accuracy, precision, etc. In the operator based framework design, the data passes through the network pipeline batch by batch. As a result, inside the operator, we only calculate the metrics for one minibatch. Thus, we need to provide a mechanism to calculate the metrics for each N pass/batch the user wants.
    
+
    
+
    
+
    Evaluator Design
    
+
    Currently, every operation is expressed in the graph. We divide the evaluator process into three steps.
    
+
      
+
    1. Initialize the metric state and add it into the block.
      2. 
+
    2. Calculate the concerned metrics for every mini-batch. The single evaluator operator is only responsible for calculating the necessary statistics for one mini-batch. For example, the accuracy operator only calculates the accuracy for a minibatch data if run once.
      3. 
+
    
+
      
+
    1. Merge the mini-batch statistics to form the evaluation result for multiple mini-batches. When it comes to distributed training/Multi-GPU training, aggregate the value from different devices.
      2. 
+
    
+
    
+
    
+
    Implementation
    
+
    This design is shown in the Python API.
+Each metric operator needs to caculate the metric statistic and return the batch-aware states. Python side is responsible for accumulating the states for each pass.
    
+
    class Evaluator(object):
+    """
+    Evaluator Base class.
+    """
+    def __init__(self, name, **kwargs):
+       """
+       Different evaluator may has different metric states. E.g, Accuracy need two variables, total and right sample counts.
+       Auc need four variables, `true_positives`,
+         `true_negatives`, `false_positives` and `false_negatives`. So every evaluator should create its needed variables and append to main_program
+
+       The initialization of Evaluator should be responsible for:
+       create metric states and append to the main_program
+       """ 
+       pass
+
+    def _update_ops(self, input, label, **kwargs)
+       """
+       Add mini-batch evaluator caculate operators to the main_program.
+       Add increment operator to accumulate the metric states.
+       """
+    
+
+    def reset(self, executor, reset_program=None):
+      """
+      Reset metric states at the begin of each pass/user specified batch number.
+      Execute the reset_program to reset the states.
+      """
+      
+
+    def eval(self, executor, eval_program=None):
+      """
+      Merge the mini-batch statistics to form the evaluation result for multiple mini-batches.
+      Execute the eval_program and return the result.
+      """
+      return eval_result
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/executor.html b/doc_cn/design/executor.html
new file mode 100644
index 0000000000000000000000000000000000000000..cba81be0c8a12366659c0ba589d82ed8cb57a685
--- /dev/null
+++ b/doc_cn/design/executor.html
@@ -0,0 +1,289 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Executor Design Doc — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Executor Design Doc
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Executor Design Doc
    
+
    
+
    Motivation
    
+
    We use executor to do the runtime evaluation of a ProgramDesc.
    
+
    
+
    
+
    Overview
    
+
    An executor takes a ProgramDesc, a block_id and a Scope.  The ProgramDesc is a list of blocks and each block contains the protobuf definition of all the parameters and operators. The block_id specifies the entrance block. And the Scope is the container of all the variable instance, which is persistent throughout different runs.
    
+
    
+
    What does executor do?
    
+
    It evaluates all the operators in the block_idth block of a ProgramDesc.
    
+
    
+
    
+
    What does executor NOT do?
    
+
    It does not do runtime optimization, meaning intelligently parse the dependency of each op a choose which one to be run and in which order they should be run.
    
+
    It does not do graph partitioning, meaning dividing the ProgramDesc into several small pieces and executing them on different devices.
    
+
    
+
    
+
    
+
    Implementation
    
+
    Executor evaluates a ProgramDesc. Essentially, it instantiates Variables and Operators, then run all the operators in sequence. [code]
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/file_manager/README.html b/doc_cn/design/file_manager/README.html
new file mode 100644
index 0000000000000000000000000000000000000000..a4614d10fe9559f77e11b082fe9c0f731431284d
--- /dev/null
+++ b/doc_cn/design/file_manager/README.html
@@ -0,0 +1,378 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  FileManager设计文档 — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • FileManager设计文档
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    FileManager设计文档
    
+
    
+
    目标
    
+
    在本文档中，我们设计说明了名为FileManager系统，方便用户上传自己的训练数据以进行分布式训练
    
+
    主要功能包括：
    
+
      
+
    • 提供常用的命令行管理命令管理文件和目录
      • 
+
    • 支持大文件的断点上传、下载
      • 
+
    
+
    
+
    
+
    名词解释
    
+
      
+
    • PFS：是Paddlepaddle cloud File System的缩写，是对用户文件存储空间的抽象，与之相对的是local filesystem。目前我们用CephFS来搭建。
      • 
+
    • CephFS：一个POSIX兼容的文件系统。
      • 
+
    • Chunk：逻辑划上文件分块的单位。
      • 
+
    
+
    
+
    
+
    模块
    
+
    
+
    架构图
    
+
    
+
    
+
    
+
    PFSClient
    
+
      
+
    • 功能： 详细设计link
        
+
      • 提供用户管理文件的命令
        • 
+
      • 需要可以跨平台执行
        • 
+
      
+
      • 
+
    • 双向验证PFSClient需要和Ingress之间做双向验证tls，所以用户需要首先在cloud.paddlepaddle.org上注册一下，申请用户空间，并且把系统生成的CA(certificate authority)、Key、CRT(CA signed certificate)下载到本地，然后才能使用PFSClient。
      • 
+
    
+
    
+
    
+
    Ingress
    
+
      
+
    • 功能：提供七层协议的反向代理、基于粘性会话的负载均衡功能。
      • 
+
    • 透传用户身份的办法Ingress需要把PFSClient的身份信息传给PFSServer，配置的方法参考link
      • 
+
    
+
    
+
    
+
    PFSServer
    
+
    PFSServer提供RESTful API接口，接收处理PFSClient端的文件管理请求，并且把结果返回PFSClient端。
    
+
    RESTful API
    
+
      
+
    • /api/v1/files
        
+
      • GET /api/v1/files: Get metadata of files or directories.
        • 
+
      • POST /api/v1/files: Create files or directories.
        • 
+
      • PATCH /api/v1/files: Update files or directories.
        • 
+
      • DELETE /api/v1/files: Delete files or directories.
        • 
+
      
+
      • 
+
    • /api/v1/file/chunks
        
+
      • GET /api/v1/storage/file/chunks: Get chunks’s metadata of a file.
        • 
+
      
+
      • 
+
    • /api/v1/storage/files
        
+
      • GET /api/v1/storage/files: Download files or directories.
        • 
+
      • POST /api/v1/storage/files: Upload files or directories.
        • 
+
      
+
      • 
+
    • /api/v1/storage/file/chunks
        
+
      • GET /api/v1/storage/file/chunks: Download chunks’s data.
        • 
+
      • POST /api/v1/storage/file/chunks: Upload chunks’s data.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    文件传输优化
    
+
    
+
    分块文件传输
    
+
    用户文件可能是比较大的，上传到Cloud或者下载到本地的时间可能比较长，而且在传输的过程中也可能出现网络不稳定的情况。为了应对以上的问题，我们提出了Chunk的概念，一个Chunk由所在的文件偏移、数据、数据长度及校验值组成。文件的上传和下载都是通过对Chunk的操作来实现的。由于Chunk比较小（默认256K），完成一个传输动作完成的时间也比较短，不容易出错。PFSClient需要在传输完毕最后一个Chunk的时候检查destination文件的MD5值是否和source文件一致。
    
+
    一个典型的Chunk如下所示：
    
+
    type Chunk struct {
+    fileOffset int64
+    checksum uint32
+    len     uint32
+    data    []byte
+}
+
    
+
    
+
    
+
    
+
    生成sparse文件
    
+
    当destination文件不存在或者大小和source文件不一致时，可以用Fallocate生成sparse文件，然后就可以并发写入多个Chunk。
    
+
    
+
    
+
    覆盖不一致的部分
    
+
    文件传输的的关键在于需要PFSClient端对比source和destination的文件Chunks的checksum是否保持一致，不一致的由PFSClient下载或者传输Chunk完成。这样已经传输成功的部分就不用重新传输了。
    
+
    
+
    
+
    
+
    用户使用流程
    
+
    参考link
    
+
    
+
    
+
    框架生成
    
+
    用swagger生成PFSClient和PFSServer的框架部分，以便我们可以把更多的精力放到逻辑本身上。
    
+
    
+
    
+
    参考文档
    
+
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/file_manager/pfs/pfsclient.html b/doc_cn/design/file_manager/pfs/pfsclient.html
new file mode 100644
index 0000000000000000000000000000000000000000..6fc8c0c09867d6439005cdaaa4e18401fbf57de8
--- /dev/null
+++ b/doc_cn/design/file_manager/pfs/pfsclient.html
@@ -0,0 +1,404 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  PFSClient — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • PFSClient
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    PFSClient
    
+
    
+
    Description
    
+
    The pfs command is a Command Line Interface to manage your files on PaddlePaddle Cloud
    
+
    
+
    
+
    Synopsis
    
+
    paddle [options] pfs <subcommand> [parameters]
+
    
+
    
+
    
+
    
+
    Options
    
+
    --profile (string)
+    Use a specific profile from your credential file.
+
+--help (string)
+    Display more information about command
+
+--version
+    Output version information and exit
+
+--debug
+    Show detailed debugging log 
+    
+--only-show-errors (boolean) 
+    Only errors and warnings are displayed. All other output is suppressed.
+
    
+
    
+
    
+
    
+
    Path Arguments
    
+
    When using a command, we need to specify path arguments. There are two path argument type: localpath and pfspath.
    
+
    A pfspath begin with /pfs, eg: /pfs/$DATACENTER/home/$USER/folder.
    
+
    Here is how to config datacenters.
    
+
    
+
    
+
    order of Path Arguments
    
+
    Commonly, if there are two path arguments, the first is the source, and the second is the destination.
    
+
    
+
    
+
    Subcommonds
    
+
      
+
    • rm - remove files or directories
      • 
+
    
+
    Synopsis:
+    rm [-r] [-v] <PFSPath> ...
+
+Options:
+    -r 
+        Remove directories and their contents recursively 
+    -v      
+        Cause rm to be verbose, showing files after they are removed.
+    
+Examples:
+    paddle pfs rm /pfs/$DATACENTER/home/$USER/file
+    paddle pfs rm -r /pfs/$DATACENTER/home/$USER/folder
+
    
+
    
+
      
+
    • mv - move (rename) files
      • 
+
    
+
    Synopsis:
+    mv [-f | -n] [-v] <LocalPath> <PFSPath>
+    mv [-f | -n] [-v] <LocalPath> ... <PFSPath>
+    mv [-f | -n] [-v] <PFSPath> <LocalPath> 
+    mv [-f | -n] [-v] <PFSPath> ... <LocalPath> 
+    mv [-f | -n] [-v] <PFSPath> <PFSPath> 
+    mv [-f | -n] [-v] <PFSPath> ... <PFSPath> 
+    
+Options:
+    -f      
+        Do not prompt for confirmation before overwriting the destination path.  (The -f option overrides previous -n options.)
+    -n      
+        Do not overwrite an existing file.  (The -n option overrides previous -f options.)
+    -v      
+        Cause mv to be verbose, showing files after they are moved.
+        
+Examples:
+    paddle pfs mv ./text1.txt /pfs/$DATACENTER/home/$USER/text1.txt
+
    
+
    
+
      
+
    • cp - copy files or directories
      • 
+
    
+
    Synopsis:
+    cp [-r] [-f | -n] [-v] [--preserve--links] <LocalPath> <PFSPath>
+    cp [-r] [-f | -n] [-v] [--preserve--links] <LocalPath> ... <PFSPath>
+    cp [-r] [-f | -n] [-v] [--preserve--links] <PFSPath> <LocalPath> 
+    cp [-r] [-f | -n] [-v] [--preserve--links] <PFSPath> ... <LocalPath>
+    cp [-r] [-f | -n] [-v] [--preserve--links] <PFSPath> <PFSPath> 
+    cp [-r] [-f | -n] [-v] [--preserve--links] <PFSPath> ... <PFSPath>
+
+Options:
+    -r
+        Copy directories recursively
+    -f      
+        Do not prompt for confirmation before overwriting the destination path.  (The -f option overrides previous -n options.)
+    -n      
+        Do not overwrite an existing file.  (The -n option overrides previous -f options.)
+    -v      
+        Cause cp to be verbose, showing files after they are copied.
+    --preserve--links
+       Reserve links when copy links
+       
+Examples:
+    paddle pfs cp ./file /pfs/$DATACENTER/home/$USER/file
+    paddle pfs cp /pfs/$DATACENTER/home/$USER/file ./file
+
    
+
    
+
      
+
    • ls- list files
      • 
+
    
+
    Synopsis:
+    ls [-r] <PFSPath> ...
+    
+Options:
+    -R
+        List directory(ies) recursively
+
+Examples:
+    paddle pfs ls  /pfs/$DATACENTER/home/$USER/file
+    paddle pfs ls  /pfs/$DATACENTER/home/$USER/folder
+
    
+
    
+
      
+
    • mkdir - mkdir directory(ies)
+Create intermediate directory(ies) as required.
      • 
+
    
+
    Synopsis:
+    mkdir <PFSPath> ...
+
+Examples:
+    paddle pfs mkdir  /pfs/$DATACENTER/home/$USER/folder
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/float16.html b/doc_cn/design/float16.html
new file mode 100644
index 0000000000000000000000000000000000000000..ab0e16ca988537161274391e559c1c3db4ce1c26
--- /dev/null
+++ b/doc_cn/design/float16.html
@@ -0,0 +1,383 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: float16 — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: float16
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: float16
    
+
    
+
    Why float16
    
+
    Half precision (float16) is a binary floating-point format that occupies 16 bits in memory. float16 is half the size of traditional 32-bit single precision format (float) and has lower precision and smaller range.
    
+
    When high precision computation is not required, using float16 data type could potentially
    
+
      
+
    • reduce storage space, memory bandwidth, and power usages;
      • 
+
    • increase the chance of data fitting into a smaller cache of lower latency;
      • 
+
    • provide arithmetic speed up if supported by hardware.
      • 
+
    
+
    
+
    
+
    Survey of current float16 support
    
+
    A brief survey of float16 support on different compilers, hardwares, and libraries can be found below. Interested readers can refer to link1 and link2 for more info.
    
+
    The goal of float16 is to serve as a key for the executor to find and run the correct version of compute method specialized for float16 in operator kernel. It should be compatible with various natively supported float16 implementations including __half for cuda, float16_t for ARM, and Eigen::half for Eigen to make writing customized float16 kernels easier.
    
+
    
+
    Compiler
    
+
      
+
    • nvcc supports __half data type after CUDA 7.5.
      • 
+
    • __fp16 or float16_t is supported as storage type for gcc >= 6.1 and clang >= 3.4.
      • 
+
    • __fp16 or float16_t is supported as arithmetic type for gcc >= 7.1 and clang >= 3.9.
      • 
+
    
+
    
+
    
+
    Hardware
    
+
      
+
    • __half is supported on GPU with compute capability >= 5.3.
      • 
+
    • __fp16 is supported as storage type for ARMv7-A, ARMv8-A, and above.
      • 
+
    • __fp16 is supported as arithmetic type after ARMv8.2-A (currently, the only microarchitecture implementing ARMv8.2-A is ARM Cortex-A75, which is announced in May 2017. There seems to be no application processors currently available on market that adopts this architecture. It is reported that Qualcomm Snapdragon 845 uses Cortex-A75 design and will be available in mobile devices in early 2018).
      • 
+
    
+
    
+
    
+
    Libraries
    
+
      
+
    • Eigen >= 3.3 supports float16 calculation on both GPU and CPU using the Eigen::half class. It is mostly useful for Nvidia GPUs because of the overloaded arithmetic operators using cuda intrinsics. It falls back to using software emulation on CPU for calculation and there is no special treatment to ARM processors.
      • 
+
    • ARM compute library >= 17.02.01 supports NEON FP16 kernels (requires ARMv8.2-A CPU).
      • 
+
    
+
    
+
    
+
    CUDA version issue
    
+
    There are currently three versions of CUDA that supports __half data type, namely, CUDA 7.5, 8.0, and 9.0.
+CUDA 7.5 and 8.0 define __half as a simple struct that has a uint16_t data (see cuda_fp16.h) as follows:
    
+
    typedef struct __align__(2) {
+   unsigned short x;
+} __half;
+
+typedef __half half;
+
    
+
    
+
    This struct does not define any overloaded arithmetic operators. So you have to directly use __hadd instead of + to correctly add two half types:
    
+
    __global__ void Add() {
+  half a, b, c;
+  c = __hadd(a, b); // correct
+  c = a + b; // compiler error: no operator "+" matches these operands
+}
+
    
+
    
+
    CUDA 9.0 provides a major update to the half data type. The related code can be found in the updated cuda_fp16.h and the newly added cuda_fp16.hpp.
    
+
    Essentially, CUDA 9.0 renames the original __half type in 7.5 and 8.0 as __half_raw, and defines a new __half class type that has constructors, conversion operators, and also provides overloaded arithmetic operators such as follows:
    
+
    typedef struct __CUDA_ALIGN__(2) {
+    unsigned short x;
+} __half_raw;
+
+
+struct __CUDA_ALIGN__(2) __half {
+protected:
+    unsigned short __x;
+public:
+    // constructors and conversion operators from/to 
+    // __half_raw and other built-in data types
+}
+
+typedef __half half;
+
+__device__ __forceinline__ 
+__half operator+(const __half &lh, const __half &rh) { 
+    return __hadd(lh, rh); 
+}
+
+// Other overloaded operators
+
    
+
    
+
    This new design makes c = a + b work correctly for CUDA half data type.
    
+
    
+
    
+
    
+
    Implementation
    
+
    The float16 class holds a 16-bit uint16_t data internally.
    
+
    struct float16 {
+  uint16_t x;
+};
+
    
+
    
+
    float16 supports the following features:
    
+
      
+
    • constructors / assignment operators that take input from primitive data types including bool, integers of various length, float, and double.
      • 
+
    • constructors / assignment operators that take input from __half on cuda, float16_t on ARM, and Eigen::half on Eigen.
      • 
+
    • conversion operators to primitive data types and half precision data types on cuda, ARM and Eigen.
      • 
+
    • overloaded arithmetic operators for cuda, arm, and non-arm cpu, respectively. These operators will take advantage of the cuda and ARM intrinsics on the corresponding hardware.
      • 
+
    
+
    To support the above features, two fundamental conversion functions are provided:
    
+
    float16 float_to_half_rn(float f);  // convert to half precision in round-to-nearest-even mode
+float half_to_float(float16 h);
+
    
+
    
+
    which provides one-to-one conversion between float32 and float16. These twos functions will do different conversion routines based on the current hardware. CUDA/ARM instrinsics will be used when the corresonding hardware is available. If the hardware or compiler level does not support float32 to float16 conversion, software emulation will be performed to do the conversion.
    
+
    
+
    
+
    To do
    
+
    After float16 class is available, some of the future items are below:
    
+
      
+
    • Update pybind/tensor_py.h to bind c++ float16 with numpy float16.
      • 
+
    • Modify GetKernelType() method in framework/operator.h to make it compatible with float16.
      • 
+
    • Create a type-casting operator that can convert the data type in tensor between float16 and other types.
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/functions_operators_layers.html b/doc_cn/design/functions_operators_layers.html
new file mode 100644
index 0000000000000000000000000000000000000000..dd2b818768f5882c54f679c8cf56b00ebd142a70
--- /dev/null
+++ b/doc_cn/design/functions_operators_layers.html
@@ -0,0 +1,344 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Functions, Operators, and Layers — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Functions, Operators, and Layers
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Functions, Operators, and Layers
    
+
    In a DL system, we can compose one or more fine grained operators into a coarse grained one.  For example, the FC layer can be composed of a multiplication operator and an add operator.
    
+
    Historically, some fine grained operations are known as operators, and some coarse level ones are known as layers.  But we need a well-defined separation.
    
+
    In general, operators are those very fine grained operations, e.g., mul and add. In the implementation, we can write them as C++ functions:
    
+
    template <typename T> T add(T x, T y) { return x + y; }
+template <typename T> T mul(T x, T y) { return x * y; }
+
    
+
    
+
    Then we can wrap them into operators which are C++ classes and can be created from Python bindings by name.  A C macro can do this. For example, the following macro invocation
    
+
    #define MAKE_FUNCTION_OPERATOR(mul);
+
    
+
    
+
    generates
    
+
    template <typename T> class mulOp : public OperatorBase {...};
+REGISTER_OP(mulOp<float32>, "mul");
+
    
+
    
+
    so that in Python we can create operator mul by:
    
+
    X1 = Var()
+X2 = Var()
+Y = Var()
+paddle.cpp.create_operator("mul", input=[X1, X2], output=Y)
+
    
+
    
+
    Also, at the same time, we can compose a coarse level C++ operator class by composing functions mul and add:
    
+
    template <typename T>
+class FCOp : public OperatorBase {
+ public:
+  void Run(...) {
+    add(mul(Input<T>("X"), Input<T>("W")), Input<T>("b");
+  }
+};
+REGISTER_OP(FCOp, "fc");
+
    
+
    
+
    We need to support such composition in Python as well.  To do so, we need a higher level Python wrapping of operator creation than paddle.cpp.create_operator.  This higher level operator API should be compatible with the layer API.
    
+
    Let’s explain using an example.  Suppose that we are going to compose the FC using mul and add in Python, we’d like to have Python functions mul and add defined in module operator:
    
+
    def operator.mul(X1, X2):
+    O = Var()
+    paddle.cpp.create_operator("mul", input={X1, Y1}, output=O)
+    return O
+
+def operator.add(X1, X2):
+    O = Var()
+    paddle.cpp.create_operator("add", input={X1, X2}, output=O)
+    return O
+
    
+
    
+
    Above code snippets are automatically generated.  Given them, users can define
    
+
    def layer.fc(X):
+    W = Var()
+    b = Var()
+    return operator.add(operator.mul(X, W), b)
+
    
+
    
+
    If we don’t have operator.mul and operator.add, the definiton of layer.fc would be complicated:
    
+
    def layer.fc(X):
+    W = Var()
+    b = Var()
+    O1 = Var()
+    paddle.cpp.create_operator("mul", input=[X, W], output=O1)
+    O2 = Var()
+    paddle.cpp.create_operator("add", input=[O1, b], output=O2)
+    return O2
+
    
+
    
+
    We’d like to have Python bindings to operators in package paddle.operator, and Python compositions of operators in package paddle.layer.  So we have the following concepts in above illustrative example:
    
+
    | C++ functions/functors | mul          | add          |             |          |
+|————————|————–|————–|————-|———-|
+| C++ operator class     | mulOp        | addOp        | FCOp        |          |
+| Python binding         | operator.mul | operator.add | operator.fc |          |
+| Python function        |              |              |             | layer.fc |
    
+
    This is how we differentiate layer and operators in PaddlePaddle:
    
+
      
+
    • those defined in C++ and have a lightweighted Python wrapper in module operators are operators; whereas
      • 
+
    • those who don’t have C++ implementations but a Python implementation that compose C++ operators are known as layers.
      • 
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/gan_api.html b/doc_cn/design/gan_api.html
new file mode 100644
index 0000000000000000000000000000000000000000..33437684e5eb1946774734ebbec5c1b5ffc2c3c1
--- /dev/null
+++ b/doc_cn/design/gan_api.html
@@ -0,0 +1,531 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design for GAN — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design for GAN
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design for GAN
    
+
    GAN (General Adversarial Net [https://arxiv.org/abs/1406.2661]) is an important model for unsupervised learning and widely used in many areas.
    
+
    It applies several important concepts in machine learning system design, including building and running subgraphs, dependency tracing, different optimizers in one executor and so forth.
    
+
    In our GAN design, we wrap it as a user-friendly easily customized python API to design different models. We take the conditional DC-GAN (Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks [https://arxiv.org/abs/1511.06434]) as an example due to its good performance on image generation.
    
+
    
+
    
+Figure 1. The overall running logic of GAN. The black solid arrows indicate the forward pass; the green dashed arrows indicate the backward pass of generator training; the red dashed arrows indicate the backward pass of the discriminator training. The BP pass of the green (red) arrow should only update the parameters in the green (red) boxes. The diamonds indicate the data providers. d\_loss and g\_loss marked in red and green are the two targets we would like to run.
+
    The operators, layers and functions required/optional to build a GAN demo is summarized in https://github.com/PaddlePaddle/Paddle/issues/4563.
    
+
    
+
    
+Figure 2. Photo borrowed from the original DC-GAN paper.
+
    
+
    The Conditional-GAN might be a class.
    
+
    This design we adopt the popular open source design in https://github.com/carpedm20/DCGAN-tensorflow and https://github.com/rajathkmp/DCGAN. It contains following data structure:
    
+
      
+
    • DCGAN(object): which contains everything required to build a GAN model. It provides following member functions methods as API:
      • 
+
    • init(...): Initialize hyper-parameters (like conv dimension and so forth), and declare model parameters of discriminator and generator as well.
      • 
+
    • generator(z, y=None): Generate a fake image from input noise z. If the label y is provided, the conditional GAN model will be chosen.
+Returns a generated image.
      • 
+
    • discriminator(image):
+Given an image, decide if it is from a real source or a fake one.
+Returns a 0/1 binary label.
      • 
+
    • build_model(self):
+build the whole GAN model, define training loss for both generator and discrimator.
      • 
+
    
+
    
+
    
+
    Discussion on Engine Functions required to build GAN
    
+
      
+
    • Trace the tensor and variable dependency in the engine executor. (Very critical, otherwise GAN can’be be trained correctly)
      • 
+
    • Different optimizers responsible for optimizing different loss.
      • 
+
    
+
    To be more detailed, we introduce our design of DCGAN as following:
    
+
    
+
    Class member Function: Initializer
    
+
      
+
    • Set up hyper-parameters, including condtional dimension, noise dimension, batch size and so forth.
      • 
+
    • Declare and define all the model variables. All the discriminator parameters are included in the list self.theta_D and all the generator parameters are included in the list self.theta_G.
      • 
+
    
+
    class DCGAN(object):
+  def __init__(self, y_dim=None):
+  
+    # hyper parameters  
+    self.y_dim = y_dim # conditional gan or not
+    self.batch_size = 100
+    self.z_dim = z_dim # input noise dimension
+
+    # define parameters of discriminators
+    self.D_W0 = pd.Variable(shape=[3,3, 1, 128], data=pd.gaussian_normal_randomizer())
+    self.D_b0 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.D_W1 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer())
+    self.D_b1 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.D_W2 = pd.Varialble(np.random.rand(128, 1))
+    self.D_b2 = pd.Variable(np.zeros(128))
+    self.theta_D = [self.D_W0, self.D_b0, self.D_W1, self.D_b1, self.D_W2, self.D_b2]
+
+    # define parameters of generators
+    self.G_W0 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer())
+    self.G_b0 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.G_W1 = pd.Variable(shape=[784, 128], data=pd.gaussian_normal_randomizer())
+    self.G_b1 = pd.Variable(np.zeros(128)) # variable also support initialization using a  numpy data
+    self.G_W2 = pd.Varialble(np.random.rand(128, 1))
+    self.G_b2 = pd.Variable(np.zeros(128))
+    self.theta_G = [self.G_W0, self.G_b0, self.G_W1, self.G_b1, self.G_W2, self.G_b2]
+
    
+
    
+
    
+
    
+
    Class member Function: Generator
    
+
      
+
    • Given a noisy input z, returns a fake image.
      • 
+
    • Concatenation, batch-norm, FC operations required;
      • 
+
    • Deconv layer required, which is missing now...
      • 
+
    
+
    class DCGAN(object):
+  def generator(self, z, y = None):
+    # input z: the random noise
+    # input y: input data label (optional)
+    # output G_im: generated fake images
+    
+    if not self.y_dim:
+      z = pd.layer.concat(1, [z, y])
+      
+    G_h0 = pd.layer.fc(z, self.G_w0, self.G_b0)
+    G_h0_bn = pd.layer.batch_norm(G_h0)
+    G_h0_relu = pd.layer.relu(G_h0_bn)
+    
+    G_h1 = pd.layer.deconv(G_h0_relu, self.G_w1, self.G_b1)
+    G_h1_bn = pd.layer.batch_norm(G_h1)
+    G_h1_relu = pd.layer.relu(G_h1_bn)
+    
+    G_h2 = pd.layer.deconv(G_h1_relu, self.G_W2, self.G_b2))
+    G_im = pd.layer.tanh(G_im)
+    return G_im
+
    
+
    
+
    
+
    
+
    Class member function: Discriminator
    
+
      
+
    • Given a noisy input z, returns a fake image.
      • 
+
    • Concatenation, Convolution, batch-norm, FC, Leaky-ReLU operations required;
      • 
+
    
+
    class DCGAN(object):
+  def discriminator(self, image):
+    # input image: either generated images or real ones
+    # output D_h2: binary logit of the label
+
+    D_h0 = pd.layer.conv2d(image, w=self.D_w0, b=self.D_b0)
+    D_h0_bn = pd.layer.batchnorm(h0)
+    D_h0_relu = pd.layer.lrelu(h0_bn)
+    
+    D_h1 = pd.layer.conv2d(D_h0_relu, w=self.D_w1, b=self.D_b1)
+    D_h1_bn = pd.layer.batchnorm(D_h1)
+    D_h1_relu = pd.layer.lrelu(D_h1_bn)
+    
+    D_h2 = pd.layer.fc(D_h1_relu, w=self.D_w2, b=self.D_b2)
+    return D_h2
+
    
+
    
+
    
+
    
+
    Class member function: Build the model
    
+
      
+
    • Define data readers as placeholders to hold the data;
      • 
+
    • Build generator and discriminators;
      • 
+
    • Define two training losses for discriminator and generator, respectively.
+If we have execution dependency engine to back-trace all tensors, the module building our GAN model will be like this:
      • 
+
    
+
    class DCGAN(object):
+  def build_model(self):
+    if self.y_dim:
+        self.y = pd.data(pd.float32, [self.batch_size, self.y_dim])
+    self.images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    self.faked_images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    self.z = pd.data(tf.float32, [None, self.z_size])
+    
+    # step 1: generate images by generator, classify real/fake images with discriminator
+    if self.y_dim: # if conditional GAN, includes label
+        self.G = self.generator(self.z, self.y)
+        self.D_t = self.discriminator(self.images)
+        # generated fake images
+        self.sampled = self.sampler(self.z, self.y)
+        self.D_f = self.discriminator(self.G)
+    else: # original version of GAN
+        self.G = self.generator(self.z)
+        self.D_t = self.discriminator(self.images)
+        # generate fake images
+        self.sampled = self.sampler(self.z)
+        self.D_f = self.discriminator(self.images)
+    
+    # step 2: define the two losses
+    self.d_loss_real = pd.reduce_mean(pd.cross_entropy(self.D_t, np.ones(self.batch_size))
+    self.d_loss_fake = pd.reduce_mean(pd.cross_entropy(self.D_f, np.zeros(self.batch_size))
+    self.d_loss = self.d_loss_real + self.d_loss_fake
+    
+    self.g_loss = pd.reduce_mean(pd.cross_entropy(self.D_f, np.ones(self.batch_szie))
+
    
+
    
+
    If we do not have dependency engine but blocks, the module building our GAN model will be like this:
    
+
    class DCGAN(object):
+  def build_model(self, default_block):
+    # input data in the default block
+    if self.y_dim:
+        self.y = pd.data(pd.float32, [self.batch_size, self.y_dim])
+    self.images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    # self.faked_images = pd.data(pd.float32, [self.batch_size, self.im_size, self.im_size])
+    self.z = pd.data(tf.float32, [None, self.z_size])
+
+    # step 1: generate images by generator, classify real/fake images with discriminator
+    with pd.default_block().g_block():
+      if self.y_dim: # if conditional GAN, includes label
+        self.G = self.generator(self.z, self.y)
+        self.D_g = self.discriminator(self.G, self.y)
+      else: # original version of GAN
+        self.G = self.generator(self.z)
+        self.D_g = self.discriminator(self.G, self.y)
+      self.g_loss = pd.reduce_mean(pd.cross_entropy(self.D_g, np.ones(self.batch_szie))
+    
+    with pd.default_block().d_block():
+      if self.y_dim: # if conditional GAN, includes label
+        self.D_t = self.discriminator(self.images, self.y)
+        self.D_f = self.discriminator(self.G, self.y)
+      else: # original version of GAN
+        self.D_t = self.discriminator(self.images)
+        self.D_f = self.discriminator(self.G)
+
+      # step 2: define the two losses
+      self.d_loss_real = pd.reduce_mean(pd.cross_entropy(self.D_t, np.ones(self.batch_size))
+      self.d_loss_fake = pd.reduce_mean(pd.cross_entropy(self.D_f, np.zeros(self.batch_size))
+      self.d_loss = self.d_loss_real + self.d_loss_fake
+
    
+
    
+
    Some small confusion and problems with this design:
    
+
      
+
    • D_g and D_f are actually the same thing, but has to be written twice; i.e., if we want to run two sub-graphs conceptually, the same codes have to be written twice if they are shared by the graph.
      • 
+
    • Requires ability to create a block anytime, rather than in if-else or rnn only;
      • 
+
    
+
    
+
    
+
    
+
    Main function for the demo:
    
+
    Generally, the user of GAN just need to the following things:
    
+
      
+
    • Define an object as DCGAN class;
      • 
+
    • Build the DCGAN model;
      • 
+
    • Specify two optimizers for two different losses with respect to different parameters.
      • 
+
    
+
    # pd for short, should be more concise.
+from paddle.v2 as pd
+import numpy as np
+import logging
+
+if __name__ == "__main__":
+    # dcgan class in the default graph/block
+    # if we use dependency engine as tensorflow
+    # the codes, will be slightly different like:
+    # dcgan = DCGAN()
+    # dcgan.build_model()
+    with pd.block() as def_block:
+      dcgan = DCGAN()
+      dcgan.build_model(def_block)
+
+    # load mnist data
+    data_X, data_y = self.load_mnist()
+    
+    # Two subgraphs required!!!
+    with pd.block().d_block():
+      d_optim = pd.train.Adam(lr = .001, beta= .1)
+      d_step = d_optim.minimize(dcgan.d_loss, dcgan.theta_D)
+    with pd.block.g_block():
+      g_optim = pd.train.Adam(lr = .001, beta= .1)
+      g_step = pd.minimize(dcgan.g_loss, dcgan.theta_G)
+
+    # executor
+    sess = pd.executor()
+    
+    # training
+    for epoch in xrange(10000):
+      for batch_id in range(N / batch_size):
+        idx = ...
+        # sample a batch
+        batch_im, batch_label = data_X[idx:idx+batch_size], data_y[idx:idx+batch_size]
+        # sample z
+        batch_z = np.random.uniform(-1., 1., [batch_size, z_dim])
+
+        if batch_id % 2 == 0:
+          sess.run(d_step, 
+                   feed_dict = {dcgan.images: batch_im,
+                                dcgan.y: batch_label,
+                                dcgan.z: batch_z})
+        else:
+          sess.run(g_step,
+                   feed_dict = {dcgan.z: batch_z})
+
    
+
    
+
    
+
    
+
    
+
    More thinking about dependency engine v.s. block design:
    
+
      
+
    • What if we just want to run an intermediate result? Do we need to run the whole block/graph?
      • 
+
    • Should we call eval() to get the fake images in the first stage? And then train the discriminator in the second stage?
      • 
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/graph.html b/doc_cn/design/graph.html
new file mode 100644
index 0000000000000000000000000000000000000000..a4964f02e33b7f9cc58e96bd2ebc9e495a9a3e36
--- /dev/null
+++ b/doc_cn/design/graph.html
@@ -0,0 +1,325 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Computations as a Graph — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Computations as a Graph
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Computations as a Graph
    
+
    A primary goal of the refactorization of PaddlePaddle is a more flexible representation of deep learning computation, in particular, a graph of operators and variables, instead of sequences of layers as before.
    
+
    This document explains that the construction of a graph as three steps:
    
+
      
+
    • construct the forward part
      • 
+
    • construct the backward part
      • 
+
    • construct the optimization part
      • 
+
    
+
    
+
    The Construction of a Graph
    
+
    Let us take the problem of image classification as a simple example.  The application program that trains the model looks like:
    
+
    x = layer.data("images")
+l = layer.data("label")
+y = layer.fc(x)
+cost = layer.mse(y, l)
+optimize(cost)
+train(cost, reader=mnist.train())
+
    
+
    
+
    
+
    Forward Part
    
+
    The first four lines of above program build the forward part of the graph.
    
+
    
+
    In particular, the first line x = layer.data("images") creates variable x and a Feed operator that copies a column from the minibatch to x.  y = layer.fc(x) creates not only the FC operator and output variable y, but also two parameters, W and b, and the initialization operators.
    
+
    Initialization operators are kind of “run-once” operators – the Run method increments a class data member counter so to run at most once.  By doing so, a parameter wouldn’t be initialized repeatedly, say, in every minibatch.
    
+
    In this example, all operators are created as OpDesc protobuf messages, and all variables are VarDesc.  These protobuf messages are saved in a BlockDesc protobuf message.
    
+
    
+
    
+
    Backward Part
    
+
    The fifth line optimize(cost) calls two functions, ConstructBackwardGraph and ConstructOptimizationGraph.
    
+
    ConstructBackwardGraph traverses the forward graph in the BlockDesc protobuf message and builds the backward part.
    
+
    
+
    According to the chain rule of gradient computation, ConstructBackwardGraph would
    
+
      
+
    1. create a gradient operator G for each operator F,
      2. 
+
    2. make all inputs, outputs, and outputs’ gradient of F as inputs of G,
      3. 
+
    3. create gradients for all inputs of F, except for those who don’t have gradients, like x and l, and
      4. 
+
    4. make all these gradients as outputs of G.
      5. 
+
    
+
    
+
    
+
    Optimization Part
    
+
    For each parameter, like W and b created by layer.fc, marked as double circles in above graphs, ConstructOptimizationGraph creates an optimization operator to apply its gradient.  Here results in the complete graph:
    
+
    
+
    
+
    
+
    
+
    Block and Graph
    
+
    The word block and graph are interchangable in the desgin of PaddlePaddle.  A Block is a metaphore of the code and local variables in a pair of curly braces in programming languages, where operators are like statements or instructions.  A graph of operators and variables is a representation of the block.
    
+
    A Block keeps operators in an array BlockDesc::ops
    
+
    message BlockDesc {
+  repeated OpDesc ops = 1;
+  repeated VarDesc vars = 2;
+}
+
    
+
    
+
    in the order that they appear in user programs, like the Python program at the beginning of this article.  We can imagine that in ops,  we have some forward operators, followed by some gradient operators, and then some optimization operators.
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/graph_survey.html b/doc_cn/design/graph_survey.html
new file mode 100644
index 0000000000000000000000000000000000000000..01fad79e371ccca7a7269a862abecd2254bac98e
--- /dev/null
+++ b/doc_cn/design/graph_survey.html
@@ -0,0 +1,458 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Survey on Graph — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Survey on Graph
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Survey on Graph
    
+
    Neural network framework often provides symbolic API for users to write network topology conveniently. This doc manily focus on symbolic API in most popular neural network frameworks, and try to find out how to parse symbolic configuration to a portable file, such as protobuf or json.
    
+
    
+
    Mxnet
    
+
    The core concept of symbolic API is Symbol. Mxnet implements Symbol class in C++, and export to Python using C-API. Please refer to the comments in Mxnet:
    
+
    Symbol is help class used to represent the operator node in Graph.
+Symbol acts as an interface for building graphs from different components like Variable, Functor and Group. Symbol is also exported to python front-end (while Graph is not) to enable quick test and deployment. Conceptually, symbol is the final operation of a graph and thus including all the information required (the graph) to evaluate its output value.
    
+
    A simple network topology wrote by Symbol is as follows:
    
+
    def get_symbol(num_classes=10, **kwargs):
+    data = mx.symbol.Variable('data')
+    data = mx.symbol.Flatten(data=data)
+    fc1  = mx.symbol.FullyConnected(data = data, name='fc1', num_hidden=128)
+    act1 = mx.symbol.Activation(data = fc1, name='relu1', act_type="relu")
+    fc2  = mx.symbol.FullyConnected(data = act1, name = 'fc2', num_hidden = 64)
+    act2 = mx.symbol.Activation(data = fc2, name='relu2', act_type="relu")
+    fc3  = mx.symbol.FullyConnected(data = act2, name='fc3', num_hidden=num_classes)
+    mlp  = mx.symbol.SoftmaxOutput(data = fc3, name = 'softmax')
+    return mlp
+
    
+
    
+
    Varible here is actually a Symbol. Every basic Symbol will correspond to one Node, and every Node has its own NodeAttr. There is a op field in NodeAttr class, when a Symbol represents Variable(often input data), the op field is null.
    
+
    Symbol contains a data member, std::vector outputs, and NodeEntry cantains a poniter to Node. We can follow the Node pointer to get all the Graph.
    
+
    And Symbol can be saved to a Json file.
    
+
    Here is a detailed example:
    
+
    >>> import mxnet as mx
+>>> data = mx.symbol.Variable('data')
+>>> print data.debug_str()
+Variable:data
+
+>>> data = mx.symbol.Flatten(data=data)
+>>> print data.debug_str()
+Symbol Outputs:
+    output[0]=flatten0(0)
+Variable:data
+--------------------
+Op:Flatten, Name=flatten0
+Inputs:
+    arg[0]=data(0) version=0
+
+>>> fc1  = mx.symbol.FullyConnected(data = data, name='fc1', num_hidden=128)
+>>> print fc1.debug_str()
+Symbol Outputs:
+    output[0]=fc1(0)
+Variable:data
+--------------------
+Op:Flatten, Name=flatten0
+Inputs:
+    arg[0]=data(0) version=0
+Variable:fc1_weight
+Variable:fc1_bias
+--------------------
+Op:FullyConnected, Name=fc1
+Inputs:
+    arg[0]=flatten0(0)
+    arg[1]=fc1_weight(0) version=0
+    arg[2]=fc1_bias(0) version=0
+Attrs:
+    num_hidden=128
+
    
+
    
+
    
+
    
+
    TensorFlow
    
+
    The core concept of symbolic API is Tensor. Tensorflow defines Tensor in Python. Please refer to the comments in TensorFlow:
    
+
    A Tensor is a symbolic handle to one of the outputs of an Operation. It does not hold the values of that operation’s output, but instead provides a means of computing those values in a TensorFlow Session.
    
+
    A simple example is as follows:
    
+
      # Build a dataflow graph.
+  c = tf.constant([[1.0, 2.0], [3.0, 4.0]])
+  d = tf.constant([[1.0, 1.0], [0.0, 1.0]])
+  e = tf.matmul(c, d)
+
+  # Construct a `Session` to execute the graph.
+  sess = tf.Session()
+
+  # Execute the graph and store the value that `e` represents in `result`.
+  result = sess.run(e)
+
    
+
    
+
    The main method of Tensor is as follows:
    
+
    @property
+def op(self):
+  """The `Operation` that produces this tensor as an output."""
+  return self._op
+
+@property
+def dtype(self):
+   """The `DType` of elements in this tensor."""
+  return self._dtype
+
+@property
+def graph(self):
+  """The `Graph` that contains this tensor."""
+  return self._op.graph
+
+@property
+def name(self):
+  """The string name of this tensor."""
+  if not self._op.name:
+    raise ValueError("Operation was not named: %s" % self._op)
+  return "%s:%d" % (self._op.name, self._value_index)
+
+@property
+def device(self):
+  """The name of the device on which this tensor will be produced, or None."""
+  return self._op.device
+
    
+
    
+
    Tensor can be taken as target to run by session. Tensor contains all the information of Graph, and tracks data dependency.
    
+
    Here is a detailed example:
    
+
    >>> import tensorflow as tf
+>>> c = tf.constant([[1.0, 2.0], [3.0, 4.0]])
+>>> print c.graph
+<tensorflow.python.framework.ops.Graph object at 0x10f256d50>
+>>> d = tf.constant([[1.0, 1.0], [0.0, 1.0]])
+>>> print d.graph
+<tensorflow.python.framework.ops.Graph object at 0x10f256d50>
+>>> e = tf.matmul(c, d)
+>>> print e.graph
+<tensorflow.python.framework.ops.Graph object at 0x10f256d50>
+
    
+
    
+
    
+
    
+
    Dynet
    
+
    The core concept of symbolic API is Expression, and Dynet defines Expression class in C++.
    
+
    A simple example is as follows:
    
+
    ComputationGraph cg;
+Expression W = parameter(cg, pW);
+
+Expression in = input(cg, xs[i]);
+Expression label = input(cg, ys[i]);
+Expression pred = W * in;
+Expression loss = square(pred - label);
+
    
+
    
+
    The input data and parameter are also represented by Expression. Every basci Expression corresponds to a Node. And input data is also a Node.
    
+
    Expression has a data member ComputationGraph, and ComputationGraph will be modified in users’ configuring process. Expression can be a running target, beacuse Expression contains all dependency.
    
+
    Here is a detailed example:
    
+
    write topology in C++
    
+
    ComputationGraph cg;
+Expression W = parameter(cg, pW);
+cg.print_graphviz();
+
+Expression pred = W * xs[i];
+cg.print_graphviz();
+
+Expression loss = square(pred - ys[i]);
+cg.print_graphviz();
+
    
+
    
+
    compile and print
    
+
    # first print
+digraph G {
+  rankdir=LR;
+  nodesep=.05;
+  N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"];
+}
+# second print
+digraph G {
+  rankdir=LR;
+  nodesep=.05;
+  N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"];
+  N1 [label="v1 = v0 * -0.98"];
+  N0 -> N1;
+}
+# third print
+digraph G {
+  rankdir=LR;
+  nodesep=.05;
+  N0 [label="v0 = parameters({1}) @ 0x7ffe4de00110"];
+  N1 [label="v1 = v0 * -0.98"];
+  N0 -> N1;
+  N2 [label="v2 = -1.88387 - v1"];
+  N1 -> N2;
+  N3 [label="v3 = -v2"];
+  N2 -> N3;
+  N4 [label="v4 = square(v3)"];
+  N3 -> N4;
+}
+
    
+
    
+
    
+
    
+
    Conclusion
    
+
    Actually, Symbol/Tensor/Expression in Mxnet/TensorFlow/Dynet are the same level concepts. We use a unified name Expression here, this level concept has following features:
    
+
      
+
    • Users wirte topoloy with symbolic API, and all return value is Expression, including input data and parameter.
      • 
+
    • Expression corresponds with a global Graph, and Expression can also be composed.
      • 
+
    • Expression tracks all dependency and can be taken as a run target
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/if_else_op.html b/doc_cn/design/if_else_op.html
new file mode 100644
index 0000000000000000000000000000000000000000..73ce4f51d2df939102ba696869305aa7660c5409
--- /dev/null
+++ b/doc_cn/design/if_else_op.html
@@ -0,0 +1,314 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  The IfElse Operator — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • The IfElse Operator
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    The IfElse Operator
    
+
    PaddlePaddle’s IfElse operator differs from TensorFlow’s:
    
+
      
+
    • the TensorFlow version takes a scalar boolean value as the condition so that the whole mini-batch goes to either the true or the false branch, whereas
      • 
+
    • the PaddlePaddle version takes a vector of boolean value as the condition, and instances corresponding to true values go to the true branch, those corresponding to false values go to the false branch.
      • 
+
    
+
    
+
    Example
    
+
    The following PaddlePaddle program shows the usage of the IfElse operator:
    
+
    import paddle as pd
+
+x = minibatch([10, 20, 30]) # shape=[None, 1]
+y = var(1) # shape=[1], value=1
+z = minibatch([10, 20, 30]) # shape=[None, 1]
+cond = larger_than(x, 15) # [false, true, true]
+
+ie = pd.ifelse()
+with ie.true_block():
+    d = pd.layer.add(x, y)
+    ie.output(d, pd.layer.softmax(d))
+with ie.false_block():
+    d = pd.layer.fc(z)
+    ie.output(d, d+1)
+o1, o2 = ie(cond)
+
    
+
    
+
    A challenge to implement the IfElse operator is to infer those variables to be split, or, say, to identify the variable of the mini-batch or those derived from the mini-batch.
    
+
    An equivalent C++ program is as follows:
    
+
    namespace pd = paddle;
+
+int x = 10;
+int y = 1;
+int z = 10;
+bool cond = false;
+int o1, o2;
+if (cond) {
+  int d = x + y;
+  o1 = z;
+  o2 = pd::layer::softmax(z);
+} else {
+  int d = pd::layer::fc(z);
+  o1 = d;
+  o2 = d+1;
+}
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/infer_var_type.html b/doc_cn/design/infer_var_type.html
new file mode 100644
index 0000000000000000000000000000000000000000..51749bc4f417bea96a472b091b92bc109b6cd995
--- /dev/null
+++ b/doc_cn/design/infer_var_type.html
@@ -0,0 +1,331 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: InferVarType — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: InferVarType
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: InferVarType
    
+
    
+
    The Problem Posed
    
+
    The variable in our design can hold variant types. Such as LoDTensor and SelectedRows. An operator should be able to inference the variable types of its output.
    
+
    For example, a lookup table operator takes two LoDTensor; one is a float tensor as the embedding table, the other is an int tensor as word ID. The gradient operator of lookup table will generate a SelectedRows as its output. A sum operator can take both LoDTensor and SelectedRows as its inputs and will generate a LoDTensor if any of its inputs is LoDTensor, otherwise, the sum operator will generate SelectedRows as its output.
    
+
    The variable type will be constant at runtime. Every variable’s type can either be set by the user (input data and parameter) or be inferred by the operator in compile time.
    
+
    
+
    
+
    Proposed Solution
    
+
    The InferVarType is a compile-time function which is registered to each operator. The inferface of that function is:
    
+
    using InferVarTypeFN = std::function<
+    void (const OpDescBind& /*op_desc*/, BlockDescBind* /*block*/)>;
+
    
+
    
+
    It takes an operator description as its input and will write the output variable type and store them in block description.
    
+
    The InferVarTypeFN will be registered in OpInfo, to replace infer_var_type_ field. The OpInfo should be
    
+
    struct OpInfo {
+  InferVarTypeFN infer_var_type_;
+  ...
+};
+
    
+
    
+
    The default InferVarType will set output type as LoDTensor. It can be done by GetInferVarType().
    
+
    void DefaultInferVarType(const OpDescBind& op_desc, BlockDescBind* block) {
+  // set the output type of variable as `LoDTensor`.
+  // ...
+}
+
+struct OpInfo {
+  InferVarTypeFN infer_var_type_;
+  InferVarTypeFN GetInferVarType() const {
+    if (infer_var_type_) {
+      return infer_var_type_;
+    } else {
+      return DefaultInferVarType;
+    }
+  }
+};
+
    
+
    
+
    
+
    
+
    Register InferVarType
    
+
    We provide a thin base class for registering an InferVarTypeFN. To use a base class will ease the implementation of registry since we can detect the registry entry is an InferVarTypeFN or not.
    
+
    class VarTypeInferer {
+public:
+  virtual void operator()(const OpDescBind& op_desc, BlockDescBind* block) const = 0;
+}
+
    
+
    
+
    Operator developers can write the specialize VarTypeInferer as follow.
    
+
    class SpecialVarTypeInferer : public VarTypeInferer {
+public:
+  virtual void operator()(const OpDescBind& op_desc, BlockDescBind* block) const {
+    // .. own logic
+  }
+}
+
    
+
    
+
    Then user can register the InferVarType just like GradOpDescMaker and OpInfoMaker.
    
+
    REGISTER_OPERATOR(some_op, OpType, SpecialVarTypeInferer, ...);
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/model_format.html b/doc_cn/design/model_format.html
new file mode 100644
index 0000000000000000000000000000000000000000..8632775ac83ba4c4e3425a893cd5d94220a57204
--- /dev/null
+++ b/doc_cn/design/model_format.html
@@ -0,0 +1,298 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Model Format — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Model Format
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Model Format
    
+
    
+
    Motivation
    
+
    A model is an output of the training process. One complete model consists of two parts, the topology and the parameters. In order to support industrial deployment, the model format must be self-complete and must not expose any training source code.
    
+
    As a result, In PaddlePaddle, the topology is represented as a  ProgramDesc, which describes the model structure. The parameters contain all the trainable weights in the model. We must support large size parameters and efficient serialization/deserialization of parameters.
    
+
    
+
    
+
    Implementation
    
+
    The topology is saved as a plain text in a detailed self-contain protobuf file.
    
+
    The parameters are saved as a binary file. As we all know, the protobuf message has a limit of 64M size. We have done a benchmark experiment, which shows that protobuf is not fit for the task.
    
+
    As a result, we design a particular format for tensor serialization. By default, an arbitrary tensor in Paddle is a LoDTensor, and has a description information proto of LoDTensorDesc. We save the DescProto as the byte string header. It contains all the necessary information, such as the dims, and the LoD information in LoDTensor. A tensor stores values in a continuous memory buffer. For speed we dump the raw memory to disk and save it as the byte string content. So, the binary format of one tensor is,
    
+
    The table below shows a tensor’s byte view in detail. Note that all the signed values are written in the little-endian format.
    
+
    |field name  | type | description |
+| — | — | — |
+| version | uint32_t | Version of saved file. Always 0 now. |
+| tensor desc length | uint32_t | TensorDesc(Protobuf message) length in bytes. |
+| tensor desc | void* | TensorDesc protobuf binary message |
+| tensor data | void* | Tensor’s data in binary format. The length of tensor_data is decided by TensorDesc.dims() and TensorDesc.data_type() |
+| lod_level | uint64_t | Level of LoD |
+| length of lod[0] | uint64_t | [Optional] length of lod[0] in bytes. |
+| data of lod[0] | uint64_t*  | [Optional] lod[0].data() |
+| ... | ... | ... |
    
+
    
+
    
+
    Summary
    
+
      
+
    • We introduce a model format.
      • 
+
    • The model represented by its forward-pass computation procedure is saved in a ProgramDesc protobuf message.
      • 
+
    • A bunch of specified format binary tensors describe the parameters.
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/multi_language_interface/why_plain_c.html b/doc_cn/design/multi_language_interface/00.why_plain_c.html
similarity index 84%
rename from doc_cn/design/multi_language_interface/why_plain_c.html
rename to doc_cn/design/multi_language_interface/00.why_plain_c.html
index 0f4844a2c40b705e4460ba497efdbd19897125a9..4cecc40fc1a5b872836a58105a93ec7d1efa37c5 100644
--- a/doc_cn/design/multi_language_interface/why_plain_c.html
+++ b/doc_cn/design/multi_language_interface/00.why_plain_c.html
@@ -62,7 +62,7 @@
     
    
     
    
       
    
-        Folk me on Github
+        Fork me on Github
         
    
           
             English
@@ -85,6 +85,7 @@
 
  • 进阶指南
    • 
 
  • API
    • 
 
  • FAQ
    • 
+
  • MOBILE
    • 
 
 
         
@@ -108,9 +109,10 @@
           
 
         
@@ -258,29 +280,29 @@
 typedef int paddle_error;
 
 extern "C"
-paddle_error paddle_matrix_shape(paddle_matrix matrix,
-                                 uint64_t* width,
-                                 uint64_t* height);
+paddle_error paddle_matrix_get_shape(paddle_matrix matrix,
+                                     uint64_t* width,
+                                     uint64_t* height);

    而在CPP里面实现这个C的接口，文件 paddle_matrix.cpp

    -
    #include "paddle/math/matrix.hpp"
+
    #include "paddle/math/matrix.h"
 extern "C"
 paddle_error paddle_matrix_shape(paddle_matrix matrix,
                                  uint64_t *width,
                                  uint64_t *height) {
-  auto m = (paddle::math::matrix*)(matrix);
+  auto m = (paddle::capi::CMatrix*)(matrix);
   *width = m->width();
   *height = m->height();
 }
 
    
 
    
-
    其中paddle/math/matrix.hpp文件内容为:
    
+
    其中paddle/capi/CMatrix.hpp文件内容为:
    
 
    namespace paddle {
 namespace math {  
 
-class Matrix {
-  //...
+class CMatrix {
+  std::shared_ptr<paddle::Matrix> mat;
 };
 
 }  // namespace math
@@ -317,8 +339,8 @@
 | 手写多语言绑定 | 不使用SWIG | 使用SWIG需要多语言绑定的开发人员熟练掌握SWIG配置，社区参与困难。SWIG生成的代码不能保证多语言代码风格的一致性 |
    
 
    
 
    
-
    简单实现
    
-
    TBD
    
+
    实现
    
+
    参考Inference implementation
    
 
    
 
    
 
diff --git a/doc_cn/design/multi_language_interface/01.inference_implementation.html b/doc_cn/design/multi_language_interface/01.inference_implementation.html
new file mode 100644
index 0000000000000000000000000000000000000000..5d56ca6137111bce62451d81d637f6a364e6c2f5
--- /dev/null
+++ b/doc_cn/design/multi_language_interface/01.inference_implementation.html
@@ -0,0 +1,403 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  C-API 模型推断实现文档 — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • C-API 模型推断实现文档
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    C-API 模型推断实现文档
    
+
    本文档描述Paddle C-API的实现细节。Paddle C-API是多语言API的基础部分。Paddle需要暴露的API很多。先实现模型推断的API，通过模型推断API的实现作为一个样例，来进行讨论。至于为什么需要C-API，请参考Why Plain C。
    
+
+
    
+
    暴露接口原则
    
+
      
+
    1. 所有的接口均为C接口。即使用extern "C"
      2. 
+
    2. 除构造某种类型的函数(paddle_matrix_create等)，其他函数均返回paddle_error。且调用时不能抛出异常或出现运行时错误。
      3. 
+
    3. 所有类型名为paddle_类型名，所有与类型相关的函数，函数名为paddle_类型名_函数名
      4. 
+
    4. 如果某一个Paddle Core概念(GradientMachine/Matrix)需要被暴露到其他语言，那么
        
+
      • 为了暴露的接口尽量简单。只暴露概念的接口，而不暴露概念的实现。即暴露GradientMachine或者Matrix但不暴露RecurrentGradientMachine和CpuSparseMatrix。
        • 
+
      • 暴露这个概念必要函数。必要是指，即完成某一个任务的最少函数。
        • 
+
      
+
      5. 
+
    5. 不在capi接口层做过多封装。
        
+
      • 如果某一个Paddle概念必须要暴露，但是又过于琐碎。不在capi这一层进行封装，而是直接修改Paddle Core。让Paddle核心中，这一概念不再琐碎。
        • 
+
      
+
      6. 
+
    
+
    
+
    
+
    目录结构
    
+
    Paddle
+  `-- paddle
+        `-- capi
+              `-- examples  # The example project for C-API.
+              `-- tests  # unittests for C-API
+              `-- capi.h  # C-API header file.
+              `-- capi_private.h  # The shared header file between implementation sources.
+              `-- matrix.{h, cpp}
+              `-- gradient_machine.{h, cpp}
+              `-- ...
+
    
+
    
+
    Paddle的C-API目录结构如上图表所示。这个目录中除了capi_private.h之外的所有头文件，均会被安装到include/paddle路径下。C-API生成的二进制文件会被安装到lib目录下。即，安装后的目录结构为
    
+
    `-- include
+      `-- paddle
+             `-- capi.h
+             `-- matrix.h
+             `-- gradient_machine.h
+             `-- ...
+`-- lib
+     `-- libpaddle_capi_shared.{so, dylib}  # In mac, dynamic libary's file name extention is `dylib`
+     `-- libpaddle_capi_whole.a  # static library for all symbols of Paddle.
+
    
+
    
+
    
+
    
+
    实现方式
    
+
    下面分别介绍某一类文件的实现方式。
    
+
    
+
    capi.h
    
+
    capi.h是用户使用C-API时所唯一需要引入的头文件。在capi.h中，引入了类型的头文件，matrix.h, gradient_machine.h。在引入其他类型的头文件时，使用相对路径的引用方式。即#include "matrix.h"
    
+
    
+
    
+
    具体某种类型的头文件
    
+
    具体某种类型的头文件，即例如matrix.h，gradient_machine.h等。在这些头文件中，包含了某种类型的类型定义和暴露的全部函数。
    
+
    这个头文件不假设其他文件的引用顺序，即使用户直接引用某种类型的头文件，也不应该报错(虽然不鼓励这样)。如果某一个类型需要引用另一个类型，例如gradient_machine需要引用matrix，则直接引入另一种类型的头文件，即#include "matrix.h"。
    
+
    
+
    
+
    capi_private.h
    
+
    capi_prviate.h是各个实现中共享的头文件，他主要包含了实际暴露的类型结构。在用户使用C-API时，Paddle的类型全部退化成void *，即typedef paddle_matrix void*。但，对于每种C-API暴露的类型，均是在capi_private.h中实现的结构体。
    
+
    struct CMatrix {
+   int type = MatrixType;
+   std::shared_ptr<paddle::Matrix> mat;
+};
+
    
+
    
+
    通常，这个结构体包含两个项目。
    
+
      
+
    • type是一个类型的标志。对于每种类型，type字段均不尽相同。这样，即使C-API接受的类型全是void *，我们也可以确定每一个参数的类型。
      
+
      void some_c_api_function(void* some_instance) {
+   int* type = (int *) some_instance;
+   switch (*type) {
+     case MatrixType:
+       CMatrix* mat = (CMatrix *) some_instance;
+       ...
+     ...
+   }
+}
+
      
+
      
+
      • 
+
    • 这个结构体中的另一个项目是，Paddle Core中这一类型接口的智能指针(shared_ptr)。
      
+
        
+
      • 使用智能指针的原因是: 用户可以安全的释放某个C-API的实例，而不必在意Paddle Core是否还在使用这个实例。
        • 
+
      • 例如，用户通过C-API获得了神经网络的参数实例。当用户使用完这个参数后，直接删除这个参数即可。即便Paddle Core中的模型还在使用这个参数，这个参数也不会一并删除。
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    具体某种类型的实现文件
    
+
    具体某种类型的实现文件，即matrix.cpp, gradient_machine.cpp等文件。在这些文件中，使用C++ 11实现了C-API的接口，并且使用extern "C"导出这些接口。在实现过程中，对输入参数的安全性进行了必要的判断，并将C-API接口的参数转发给Paddle Core。
    
+
    
+
    
+
    libpaddle_capi_shared.{so, dylib}
    
+
    libpaddle_capi_shared是C-API导出的动态库。这个动态库的连接参数与Paddle的其他二进制(例如paddle_trainer)类似。用户可以直接使用这个动态库来引入Paddle C-API。具体使用方法为-lpaddle_capi_shared。
    
+
    
+
    
+
    libpaddle_capi_whole.a
    
+
    libpaddle_capi_whole是C-API导出的静态库。这个静态库包含了Paddle的全部符号。他是将libpaddle_gserver.a, libpaddle_math.a, libpaddle_capi.a等全部静态库中的目标文件全部打包后产生的文件。具体使用方法为--whole-archive -lpaddle_capi_whole --no-whole-archive。
    
+
    
+
    
+
    examples
    
+
    在样例中，使用C99开发了模型预测的样例代码。具体请参考example/README.md。
    
+
    
+
    
+
    
+
    编译选项
    
+
    C-API的编译选项默认关闭，打开这个编译选项，需要在cmake的时候，设置
    
+
    cmake ${YOUR_SOURCE_ROOT} -DWITH_C_API=ON -DWITH_PYTHON=OFF -DWITH_SWIG_PY=OFF
+
    
+
    
+
    编译C-API的时候推荐Paddle不嵌入Python解释器，也不生成SWIG接口，具体原因参考Why Plain C。
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/ops/rnn.html b/doc_cn/design/ops/rnn.html
new file mode 100644
index 0000000000000000000000000000000000000000..6a8005544a6f473e2060d9ede67c877022128121
--- /dev/null
+++ b/doc_cn/design/ops/rnn.html
@@ -0,0 +1,399 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  RNNOp design — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • RNNOp design
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    RNNOp design
    
+
    This document describes the RNN (Recurrent Neural Network) operator and how it is implemented in PaddlePaddle. The RNN op requires that all instances in a mini-batch have the same length. We will have a more flexible dynamic RNN operator in the future.
    
+
    
+
    RNN Algorithm Implementation
    
+
    
+
+
    The above diagram shows an RNN unrolled into a full network.
    
+
    There are several important concepts here:
    
+
      
+
    • step-net: the sub-graph that runs at each step.
      • 
+
    • memory, $h_t$, the state of the current step.
      • 
+
    • ex-memory, $h_{t-1}$, the state of the previous step.
      • 
+
    • initial memory value, the memory of the first (initial) step.
      • 
+
    
+
    
+
    Step-scope
    
+
    There could be local variables defined in each step-net.  PaddlePaddle runtime realizes these variables in step-scopes which are created for each step.
    
+
    
+
    
+Figure 2 illustrates the RNN's data flow
+
    Please be aware that every step runs the same step-net.  Each step does the following:
    
+
      
+
    1. Creates the step-scope.
      2. 
+
    2. Initializes the local variables including step-outputs, in the step-scope.
      3. 
+
    3. Runs the step-net, which uses the above mentioned variables.
      4. 
+
    
+
    The RNN operator will compose its output from step outputs in each of the step scopes.
    
+
    
+
    
+
    Memory and Ex-memory
    
+
    Let’s give more details about memory and ex-memory using a simple example:
    
+
    $$
+h_t = U h_{t-1} + W x_t
+$$,
    
+
    where $h_t$ and $h_{t-1}$ are the memory and ex-memory (previous memory) of step $t$ respectively.
    
+
    In the implementation, we can make an ex-memory variable either “refer to” the memory variable of the previous step,
+or copy the memory value of the previous step to the current ex-memory variable.
    
+
    
+
    
+
    Usage in Python
    
+
    For more information on Block, please refer to the design doc.
    
+
    We can define an RNN’s step-net using a Block:
    
+
    import paddle as pd
+
+X = some_op() # x is some operator's output and is a LoDTensor
+a = some_op()
+
+# declare parameters
+W = pd.Variable(shape=[20, 30])
+U = pd.Variable(shape=[20, 30])
+
+rnn = pd.create_rnn_op(output_num=1)
+with rnn.stepnet():
+    x = rnn.add_input(X)
+    # declare a memory (rnn's step)
+    h = rnn.add_memory(init=a)
+    # h.pre_state(), the previous memory of rnn
+    new_state = pd.add_two( pd.matmul(W, x) + pd.matmul(U, h.pre_state()))
+    # update current memory
+    h.update(new_state)
+    # indicate that h variables in all step scopes should be merged
+    rnn.add_outputs(h)
+
+out = rnn()
+
    
+
    
+
    Python API functions in above example:
    
+
      
+
    • rnn.add_input: indicates that the parameter is a variable that will be segmented into step-inputs.
      • 
+
    • rnn.add_memory: creates a variable used as the memory.
      • 
+
    • rnn.add_outputs: marks the variables that will be concatenated across steps into the RNN output.
      • 
+
    
+
    
+
    
+
    Nested RNN and LoDTensor
    
+
    An RNN whose step-net includes other RNN operators is known as an nested RNN.
    
+
    For example, we could have a 2-level RNN, where the top level corresponds to paragraphs, and the lower level corresponds to sentences. Each step of the higher level RNN also receives an input from the corresponding step of the lower level, and additionally the output from the previous time step at the same level.
    
+
    The following figure illustrates feeding in text into the lower level, one sentence at a step, and the feeding in step outputs to the top level. The final top level output is about the whole text.
    
+
    
+
+
    import paddle as pd
+
+W = pd.Variable(shape=[20, 30])
+U = pd.Variable(shape=[20, 30])
+
+W0 = pd.Variable(shape=[20, 30])
+U0 = pd.Variable(shape=[20, 30])
+
+# a is output of some op
+a = some_op()
+
+# chapter_data is a set of 128-dim word vectors
+# the first level of LoD is sentence
+# the second level of LoD is a chapter
+chapter_data = pd.Variable(shape=[None, 128], type=pd.lod_tensor, level=2)
+
+def lower_level_rnn(paragraph):
+    '''
+    x: the input
+    '''
+    rnn = pd.create_rnn_op(output_num=1)
+    with rnn.stepnet():
+        sentence = rnn.add_input(paragraph, level=0)
+        h = rnn.add_memory(shape=[20, 30])
+        h.update(
+            pd.matmul(W, sentence) + pd.matmul(U, h.pre_state()))
+        # get the last state as sentence's info
+        rnn.add_outputs(h)
+    return rnn
+
+top_level_rnn = pd.create_rnn_op(output_num=1)
+with top_level_rnn.stepnet():
+    paragraph_data = rnn.add_input(chapter_data, level=1)
+    low_rnn = lower_level_rnn(paragraph_data)
+    paragraph_out = low_rnn()
+
+    h = rnn.add_memory(init=a)
+    h.update(
+        pd.matmul(W0, paragraph_data) + pd.matmul(U0, h.pre_state()))
+    top_level_rnn.add_outputs(h)
+
+# output the last step
+chapter_out = top_level_rnn(output_all_steps=False)
+
    
+
    
+
    In the above example, the construction of the top_level_rnn calls  lower_level_rnn.  The input is an LoD Tensor. The top level RNN segments input text data into paragraphs, and the lower level RNN segments each paragraph into sentences.
    
+
    By default, the RNNOp will concatenate the outputs from all the time steps.
+If the output_all_steps is set to False, it will only output the final time step.
    
+
    
+
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/ops/sequence_decoder.html b/doc_cn/design/ops/sequence_decoder.html
new file mode 100644
index 0000000000000000000000000000000000000000..0c878ac01da0b2631ecad42c6959f5705dd826e8
--- /dev/null
+++ b/doc_cn/design/ops/sequence_decoder.html
@@ -0,0 +1,474 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design: Sequence Decoder Generating LoDTensors — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design: Sequence Decoder Generating LoDTensors
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design: Sequence Decoder Generating LoDTensors
    
+
    In tasks such as machine translation and visual captioning,
+a sequence decoder is necessary to generate sequences, one word at a time.
    
+
    This documentation describes how to implement the sequence decoder as an operator.
    
+
    
+
    Beam Search based Decoder
    
+
    The beam search algorithm is necessary when generating sequences. It is a heuristic search algorithm that explores the paths by expanding the most promising node in a limited set.
    
+
    In the old version of PaddlePaddle, the C++ class RecurrentGradientMachine implements the general sequence decoder based on beam search, due to the complexity involved, the implementation relies on a lot of special data structures that are quite trivial and hard to be customized by users.
    
+
    There are a lot of heuristic tricks in the sequence generation tasks, so the flexibility of sequence decoder is very important to users.
    
+
    During the refactoring of PaddlePaddle, some new concepts are proposed such as:  LoDTensor and TensorArray that can better support the sequence usage, and they can also help make the implementation of beam search based sequence decoder more transparent and modular .
    
+
    For example, the RNN states, candidates IDs and probabilities of beam search can be represented all as LoDTensors;
+the selected candidate’s IDs in each time step can be stored in a TensorArray, and Packed to the sentences translated.
    
+
    
+
    
+
    Changing LoD’s absolute offset to relative offsets
    
+
    The current LoDTensor is designed to store levels of variable-length sequences. It stores several arrays of integers where each represents a level.
    
+
    The integers in each level represent the begin and end (not inclusive) offset of a sequence in the underlying tensor,
+let’s call this format the absolute-offset LoD for clarity.
    
+
    The relative-offset LoD can retrieve any sequence very quickly but fails to represent empty sequences, for example, a two-level LoD is as follows
    
+
    [[0, 3, 9]
+ [0, 2, 3, 3, 3, 9]]
+
    
+
    
+
    The first level tells that there are two sequences:
    
+
      
+
    • the first’s offset is [0, 3)
      • 
+
    • the second’s offset is [3, 9)
      • 
+
    
+
    while on the second level, there are several empty sequences that both begin and end at 3.
+It is impossible to tell how many empty second-level sequences exist in the first-level sequences.
    
+
    There are many scenarios that rely on empty sequence representation, for example in machine translation or visual captioning, one instance has no translation or the empty candidate set for a prefix.
    
+
    So let’s introduce another format of LoD,
+it stores the offsets of the lower level sequences and is called relative-offset LoD.
    
+
    For example, to represent the same sequences of the above data
    
+
    [[0, 3, 6]
+ [0, 2, 3, 3, 3, 9]]
+
    
+
    
+
    the first level represents that there are two sequences,
+their offsets in the second-level LoD is [0, 3) and [3, 5).
    
+
    The second level is the same with the relative offset example because the lower level is a tensor.
+It is easy to find out the second sequence in the first-level LoD has two empty sequences.
    
+
    The following examples are based on relative-offset LoD.
    
+
    
+
    
+
    Usage in a simple machine translation model
    
+
    Let’s start from a simple machine translation model that is simplified from the machine translation chapter to draw a blueprint of what a sequence decoder can do and how to use it.
    
+
    The model has an encoder that learns the semantic vector from a sequence, and a decoder which uses the sequence encoder to generate new sentences.
    
+
    Encoder
    
+
    import paddle as pd
+
+dict_size = 8000
+source_dict_size = dict_size
+target_dict_size = dict_size
+word_vector_dim = 128
+encoder_dim = 128
+decoder_dim = 128
+beam_size = 5
+max_length = 120
+
+# encoder
+src_word_id = pd.data(
+    name='source_language_word',
+    type=pd.data.integer_value_sequence(source_dict_dim))
+src_embedding = pd.embedding(size=source_dict_size, size=word_vector_dim)
+
+src_word_vec = pd.lookup(src_embedding, src_word_id)
+
+encoder_out_seq = pd.gru(input=src_word_vec, size=encoder_dim)
+
+encoder_ctx = pd.last_seq(encoder_out_seq)
+# encoder_ctx_proj is the learned semantic vector
+encoder_ctx_proj = pd.fc(
+    encoder_ctx, size=decoder_dim, act=pd.activation.Tanh(), bias=None)
+
    
+
    
+
    Decoder
    
+
    def generate():
+    decoder = pd.while_loop()
+    with decoder.step():
+        decoder_mem = decoder.memory(init=encoder_ctx)  # mark the memory
+        generated_ids = decoder.memory() # TODO init to batch_size <s>s
+        generated_scores = decoder.memory() # TODO init to batch_size 1s or 0s
+
+        target_word = pd.lookup(trg_embedding, gendrated_ids)
+        # expand encoder_ctx's batch to fit target_word's lod
+        # for example
+        # decoder_mem.lod is
+        # [[0 1 3],
+        #  [0 1 3 6]]
+        # its tensor content is [a1 a2 a3 a4 a5]
+        # which means there are 2 sentences to translate
+        #   - the first sentence has 1 translation prefixes, the offsets are [0, 1)
+        #   - the second sentence has 2 translation prefixes, the offsets are [1, 3) and [3, 6)
+        # the target_word.lod is
+        # [[0, 1, 6]
+        #  [0, 2, 4, 7, 9 12]]
+        # which means 2 sentences to translate, each has 1 and 5 prefixes
+        # the first prefix has 2 candidates
+        # the following has 2, 3, 2, 3 candidates
+        # the encoder_ctx_expanded's content will be
+        # [a1 a1 a2 a2 a3 a3 a3 a4 a4 a5 a5 a5]
+        encoder_ctx_expanded = pd.lod_expand(encoder_ctx, target_word)
+        decoder_input = pd.fc(
+            act=pd.activation.Linear(),
+            input=[target_word, encoder_ctx],
+            size=3 * decoder_dim)
+        gru_out, cur_mem = pd.gru_step(
+            decoder_input, mem=decoder_mem, size=decoder_dim)
+        scores = pd.fc(
+            gru_out,
+            size=trg_dic_size,
+            bias=None,
+            act=pd.activation.Softmax())
+        # K is an config
+        topk_scores, topk_ids = pd.top_k(scores, K)
+        topk_generated_scores = pd.add_scalar(topk_scores, generated_scores)
+
+        selected_ids, selected_generation_scores = decoder.beam_search(
+            topk_ids, topk_generated_scores)
+
+        # update the states
+        decoder_mem.update(cur_mem)  # tells how to update state
+        generated_ids.update(selected_ids)
+        generated_scores.update(selected_generation_scores)
+
+        decoder.output(selected_ids)
+        decoder.output(selected_generation_scores)
+
+translation_ids, translation_scores = decoder()
+
    
+
    
+
    The decoder.beam_search is an operator that, given the candidates and the scores of translations including the candidates,
+returns the result of the beam search algorithm.
    
+
    In this way, users can customize anything on the input or output of beam search, for example:
    
+
      
+
    1. Make the corresponding elements in topk_generated_scores zero or some small values, beam_search will discard this candidate.
      2. 
+
    2. Remove some specific candidate in selected_ids.
      3. 
+
    3. Get the final translation_ids, remove the translation sequence in it.
      4. 
+
    
+
    The implementation of sequence decoder can reuse the C++ class:  RNNAlgorithm,
+so the python syntax is quite similar to that of an  RNN.
    
+
    Both of them are two-level LoDTensors:
    
+
      
+
    • The first level represents batch_size of (source) sentences.
      • 
+
    • The second level represents the candidate ID sets for translation prefix.
      • 
+
    
+
    For example, 3 source sentences to translate, and has 2, 3, 1 candidates.
    
+
    Unlike an RNN, in sequence decoder, the previous state and the current state have different LoD and shape, and an lod_expand operator is used to expand the LoD of the previous state to fit the current state.
    
+
    For example, the previous state:
    
+
      
+
    • LoD is [0, 1, 3][0, 2, 5, 6]
      • 
+
    • content of tensor is a1 a2 b1 b2 b3 c1
      • 
+
    
+
    the current state is stored in encoder_ctx_expanded:
    
+
      
+
    • LoD is [0, 2, 7][0 3 5 8 9 11 11]
      • 
+
    • the content is
        
+
      • a1 a1 a1 (a1 has 3 candidates, so the state should be copied 3 times for each candidates)
        • 
+
      • a2 a2
        • 
+
      • b1 b1 b1
        • 
+
      • b2
        • 
+
      • b3 b3
        • 
+
      • None (c1 has 0 candidates, so c1 is dropped)
        • 
+
      
+
      • 
+
    
+
    The benefit from the relative offset LoD is that the empty candidate set can be represented naturally.
    
+
    The status in each time step can be stored in TensorArray, and Packed to a final LoDTensor. The corresponding syntax is:
    
+
    decoder.output(selected_ids)
+decoder.output(selected_generation_scores)
+
    
+
    
+
    The selected_ids are the candidate ids for the prefixes, and will be Packed by TensorArray to a two-level LoDTensor, where the first level represents the source sequences and the second level represents generated sequences.
    
+
    Packing the selected_scores will get a LoDTensor that stores scores of each translation candidate.
    
+
    Packing the selected_generation_scores will get a LoDTensor, and each tail is the probability of the translation.
    
+
    
+
    
+
    LoD and shape changes during decoding
    
+
    
+  
+
    According to the image above, the only phase that changes the LoD is beam search.
    
+
    
+
    
+
    Beam search design
    
+
    The beam search algorithm will be implemented as one method of the sequence decoder and has 3 inputs:
    
+
      
+
    1. topk_ids, the top K candidate ids for each prefix.
      2. 
+
    2. topk_scores, the corresponding scores for topk_ids
      3. 
+
    3. generated_scores, the score of the prefixes.
      4. 
+
    
+
    All of these are LoDTensors, so that the sequence affiliation is clear. Beam search will keep a beam for each prefix and select a smaller candidate set for each prefix.
    
+
    It will return three variables:
    
+
      
+
    1. selected_ids, the final candidate beam search function selected for the next step.
      2. 
+
    2. selected_scores, the scores for the candidates.
      3. 
+
    3. generated_scores, the updated scores for each prefix (with the new candidates appended).
      4. 
+
    
+
    
+
    
+
    Introducing the LoD-based Pack and Unpack methods in TensorArray
    
+
    The selected_ids, selected_scores and generated_scores are LoDTensors that exist at each time step,
+so it is natural to store them in arrays.
    
+
    Currently, PaddlePaddle has a module called TensorArray which can store an array of tensors. It is better to store the results of beam search in a TensorArray.
    
+
    The Pack and UnPack in TensorArray are used to pack tensors in the array to an LoDTensor or split the LoDTensor to an array of tensors.
+It needs some extensions to support the packing or unpacking an array of LoDTensors.
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/optimizer.html b/doc_cn/design/optimizer.html
new file mode 100644
index 0000000000000000000000000000000000000000..2246e6952ddebd55079c2ec1e96370f9de0a454f
--- /dev/null
+++ b/doc_cn/design/optimizer.html
@@ -0,0 +1,354 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Optimizer Design — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Optimizer Design
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Optimizer Design
    
+
    
+
    The Problem
    
+
    A PaddlePaddle program, or a block, is a sequence of operators operating variables.  A training program needs to do three kinds of works:
    
+
      
+
    1. the forward pass, which computes intermediate results and the cost(s),
      2. 
+
    2. the backward pass, which derives gradients from intermediate results and costs, and
      3. 
+
    3. the optimization pass, which update model parameters to optimize the cost(s).
      4. 
+
    
+
    These works rely on three kinds of operators:
    
+
      
+
    1. forward operators,
      2. 
+
    2. gradient operators, and
      3. 
+
    3. optimization operators.
      4. 
+
    
+
    It’s true that users should be able to create all these operators manually by calling some low-level API, but it would be much more convenient if they could only describe the forward pass and let PaddlePaddle create the backward and optimization operators automatically.
    
+
    In this design, we propose a high-level API that automatically derives the optimisation pass and operators from the forward pass.
    
+
    
+
    
+
    High-level Python API to describe the training process
    
+
      
+
    1. User write code to describe the network:
      
+
      images = layer.data("images")
+labels = layer.data("labels")
+w1 = pd.var("w1")
+b1 = pd.var("b1")
+hidden = layer.fc(images, w=w1, b=b1)
+cost = layer.mse(hidden, labels)
+
      
+
      
+
      The above code snippet will create forward operators in Block.
      
+
      2. 
+
    
+
      
+
    1. Users create a certain kind of Optimizer with some argument.
      
+
      optimizer = AdagradOptimizer(learing_rate=0.001)
+
      
+
      
+
      2. 
+
    2. Users use the optimizer to minimize a certain cost through updating parameters in parameter_list.
      
+
      opt_op_list = optimizer.minimize(cost, parameter_list=[w1, b1])
+
      
+
      
+
      The above code snippet will create gradient and optimization operators in Block. The return value of minimize() is list of optimization operators that will be run by session.
      
+
      3. 
+
    3. Users use Session/Executor to run this opt_op_list as target to do training.
      
+
      sess.run(target= opt_op_list, ...)
+
      
+
      
+
      4. 
+
    
+
    
+
    Optimizer Python interface:
    
+
    class Optimizer(object):
+    """Optimizer Base class.
+
+    """
+
+    def __init__(self):
+        pass
+
+    def create_optimization_pass(self, parameters_and_grads):
+        """Add optimization operators to update gradients to variables.
+
+        Args:
+          parameters_and_grads: a list of (variable, gradient) pair to update.
+
+        Returns:
+          optmization_op_list: a list of optimization operator that will update parameter using gradient.
+        """
+        return None
+
+    def minimize(self, loss, parameter_list):
+        """Add operations to minimize `loss` by updating `parameter_list`.
+
+        This method combines interface `append_backward_ops()` and
+        `create_optimization_pass()` into one.
+        """
+        params_grads = self.create_backward_pass(loss, parameter_list)
+        update_ops = self.create_optimization_pass(params_grads)
+        return update_ops
+
+
    
+
    
+
    Users can inherit the Optimizer above to create their own Optimizer with some special logic, such as AdagradOptimizer.
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/parameter_average.html b/doc_cn/design/parameter_average.html
new file mode 100644
index 0000000000000000000000000000000000000000..57ff2cfa426e6ee9e8b93acc0c48312a75387176
--- /dev/null
+++ b/doc_cn/design/parameter_average.html
@@ -0,0 +1,344 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Averaging Parameter in PaddlePaddle — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Averaging Parameter in PaddlePaddle
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Averaging Parameter in PaddlePaddle
    
+
    
+
    Why Averaging
    
+
    In a large scale machine learning setup where the size of the training data is huge, it could take us a large number of iterations over the training data before we can achieve the optimal values of parameters of our model. Looking at the problem setup, it is desirable if we can obtain the optimal values of parameters by going through the data in as few passes as we can.
    
+
    Polyak and Juditsky (1992) showed that the test performance of simple average of parameters obtained by Stochastic Gradient Descent (SGD) is as good as that of parameter values that are obtained by training the model over and over again, over the training dataset.
    
+
    Hence, to accelerate the speed of Stochastic Gradient Descent, Averaged Stochastic Gradient Descent (ASGD) was proposed in Polyak and Juditsky (1992). For ASGD, the running average of parameters obtained by SGD, is used as the estimator for 
     . The averaging is done as follows:
    
+

    
+
    We propose averaging for any optimizer similar to how ASGD performs it, as mentioned above.
    
+
    
+
    How to perform Parameter Averaging in PaddlePaddle
    
+
    Parameter Averaging in PaddlePaddle works in the following way during training :
    
+
      
+
    1. It will take in an instance of a normal optimizer as an input, e.g. RMSPropOptimizer
      2. 
+
    2. The optimizer itself is responsible for updating the parameters.
      3. 
+
    3. The ParameterAverageOptimizer maintains a separate copy of the parameters for itself:
        
+
      1. In concept, the values of this copy are the average of the values of the parameters in the most recent N batches.
        2. 
+
      2. However, saving all the N instances of the parameters in memory is not feasible.
        3. 
+
      3. Therefore, an approximation algorithm is used.
        4. 
+
      
+
      4. 
+
    
+
    Hence, overall we have have two copies of the parameters: one for the optimizer itself, and one for the ParameterAverageOptimizer. The former should be used in back propagation, while the latter should be used during testing and should be saved.
    
+
    During the testing/ saving the model phase, we perform the following steps:
    
+
      
+
    1. Perform the delayed operations.
      2. 
+
    2. Save current values of the parameters to a temporary variable.
      3. 
+
    3. Replace the values of the parameters with the averaged values.
      4. 
+
    4. Perform testing and/or save the parameters.
      5. 
+
    5. Restore the values of the parameters once done.
      6. 
+
    
+
    
+
    
+
    How to implement Averaging of Parameter in PaddlePaddle
    
+
    We can add the ParameterAverageOptimizer op to the graph through Python API. Using this approach, we manually add this op to the graph and direct the output of the optimizer op to this op during training.
    
+
    **Advantages**:
+- Allows for greater flexibility to the users of PaddlePaddle. Using this approach, the users can plug different optimizers into ParameterAverageOptimizer by passing in the optimizer to the op.
+- Makes it easy for the users to customize and extend the framework.
+
+**Disadvantages**:
+- Implementation requires re-writing the averaging methodology in Python.  
+
    
+
    
+
    
+
    
+
    Low-Level implementation
    
+
    In the new design, we propose to create a new operation for averaging parameter updates (ParameterAverageOptimizer). For now, we can add an op that takes in the following as input:
    
+
      
+
    • the optimizer
      • 
+
    • the window_size to keep the updates
      • 
+
    
+
    The ParameterAverageOptimizer op can be like any other operator with its own CPU/GPU implementation either using Eigen or separate CPU and GPU kernels. As the initial implementation, we can implement the kernel using Eigen following the abstraction pattern implemented for Operators. We also want to support the case when the Trainer/Optimizer runs on the GPU while ParameterAverageOptimizer runs on a CPU.
    
+
    The idea of building an op for averaging is in sync with the refactored PaddlePaddle philosophy of using operators to represent any computation unit. The way the op will be added to the computation graph will be decided by the layer functions in Python API.
    
+
    
+
    
+
    Python API implementation for ParameterAverageOptimizer
    
+
    Based on Polyak and Juditsky (1992), we can generalize the averaging of updates to any optimizer. The input to the op would be the following:
    
+
      
+
    • Any optimizer (RMSProp , AdaGrad etc.)
      • 
+
    • A window size. The op keeps accumulating updated parameter values over a window of N batches and takes an average. Move the averaged value to a buffer when window is full to avoid loss of precision.
      • 
+
    
+
    Using the ParameterAverageOptimizer op, any user can add the operation to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support averaging. As per the PaddlePaddle Python API design, the layer functions are responsible for creating operators, operator parameters and variables. Since ParameterAverageOptimizer will be an operator, it makes sense to create it in the layer functions.
+We will have a wrapper written in Python that will support the functionality and implement the actual core computation in C++ core as we have done for other Optimizers
    
+
    
+
    Creation of the ParameterAverageOptimizer operator
    
+
    There are two ways for creating the ParameterAverageOptimizer op:
    
+
      
+
    1. We create the op immediately while building the computation graph.
      2. 
+
    2. We add the op in a lazy manner, just before the backward pass, similar to the way the optimization ops are added.
      3. 
+
    
+
    The proposal is to add the op immediately while building the computation graph.
    
+
    
+
    
+
    High-level API
    
+
    In PaddlePaddle Python API, users will primarily rely on layer functions to create neural network layers. Hence, we also need to provide parameter average functionality in layer functions.
    
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/parameters_in_cpp.html b/doc_cn/design/parameters_in_cpp.html
new file mode 100644
index 0000000000000000000000000000000000000000..adce0cfe70b073720706f3a2619da9b4daf48330
--- /dev/null
+++ b/doc_cn/design/parameters_in_cpp.html
@@ -0,0 +1,303 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: The C++ Class Parameters — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: The C++ Class Parameters
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: The C++ Class Parameters
    
+
    Parameters is a concept we designed in PaddlePaddle V2 API. Parameters is a container of parameters, which makes PaddlePaddle capable of  sharing parameter between topologies. We described usages of Parameter in api.md.
    
+
    We used Python to implement Parameters when designing V2 API before. There are several defects for the current implementation:
    
+
      
+
    • We just use memcpy to share Parameters between topologies, but this is very inefficient.
      • 
+
    • We did not support sharing Parameters while training. We just trigger memcpy when start training.
      • 
+
    
+
    It is necessary that we implement Parameters in CPP side. However, it could result a code refactoring for PaddlePaddle, because PaddlePaddle was designed for training only one topology before, i.e., each GradientMachine contains its Parameter as a data member. In current PaddlePaddle implementation, there are three concepts associated with Parameters:
    
+
      
+
    1. paddle::Parameter. A Parameters is a container for paddle::Parameter.
+It is evident that we should use paddle::Parameter when developing Parameters.
+However, the Parameter class contains many functions and does not have a clear interface.
+It contains create/store Parameter, serialize/deserialize, optimize(i.e SGD), randomize/zero.
+When we developing Parameters, we only use create/store Parameter functionality.
+We should extract functionalities of Parameter into many classes to clean PaddlePaddle CPP implementation.
      2. 
+
    2. paddle::GradientMachine and its sub-classes, e.g., paddle::MultiGradientMachine, paddle::NeuralNetwork.
+We should pass Parameters to paddle::GradientMachine when forward/backward to avoid memcpy between topologies.
+Also, we should handle multi-GPU/CPU training, because forward and backward would perform on multi-GPUs and multi-CPUs.
+Parameters should dispatch the parameter value to each device, and gather the parameter gradient from each device.
      3. 
+
    3. paddle::ParameterUpdater. The ParameterUpdater is used to update parameters in Paddle.
+So Parameters should be used by paddle::ParameterUpdater, and paddle::ParameterUpdater should optimize Parameters (by SGD).
      4. 
+
    
+
    The step by step approach for implementation Parameters in PaddlePaddle C++ core is listed below. Each step should be a PR and could be merged into PaddlePaddle one by one.
    
+
      
+
    1. Clean paddle::Parameter interface. Extract the functionalities of paddle::Parameter to prepare for the implementation of Parameters.
      2. 
+
    2. Implementation a Parameters class. It just stores the paddle::Parameter inside. Make GradientMachine uses Parameters as a class member.
      3. 
+
    3. Make Parameters support Multi-CPU and Multi-GPU training to prepare for sharing Parameter between topologies.
+Because we need share Parameters between topologies, it is Parameters‘s response to exchange Parameters between GPUs.
+GradientMachine should not handle how to exchange Parameters because GradientMachine only used to train one topology and we need to support train many topologies in Paddle, i.e., there could be many GradientMachines use one Parameters.
        
+
      • We should use a global function to exchange Parameters between GPUs, not a member function in Parameters. The MultiGradientMachine invoke this function, which uses Parameters as this function inputs.
        • 
+
      • The MultiGradientMachine contains many functionalities. Extracting the Parameters exchanging logic could make MultiGradientMachine clearer and simpler.
        • 
+
      
+
      4. 
+
    4. Make Parameters as an argument for forward/backward function, not a data member for GradientMachine. For example, forward could be forward(const Parameters& params, ...) and backward could be backward(Parameters* params, ...). After this step, Paddle could share Parameters between topologies.
      5. 
+
    5. ParameterUpdater is invoked by GradientMachine and Trainer, but it updates Parameters. In the end of this code refactoring, we could change ParameterUpdater directly uses Parameters to make ParameterUpdater‘s implementation clear.
      6. 
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/program.html b/doc_cn/design/program.html
new file mode 100644
index 0000000000000000000000000000000000000000..b98bde47e8ba779794e9c694b4788e9c89f73b74
--- /dev/null
+++ b/doc_cn/design/program.html
@@ -0,0 +1,391 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: PaddlePaddle Programs — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: PaddlePaddle Programs
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: PaddlePaddle Programs
    
+
    
+
    Compile and Execution
    
+
    A PaddlePaddle program consists of two parts – the first generates a ProgramDesc protobuf message that describes the program, and the second runs this message using a C++ class Executor.
    
+
    A simple example PaddlePaddle program can be found in graph.md:
    
+
    x = layer.data("images")
+l = layer.data("label")
+y = layer.fc(x)
+cost = layer.mse(y, l)
+optimize(cost)
+train(cost, reader=mnist.train())
+
    
+
    
+
    The first five lines of the following PaddlePaddle program generates, or, compiles, the ProgramDesc message.  The last line runs it.
    
+
    
+
    
+
    Programs and Blocks
    
+
    The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.
    
+
      
+
    • program: some nested blocks
      • 
+
    • block:
        
+
      • some local variable definitions, and
        • 
+
      • a sequence of operators
        • 
+
      
+
      • 
+
    
+
    The concept of block comes from usual programs.  For example, the following C++ program has three blocks:
    
+
    int main() { // block 0
+  int i = 0;
+  if (i < 10) { // block 1
+    for (int j = 0; j < 10; j++) { // block 2
+    }
+  }
+  return 0;
+}
+
    
+
    
+
    The following PaddlePaddle program has three blocks:
    
+
    import paddle as pd  // block 0
+
+x = minibatch([10, 20, 30]) # shape=[None, 1]
+y = var(1) # shape=[1], value=1
+z = minibatch([10, 20, 30]) # shape=[None, 1]
+cond = larger_than(x, 15) # [false, true, true]
+
+ie = pd.ifelse()
+with ie.true_block():  // block 1
+    d = pd.layer.add_scalar(x, y)
+    ie.output(d, pd.layer.softmax(d))
+with ie.false_block():  // block 2
+    d = pd.layer.fc(z)
+    ie.output(d, d+1)
+o1, o2 = ie(cond)
+
    
+
    
+
    
+
    
+
    BlockDesc and ProgramDesc
    
+
    All protobuf messages are defined in framework.proto.
    
+
    BlockDesc is straight-forward – it includes local variable definitions, vars, and a sequence of operators, ops.
    
+
    message BlockDesc {
+  required int32 parent = 1;
+  repeated VarDesc vars = 2;
+  repeated OpDesc ops = 3;
+}
+
    
+
    
+
    The parent ID indicates the parent block so that operators in a block can refer to variables defined locally and also those defined in their ancestor blocks.
    
+
    All hierarchical blocks in a program are flattened and stored in an array. The block ID is the index of the block in this array.
    
+
    message ProgramDesc {
+  repeated BlockDesc blocks = 1;
+}
+
    
+
    
+
    
+
    Global Block
    
+
    The global block is the first one in the above array.
    
+
    
+
    
+
    
+
    Operators that Use Blocks
    
+
    In the above example, the operator IfElseOp has two blocks – the true branch and the false branch.
    
+
    The definition of OpDesc shows that an operator could have some attributes:
    
+
    message OpDesc {
+  AttrDesc attrs = 1;
+  ...
+}
+
    
+
    
+
    and an attribute could be of type block, which is, in fact, a block ID as described above:
    
+
    message AttrDesc {
+  required string name = 1;
+
+  enum AttrType {
+    INT = 1,
+    STRING = 2,
+    ...
+    BLOCK = ...
+  }
+  required AttrType type = 2;
+
+  optional int32 block = 10; // when type == BLOCK
+  ...
+}
+
    
+
    
+
    
+
    
+
    InferShape
    
+
    With this design, the InferShape function should take the following parameters:
    
+
    void InferShape(int current_block,
+                int current_operator,
+                ProgramDesc* program // might change VarDesc values.
+                ) {
+  ...
+}
+
    
+
    
+
    where
    
+
      
+
    • current_block indices into ProgramDesc::blocks,
      • 
+
    • current_operator indices into BlockDesc::ops.
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/prune.html b/doc_cn/design/prune.html
new file mode 100644
index 0000000000000000000000000000000000000000..d35ec3a05ec9d2dbf7df5353a58a510b7c077d23
--- /dev/null
+++ b/doc_cn/design/prune.html
@@ -0,0 +1,325 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Prune — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Prune
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Prune
    
+
    
+
    Motivation
    
+
    We want to support running inference, training and checkpointing in one ProgramDesc. We implement
+void Prune(const ProgramDesc* input, ProgramDesc* output) function, which takes a ProgramDesc
+and generate a pruned ProgramDesc.
    
+
    
+
    
+
    Challenge
    
+
    Pruning need to support both variables and operators being evaluation targets. Consider the following
+different situations.
    
+
    # Case 1: run foward pass.
+cost_np = session.run(target=cost)
+# Case 2: run backward passing.
+opts_np, _ = session.run(target=[cost, opt])
+# Case 3: run checkpointing
+_ = session.run(target=checkpoint)
+
    
+
    
+
    
+
    
+
    Solution
    
+
    To support evaluation of operators, we add is_target field in the OpDesc.
    
+
    message OpDesc {
+  required string type = 3;
+  repeated Var inputs = 1;
+  repeated Var outputs = 2;
+  repeated Attr attrs = 4;
+  optional bool is_target = 5 [ default = false ];
+};
+
    
+
    
+
    To support evaluation of variables, we add fetch_op.
+For each variable in the target, we insert a fetch_op into the ProgramDesc with variable being
+fetch_op‘s input. Then we also set fetch_op is a target.
    
+
    
+
    Algorithm
    
+
    If an operator needs to be run, it must fall into one of the following cases:
    
+
      
+
    1. It is the target.
      2. 
+
    2. It is depended by some other ops, meaning its output is some other op’s input.
      3. 
+
    
+
    The first case can be checked by op_desc.is_traget() . The second case can be implement as
    
+
    bool HasDependentVar(const OpDesc& op_desc, const std::set<string>& dependent_vars) {
+  for (auto& var : op_desc.outputs()) {
+    for (auto& argu : var.arguments()) {
+      if (dependent_vars.count(argu) != 0) {
+        return true;
+      }
+    }
+  }
+  return false;
+}
+
    
+
    
+
    Then the whole algorithm can be implemented as the following code.
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/python_api.html b/doc_cn/design/python_api.html
new file mode 100644
index 0000000000000000000000000000000000000000..727d6e4bcbc9c9e41c2d512b3a002d95f463f8fd
--- /dev/null
+++ b/doc_cn/design/python_api.html
@@ -0,0 +1,522 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Python API — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Python API
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Python API
    
+
    Due to the refactorization of the PaddlePaddle core, we need Python classes to construct corresponding protobuf messages that describe a DL program.
    
+
    | Python classes | Protobuf messages |
+| — | — |
+| Program | ProgramDesc |
+| Block | BlockDesc |
+| Operator | OpDesc |
+| Variable | VarDesc |
    
+
    Please be aware that these Python classes need to maintain some construction-time information, which are not part of the protobuf messages.
    
+
    
+
    Core Concepts
    
+
    
+
    Program
    
+
    A ProgramDesc describes a DL program, which is composed of an array of BlockDescs.  The BlockDescs in a ProgramDesc can have a tree-like hierarchical structure. However, the ProgramDesc onlys stores a flattened array of BlockDescs. A BlockDesc refers to its parent block by its index in the array.  For example, operators in the step block of an RNN operator need to be able to access variables in its ancestor blocks.
    
+
    Whenever we create a block, we need to set its parent block to the current block, hence the Python class Program needs to maintain a data member current_block.
    
+
    class Program(objects):
+    def __init__(self):
+        self.desc = core.NewProgram() # a C++ ProgramDesc pointer.
+        self.blocks = vector<Block>()
+        self.blocks.append(Block(self, -1)) # the global block
+        self.current_block = 0          # initialized to the global block
+
+    def global_block():
+        return self.blocks[0]
+
+    def current_block():
+        return self.get_block(self.current_block)
+
+    def rollback():
+        self.current_block = self.current_block().parent_idx
+
+    def create_block():
+        new_block_idx = len(self.block)
+        self.blocks.append(Block(self, self.current_block))
+        self.current_block = new_block_idx
+        return current_block()
+
    
+
    
+
    Program is an accessor to the protobuf message ProgramDesc, which is created in C++ space, because the InferShape function is in C++, which manipulates VarDesc messages, which are in turn members of BlockDesc, which is a member of ProgramDesc.
    
+
    Program creates the first block as the global block in its constructor.  All parameters and their initializer operators are in the global block.
    
+
    
+
    
+
    Block
    
+
    A Block includes
    
+
      
+
    1. a map from variable names to an instance of the Python Variable class, and
      2. 
+
    2. a list of Operator instances.
      3. 
+
    
+
    class Block(objects):
+    def __init__(self, program, parent_idx):
+        self.desc = core.NewBlock(program.desc)
+        self.program = program
+        self.vars = map<string, Variable>()
+        self.ops = vector<Operator>()
+        self.parent_idx = parent_idx
+
+    def create_var(self, ...):
+        return Variable(self, ...)
+
+    def _create_global_var(self, ...):
+        program.global_block().create_var(...)
+
+    def create_parameter(self, name, ...):
+        # Parameter is a subclass of variable. See Parameter section for details.
+        self.vars[name] = Parameter(self._create_global_var(...), ...)
+        return self.vars[name]
+
+    def append_operator(self, ...):
+        self.ops.append(Operator(self, ...))
+
+    def prepend_operator(self, ...): # Parameter's ctor prepands initialize operators.
+       self.ops.prepend(Operator(self, ...))
+
    
+
    
+
    create_parameter is necessary because parameters are global variables, defined in the global block, but can be created in some sub-blocks. For example, an FC layer in the step block of an RNN operator.
    
+
    prepend_operator is necessary because the constructor of Parameter needs to create the initialize (or load) operator of the parameter, and would like to put it in the preamble of the global block.
    
+
    
+
    
+
    Operator
    
+
    The Operator class fills in the OpDesc message and calls the C++ function InferShape to infer the output shapes from the input shapes.
    
+
    class Operator(object):
+    def __init__(self,
+                 block,  # Block
+                 type,   # string
+                 inputs, # dict<string, Variable>
+                 outputs,# dict<stirng, Variable>
+                 attrs   # dict<string, Any>
+                 ):
+        self.desc = core.NewOpDesc(block.desc, type, inputs, outputs, attrs)
+        core.infer_shape(self.desc, inputs, outputs)
+
+    def type(self):
+        return self.desc.type()
+
    
+
    
+
    Operator creates the OpDesc message in C++ space, so that it can call the InferShape function, which is in C++.
    
+
    
+
    
+
    Variable
    
+
    Operators take Variables as its inputs and outputs.
    
+
    class Variable(object):
+    def __init__(self,
+                 block=None,      # Block
+                 name=None,       # string
+                 shape,           # tuple
+                 dtype="float32", # string
+                 lod_level=None   # int
+                 ):
+        if name is None:
+            name = unique_name_generator()
+        self.name = name
+        self.block = block
+        self.desc = core.NewVarDesc(block.desc, name, shape, lod_level)
+        self.writer = None
+
    
+
    
+
    Please be aware of self.writer, that tracks operator who creates the variable.  It possible that there are more than one operators who write a variable, but in Python space, each write to a variable is represented by a Variable class.  This is guaranteed by the fact that core.NewVarDesc must NOT create a new VarDesc message if its name already exists in the specified block.
    
+
    
+
    
+
    Parameter
    
+
    A parameter is a global variable with an initializer (or load) operator.
    
+
    class Parameter(Variable):
+    def __init__(self,
+                 block=None,      # Block
+                 name=None,       # string
+                 shape,           # tuple
+                 dtype="float32", # string
+                 lod_level=None   # int
+                 trainable,       # bool
+                 initialize_op_attrs,
+                 optimize_op_attrs):
+        super(Parameter, self).__init__(block, name, shape, dtype, lod_level)
+        self.trainable = trainable
+        self.optimize_op_attrs = optimize_op_attrs
+        block.prepend(Operator(block,  # Block
+                               initialize_op_attrs['type'],   # string
+                               None,   # no inputs
+                               self,   # output is the parameter
+                               initialize_op_attrs)
+
    
+
    
+
    When users create a parameter, they can call
    
+
    program.create_parameter(
+  ...,
+  init_attr={
+    type: "uniform_random",
+    min: -1.0,
+    max: 1.0,
+  })
+)
+
    
+
    
+
    In above example, init_attr.type names an initialize operator.  It can also name the load operator
    
+
    init_attr={
+ type: "load",
+ filename: "something.numpy",
+}
+
    
+
    
+
    optimize_op_attrs is not in the VarDesc message, but kept in the Python instance, as it will be used in the Python space when creating the optimize operator’s OpDesc, and will be in the OpDesc message.
    
+
    
+
    
+
    
+
    Layer Function
    
+
    A layer is a Python function that creates some operators and variables. Layers simplify the work of application programmers.
    
+
    Layer functions take Variable and configuration parameters as its input and return the output variable(s).
    
+
    For example, FullyConnected take one or more variable as its input. The input could be input data or another layer’s output. There are many configuration options for a FullyConnected layer, such as layer size, activation, parameter names, initialization strategies of parameters, and so on. The FullyConnected layer will return an output variable.
    
+
    
+
    Necessity for reusing code between layer functions
    
+
    There are a lot of code that can be reused. Such as
    
+
      
+
    • Give the default value of configuration. e.g., default initialize strategy for parameters is uniform random with min = -1.0, max = 1.0. and default initialize strategy for bias is to fill zero.
      • 
+
    • Append the activation operator.
      • 
+
    • Create a temporary variable.
      • 
+
    • Create parameter.
      • 
+
    • Generate a unique name.
      • 
+
    • Add a bias.
      • 
+
    • ...
      • 
+
    
+
    A mechanism to reuse code between layer functions is necessary. It will be around 150 lines of code if we write a FullyConnected layer without any helper functions.
    
+
    
+
    
+
    Comparision between global functions and helper class
    
+
    The FullyConnected layer will be as follow when we provide global functions:
    
+
    def fc_layer(input, size, param_attr=None, bias_attr=None, act=None, name=None):
+  if name is None:
+    name = unique_name("fc")
+  input = multiple_input(input)
+  param_attr = default_param_attr(param_attr)
+  param_attr = multiple_param_attr(param_attr, len(input))
+
+  # mul
+  mul_results = []
+  for ipt, attr in zip(input, param_attr):
+    shape = ipt.shape[1:] + [size]
+    w = g_program.global_block().create_parameter(shape, ipt.dtype, name, attr)
+    tmp = create_tmp_var(name)
+    g_program.current_block().append_op("mul", {ipt, w}, {tmp})
+  mul_results.append(tmp)
+
+  # add sum
+  ...
+  # add bias
+  ...
+  # add activation
+  ...
+  return out
+
    
+
    
+
    We can provide many helpers functions for layer developers. However, there are several disadvantages for global helper functions:
    
+
      
+
    1. We need a namespace for these methods, then layer developers can quickly figure out what method they can use.
      2. 
+
    2. Global functions will force layer developers to pass its parameter time by time.
      3. 
+
    
+
    So we provide a helper class, LayerHelper, to share code between layer functions. The FullyConnected Layer will be as follow.
    
+
    def fc_layer(input, size, param_attr=None, bias_attr=None, act=None, name=None):
+  helper = LayerHelper(locals())  # pass all parameter to LayerHelper
+
+  mul_results = []
+  for ipt, param in helper.iter_multiple_input_and_param():
+    w = helper.create_parameter(shape=ipt.shape[1:] + [size], dtype = ipt.dtype)
+    tmp = helper.create_tmp_variable()
+    helper.append_op('mul', {ipt, w}, {tmp})
+    mul_results.append(tmp)
+
+  pre_bias = helper.add_sum(mul_results)
+  pre_activation = helper.add_bias(pre_bias)
+  return helper.add_activation(pre_activation)
+
    
+
    
+
    We not only use the fewer lines of code to write fc_layer but also make the code clearer to understand. At the same time, layer developers can figure out what function they can invoke by typing helper. in a python editor.
    
+
    
+
    
+
    Implementation of layer helper
    
+
    We just keep all parameters of a layer function as a dictionary in layer helper as a private data member. Every method of layer helper will look up the dictionary after it is invoked. In that way, we can implement a layer helper for all layer functions even some layer does not contain some operator. For example, The activation is used by the FullyConnected layer or convolution layers, but a cross-entropy layer does not use it. The example code of add_activation are:
    
+
    class LayerHelper(object):
+  def __init__(self, **kwargs):  # kwargs is short for `keyword arguments`
+    self.kwargs = kwargs
+
+  def add_activation(self, input_var):
+    act = self.kwargs.get("act", None)  # default value is None
+    if act is None:  # do nothing if no act
+      return input_var
+
+    tmp = self.create_tmp_var(self)
+    self.append_op(type=act, input=input_var, output=tmp)
+    return tmp
+
    
+
    
+
    
+
    
+
    
+
    Optimizer
    
+
    Optimizer Design Doc
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/reader/README.html b/doc_cn/design/reader/README.html
index 7df8af41f174dc2f744e8cc129f3c6558766f0e1..8ca0817100c0d913cb505d0da4f63a0101c78a30 100644
--- a/doc_cn/design/reader/README.html
+++ b/doc_cn/design/reader/README.html
@@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -85,6 +85,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -108,9 +109,10 @@ @@ -184,22 +206,22 @@

    Python Data Reader Design Doc

    -

    At training and testing time, PaddlePaddle programs need to read data. To ease the users’ work to write data reading code, we define that

    +

    During the training and testing phases, PaddlePaddle programs need to read data. To help the users write code that performs reading input data, we define the following:

      -
    • A reader is a function that reads data (from file, network, random number generator, etc) and yields data items.
      • -
    • A reader creator is a function that returns a reader function.
      • -
    • A reader decorator is a function, which accepts one or more readers, and returns a reader.
      • -
    • A batch reader is a function that reads data (from reader, file, network, random number generator, etc) and yields a batch of data items.
      • +
    • A reader: A function that reads data (from file, network, random number generator, etc) and yields the data items.
      • +
    • A reader creator: A function that returns a reader function.
      • +
    • A reader decorator: A function, which takes in one or more readers, and returns a reader.
      • +
    • A batch reader: A function that reads data (from reader, file, network, random number generator, etc) and yields a batch of data items.
    -

    and provide function which converts reader to batch reader, frequently used reader creators and reader decorators.

    +

    and also provide a function which can convert a reader to a batch reader, frequently used reader creators and reader decorators.

    Data Reader Interface

    -

    Indeed, data reader doesn’t have to be a function that reads and yields data items. It can be any function with no parameter that creates a iterable (anything can be used in for x in iterable):

    +

    Data reader doesn’t have to be a function that reads and yields data items. It can just be any function without any parameters that creates an iterable (anything can be used in for x in iterable) as follows:

    iterable = data_reader()
    -

    Element produced from the iterable should be a single entry of data, not a mini batch. That entry of data could be a single item, or a tuple of items. Item should be of supported type (e.g., numpy 1d array of float32, int, list of int)

    -

    An example implementation for single item data reader creator:

    +

    The item produced from the iterable should be a single entry of data and not a mini batch. The entry of data could be a single item or a tuple of items. Item should be of one of the supported types (e.g., numpy 1d array of float32, int, list of int etc.)

    +

    An example implementation for single item data reader creator is as follows:

    def reader_creator_random_image(width, height):
     def reader():
         while True:
@@ -207,7 +229,7 @@
     return reader
    -

    An example implementation for multiple item data reader creator:

    +

    An example implementation for multiple item data reader creator is as follows:

    def reader_creator_random_image_and_label(width, height, label):
     def reader():
         while True:
@@ -218,8 +240,8 @@

    Batch Reader Interface

    -

    batch reader can be any function with no parameter that creates a iterable (anything can be used in for x in iterable). The output of the iterable should be a batch (list) of data items. Each item inside the list must be a tuple.

    -

    Here are valid outputs:

    +

    Batch reader can be any function without any parameters that creates an iterable (anything can be used in for x in iterable). The output of the iterable should be a batch (list) of data items. Each item inside the list should be a tuple.

    +

    Here are some valid outputs:

    # a mini batch of three data items. Each data item consist three columns of data, each of which is 1.
 [(1, 1, 1),
 (2, 2, 2),
@@ -228,24 +250,24 @@
 # a mini batch of three data items, each data item is a list (single column).
 [([1,1,1],),
 ([2,2,2],),
-([3,3,3],),
+([3,3,3],)]

    Please note that each item inside the list must be a tuple, below is an invalid output:

     # wrong, [1,1,1] needs to be inside a tuple: ([1,1,1],).
- # Otherwise it's ambiguous whether [1,1,1] means a single column of data [1, 1, 1],
- # or three column of datas, each of which is 1.
+ # Otherwise it is ambiguous whether [1,1,1] means a single column of data [1, 1, 1],
+ # or three columns of data, each of which is 1.
 [[1,1,1],
 [2,2,2],
 [3,3,3]]
    -

    It’s easy to convert from reader to batch reader:

    +

    It is easy to convert from a reader to a batch reader:

    mnist_train = paddle.dataset.mnist.train()
 mnist_train_batch_reader = paddle.batch(mnist_train, 128)
    -

    Also easy to create custom batch reader:

    +

    It is also straight forward to create a custom batch reader:

    def custom_batch_reader():
     while True:
         batch = []
@@ -259,7 +281,8 @@

    Usage

    -

    batch reader, mapping from item(s) read to data layer, batch size and number of total pass will be passed into paddle.train:

    +

    Following is how we can use the reader with PaddlePaddle: +The batch reader, a mapping from item(s) to data layer, the batch size and the number of total passes will be passed into paddle.train as follows:

    # two data layer is created:
 image_layer = paddle.layer.data("image", ...)
 label_layer = paddle.layer.data("label", ...)
@@ -272,11 +295,11 @@

    Data Reader Decorator

    -

    Data reader decorator takes a single or multiple data reader, returns a new data reader. It is similar to a python decorator, but it does not use @ syntax.

    -

    Since we have a strict interface for data readers (no parameter, return a single data item). Data reader can be used flexiable via data reader decorators. Following are a few examples:

    +

    The Data reader decorator takes in a single reader or multiple data readers and returns a new data reader. It is similar to a python decorator, but it does not use @ in the syntax.

    +

    Since we have a strict interface for data readers (no parameters and return a single data item), a data reader can be used in a flexible way using data reader decorators. Following are a few examples:

    Prefetch Data

    -

    Since reading data may take time and training can not proceed without data. It is generally a good idea to prefetch data.

    +

    Since reading data may take some time and training can not proceed without data, it is generally a good idea to prefetch the data.

    Use paddle.reader.buffered to prefetch data:

    buffered_reader = paddle.reader.buffered(paddle.dataset.mnist.train(), 100)
    @@ -285,8 +308,8 @@

    Compose Multiple Data Readers

    -

    For example, we want to use a source of real images (reusing mnist dataset), and a source of random images as input for Generative Adversarial Networks.

    -

    We can do:

    +

    For example, if we want to use a source of real images (say reusing mnist dataset), and a source of random images as input for Generative Adversarial Networks.

    +

    We can do the following :

    def reader_creator_random_image(width, height):
     def reader():
         while True:
@@ -304,14 +327,14 @@
 
 reader = paddle.reader.compose(paddle.dataset.mnist.train(), data_reader_creator_random_image(20, 20), true_reader, false_reader)
 # Skipped 1 because paddle.dataset.mnist.train() produces two items per data entry.
-# And we don't care second item at this time.
+# And we don't care about the second item at this time.
 paddle.train(paddle.batch(reader, 128), {"true_image":0, "fake_image": 2, "true_label": 3, "false_label": 4}, ...)

    Shuffle

    -

    Given shuffle buffer size n, paddle.reader.shuffle will return a data reader that buffers n data entries and shuffle them before a data entry is read.

    +

    Given the shuffle buffer size n, paddle.reader.shuffle returns a data reader that buffers n data entries and shuffles them before a data entry is read.

    Example:

    reader = paddle.reader.shuffle(paddle.dataset.mnist.train(), 512)
    @@ -320,21 +343,21 @@

    Q & A

    -
    -

    Why reader return only a single entry, but not a mini batch?

    -

    Always returning a single entry make reusing existing data readers much easier (e.g., if existing reader return not a single entry but 3 entries, training code will be more complex because it need to handle cases like batch size 2).

    -

    We provide function paddle.batch to turn (single entry) reader into batch reader.

    +
    +

    Why does a reader return only a single entry, and not a mini batch?

    +

    Returning a single entry makes reusing existing data readers much easier (for example, if an existing reader returns 3 entries instead if a single entry, the training code will be more complicated because it need to handle cases like a batch size 2).

    +

    We provide a function: paddle.batch to turn (a single entry) reader into a batch reader.

    -
    -

    Why do we need batch reader, isn’t train take reader and batch_size as arguments sufficient?

    -

    In most of the case, train taking reader and batch_size as arguments would be sufficent. However sometimes user want to customize order of data entries inside a mini batch. Or even change batch size dynamically.

    +
    +

    Why do we need a batch reader, isn’t is sufficient to give the reader and batch_size as arguments during training ?

    +

    In most of the cases, it would be sufficient to give the reader and batch_size as arguments to the train method. However sometimes the user wants to customize the order of data entries inside a mini batch, or even change the batch size dynamically. For these cases using a batch reader is very efficient and helpful.

    -
    -

    Why use a dictionary but not a list to provide mapping?

    -

    We decided to use dictionary ({"image":0, "label":1}) instead of list (["image", "label"]) is because that user can easily resue item (e.g., using {"image_a":0, "image_b":0, "label":1}) or skip item (e.g., using {"image_a":0, "label":2}).

    +
    +

    Why use a dictionary instead of a list to provide mapping?

    +

    Using a dictionary ({"image":0, "label":1}) instead of a list (["image", "label"]) gives the advantage that the user can easily reuse the items (e.g., using {"image_a":0, "image_b":0, "label":1}) or even skip an item (e.g., using {"image_a":0, "label":2}).

    -
    -

    How to create custom data reader creator

    +
    +

    How to create a custom data reader creator ?

    def image_reader_creator(image_path, label_path, n):
     def reader():
         f = open(image_path)
@@ -357,7 +380,7 @@

    How is paddle.train implemented

    -

    An example implementation of paddle.train could be:

    +

    An example implementation of paddle.train is:

    def train(batch_reader, mapping, batch_size, total_pass):
     for pass_idx in range(total_pass):
         for mini_batch in batch_reader(): # this loop will never end in online learning.
diff --git a/doc_cn/design/refactor/distributed_architecture.html b/doc_cn/design/refactor/distributed_architecture.html
new file mode 100644
index 0000000000000000000000000000000000000000..7a06b728de132078585fd952a3fe9a53b5532fe4
--- /dev/null
+++ b/doc_cn/design/refactor/distributed_architecture.html
@@ -0,0 +1,381 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Distributed Training Architecture — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Distributed Training Architecture
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Distributed Training Architecture
    
+
    
+
    Abstract
    
+
    PaddlePaddle version 0.10.0 uses the “trainer-parameter server” architecture. We run multiple instances of trainers (where each trainer runs the same model) and parameter servers for distributed training. This architecture serves well, but has few limitations:
    
+
      
+
    1. There is a need to write special code that handles tasks which should only be run on a single trainer. E.g., initializing the model, saving the model etc.
      2. 
+
    2. Model parallelism is hard: It would need all the if-else branches conditioned on the trainer ID to partition the model onto the trainers, and eventually manually writing out the inter-model-shard communication code to communicate between different trainers.
      3. 
+
    3. The user can not directly specify the parameter update rule: This would need to modify the parameter server code and compile a new binary. This makes things more complicated for researchers: A lot of extra effort is required to make this work. Besides, the training job submission program may not allow running arbitrary binaries.
      4. 
+
    
+
    This design doc discusses PaddlePaddle’s new distributed training architecture that addresses the above mentioned limitations.
    
+
    
+
    
+
    Analysis
    
+
    The assumption is that the user writes the trainer program in either Python or C++.
    
+
    
+
    Limitation 1
    
+
    There are two basic functionalities in the trainer program:
    
+
      
+
    1. The training logic such as loading / saving the model and printing out the logs.
      2. 
+
    2. The neural network definition such as the definition of the data layer, the fully connected layer, the cost function and the
+optimizer.
      3. 
+
    
+
    When we train using PaddlePaddle v0.10.0 in a distributed fashion, multiple instances of the same Python code are run on different nodes, hence both: the
+training logic as well as the neural network computation logic, is replicated.
    
+
    The tasks that only need to be run once belong to the training logic. Hence if we only replicate the neural network computation part, and do not
+replicate the training logic, the limitation mentioned above can be avoided.
    
+
    
+
    
+
    Limitation 2
    
+
    Model parallelism means that a single model is partitioned into different components and each node runs one of the component separately. This comes at the extra cost of managing the
+inter-model-shard communication between nodes.
    
+
    PaddlePaddle should ideally be able to modify the neural network computation and figure out the support for model parallelism automatically. However, the
+computation is only specified in Python code which sits outside of PaddlePaddle, hence PaddlePaddle can not support the feature in this setup.
    
+
    Similar to how a compiler uses an intermediate representation (IR) so that the programmer does not need to manually optimize their code for most of the cases, we can have an intermediate representation in PaddlePaddle as well. The compiler optimizes the IR as follows:
    
+
    
+
    PaddlePaddle can support model parallelism by converting the IR so that the user no longer needs to manually perform the computation and operations in the Python component:
    
+
    
+
    The IR for PaddlePaddle after refactoring is called a Block, it specifies the computation dependency graph and the variables used in the computation.
    
+
    
+
    
+
    Limitation 3
    
+
    The user can not directly specify the parameter update rule for the parameter server in the Python module, since the parameter server does not use the same computation definition as the trainer. Instead, the update rule is baked inside the parameter server. The user can not specify the update rule explicitly.
    
+
    This could be fixed by making the parameter server run the same computation definition as the trainer (the user’s Python module). For a detailed explanation, refer to this document -
+Design Doc: Operation Graph Based Parameter Server
    
+
    
+
    
+
    
+
    Distributed Training Architecture
    
+
    The revamped distributed training architecture can address the above discussed limitations. Below is the illustration of how it does so:
    
+
    
+
    The major components in the architecture are: PaddlePaddle Python, PaddlePaddle converter and PaddlePaddle runtime.
    
+
    
+
    PaddlePaddle Python
    
+
    PaddlePaddle Python is the Python library that user’s Python code invokes, to read the data. build the neural network topology, start training, etc.
    
+
    paddle.init()
+input = paddle.op.recordIO("/home/data/mnist.recordio") # file stored on the cluster
+img, label = input[0], input[1]
+hidden = paddle.layer.fc(input=img, size=200, act=paddle.activation.Tanh())
+prediction = paddle.layer.fc(input=img, size=10, act=paddle.activation.Softmax())
+cost = paddle.layer.classification_cost(input=prediction, label=label)
+optimizer = paddle.optimizer.SGD(cost, learning_rate=0.01)
+session = paddle.session.NewRemote(num_trainer=3, num_ps=2, GPU_per_trainer=1)
+for i in range(1000):
+    _, cost_val = session.eval(targets=[cost, optimizer])
+    print cost_val
+
    
+
    
+
    The above code is what a typical Python trainer code is, the neural network topology is built using the helper functions such as paddle.layer.fc. Training is done by calling session.eval iteratively.
    
+
    
+
    session.eval
    
+
    As shown in the graph, session.eval sends the IR and the evaluation inputs or targets to the PaddlePaddle cluster for evaluation.
+The targets can be any variable in the computation graph. When the target is say, the optimizer variable, the neural network will be optimized once. When the target is the cost variable, session.eval returns the cost value. Based on what the target is, an appropriate action is taken.
    
+
    The Python session is a wrapper of the C++ Session class. For more information about Session, refer to this document - Design Doc: Session.
    
+
    
+
    
+
    
+
    PaddlePaddle Converter
    
+
    The PaddlePaddle converter automatically converts the IR in the request (IR and evaluation inputs/targets) from PaddlePaddle Python to partitioned IRs and dispatches the new IRs and evaluation inputs/targets to different PaddlePaddle runtimes. Below are the steps that are followed :
    
+
      
+
    1. Add a feed OP that feeds the eval inputs, and a fetch OP that fetches the eval targets to the IR.
      2. 
+
    2. Extract a new computation (sub)graph with the feed and fetch OPs as the boundary. The runtime does not need to run the OP that is not dependent on the fetch OP.
      3. 
+
    3. Optimize the computation graph.
      4. 
+
    4. Place the OPs in the graph onto different devices on different PaddlePaddle runtime according to a placement algorithm and the device constraints specified by the user.
      5. 
+
    5. Partition the graph according to runtime boundaries and add send / recv OP pair on the runtime boundaries.
      6. 
+
    6. Dispatch the partitioned graph to different PaddlePaddle runtimes.
      7. 
+
    7. PaddlePaddle runtimes with the fetch OP reports evaluation results back to the converter, the converter reports the evaluation results back to the PaddlePaddle Python.
      8. 
+
    
+
    The output IRs will be cached to optimize the conversion latency.
    
+
    
+
    Placement Algorithm
    
+
    Our first implementation will only support “trainer-parameter server” placement: the parameters, initializers, and optimizers are all placed on the PaddlePaddle runtimes with the parameter server role. Everything else will be placed on the PaddlePaddle runtimes with the trainer role. This has the same functionality as the “trainer-parameter server” architecture of PaddlePaddle v0.10.0, but is more generic and flexible.
    
+
    In the future, a more general placement algorithm should be implemented, which makes placements according to the input IR, and a model of device computation time and device communication time. Model parallelism requires the generic placement algorithm.
    
+
    
+
    
+
    
+
    PaddlePaddle Runtime
    
+
    The PaddlePaddle runtime owns multiple devices (e.g., CPUs, GPUs) and runs the IR. The runtime does not need to do OP placement since it is already done by the converter.
    
+
    
+
    
+
    Local Training Architecture
    
+
    The local training architecture will be the same as the distributed training architecture, the difference is that everything runs locally, and there is just one PaddlePaddle runtime:
    
+
    
+
    
+
    
+
    Training Data
    
+
    In PaddlePaddle v0.10.0, training data is typically read with a data reader from Python. This approach is no longer efficient when training in a distributed fashion since the Python process no longer runs on the same node with the trainer processes. The Python reader will need to read from the distributed filesystem (assuming it has the required access) and send to the trainers, doubling the network traffic.
    
+
    When doing distributed training, the user can still use Python data reader: the training data are sent with session.eval. However this should be used for debugging purpose only. The users are encouraged to use the read data OPs.
    
+
    
+
    
+
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/refactor/parameter_server.html b/doc_cn/design/refactor/parameter_server.html
new file mode 100644
index 0000000000000000000000000000000000000000..289b08e0e898f61572230e13c63f755dc955e3e7
--- /dev/null
+++ b/doc_cn/design/refactor/parameter_server.html
@@ -0,0 +1,371 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Operation Graph Based Parameter Server — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Operation Graph Based Parameter Server
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Operation Graph Based Parameter Server
    
+
    
+
    Abstract
    
+
    We propose an approach to implement the parameter server. In this
+approach, there is no fundamental difference between the trainer and
+the parameter server: they both run subgraphs, but subgraphs of
+different purposes.
    
+
    
+
    
+
    Background
    
+
    The previous implementations of the parameter server does not run a
+subgraph. parameter initialization, optimizer computation, network
+communication and checkpointing are implemented twice on both the
+trainer and the parameter server.
    
+
    It would be great if we can write code once and use them on both the
+trainer and the parameter server: reduces code duplication and
+improves extensibility. Given that after the current refactor, we are
+representing everything as a computing graph on the
+trainer. Representing everything as a computing graph on the parameter
+server becomes a natural extension.
    
+
    
+
    
+
    Design
    
+
    
+
    Graph Converter
    
+
    The graph converter converts the user-defined operation (OP) graph
+into subgraphs to be scheduled on different nodes with the following
+steps:
    
+
      
+
    1. OP placement: the OPs will be placed on different nodes according
+to heuristic that minimizes estimated total computation
+time. Currently we will use a simple heuristic that puts parameter
+varable on parameter server workers and everything else on trainer
+workers.
      2. 
+
    2. Add communication OPs to enable the communication between nodes.
      3. 
+
    
+
    We will need these OPs: Send, Recv, Enqueue, Dequeue.
    
+
    Below is an example of converting the user defined graph to the
+subgraphs for the trainer and the parameter server:
    
+
    
+
    After converting:
    
+
    
+
      
+
    1. The parameter variable W and it’s optimizer subgraph are placed on the parameter server.
      2. 
+
    2. Operators are added to the subgraphs.
        
+
      • Send sends data to the connected Recv operator.  The
+scheduler on the receive node will only schedule Recv operator
+to run when the Send operator has ran (the Send OP will mark
+the Recv OP runnable automatically).
        • 
+
      • Enueue enqueues the input variable, it can block until space
+become available in the queue.
        • 
+
      • Dequeue outputs configurable numbers of tensors from the
+queue. It will block until the queue have the required number of
+tensors.
        • 
+
      
+
      3. 
+
    
+
    
+
    
+
    Benefits
    
+
      
+
    • Model parallelism become easier to implement: it’s an extension to
+the trainer - parameter server approach. we already have the
+communication OPs, but need to extend the graph converter’s
+placement functionality.
      • 
+
    • User-defined optimizer is easier to add - user can now express it as
+a subgraph.
      • 
+
    • No more duplication logic inside the trainer and the parameter
+server mentioned in the background section.
      • 
+
    
+
    
+
    
+
    Challenges
    
+
      
+
    • It might be hard for the graph converter to cut a general graph
+(without any hint for which subgraph is the optimizer). We may need
+to label which subgraph inside the OP graph is the optimizer.
      • 
+
    • It’s important to balance the parameter shards of on multiple
+parameter server. If a single parameter is very big (some
+word-embedding, fully connected, softmax layer), we need to
+automatically partition the single parameter onto different
+parameter servers when possible (only element-wise optimizer depends
+on the parameter variable).
      • 
+
    
+
    
+
    
+
    Discussion
    
+
      
+
    • In the “Aync SGD” figure, the “W” variable on the parameter server
+could be read and wrote concurrently, what is our locking strategy?
+E.g., each variable have a lock cpp method to be invoked by every
+OP, or, have a lock OP.
      • 
+
    • Can the Enqueue OP be implemented under our current tensor design
+(puts the input tensor into the queue tensor)?
      • 
+
    • Dequeue OP will have variable numbers of output (depends on the
+min_count attribute), does our current design support it? (similar
+question for the Add OP)
      • 
+
    
+
    
+
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/refactor/session.html b/doc_cn/design/refactor/session.html
new file mode 100644
index 0000000000000000000000000000000000000000..585285b5c27f384baab3eef54586801ce7126184
--- /dev/null
+++ b/doc_cn/design/refactor/session.html
@@ -0,0 +1,429 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Session — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Session
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Session
    
+
    
+
    Abstract
    
+
    The session object encapsulates the environment in which the
+computation graph is executed.
    
+
    We will have the local session and remote session, they offer the
+same interface. The local session encapsulates the local
+runtime environment and the remote session encapsulates the cluster
+runtime environment.
    
+
    The local runtime environment contains:
    
+
      
+
    1. computation devices (i.e., CPU, GPU) handles, and
      2. 
+
    2. the scope which holds all variables.
      3. 
+
    
+
    The remote runtime environment contains:
    
+
      
+
    1. computation devices (i.e., CPU and GPU on node 0, 1) in a cluster,
+and
      2. 
+
    2. the distributed scope in a cluster which holds all
+variables.
      3. 
+
    
+
    The user can create a remote session on Paddle Cloud and evaluate the
+computation graph with it. In this way, the user can control the
+remote computation resource in a cluster from his local computer.
    
+
    
+
    
+
    Background
    
+
    The current design has an implicit global session in which
+paddle.eval() is executed. The pain point is:
    
+
    Since the user is not able to explicitly switch between runtime
+environments, the user cannot run a topology in two independent
+environments.
    
+
    For example, in reinforcement learning, the user may want to have a
+stale model for inference and a fresh model for training, and only
+replace the stale model with the fresh model periodically.
    
+
    Furthermore, we have no concept that encapsulates a remote environment
+that executes a computation graph.
    
+
    We need the session object to address above issues.
    
+
    
+
    
+
    Session
    
+
    A session is an object that owns the runtime environment. All
+computations are executed through session.eval().
    
+
    
+
    Interface
    
+
    eval(
+    targets,
+    feed_dict=None,
+)
+
    
+
    
+
    Evaluates the target Operations or Variables in targets.
    
+
      
+
    • targets: the evaluation targets. Can be a single Operation or
+Variable, or a list with the Operations or Variables as
+elements. The value returned by eval() has the same shape as the
+target argument.
      
+
      The PaddlePaddle program is represented by
+the ProgramDesc, eval() will infer the
+ProgramDesc from the given targets and run the PaddlePaddle
+program. Please
+see
+this graph for
+the detailed illustration for the local session
+and
+this graph for
+the detailed illustration for the remote session.
      
+
      • 
+
    • feed_dict: a dictionary that contains the tensors which override
+the edges of the computation graph.
      
+
      feed_dict not only can provide the input data, it can override any
+OP’s input as well:
      
+
      a = pd.constant(2.0, name="a")
+b = pd.variable(name="b")
+c = pd.mul(a,b)
+sess.eval(targets=c, feed_dict={"b":3.0}) # returns 6.0
+
      
+
      
+
      • 
+
    
+
    close()
+
    
+
    
+
    Closes the session and releases the scope that the session owns.
    
+
    
+
    
+
    Create a Local Session
    
+
    session(
+    devices=None
+)
+
    
+
    
+
    Creates a new session. One session owns one global scope, so creating
+multiple sessions will create different scopes.
    
+
      
+
    • devices: a single string or a list of string of device names,
+the corresponding devices will be the computation devices for
+eval(). If not specified, all available devices (e.g., all GPUs)
+will be used. The user doesn’t need to specify the CPU device since
+it will be always used. Multiple sessions can use the same device.
      • 
+
    
+
    
+
    Example
    
+
    a = paddle.constant(1.0)
+b = paddle.constant(2.0)
+c = a + b
+sess = paddle.session(devices=["gpu:0", "gpu:1", "fpga:0"])
+sess.eval(c)
+sess.close()
+
    
+
    
+
    
+
    
+
    
+
    Create a Remote Session
    
+
    create_cloud_job(
+    name,
+    num_trainer,
+    mem_per_trainer,
+    gpu_per_trainer,
+    cpu_per_trainer,
+    num_ps,
+    mem_per_ps,
+    cpu_per_ps,
+)
+
    
+
    
+
    Creates a Paddle Cloud job. Fails if the job name exists.
    
+
    get_cloud_job(
+    name
+)
+
    
+
    
+
    Gets a Paddle Cloud job.
    
+
    remote_session(
+    job
+)
+
    
+
    
+
      
+
    • job: the Paddle Cloud job.
      • 
+
    
+
    
+
    Example
    
+
    reader = paddle.reader.recordio("/pfs/home/peter/mnist-train-*") # data stored on Paddle Cloud
+image = reader.column(0)
+label = reader.column(1)
+fc1 = paddle.op.fc(image, size=256, act="sigmoid")
+fc2 = paddle.op.fc(fc1, size=10, act="softmax")
+cost = paddle.op.cross_entropy(fc2, label)
+opt = paddle.optimizer.sgd(cost)
+
+job = paddle.create_cloud_job("test", 3, "1G", 1, 1, 2, "1G", 1)
+sess = paddle.remote_ession(job)
+for i in range(1000):
+    sess.eval(opt)
+sess.close()
+
    
+
    
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/refactorization.html b/doc_cn/design/refactorization.html
new file mode 100644
index 0000000000000000000000000000000000000000..15517fb2bc41d1dd1c6991b10ff457bef5dd0f1b
--- /dev/null
+++ b/doc_cn/design/refactorization.html
@@ -0,0 +1,596 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Refactorization Overview — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Refactorization Overview
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Refactorization Overview
    
+
    The goals of refactoring include:
    
+
      
+
    1. Making it easy for external contributors to write new elementary computation operations.
      2. 
+
    2. Making the codebase clean and readable.
      3. 
+
    3. Designing a new computation representation – a computation graph of operators and variables.
      4. 
+
    4. Implementing auto-scalability and auto fault recoverable distributed computing with the help of computation graphs.
      5. 
+
    
+
    
+
    Computation Graphs
    
+
      
+
    1. PaddlePaddle represents the computation, training and inference of Deep Learning models, by computation graphs.
      2. 
+
    2. Please refer to computation graphs for a concrete example.
      3. 
+
    3. Users write Python programs to describe the graphs and run them (locally or remotely).
      4. 
+
    4. A graph is composed of variables and operators.
      5. 
+
    5. The description of graphs must be serializable/deserializable, so that:
        
+
      1. It can be sent to the cloud for distributed execution, and
        2. 
+
      2. It can be sent to clients for mobile or enterprise deployment.
        3. 
+
      
+
      6. 
+
    6. The Python program does two things
        
+
      1. Compilation runs a Python program to generate a protobuf message representation of the graph and send it to
          
+
        1. the C++ library libpaddle.so for local execution,
          2. 
+
        2. the master process of a distributed training job for training, or
          3. 
+
        3. the server process of a Kubernetes serving job for distributed serving.
          4. 
+
        
+
        2. 
+
      2. Execution executes the graph by constructing instances of class Variable and OperatorBase, according to the protobuf message.
        3. 
+
      
+
      7. 
+
    
+
    
+
    
+
    Description and Realization of Computation Graph
    
+
    At compile time, the Python program generates a protobuf message representation of the graph, or a description of the graph.
    
+
    At runtime, the C++ program realizes the graph and runs it.
    
+
    | | Representation (protobuf messages) | Realization (C++ class objects) |
+|—|—|—|
+|Data|VarDesc|Variable|
+|Operation|OpDesc|Operator|
+|Block|BlockDesc|Block|
    
+
    The word graph is interchangeable with block in this document.  A graph consists of computation steps and local variables similar to a C++/Java program block, or a pair of parentheses({ and }).
    
+
    
+
    
+
    Compilation and Execution
    
+
      
+
    1. Run a Python program to describe the graph.  In particular, the Python application program does the following:
        
+
      1. Create VarDesc to represent local/intermediate variables,
        2. 
+
      2. Create operators and set attributes,
        3. 
+
      3. Validate attribute values,
        4. 
+
      4. Infer the type and the shape of variables,
        5. 
+
      5. Plan memory-reuse for variables,
        6. 
+
      6. Generate the backward graph
        7. 
+
      7. Add optimization operators to the computation graph.
        8. 
+
      8. Optionally, split the graph for distributed training.
        9. 
+
      
+
      2. 
+
    2. The invocation of train or infer methods in the Python program does the following:
        
+
      1. Create a new Scope instance in the scope hierarchy for each run of a block,
          
+
        1. realize local variables defined in the BlockDesc message in the new scope,
          2. 
+
        2. a scope is similar to the stack frame in programming languages,
          3. 
+
        
+
        2. 
+
      2. Create an instance of class Block, in which,
          
+
        1. realize operators in the BlockDesc message,
          2. 
+
        
+
        3. 
+
      3. Run the Block by calling
          
+
        1. Block::Eval(vector<Variable>* targets) for forward and backward computations, or
          2. 
+
        2. Block::Eval(vector<Operator>* targets) for optimization.
          3. 
+
        
+
        4. 
+
      
+
      3. 
+
    
+
    
+
    
+
    Intermediate Representation (IR)
    
+
    Compile Time -> IR -> Runtime
+
    
+
    
+
    
+
    Benefits of IR
    
+
      
+
    • Optimization
      
+
      Compile Time -> IR -> Optimized IR -> Runtime
+
      
+
      
+
      • 
+
    • Automatically send partitioned IR to different nodes.
      
+
        
+
      • Automatic Data Parallelism
        
+
        Compile Time
+|-> Single GPU IR
+    |-> [trainer-IR-0, trainer-IR-1, pserver-IR]
+        |-> Node-0 (runs trainer-IR-0)
+        |-> Node-1 (runs trainer-IR-1)
+        |-> Node-2 (runs pserver-IR)
+
        
+
        
+
        • 
+
      • Automatic Model Parallelism (planned for future)
        
+
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    
+
    
+
    Operator/OpWithKernel/OpKernel
    
+
    class_diagram
    
+
    
+
    
+
    
+
    Operator
    
+
    class_diagram
    
+
      
+
    • Operator is the fundamental building block of the user interface.
        
+
      • Operator stores input/output variable names and attributes.
        • 
+
      • The InferShape interface is used to infer the shape of the output variables based on the shapes of the input variables.
        • 
+
      • Use Run to compute the output variables from the input variables.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    OpWithKernel/Kernel
    
+
    class_diagram
    
+
      
+
    • OpWithKernel inherits Operator.
      • 
+
    • OpWithKernel contains a Kernel map.
        
+
      • OpWithKernel::Run get device’s kernel, and invoke OpKernel::Compute.
        • 
+
      • OpKernelKey is the map key. Only device place now, but may be data type later.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    Why separate Kernel and Operator
    
+
      
+
    • Separate GPU and CPU code.
        
+
      • Make Paddle capable of running without GPU.
        • 
+
      
+
      • 
+
    • Make one operator (which is a user interface) and create many implementations.
        
+
      • For example, same multiplication op can have different implementations kernels such as FP16 kernel, FP32 kernel, MKL, eigen kernel.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    Libraries for Kernel development
    
+
      
+
    • Eigen::Tensor contains basic math and element-wise functions.
        
+
      • Note that Eigen::Tensor has broadcast implementation.
        • 
+
      • Limit the number of tensor.device(dev) = in your code.
        • 
+
      
+
      • 
+
    • thrust::transform and std::transform.
        
+
      • thrust has the same API as C++ standard library. Using transform, one can quickly implement customized element-wise kernels.
        • 
+
      • thrust, in addition, supports more complex APIs, like scan, reduce, reduce_by_key.
        • 
+
      
+
      • 
+
    • Hand-writing GPUKernel and CPU code
        
+
      • Do not write in header (.h) files. CPU Kernel should be in cpp source (.cc) and GPU kernels should be in cuda (.cu) files. (GCC cannot compile GPU code.)
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    Operator Registration
    
+
    
+
    Why is registration necessary?
    
+
    We need a method to build mappings between Op type names and Op classes.
    
+
    
+
    
+
    How is registration implemented?
    
+
    Maintaining a map, whose key is the type name and the value is the corresponding Op constructor.
    
+
    
+
    
+
    
+
    
+
    The Registry Map
    
+
    
+
    OpInfoMap
    
+
    op_type(string) -> OpInfo
    
+
    OpInfo:
    
+
      
+
    • creator: The Op constructor.
      • 
+
    • grad_op_type: The type of the gradient Op.
      • 
+
    • proto: The Op’s Protobuf, including inputs, outputs and required attributes.
      • 
+
    • checker: Used to check attributes.
      • 
+
    
+
    
+
    
+
    
+
+
    
+
    
+
    Registration Process
    
+
      
+
    1. Write an Op class and its gradient Op class, if required.
      2. 
+
    2. Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator.
      3. 
+
    3. Invoke the macro REGISTER_OP. This macro will
        
+
      1. Call maker class to complete proto and checker
        2. 
+
      2. Using the completed proto and checker, it will add a new key-value pair to the OpInfoMap
        3. 
+
      
+
      4. 
+
    
+
    
+
    
+
    
+
    Backward Module (1/2)
    
+
    
+
    Create Backward Operator
    
+
      
+
    • Mapping from forward Op to backward Op
+backward
      • 
+
    
+
    
+
    
+
    
+
    
+
    Backward Module (2/2)
    
+
    
+
    Build Backward Network
    
+
      
+
    • Input: a graph of forward operators
      • 
+
    • Output: a graph of backward operators
      • 
+
    • Corner cases in construction
        
+
      • Shared Variables => insert an Add operator to combine gradients
        • 
+
      • No Gradient => insert a fill_zero_grad operator
        • 
+
      • Recursive NetOp => call Backward recursively
        • 
+
      • RNN Op => recursively call Backward on stepnet
        • 
+
      • RNN Op => recursively call Backward on stepnet
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    
+
    Scope, Variable, Tensor
    
+
      
+
    • Tensor is an n-dimension array with type.
        
+
      • Only dims and data pointers are stored in Tensor.
        • 
+
      • All operations on Tensor are written in Operator or global functions.
        • 
+
      • Variable length Tensor design LoDTensor
        • 
+
      
+
      • 
+
    • Variable instances are the inputs and the outputs of an operator, not just Tensor.
        
+
      • step_scopes in RNN is a variable and not a tensor.
        • 
+
      
+
      • 
+
    • Scope is where variables are stored.
        
+
      • map<string var name, Variable>
        • 
+
      • Scope has a hierarchical structure. The local scope can get variables from its parent scope.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    Block (in design)
    
+
    
+
    the difference between original RNNOp and Block
    
+
      
+
    • As an operator is more intuitive than RNNOp,
      • 
+
    • Offers a new interface Eval(targets) to deduce the minimal block to Run,
      • 
+
    • Fits the compile-time/ runtime separation design paradigm.
        
+
      • During the compilation, SymbolTable stores VarDescs and OpDescs and serialize to a BlockDesc
        • 
+
      • When graph executes, a Block with BlockDesc is passed. It then creates Op and Var instances and then invokes Run.
        • 
+
      
+
      • 
+
    
+
    
+
    
+
    
+
    
+
    Milestone
    
+
      
+
    • Take Paddle/books as the main line, the requirement of the models motivates framework refactoring,
      • 
+
    • Model migration
        
+
      • Framework development gives priority support to model migration, for example,
          
+
        • the MNIST demo needs a Python interface,
          • 
+
        • the RNN models require the framework to support LoDTensor.
          • 
+
        
+
        • 
+
      • Determine some timelines,
        • 
+
      • Frequently used Ops need to be migrated first,
        • 
+
      • Different models can be migrated in parallel.
        • 
+
      
+
      • 
+
    • Improve the framework at the same time
      • 
+
    • Accept imperfection, concentrate on solving the specific problem at the right price.
      • 
+
    
+
    
+
    
+
    
+
    Control the migration quality
    
+
      
+
    • Compare the performance of migrated models with old ones.
      • 
+
    • Follow the google C++ style guide.
      • 
+
    • Build the automatic workflow of generating Python/C++ documentations.
        
+
      • The documentation of layers and ops should be written inside the code.
        • 
+
      • Take the documentation quality into account when submitting pull requests.
        • 
+
      • Preview the documentations, read and improve them from a user’s perspective.
        • 
+
      
+
      • 
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/register_grad_op.html b/doc_cn/design/register_grad_op.html
new file mode 100644
index 0000000000000000000000000000000000000000..9378d150045adb03104d2728e085382ac6f1d404
--- /dev/null
+++ b/doc_cn/design/register_grad_op.html
@@ -0,0 +1,341 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Gradient Operators Registration — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Gradient Operators Registration
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Gradient Operators Registration
    
+
    
+
    The Problem Posed
    
+
    Currently, for each C++ operator class definition, a gradient operator creator function is registered, which takes as input a C++ operator instance and returns the corresponding gradient operator instance.
    
+
    However, we noticed two problems with the current design:
    
+
      
+
    1. As we decided to separate the compilation and the execution phases, we need to change the creator to take an OpDesc protobuf message in a ProgramDesc and inserts corresponding OpDesc messages into the ProgramDesc message.
      2. 
+
    2. For some operators, the gradient computation can be written in terms of existing operators.  For example, the gradient of minus operator consists of two operators – an identity operator followed by a scale operator.  Hence the registration mechanism needs to support mapping from an operator to a set of operators for the gradient computation.
      3. 
+
    
+
    
+
    
+
    The Current Implementation
    
+
    Instances of the C++ class OpInfo are stored an associative map whose key is the operator type. The grad_op_type indicates the associated gradient operator type. An operator can create the gradient operator by invoking OpInfo::creator_ of the gradient operator. The pseudo code is as follows
    
+
    struct OpInfo {
+  std::function<OperatorBase*(...)> creator_;
+  std::string grad_op_type_;
+  ...
+};
+
+map<string, OpInfo> OpInfoMap;
+
+OperatorBase* CreateGradientOperator(const OperatorBase& op) {
+  return OpInfoMap.at(op.Type()).creator_(...);
+}
+
    
+
    
+
    
+
    
+
    Proposed Solution
    
+
    The mapping relationship between an operator and its gradient operators is a function. The interface of this function is:
    
+
    // (OpDesc) --> vector<OpDesc>
+std::function<std::vector<OpDescBind>(const OpDescBind&)>;
+
    
+
    
+
    The function takes an OpDescBind of the forward operator and returns one or many gradient operator descriptions. OpDescBind is a C++ wrapper for  the protobuf message OpDesc for rapid manipulation of OpDesc.
    
+
    The GradOpDescMaker will be registered in OpInfo and will replace the grad_op_type_ field. The OpInfo should look like
    
+
    struct OpInfo {
+  std::function<std::vector<std::unique_ptr<OpDescBind>>(const OpDescBind&)>  grad_op_maker_;
+  ...
+};
+
    
+
    
+
    The grad_op_maker_ is a nullptr if the operator does not have any associated gradient operators.
    
+
    We propose a base class called GradOpDescMakerBase to let operator developers generate Gradient Operators easily. The public interface of that class is
    
+
    class GradOpDescMakerBase {
+public:
+  GradOpDescMakerBase(const OpDescBind& );
+  virtual std::vector<std::unique_ptr<OpDescBind>> operator()()const = 0;
+};
+
    
+
    
+
    We can convert GradOpDescMakerBase to std::function<std::vector<std::unique_ptr<OpDescBind>>(const OpDescBind&)> by
    
+
    using GradOpMaker = ...;
+std::function<std::vector<OpDescBind>(const OpDescBind&)> func;
+func = [] (const OpDescBind& fwd_op) {
+  GradOpMaker maker(fwd_op);
+  return maker();
+};
+
    
+
    
+
    We can write many helper functions since the GradOpDescMakerBase is a class now. The basic helper functions get the variables of Input, Output, InputGradient and OutputGradient in the forwarding operator.
    
+
    We should change register macros at the same time. In the current solution, there is no difference between forwarding operators and backward operators. So REGISTER_OP just register one operator. If the REGISTER_OPERATOR contains OpProtoAndCheckerMaker and GradOpDescMaker, we just list them in the same macro. It can be done by a macro contains __VA_ARGS__.
    
+
    The user interface should be
    
+
    vector<OpDesc> MinusOpGradMaker(OpDesc) {...}
+REGISTER_OPERATOR(minus, MinusOp, MinusOpProtoAndCheckerMaker, SumOpGradMaker);
+// Developers can still manually implement gradient operator.
+REGISTER_OPERATOR(minus_grad, MinusGradOp);
+
    
+
    
+
    The interface of current REGISTER_OP macro could not be changed. In REGISTER_OP, it will invoke REGISTER_OPERATOR two times and generate GradOpDescMaker inside.
    
+
    REGISTER_OP(minus, MinusOp, MinusOpProtoAndCheckerMaker, minus_grad, MinusGradOp);
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/regularization.html b/doc_cn/design/regularization.html
new file mode 100644
index 0000000000000000000000000000000000000000..f5586ad554f8d08d455ac53c4142df09300f7d8e
--- /dev/null
+++ b/doc_cn/design/regularization.html
@@ -0,0 +1,333 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Regularization in PaddlePaddle — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Regularization in PaddlePaddle
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Regularization in PaddlePaddle
    
+
    
+
    Introduction to Regularization
    
+
    A central problem in machine learning is how to design an algorithm that will perform well not just on the training data, but also on new data. A frequently faced problem is the problem of overfitting, where the model does not make reliable predictions on new unseen data. Regularization is the process of introducing additional information in order to prevent overfitting. This is usually done by adding extra penalties to the loss function that restricts the parameter spaces that an optimization algorithm can explore.
    
+
    
+
    Parameter Norm Penalties
    
+
    Most common regularization approaches in deep learning are based on limiting the capacity of the models by adding a parameter norm penalty to the objective function J. This is given as follows:
    
+

    
+
    The parameter alpha is a hyperparameter that weights the relative contribution of the norm penalty term, omega, relative to the standard objective function J.
    
+
    The most commonly used norm penalties are the L2 norm penalty and the L1 norm penalty. These are given as follows:
    
+
    
+
    L2 Regularization:
    
+

    
+
    
+
    
+
    L1 Regularization
    
+

    
+
    A much more detailed mathematical background of regularization can be found here.
    
+
    
+
    
+
    
+
    
+
    Regularization Survey
    
+
    A detailed survey of regularization in various deep learning frameworks can be found here.
    
+
    
+
    
+
    Proposal for Regularization in PaddlePaddle
    
+
    
+
    Low-Level implementation
    
+
    In the new design, we propose to create new operations for regularization. For now, we can add 2 ops that correspond to the most frequently used regularizations:
    
+
      
+
    • L2_regularization_op
      • 
+
    • L1_regularization_op
      • 
+
    
+
    These ops can be like any other ops with their own CPU/GPU implementations either using Eigen or separate CPU and GPU kernels. As the initial implementation, we can implement their kernels using Eigen following the abstraction pattern implemented for Activation Ops. This abstraction pattern can make it very easy to implement new regularization schemes other than L1 and L2 norm penalties.
    
+
    The idea of building ops for regularization is in sync with the refactored Paddle philosophy of using operators to represent any computation unit. The way these ops will be added to the computation graph, will be decided by the layer functions in Python API.
    
+
    
+
    
+
    Computation Graph
    
+
    Below is an example of a really simple feed forward neural network.
    
+

    
+
    The Python API will modify this computation graph to add regularization operators. The modified computation graph will look as follows:
    
+

    
+
    
+
    
+
    Python API implementation for Regularization
    
+
    Using the low level ops, L2_regularization_op and L1_regularization_op, any user can add regularization to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support regularization. An example of such an API can be seen in Keras. As per the PaddlePaddle Python API design, the layer functions are responsible for creating operators, operator parameters and variables. Since regularization is a property of parameters, it makes sense to create these in the layer functions.
    
+
    
+
    Creation of Regularization ops
    
+
    There are two possibilities for creating the regularization ops:
    
+
      
+
    1. We create these ops immediately while building the computation graph.
      2. 
+
    2. We add these ops in a lazy manner, just before the backward, similar to the way the optimization ops are added.
      3. 
+
    
+
    The proposal is to add these ops in a lazy manner just before the backward pass.
    
+
    
+
    
+
    Storage of Regularization attributes
    
+
    Since we want to create the regularization ops in a lazy manner, the regularization attributes (type of regularization and weight of regularization penalty) can be stored as attributes of the Parameter class. This is because regularization is a property of the parameters and storing regularization properties with Parameters also allows for shared parameters.
    
+
    
+
    
+
    High-level API
    
+
    In PaddlePaddle Python API, users will primarily rely on layer functions to create neural network layers. Hence, we also need to provide regularization functionality in layer functions. The design of these APIs can be postponed for later right now. A good reference for these APIs can be found in Keras and also by looking at Tensorflow in tf.contrib.layers.
    
+
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/releasing_process.html b/doc_cn/design/releasing_process.html
new file mode 100644
index 0000000000000000000000000000000000000000..d89a53430ff8d160025f6965aff30eb58b40d14b
--- /dev/null
+++ b/doc_cn/design/releasing_process.html
@@ -0,0 +1,357 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  PaddlePaddle发行规范 — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • PaddlePaddle发行规范
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    PaddlePaddle发行规范
    
+
    PaddlePaddle使用git-flow branching model做分支管理，使用Semantic Versioning标准表示PaddlePaddle版本号。
    
+
    PaddlePaddle每次发新的版本，遵循以下流程:
    
+
      
+
    1. 从develop分支派生出新的分支，分支名为release/版本号。例如，release/0.10.0
      
+
      2. 
+
    2. 将新分支的版本打上tag，tag为版本号rc.Patch号。第一个tag为0.10.0rc1，第二个为0.10.0rc2，依次类推。
      
+
      3. 
+
    3. 对这个版本的提交，做如下几个操作:
      
+
        
+
      • 修改python/setup.py.in中的版本信息,并将istaged字段设为True。
        
+
        • 
+
      • 编译这个版本的Docker发行镜像，发布到dockerhub。如果失败，修复Docker编译镜像问题，Patch号加一，返回第二步
        
+
        • 
+
      • 编译这个版本的Ubuntu Deb包。如果失败，修复Ubuntu Deb包编译问题，Patch号加一，返回第二步。
        
+
        • 
+
      • 使用Regression Test List作为检查列表，测试Docker镜像/ubuntu安装包的功能正确性
        
+
          
+
        • 如果失败，记录下所有失败的例子，在这个release/版本号分支中，修复所有bug后，Patch号加一，返回第二步
          • 
+
        
+
        • 
+
      • 编译这个版本的python wheel包，并发布到pypi。
        
+
          
+
        • 由于pypi.python.org目前遵循严格的命名规范PEP 513，在使用twine上传之前，需要重命名wheel包中platform相关的后缀，比如将linux_x86_64修改成manylinux1_x86_64。
          
+
          • 
+
        • pypi上的package名称为paddlepaddle和paddlepaddle_gpu，如果要上传GPU版本的包，需要修改build/python/setup.py中，name: “paddlepaddle_gpu”并重新打包wheel包：python setup.py bdist_wheel。
          
+
          • 
+
        • 上传方法：
          
+
          cd build/python
+pip install twine
+twine upload dist/[package to upload]
+
          
+
          
+
          • 
+
        
+
        • 
+
      
+
      4. 
+
    4. 第三步完成后，将release/版本号分支合入master分支，并删除release/版本号分支。将master分支的合入commit打上tag，tag为版本号。同时再将master分支合入develop分支。最后删除release/版本号分支。
      
+
      5. 
+
    5. 编译master分支的Docker发行镜像，发布到dockerhub。编译ubuntu的deb包，发布到github release页面
      
+
      6. 
+
    6. 协同完成Release Note的书写
      
+
      7. 
+
    
+
    需要注意的是:
    
+
      
+
    • release/版本号分支一旦建立，一般不允许再从develop分支合入release/版本号。这样保证release/版本号分支功能的封闭，方便测试人员测试PaddlePaddle的行为。
      • 
+
    • 在release/版本号分支存在的时候，如果有bugfix的行为，需要将bugfix的分支同时merge到master, develop和release/版本号这三个分支。
      • 
+
    
+
    
+
    PaddlePaddle 分支规范
    
+
    PaddlePaddle开发过程使用git-flow分支规范，并适应github的特性做了一些区别。
    
+
      
+
    • PaddlePaddle的主版本库遵循git-flow分支规范。其中:
        
+
      • master分支为稳定(stable branch)版本分支。每一个master分支的版本都是经过单元测试和回归测试的版本。
        • 
+
      • develop分支为开发(develop branch)版本分支。每一个develop分支的版本都经过单元测试，但并没有经过回归测试。
        • 
+
      • release/版本号分支为每一次Release时建立的临时分支。在这个阶段的代码正在经历回归测试。
        • 
+
      
+
      • 
+
    • 其他用户的fork版本库并不需要严格遵守git-flow分支规范，但所有fork的版本库的所有分支都相当于特性分支。
        
+
      • 建议，开发者fork的版本库使用develop分支同步主版本库的develop分支
        • 
+
      • 建议，开发者fork的版本库中，再基于develop版本fork出自己的功能分支。
        • 
+
      • 当功能分支开发完毕后，向PaddlePaddle的主版本库提交Pull Reuqest，进而进行代码评审。
          
+
        • 在评审过程中，开发者修改自己的代码，可以继续在自己的功能分支提交代码。
          • 
+
        
+
        • 
+
      
+
      • 
+
    • BugFix分支也是在开发者自己的fork版本库维护，与功能分支不同的是，BugFix分支需要分别给主版本库的master、develop与可能有的release/版本号分支，同时提起Pull Request。
      • 
+
    
+
    
+
    
+
    PaddlePaddle回归测试列表
    
+
    本列表说明PaddlePaddle发版之前需要测试的功能点。
    
+
    
+
    PaddlePaddle Book中所有章节
    
+
    PaddlePaddle每次发版本首先要保证PaddlePaddle Book中所有章节功能的正确性。功能的正确性包括验证PaddlePaddle目前的paddle_trainer训练和纯使用Python训练模型正确性。
    
+
    | | 新手入门章节 | 识别数字 | 图像分类 | 词向量 | 情感分析 | 语意角色标注 | 机器翻译 | 个性化推荐 |
+| — | — | — | — | — | — | — | — | — |
+| API.V2 + Docker + GPU  |  |  |  |  |  |  |  |  |
+| API.V2 + Docker + CPU  |  |  |  |  |  |  |  |  |
+| paddle_trainer + Docker + GPU |  |  |  |  |  |  |  |  |
+| paddle_trainer + Docker + CPU |  |  |  |  |  |  |  |  |
+| API.V2 + Ubuntu + GPU |  |  |  |  |  |  |  |  |
+| API.V2 + Ubuntu + CPU |  |  |  |  |  |  |  |  |
+| paddle_trainer + Ubuntu + GPU |  |  |  |  |  |  |  |  |
+| paddle_trainer + Ubuntu + CPU |  |  |  |  |  |  |  |  |
    
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/scope.html b/doc_cn/design/scope.html
new file mode 100644
index 0000000000000000000000000000000000000000..74d6f60a53983992848ecd0524213ae806aeb4be
--- /dev/null
+++ b/doc_cn/design/scope.html
@@ -0,0 +1,386 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design of Scope in Paddle — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design of Scope in Paddle
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design of Scope in Paddle
    
+
    
+
    Overview
    
+
    Scope is an important concept in programming languages, which defines a program region that a set of bindings between names and entities applies. In a specific scope, a valid name is uniquely associated with an entity, such as a variable. And in another scope, this name may refer to other entity or nothing at all. It clearly restricts the visibility and validity of names in a program. Hence Scope is introduced to PaddlePaddle to manage variables in context. But different from the original abstract concept, Scope now becomes an object with two important attributes:
    
+
      
+
    • Scope is an association of a name to variable.
      • 
+
    • Variables in a parent scope can be retrieved from local scope.
      • 
+
    
+
    A detailed explanation of these two attributes goes as following.
    
+
    
+
    
+
    Scope is an association of a name to variable.
    
+
    Scope is an association of a name to variable. All variables belong to Scope. You need to specify a scope to run a Net, i.e., net.Run(&scope). One net can run in different scopes and update different variable in the scope.
    
+
      
+
    1. Scope only contains a map of a name to variable.
      
+
      All parameters, data, states in a Net should be variables and stored inside a scope. Each op should get inputs and outputs to do computation from a scope, such as data buffer, state (momentum) etc.
      
+
      2. 
+
    2. Variable can only be created by Scope and a variable can only be got from Scope. User cannot create or get a variable outside a scope. This is a constraints of our framework, and will keep our framework simple and clear.
      
+
      3. 
+
    3. Scope only contains methods that are used to Create and Get Variables. Scope do not contain Operators and have no information to run them.
+Net is designed to drive the computation and Scope only contains a map of variables. There is no computation logic inside a Scope. Scope just handles the lifetime management of variables.
      
+
        
+
      • Create is used to create a Variable by its name and add the mapping relation.
        • 
+
      • Get is used to find a Variable by name.
        • 
+
      
+
      4. 
+
    4. Every variable only belongs to one certain Scope.
      
+
      Variable can not belong to many scopes. If you want to use variables from parent scope, you can use parent scope.
      
+
      5. 
+
    5. Scope should destruct all Variables inside it when itself is destructed. User can never store Variable pointer somewhere else.
      
+
      Because Variable can only be got from Scope. When destroying Scope, we also need to destroy all the Variables in it. If user store Variable pointer to private data member or some global variable, the pointer will be an invalid pointer when associated Scope is destroyed.
      
+
      6. 
+
    
+
    class Scope {
+ public:
+  Variable* Var(const std::string& name);
+  const Variable* FindVar(const std::string& name) const;
+
+ private:
+    std::unordered_map<std::string, std::unique_ptr<Variable>> vars_;
+};
+
    
+
    
+
    
+
    
+
    Parent scope and local scope
    
+
    Just like scope in programming languages, Scope in the neural network can also be a local scope. There are two attributes about local scope.
    
+
      
+
    1. We can create local variables in a local scope. When that local scope is destroyed, all local variables should also be destroyed.
      2. 
+
    2. Variables in a parent scope can be retrieved from local scopes of that parent scope, i.e., when user get a variable from a scope, it will try to search this variable in current scope. If there is no such variable in the local scope, scope will keep searching from its parent, until the variable is found or there is no parent.
      3. 
+
    
+
    class Scope {
+ public:
+  Scope(const std::shared_ptr<Scope>& scope): parent_(scope) {}
+
+  Variable* FindVar(const std::string& name) const {
+    auto it = vars_.find(name);
+    if (it != vars_.end()) {
+      return it->second.get();
+    } else if (parent_ != nullptr) {
+      return parent_->FindVar(name);
+    } else {
+      return nullptr;
+    }
+  }
+
+ private:
+  std::shared_ptr<Scope> parent_ {nullptr};
+};
+
    
+
    
+
    In Scope class, there is a private data member called parent_. parent_ is a smart pointer to its parent scope. When user Get a variable by its name, the name will be searched inside the current scope. If the variable cannot be found locally and parent scope is not a nullptr, the variable will be searched inside that parent scope. parent_ pointer’s default value is nullptr. It means that the scope is a global scope when parent_ is nullptr.
    
+
    A local scope is very useful when we implement Recurrent Neural Network. Each timestep of an RNN should be a Net. Each Net of timestep (StepNet for short) should use an independent local scope. Just like variables in a while loop is inside a local scope in programming languages. By using a single StepNet and changing local scope, we can implement an RNN easily.
    
+
    
+
    
+
    
+
    Interface Design
    
+
    class Variable {
+ private:
+  Variable() = default;
+  friend class Scope;
+};
+
+class Scope {
+ private:
+  Scope(const std::shared_ptr<Scope>& parent = nullptr);
+
+ public:
+  static std::shared_ptr<Scope> Create(const std::shared_ptr<Scope>& parent = nullptr);
+
+  // return nullptr if not found.
+  Variable* FindVar(const std::string& name) const;
+
+  // return if already contains same name variable.
+  Variable* Var(const std::string& name);
+
+ private:
+  std::shared_ptr<Scope> parent_;
+  std::unordered_map<std::string, std::unique_ptr<Variable>> vars_;
+};
+
    
+
    
+
    
+
    Only scope can create a variable
    
+
    To ensure only scope can create a variable, we should mark Variable‘s constructor as a private member function, and Scope is a friend class of Variable. And then only Var can construct Variable.
    
+
    
+
    
+
    When scope destroyed, all variables inside this scope should be destroyed together
    
+
    The scope hold unique pointers for all variables. User can FindVar from scope, but he should not hold this pointer as a member variable. Because when scope is destroyed, all variables inside this scope will be destroyed together.
    
+
    
+
    
+
    Sharing a parent scope
    
+
    Local scope contains a parent_ pointer. It is a linked-list for scopes. Using a shared_ptr because when a local scope is using, its parents cannot be destroyed.
    
+
    Also, as the parent scope is a shared_ptr, we can only Create() a scope shared pointer. We cannot construct a scope variable, because it cannot be passed to other scope as parent pointer.
    
+
    
+
    
+
    Orthogonal interface
    
+
    FindVar will return nullptr when name is not found. It can be used as Contains method. Var will return an Error when there is a name conflict locally. Combine FindVar and Var, we can implement Var easily.
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/selected_rows.html b/doc_cn/design/selected_rows.html
new file mode 100644
index 0000000000000000000000000000000000000000..a98dc73b3ffac3912d8024c18d125b1f50dcf978
--- /dev/null
+++ b/doc_cn/design/selected_rows.html
@@ -0,0 +1,332 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design Doc: Selected Rows — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design Doc: Selected Rows
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design Doc: Selected Rows
    
+
    SelectedRows is a type of sparse tensor data type, which is designed to support embedding operators. The gradient of embedding table is a sparse tensor. Only a few rows are non-zero values in this tensor. It is straight-forward to represent a sparse tensor by the following sparse tensor data structure:
    
+
    class SelectedRows {
+ private:
+  vector<int> rows_;
+  Tensor value_;
+  int height_;
+};
+
    
+
    
+
    The field height_ is the first dimension of SelectedRows. The rows are the indices of the non-zero rows of SelectedRows. The value_ field is an N-dim tensor of shape [rows.size() /* NUM_ROWS */, ...], which supplies values for each row. The dimension of SelectedRows satisfies [height_] + value_.shape[1:].
    
+
    Suppose that a SelectedRows-typed variable x has many rows, but only two of them have values – row 73 is [1, 2] and row 84 is [3, 4], the SelectedRows representation would be:
    
+
    x = SelectedRow {
+  rows = [73, 84],
+  value = [[1, 2], [3,4]]
+}
+
    
+
    
+
    
+
    SelectedRows in Protobuf
    
+
    SelectedRows is a type of Variable. VarDesc in protobuf should describe the SelectedRows information. Only the tensor dimension of a SelectedRows will be described in compile-time because the rows_ and value_ are dependent on the training data.
+So we use TensorDesc to unify data_type and dims. A LodTensorDesc contains a TensorDesc and lod_level. The description of SelectedRows is a Tensor description.
    
+
    message TensorDesc {
+  required DataType data_type = 1;
+  repeated int64 dims = 2; // [UNK, 640, 480] is saved as [-1, 640, 480]
+}
+
+message LodTensorDesc {
+  required TensorDesc tensor = 1;
+  optional int lod_level = 2;
+}
+
+message VarDesc {
+  required string name = 1;
+  enum VarType { 
+    LOD_TENSOR = 0;
+    SELECTED_ROWS = 1;
+  }
+  required VarType type = 2;
+  optional LodTensorDesc lod_desc = 3;
+  optional TensorDesc selected_rows_desc = 4;
+  optional bool persistable = 5 [ default = false ];
+}
+
    
+
    
+
    
+
    
+
    InferShape for Selected Rows
    
+
    Just like LoD information, InferShape method will infer the output tensor type as well. The operator should decide whether its output is a SelectedRows or Dense tensor.
    
+
    For example, the gradient operator of TableLookup will always generate SelectedRows. Its InferShape method should be like following
    
+
    void TableLookupGrad::InferShape(context) {
+  ...
+  context.SetDataType("Embedding.Grad", kSelectedRows);
+}
+
    
+
    
+
    
+
    
+
    Sparse Operators
    
+
    There are several operators that need to be written to support SelectedRows. These are:
    
+
      
+
    1. Operators which generate SelectedRows gradient. e.g. Gradient of TableLookupOp.
      2. 
+
    2. Optimize operators which support SelectedRows gradient. e.g. SGD or AdaGrad for SelectedRows. However, there should be only one SGD operator. OpWithKernel::Run should select a suitable kernel for both dense tensor or SelectedRows.
      3. 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/simple_op_design.html b/doc_cn/design/simple_op_design.html
new file mode 100644
index 0000000000000000000000000000000000000000..0f3ba4c48396428ff49e4a66105148450bccd4ff
--- /dev/null
+++ b/doc_cn/design/simple_op_design.html
@@ -0,0 +1,454 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Interaction between C++ and Python — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Interaction between C++ and Python
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Interaction between C++ and Python
    
+
    Users employ API in Python to describe their own network, however, the network construction actually happens in C++. so Protobuf is introduced to send the message between Python and C++.
    
+
    The Interaction between Python and C++ can be simplified as two steps:
    
+
      
+
    1. C++ tells Python how many Ops there are, and what parameter do users need to offer to initialize a new Op. Python then builds API for each Op at compile time.
      2. 
+
    2. Users invoke APIs built by Python and provide necessary parameters. These parameters will be sent to C++ for finishing the Op construction task.
      3. 
+
    
+
    
+
    Message from C++ to Python
    
+
    We define a Protobuf message class OpProto to hold message needed in the first step. What should an OpProto contain? This question is equivalent to “What message do we need to offer, to build a Python API which is legal and user oriented and can use to describe a whole Op.”
    
+
    Following message are necessary:
    
+
      
+
    1. Op’s name, and its simple comment.
      2. 
+
    2. Input and output variable number; each variable’s name, type, and comment.
      3. 
+
    3. Op’s attributes; each attribute includes name, type, comment, default value and value range.
      4. 
+
    
+
    So OpProto can be defined as follows:
    
+
    enum AttrType {
+    INT = 1;
+    FLOAT = 2;
+    STRING = 3;
+    INTS = 4;
+    FLOATS = 5;
+    STRINGS = 6;
+};
+
+message AttrValue {
+    AttrType type = 1;
+    optional int iv = 2;
+    optional float fv = 3;
+    optional string sv = 4;
+    repeated int ivs = 5;
+    repeated float fvs = 6;
+    repeated string svs = 7;
+};
+
+message AttrProto {
+    required string name = 1;
+    required string comment = 2;
+    required AttrType type = 3;
+};
+
+message VarProto {
+    required string name = 1;
+    required string comment = 2;
+    required bool is_tensor = 3;
+};
+
+message OpProto {
+    repeated VarProto inputs = 1;
+    repeated VarProto outputs = 2;
+    repeated AttrProto attrs = 3;
+    required string type = 4;
+    required string comment = 5;
+};
+
    
+
    
+
    To generate Python code automatically:
    
+
    def create_python_ops_creatation_functions():
+    op_protos = paddle.framework.OpRegistry.get_all_op_proto()
+    for type_name in op_protos:
+        op_proto = op_protos[type_name]
+        def __impl__(**kwargs):  # User must use key word args in Paddle API
+            inputs = [kwargs.get(ipt.name, "") for ipt in op_proto.inputs]
+            outputs = [kwargs.get(opt.name, "") for opt in op_proto.outputs]
+            attrs = [cast_to_op_attr(attr, kwargs.get(attr.name, None)) for attr in op_proto.attrs]
+            opdesc = （input, outputs, type_name, attrs）
+            return paddle.framework.OpRegistry.CreateOp(opdesc)
+        __impl__.__doc__ = create_doc_string(op_proto)
+        globals()[type_name] = __impl__
+
+create_python_ops_creatation_functions()
+
    
+
    
+
    
+
    
+
    Message from Python to C++
    
+
    To hold message needed in the above second step, we define Protobuf message class OpDesc. It is used to hold user-specified parameters in Op describing.
    
+
    message OpDesc {
+    required string type = 1;   
+    repeated string inputs = 2;
+    repeated string outputs = 3;
+    map<string, AttrValue> attrs = 4;
+};
+
    
+
    
+
    
+
    
+
    
+
    OpProto Register
    
+
    Every Op has its own OpProto. For using convenience, we need to register them and record all their messages. For each Op class, we define a corresponding OpMaker class, in whose constructor we implement the OpProto‘s building process. OpMaker‘s constructor will be invoked by another function OpRegistry::RegisterOp().
    
+
    class OpProtoMaker {
+public:
+    OpProtoMaker(OpProto* proto): proto_(proto) {}
+protected:
+    OpProto* proto_;
+    void AddInput(const std::string& name, const std::string& desc) {...}
+    void AddAttr(const std::string& name, const std::string& desc, TypeId type) {...}
+    void AddComment(const std::string& comment) { ... }
+};
+
+class OpRegistry {
+public:
+    using OpCreator = std::function<OperatorBase* (OpDesc& desc)>;
+    
+    template <typename OpType, typename OpMaker>
+    static void RegisterOp(const std::string& name) {
+        gCreators_[name] = [](const OpDesc& desc) {
+            return new OpType(desc);
+        };
+        OpProto& opProto = gProtos_[name];
+        OpMaker()(&opProto);
+    }
+
+    static map<string, OpCreator> gCreators_;
+    static map<string, OpProto> gProtos_;
+};
+
+template <typename OpType, typename OpMaker>
+class OpRegister {
+  public:
+    OpRegister(std::string type) {
+        OpRegistry::RegisterOp<OpType, OpMaker>(type);
+    }
+};
+
+#define REGISTER_OP(op_class, op_maker_class, type_name)         \
+    class op_class##Register {                                   \
+      private:                                                   \
+        const static OpRegister<#op_class, #op_maker_class> reg; \
+    };                                                           \
+    const Register op_class##Register::reg(#type_name);
+    
+class CosineOp {
+// ...
+}
+
+struct CosineOpProtoMaker : public OpProtoMaker {
+    CosineOpProtoMaker(OpProto* proto) : OpProtoMaker(proto) {
+        AddInput("input", "input of cosine op");
+        AddAttr("scale", "scale of cosine op", float).Default(1.0).GreaterThan(0.0);
+        AddType("cos");
+        AddComment("This is cos op");
+    }
+}
+
+REGISTER_OP(CosineOp, CosineOpProtoMaker, cos);
+
    
+
    
+
    In REGISTER_OP(CosineOp, CosineOpProtoMaker, cos), we register not only CosineOp but also CosineOpProto. As fields of CosineOpProto, the default value and value range of scale are also registered here.
    
+
    
+
    
+
    Python API
    
+
    Python  APIs are divided into two types, high-level API and low-level API.
    
+
    
+
    High-Level API
    
+
    High-level API is called by users directly, so it should keep its style consistent with existing V2 APIs.
    
+
    Here is a sample about how a define a fc layer:
    
+
    hd = fc_layer(input=data, size=56, with_bias=True, activation="sigmoid");
+
    
+
    
+
    hd is the output of fc_layer and it’s a variable. It can be further sent into other layers as input.
    
+
    The definition of fc_layer():
    
+
    def fc_layer(input, size, with_bias, activation):
+    attr_map = {"size":size}
+    check_attrs(attr_map)
+    w = make_variable('w')
+    if with_bias:
+        b = make_variable('b')
+    else:
+        b = None
+    fc_output = make_variable('fc_output');
+    fc_op(input, w, b, fc_output, attr_map)
+    act_output = make_variable('sigmod_output');
+    if activation == "sigmod":
+        sigmod_op(fc_output, act_output);
+    elif:
+        # ...
+    return act_output;
+
    
+
    
+
    
+
    
+
    Low Leval API
    
+
    In above sample, fc_op and sigmod_op are low-level API. They build OpDesc and invoke corresponding C++ code.
    
+
    TODO
    
+
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/tensor_array.html b/doc_cn/design/tensor_array.html
new file mode 100644
index 0000000000000000000000000000000000000000..e0a5e459a8b9c174748b15cd94e2f6116dd0f585
--- /dev/null
+++ b/doc_cn/design/tensor_array.html
@@ -0,0 +1,516 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Design for TensorArray — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Design for TensorArray
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Design for TensorArray
    
+
    This design doc presents the necessity of a new C++ class TensorArray.
+In addition to the very simple C++ implementation
    
+
    class TensorArray {
+ public:
+  explicit TensorArray(const LoDTensor&);
+  explicit TensorArray(size_t size);
+
+ private:
+  vector<LoDTensor> values_;
+};
+
    
+
    
+
    We also need to expose it to PaddlePaddle’s Python API,
+because users would want to use it with our very flexible operators WhileLoop.
+An example for a RNN based on dynamic operators is
    
+
    input = pd.data(...)
+num_steps = Var(12)
+
+TensorArray states(size=num_steps)
+TensorArray step_inputs(unstack_from=input)
+TensorArray step_outputs(size=num_steps)
+
+W = Tensor(...)
+U = Tensor(...)
+default_state = some_op()
+
+step = Var(1)
+
+wloop = paddle.create_whileloop(loop_vars=[step])
+with wloop.frame():
+    wloop.break_if(pd.equal(step, num_steps)
+    pre_state = states.read(step-1, default_state)
+    step_input = step_inputs.read(step)
+    state = pd.sigmoid(pd.matmul(U, pre_state) + pd.matmul(W, step_input))
+    states.write(step, state)
+    step_outputs.write(step, state) # output state
+    step.update(state+1)
+
+output = step_outputs.stack()
+
    
+
    
+
    
+
    Background
    
+
    Steps are one of the core concepts of RNN. In each time step of RNN, there should be several input segments, states, and output segments; all these components act like arrays, for example, call states[step_id] will get the state in step_idth time step.
    
+
    An RNN can be implemented with the following pseudocode
    
+
    Array states;
+Array input_segments;
+Array output_segments;
+Parameter W, U;
+
+step = 1
+seq_len = 12
+while_loop {
+   if (step == seq_len) break;
+    states[step] = sigmoid(W * states[step-1] + U * input_segments[step]);
+    output_segments[step] = states[step] // take state as output
+   step++;
+}
+
    
+
    
+
    According to the RNN roadmap, there are several different RNNs that PaddlePaddle will eventually support.
    
+
    Currently, the basic RNN implementation supported by PaddlePaddle is the recurrent_op which takes tensors as input and splits them into input_segments.
    
+
    Since a tensor cannot store variable-length sequences directly, PaddlePaddle implements the tensor with level of details (LoDTensor for short).
+Segmenting the LoDTensor is much more complicated than splitting a tensor, that makes it necessary to refactor the recurrent_op with LoDTensor segmenting support.
    
+
    As the next step in RNN support, dynamic_recurrent_op should be introduced to handle inputs with variable-length sequences.
    
+
    The implementation is similar to recurrent_op.
+The key difference is the way the original input LoDTensors and outupts are split to get the input_segments and the output_segments.
    
+
    Though it can’t be built over recurrent_op or dynamic_recurrent_op directly,
+the logic behind splitting a tensor or a LoD tensor into input_segments remains the same.
    
+
    
+
    
+
    Why TensorArray
    
+
    The logic behind splitting the inputs to segments, states and outputs is similar and can be shared in a seperate module.
    
+
    The array of states, input_segments and output_segments would be exposed to users when writing a dynamic RNN model similar to the above pseudo codes.
    
+
    So there should be an array-like container, which can store the segments of a tensor or LoD tensor.
    
+
    This container can store an array of tensors and provides several methods to split a tensor or a LoD tensor .
+This is where the notion of TensorArray comes from.
    
+
    
+
    
+
    Introduce TensorArray to uniform all the three RNNs
    
+
    TensorArray as a new concept is borrowed from TensorFlow,
+it is meant to be used with dynamic iteration primitives such as while_loop and map_fn.
    
+
    This concept can be used to support our new design of dynamic operations, and help to refactor some existing variant-sentence-related layers,
+such as recurrent_op, RecurrentGradientMachine.
    
+
    In our design for dynamic RNN,
+TensorArray is used to segment inputs and store states in all time steps.
+By providing some methods similar to a C++ array,
+the definition of some state-based dynamic models such as RNN can be more natural and highly flexible.
    
+
    
+
    
+
    Dynamic-operations on TensorArray
    
+
    TensorArray will be used directly when defining dynamic models, so some operators listed below should be implemented
    
+
    # several helper operators for TensorArray
+def tensor_array_stack(ta, tensor):
+    '''
+    get a tensor array `ta`, return a packed `tensor`.
+    '''
+    pass
+
+def tensor_array_unstack(tensor, ta):
+    '''
+    get a `tensor`, unstack it and get a tensor array `ta`.
+    '''
+    pass
+
+def tensor_array_write(ta, index, tensor, data_shared):
+    '''
+    get a `tensor` and a scalar tensor `index`, write `tensor` into index-th
+    value of the tensor array `ta`.
+    `data_shared` is an attribute that specifies whether to copy or reference the tensors.
+    '''
+    pass
+
+def tensor_array_read(ta, index, tensor):
+    '''
+    get a tensor array `ta`, a scalar tensor `index`, read the index-th value of
+    `ta` and return as the `tensor`.
+    '''
+    pass
+
+def tensor_array_size(ta, tensor):
+    '''
+    get a tensor array `ta`, return the size of `ta` and return as the scalar `tensor`.
+    '''
+    pass
+
    
+
    
+
    It is trivial for users to use so many low-level operators, so some helper methods should be proposed in python wrapper to make TensorArray easier to use,
+for example
    
+
    class TensorArray:
+    def __init__(self, name):
+        self.name = name
+        self.desc = TensorArrayDesc()
+
+    def stack(self, name=None):
+        '''
+        Pack the values in a `TensorArray` into a tensor with rank one higher
+        than each tensor in `values`.
+        `stack` can be used to split tensor into time steps for RNN or whileloop.
+
+        @name: str
+            the name of the variable to output.
+        '''
+        tensor = Var(name)
+        tensor_array_stack(self.name, tensor)
+        return tensor
+
+    def unstack(self, input):
+        '''
+        Unpacks the given dimension of a rank-`R` tensor into rank-`(R-1)` tensors.
+        `unstack` can be used to concatenate all the time steps for RNN or whileloop.
+
+        @input: str
+            the name of input tensor
+        '''
+        tensor_array_unstack(tensor, self.name)
+
+    def write(self, index, value, data_shared=True):
+        '''
+        Write value into index of the TensorArray.
+        If `data_shared` is set to True, than the index-th value in TensorArray will
+        be shared with the tensor passed in.
+
+        @index: str
+            name of a scalar tensor
+        @value: str
+            name of a tensor
+        @data_shared: bool
+        '''
+        tensor_array_write(self.name, index, value, data_shared)
+
+    def read(self, index, output):
+        '''
+        Read the value at location `index` in the `TensorArray`.
+
+        @index: str
+            name of a scalar tensor
+        @output:
+            name of a output variable
+        '''
+        tensor_array_read(self.name, index, output)
+
+
+    def size(self, output):
+        '''
+        Return the number of values.
+
+        @output: str
+            name of a scalar tensor
+        '''
+        tensor_array_size(self.name, output)
+
    
+
    
+
    
+
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/design/var_desc.html b/doc_cn/design/var_desc.html
new file mode 100644
index 0000000000000000000000000000000000000000..c5edef89fa1a6c63b6c867b0585ad0563701a3c6
--- /dev/null
+++ b/doc_cn/design/var_desc.html
@@ -0,0 +1,330 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  Background — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+     
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+    
    • Background
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    Background
    
+
    PaddlePaddle divides the description of neural network computation graph into two stages: compile time and runtime.
    
+
    PaddlePaddle use proto message to describe compile time graph because
    
+
      
+
    1. Computation graph should be able to be saved to a file.
      2. 
+
    2. In distributed training, the graph will be serialized and send to multiple workers.
      3. 
+
    
+
    The computation graph is constructed by Data Node and Operation Node. The concept to represent them is in the table below.
    
+
    | |compile time|runtime|
+|—|—|—|
+|Data|VarDesc(proto)|Variable(cpp)|
+|Operation|OpDesc(proto)|Operator(cpp)|
    
+
    
+
    
+
    Definition of VarDesc
    
+
    A VarDesc should have a name, and value. The are two kinds of variable type in compile time, they are LoDTensor and SelectedRows.
    
+
    message VarDesc {
+  required string name = 1;
+  enum VarType {
+    LOD_TENSOR = 0;
+    SELECTED_ROWS = 1;
+  }
+  required VarType type = 2;
+  optional LoDTensorDesc lod_desc = 3;
+  optional TensorDesc selected_rows_desc = 4;
+  optional bool persistable = 5 [ default = false ];
+}
+
    
+
    
+
    
+
    
+
    Definition of TensorDesc
    
+
    enum DataType {
+  BOOL = 0;
+  INT16 = 1;
+  INT32 = 2;
+  INT64 = 3;
+  FP16 = 4;
+  FP32 = 5;
+  FP64 = 6;
+}
+
+message TensorDesc {
+  required DataType data_type = 1;
+  repeated int64 dims = 2; // [UNK, 640, 480] is saved as [-1, 640, 480]
+}
+
    
+
    
+
    A TensorDesc describes SelectedRows and LoDTensor. For details of SelectedRows, please reference SelectedRows.
    
+
    
+
    
+
    Definition of LodTensorDesc
    
+
    message LoDTensorDesc {
+  required TensorDesc tensor = 1;
+  optional int lod_level = 2;
+}
+
    
+
    
+
    A LoDTensorDesc contains a tensor and a lod_level.
    
+
    
+
    
+
    Definition of Variable in Python
    
+
    For Variable in Python, please reference Python API.
    
+
    
+
+
+           
    
+          
    
+          
    
+  
+
+  
    
+
+  
    
+    
    
+        © Copyright 2016, PaddlePaddle developers.
+
+    
    
+  
    
+  Built with Sphinx using a theme provided by Read the Docs. 
+
+
    
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/faq/build_and_install/index_cn.html b/doc_cn/faq/build_and_install/index_cn.html
new file mode 100644
index 0000000000000000000000000000000000000000..561a9274a0880aeda067f76d69f04e922e300ac3
--- /dev/null
+++ b/doc_cn/faq/build_and_install/index_cn.html
@@ -0,0 +1,382 @@
+
+
+
+
+  
+
+  
+  
+  
+  
+  编译安装与单元测试 — PaddlePaddle  文档
+  
+
+  
+  
+
+  
+
+  
+  
+    
+
+  
+
+  
+  
+    
+  
+
+  
+  
+        
+        
+    
+        
+        
+         
+
+  
+  
+  
+
+  
+
+  
+  
+
+
+
+
+
+  
+  
    
+    
+    
    
+      
    
+        Fork me on Github
+        
    
+          
+            English
+            
+            
+          
+          
+        
    
+        
+      
    
+      
    
+        
+        
+
+        
+
    
+  
    
+    
+    
+    
+  
    
+
            
+      
    
+    
    
+  
    
+  
+  
    
+
+    
+    
+    
+    
    
+
+      
+
+ 
+
+
+
+
+
+
+
+
    
+  
      
+      
+        
    • FAQ > 
      • 
+      
+    
    • 编译安装与单元测试
      • 
+  
    
+
    
+      
+      
    
+        
    
+          
    
+           
    
+            
+  
    
+
    编译安装与单元测试
    
+
+
    
+
    1. 运行Docker GPU镜像出现 “CUDA driver version is insufficient”
    
+
    用户在使用PaddlePaddle GPU的Docker镜像的时候，常常出现 Cuda Error: CUDA driver version is insufficient for CUDA runtime version, 原因在于没有把机器上CUDA相关的驱动和库映射到容器内部。
+具体的解决方法是：
    
+
    $ export CUDA_SO="$(\ls usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
+$ export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
+$ docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddlepaddle:latest-gpu
+
    
+
    
+
    更多关于Docker的安装与使用, 请参考 PaddlePaddle Docker 文档 。
    
+
    
+
    
+
    2. CMake源码编译, 找到的PythonLibs和PythonInterp版本不一致
    
+
    这是目前CMake寻找Python的逻辑存在缺陷，如果系统安装了多个Python版本，CMake找到的Python库和Python解释器版本可能有不一致现象，导致编译PaddlePaddle失败。正确的解决方法是，
+用户强制指定特定的Python版本，具体操作如下：
    
+
    
+
    cmake .. -DPYTHON_EXECUTABLE=<exc_path> -DPYTHON_LIBRARY=<lib_path>  -DPYTHON_INCLUDE_DIR=<inc_path>
+
    
+
    
+
    
+
    用户需要指定本机上Python的路径：<exc_path>, <lib_path>, <inc_path>
    
+
    
+
    
+
    3. CMake源码编译，Paddle版本号为0.0.0
    
+
    如果运行 paddle version, 出现 PaddlePaddle 0.0.0；或者运行 cmake ..，出现
    
+
    CMake Warning at cmake/version.cmake:20 (message):
+  Cannot add paddle version from git tag
+
    
+
    
+
    那么用户需要拉取所有的远程分支到本机，命令为 git fetch upstream，然后重新cmake即可。
    
+
    
+
    
+
    4. paddlepaddle*.whl is not a supported wheel on this platform.
    
+
    出现这个问题的主要原因是，没有找到和当前系统匹配的paddlepaddle安装包。最新的paddlepaddle python安装包支持Linux x86_64和MacOS 10.12操作系统，并安装了python 2.7和pip 9.0.1。
    
+
    更新 pip 包的方法是:
    
+
    pip install --upgrade pip
+
    
+
    
+
    如果还不行，可以执行 python -c "import pip; print(pip.pep425tags.get_supported())" 获取当前系统支持的python包的后缀，
+并对比是否和正在安装的后缀一致。
    
+
    如果系统支持的是 linux_x86_64 而安装包是 manylinux1_x86_64 ，需要升级pip版本到最新；
+如果系统支持 manylinux1_x86_64 而安装包（本地）是 linux_x86_64 ，可以重命名这个whl包为 manylinux1_x86_64 再安装。
    
+
    
+
    
+
    5. 编译安装后执行 import paddle.v2 as paddle 报ImportError: No module named v2
    
+
    先查看一下是否曾经安装过paddle v1版本，有的话需要先卸载：
    
+
    pip uninstall py_paddle paddle
    
+
    然后安装paddle的python环境, 在build目录下执行
    
+
    pip install python/dist/paddle*.whl && pip install ../paddle/dist/py_paddle*.whl
    
+
    
+
    
+
    6. 遇到“非法指令”或者是“illegal instruction”
    
+
    PaddlePaddle使用avx SIMD指令提高cpu执行效率，因此错误的使用二进制发行版可能会导致这种错误，请选择正确的版本。
    
+
    
+
    
+
    7.  python相关的单元测试都过不了
    
+
    如果出现以下python相关的单元测试都过不了的情况：
    
+
    24 - test_PyDataProvider (Failed)
+26 - test_RecurrentGradientMachine (Failed)
+27 - test_NetworkCompare (Failed)
+28 - test_PyDataProvider2 (Failed)
+32 - test_Prediction (Failed)
+33 - test_Compare (Failed)
+34 - test_Trainer (Failed)
+35 - test_TrainerOnePass (Failed)
+36 - test_CompareTwoNets (Failed)
+37 - test_CompareTwoOpts (Failed)
+38 - test_CompareSparse (Failed)
+39 - test_recurrent_machine_generation (Failed)
+40 - test_PyDataProviderWrapper (Failed)
+41 - test_config_parser (Failed)
+42 - test_swig_api (Failed)
+43 - layers_test (Failed)
+
    
+
    
+
    并且查询PaddlePaddle单元测试的日志，提示：
    
+
    paddle package is already in your PYTHONPATH. But unittest need a clean environment.
+Please uninstall paddle package before start unittest. Try to 'pip uninstall paddle'.
+
    
+
    
+
    解决办法是：
    
+
      
+
    • 卸载PaddlePaddle包 pip uninstall paddle, 清理掉老旧的PaddlePaddle安装包，使得单元测试有一个干净的环境。如果PaddlePaddle包已经在python的site-packages里面，单元测试会引用site-packages里面的python包，而不是源码目录里 /python 目录下的python包。同时，即便设置 PYTHONPATH 到 /python 也没用，因为python的搜索路径是优先已经安装的python包。
      • 
+
    
+
    
+
    
+
+
+           
    
+          
    
+          
+
+        
    
+      
    
+
+    
    
+
+  
    
+  
+
+
+  
+
+    
+      
+      
+      
+      
+      
+       
+  
+
+  
+  
+    
+  
+  
+  
+  
+   
+
+
+
\ No newline at end of file
diff --git a/doc_cn/getstarted/build_and_install/ubuntu_install_cn.html b/doc_cn/faq/cluster/index_cn.html
similarity index 60%
rename from doc_cn/getstarted/build_and_install/ubuntu_install_cn.html
rename to doc_cn/faq/cluster/index_cn.html
index 876316aca7c8214fd26d41b479fe85622935d812..2202611b8bc7f4e6f79af7b7c3ad7ac9361283a9 100644
--- a/doc_cn/getstarted/build_and_install/ubuntu_install_cn.html
+++ b/doc_cn/faq/cluster/index_cn.html
@@ -8,7 +8,7 @@
   
   
   
-  Ubuntu部署PaddlePaddle — PaddlePaddle  文档
+  集群训练与预测 — PaddlePaddle  文档
   
 
   
@@ -33,9 +33,9 @@
               href="../../genindex.html"/>
         
     
-        
-        
-         
+        
+        
+         
 
   
   
@@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -84,10 +84,11 @@
    @@ -109,14 +110,15 @@
      -
    • 新手入门 @@ -109,9 +111,10 @@ @@ -184,346 +207,16 @@
      -

      FAQ

      - -
      -

      1. 如何减少内存占用

      -

      神经网络的训练本身是一个非常消耗内存和显存的工作，经常会消耗数10GB的内存和数GB的显存。 -PaddlePaddle的内存占用主要分为如下几个方面:

      -
        -
      • DataProvider缓冲池内存（只针对内存）
        • -
      • 神经元激活内存（针对内存和显存）
        • -
      • 参数内存 （针对内存和显存）
        • -
      • 其他内存杂项
        • -
      -

      其中，其他内存杂项是指PaddlePaddle本身所用的一些内存，包括字符串分配，临时变量等等，暂不考虑在内。

      -
      -

      减少DataProvider缓冲池内存

      -

      PyDataProvider使用的是异步加载，同时在内存里直接随即选取数据来做Shuffle。即

      -digraph { - rankdir=LR; - 数据文件 -> 内存池 -> PaddlePaddle训练 -} -

      所以，减小这个内存池即可减小内存占用，同时也可以加速开始训练前数据载入的过程。但是，这 -个内存池实际上决定了shuffle的粒度。所以，如果将这个内存池减小，又要保证数据是随机的， -那么最好将数据文件在每次读取之前做一次shuffle。可能的代码为

      -
      @provider(min_pool_size=0, ...)
-def process(settings, filename):
-    os.system('shuf %s > %s.shuf' % (filename, filename))  # shuffle before.
-    with open('%s.shuf' % filename, 'r') as f:
-        for line in f:
-            yield get_sample_from_line(line)
-
      -
      -

      这样做可以极大的减少内存占用，并且可能会加速训练过程，详细文档参考 PyDataProvider2的使用 。

      -
      -
      -

      神经元激活内存

      -

      神经网络在训练的时候，会对每一个激活暂存一些数据，如神经元激活值等。 -在反向传递的时候，这些数据会被用来更新参数。这些数据使用的内存主要和两个参数有关系， -一是batch size，另一个是每条序列(Sequence)长度。所以，其实也是和每个mini-batch中包含 -的时间步信息成正比。

      -

      所以做法可以有两种：

      -
        -
      • 减小batch size。 即在网络配置中 settings(batch_size=1000) 设置成一个小一些的值。但是batch size本身是神经网络的超参数，减小batch size可能会对训练结果产生影响。
        • -
      • 减小序列的长度，或者直接扔掉非常长的序列。比如，一个数据集大部分序列长度是100-200, -但是突然有一个10000长的序列，就很容易导致内存超限，特别是在LSTM等RNN中。
        • -
      -
      -
      -

      参数内存

      -

      PaddlePaddle支持非常多的优化算法(Optimizer)，不同的优化算法需要使用不同大小的内存。 -例如使用 adadelta 算法，则需要使用等于权重参数规模大约5倍的内存。举例，如果参数保存下来的模型目录 -文件为 100M， 那么该优化算法至少需要 500M 的内存。

      -

      可以考虑使用一些优化算法，例如 momentum。

      -
      -
      -
      -

      2. 如何加速PaddlePaddle的训练速度

      -

      加速PaddlePaddle训练可以考虑从以下几个方面：

      -
        -
      • 减少数据载入的耗时
        • -
      • 加速训练速度
        • -
      • 利用分布式训练驾驭更多的计算资源
        • -
      -
      -

      减少数据载入的耗时

      -

      使用pydataprovider时，可以减少缓存池的大小，同时设置内存缓存功能，即可以极大的加速数据载入流程。 -DataProvider 缓存池的减小，和之前减小通过减小缓存池来减小内存占用的原理一致。

      -
      @provider(min_pool_size=0, ...)
-def process(settings, filename):
-    os.system('shuf %s > %s.shuf' % (filename, filename))  # shuffle before.
-    with open('%s.shuf' % filename, 'r') as f:
-        for line in f:
-            yield get_sample_from_line(line)
-
      -
      -

      同时 @provider 接口有一个 cache 参数来控制缓存方法，将其设置成 CacheType.CACHE_PASS_IN_MEM 的话，会将第一个 pass (过完所有训练数据即为一个pass)生成的数据缓存在内存里，在之后的 pass 中，不会再从 python 端读取数据，而是直接从内存的缓存里读取数据。这也会极大减少数据读入的耗时。

      -
      -
      -

      加速训练速度

      -

      PaddlePaddle支持Sparse的训练，sparse训练需要训练特征是 sparse_binary_vector 、 sparse_vector 、或者 integer_value 的任一一种。同时，与这个训练数据交互的Layer，需要将其Parameter设置成 sparse 更新模式，即设置 sparse_update=True

      -

      这里使用简单的 word2vec 训练语言模型距离，具体使用方法为:

      -

      使用一个词前两个词和后两个词，来预测这个中间的词。这个任务的DataProvider为:

      -
      DICT_DIM = 3000
-
-
-@provider(input_types=[integer_sequence(DICT_DIM), integer_value(DICT_DIM)])
-def process(settings, filename):
-    with open(filename) as f:
-        # yield word ids to predict inner word id
-        # such as [28, 29, 10, 4], 4
-        # It means the sentance is  28, 29, 4, 10, 4.
-        yield read_next_from_file(f)
-
      -
      -

      这个任务的配置为:

      -
      ...  # the settings and define data provider is omitted.
-DICT_DIM = 3000  # dictionary dimension.
-word_ids = data_layer('word_ids', size=DICT_DIM)
-
-emb = embedding_layer(
-    input=word_ids, size=256, param_attr=ParamAttr(sparse_update=True))
-emb_sum = pooling_layer(input=emb, pooling_type=SumPooling())
-predict = fc_layer(input=emb_sum, size=DICT_DIM, act=Softmax())
-outputs(
-    classification_cost(
-        input=predict, label=data_layer(
-            'label', size=DICT_DIM)))
-
      -
      -
      -
      -

      利用更多的计算资源

      -

      利用更多的计算资源可以分为一下几个方式来进行:

      -
        -
      • 单机CPU训练
          -
        • 使用多线程训练。设置命令行参数 trainer_count。
          • -
        -
        • -
      • 单机GPU训练
          -
        • 使用显卡训练。设置命令行参数 use_gpu。
          • -
        • 使用多块显卡训练。设置命令行参数 use_gpu 和 trainer_count 。
          • -
        -
        • -
      • 多机训练 -
        • -
      -
      -
      -
      -

      3. 遇到“非法指令”或者是“illegal instruction”

      -

      PaddlePaddle使用avx SIMD指令提高cpu执行效率，因此错误的使用二进制发行版可能会导致这种错误，请选择正确的版本。

      -
      -
      -

      4. 如何选择SGD算法的学习率

      -

      在采用sgd/async_sgd进行训练时，一个重要的问题是选择正确的learning_rate。如果learning_rate太大，那么训练有可能不收敛，如果learning_rate太小，那么收敛可能很慢，导致训练时间过长。

      -

      通常做法是从一个比较大的learning_rate开始试，如果不收敛，那减少学习率10倍继续试验，直到训练收敛为止。那么如何判断训练不收敛呢？可以估计出如果模型采用不变的输出最小的cost0是多少。

      -

      如果训练过程的的cost明显高于这个常数输出的cost，那么我们可以判断为训练不收敛。举一个例子，假如我们是三分类问题，采用multi-class-cross-entropy作为cost，数据中0,1,2三类的比例为 0.2, 0.5, 0.3 , 那么常数输出所能达到的最小cost是 -(0.2*log(0.2)+0.5*log(0.5)+0.3*log(0.3))=1.03 。如果训练一个pass（或者更早）后，cost还大于这个数，那么可以认为训练不收敛，应该降低学习率。

      -
      -
      -

      5. 如何初始化参数

      -

      默认情况下，PaddlePaddle使用均值0，标准差为 \(\frac{1}{\sqrt{d}}\) 来初始化参数。其中 \(d\) 为参数矩阵的宽度。这种初始化方式在一般情况下不会产生很差的结果。如果用户想要自定义初始化方式，PaddlePaddle目前提供两种参数初始化的方式:

      -
        -
      • 高斯分布。将 param_attr 设置成 param_attr=ParamAttr(initial_mean=0.0, initial_std=1.0)
        • -
      • 均匀分布。将 param_attr 设置成 param_attr=ParamAttr(initial_max=1.0, initial_min=-1.0)
        • -
      -

      比如设置一个全连接层的参数初始化方式和bias初始化方式，可以使用如下代码。

      -
      hidden = fc_layer(input=ipt, param_attr=ParamAttr(initial_max=1.0, initial_min=-1.0),
-                  bias_attr=ParamAttr(initial_mean=1.0, initial_std=0.0))
-
      -
      -

      上述代码将bias全部初始化为1.0, 同时将参数初始化为 [1.0, -1.0] 的均匀分布。

      -
      -
      -

      6. 如何共享参数

      -

      PaddlePaddle的参数使用名字 name 作为参数的ID，相同名字的参数，会共享参数。设置参数的名字，可以使用 ParamAttr(name="YOUR_PARAM_NAME") 来设置。更方便的设置方式，是使得要共享的参数使用同样的 ParamAttr 对象。

      -

      简单的全连接网络，参数共享的配置示例为:

      -
      from paddle.trainer_config_helpers import *
-
-settings(learning_rate=1e-4, batch_size=1000)
-
-a = data_layer(name='feature_a', size=200)
-b = data_layer(name='feature_b', size=200)
-
-fc_param = ParamAttr(name='fc_param', initial_max=1.0, initial_min=-1.0)
-bias_param = ParamAttr(name='bias_param', initial_mean=0.0, initial_std=0.0)
-
-softmax_param = ParamAttr(
-    name='softmax_param', initial_max=1.0, initial_min=-1.0)
-
-hidden_a = fc_layer(
-    input=a, size=200, param_attr=fc_param, bias_attr=bias_param)
-hidden_b = fc_layer(
-    input=b, size=200, param_attr=fc_param, bias_attr=bias_param)
-
-predict = fc_layer(
-    input=[hidden_a, hidden_b],
-    param_attr=[softmax_param, softmax_param],
-    bias_attr=False,
-    size=10,
-    act=SoftmaxActivation())
-
-outputs(
-    classification_cost(
-        input=predict, label=data_layer(
-            name='label', size=10)))
-
      -
      -

      这里 hidden_a 和 hidden_b 使用了同样的parameter和bias。并且softmax层的两个输入也使用了同样的参数 softmax_param。

      -
      -
      -

      7. *-cp27mu-linux_x86_64.whl is not a supported wheel on this platform.

      -

      出现这个问题的主要原因是，系统编译wheel包的时候，使用的 wheel 包是最新的， -而系统中的 pip 包比较老。具体的解决方法是，更新 pip 包并重新编译PaddlePaddle。 -更新 pip 包的方法是:

      -
      pip install --upgrade pip
-
      -
      -
      -
      -

      8. python相关的单元测试都过不了

      -

      如果出现以下python相关的单元测试都过不了的情况：

      -
      24 - test_PyDataProvider (Failed)
-26 - test_RecurrentGradientMachine (Failed)
-27 - test_NetworkCompare (Failed)
-28 - test_PyDataProvider2 (Failed)
-32 - test_Prediction (Failed)
-33 - test_Compare (Failed)
-34 - test_Trainer (Failed)
-35 - test_TrainerOnePass (Failed)
-36 - test_CompareTwoNets (Failed)
-37 - test_CompareTwoOpts (Failed)
-38 - test_CompareSparse (Failed)
-39 - test_recurrent_machine_generation (Failed)
-40 - test_PyDataProviderWrapper (Failed)
-41 - test_config_parser (Failed)
-42 - test_swig_api (Failed)
-43 - layers_test (Failed)
-
      -
      -

      并且查询PaddlePaddle单元测试的日志，提示：

      -
      paddle package is already in your PYTHONPATH. But unittest need a clean environment.
-Please uninstall paddle package before start unittest. Try to 'pip uninstall paddle'.
-
      -
      -

      解决办法是：

      -
        -
      • 卸载PaddlePaddle包 pip uninstall paddle, 清理掉老旧的PaddlePaddle安装包，使得单元测试有一个干净的环境。如果PaddlePaddle包已经在python的site-packages里面，单元测试会引用site-packages里面的python包，而不是源码目录里 /python 目录下的python包。同时，即便设置 PYTHONPATH 到 /python 也没用，因为python的搜索路径是优先已经安装的python包。
        • +

        FAQ

        +
        +
        -
        -

        9. 运行Docker GPU镜像出现 “CUDA driver version is insufficient”

        -

        用户在使用PaddlePaddle GPU的Docker镜像的时候，常常出现 Cuda Error: CUDA driver version is insufficient for CUDA runtime version, 原因在于没有把机器上CUDA相关的驱动和库映射到容器内部。 -具体的解决方法是：

        -
        $ export CUDA_SO="$(\ls usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
-$ export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
-$ docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddlepaddle:latest-gpu
-
        -
        -

        更多关于Docker的安装与使用, 请参考 PaddlePaddle Docker 文档 。

        -
        -
        -

        10. CMake源码编译, 找到的PythonLibs和PythonInterp版本不一致

        -

        这是目前CMake寻找Python的逻辑存在缺陷，如果系统安装了多个Python版本，CMake找到的Python库和Python解释器版本可能有不一致现象，导致编译PaddlePaddle失败。正确的解决方法是， -用户强制指定特定的Python版本，具体操作如下：

        -
        -
        cmake .. -DPYTHON_EXECUTABLE=<exc_path> -DPYTHON_LIBRARY=<lib_path>  -DPYTHON_INCLUDE_DIR=<inc_path>
-
        -
        -
        -

        用户需要指定本机上Python的路径：<exc_path>, <lib_path>, <inc_path>

        -
        -
        -

        10. A protocol message was rejected because it was too big

        -

        如果在训练NLP相关模型时，出现以下错误：

        -
        [libprotobuf ERROR google/protobuf/io/coded_stream.cc:171] A protocol message was rejected because it was too big (more than 67108864 bytes).  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
-F1205 14:59:50.295174 14703 TrainerConfigHelper.cpp:59] Check failed: m->conf.ParseFromString(configProtoStr)
-
        -
        -

        可能的原因是：传给dataprovider的某一个args过大，一般是由于直接传递大字典导致的。错误的define_py_data_sources2类似：

        -
        src_dict = dict()
-for line_count, line in enumerate(open(src_dict_path, "r")):
-   src_dict[line.strip()] = line_count
-
-define_py_data_sources2(
-   train_list,
-   test_list,
-   module="dataprovider",
-   obj="process",
-   args={"src_dict": src_dict})
-
        -
        -

        解决方案是：将字典的地址作为args传给dataprovider，然后在dataprovider里面根据该地址加载字典。即define_py_data_sources2应改为：

        -
        define_py_data_sources2(
-   train_list,
-   test_list,
-   module="dataprovider",
-   obj="process",
-   args={"src_dict_path": src_dict_path})
-
        -
        -

        完整源码可参考 seqToseq 示例。

        -
        -
        -

        11. 如何指定GPU设备

        -

        例如机器上有4块GPU，编号从0开始，指定使用2、3号GPU：

        - -
        env CUDA_VISIBLE_DEVICES=2,3 paddle train --use_gpu=true --trainer_count=2
-
        -
        -
          -
        • 方式2：通过命令行参数 --gpu_id 指定。
          • -
        -
        paddle train --use_gpu=true --trainer_count=2 --gpu_id=2
-
        -
        -
        -
        -

        12. 训练过程中出现 Floating point exception, 训练因此退出怎么办?

        -

        Paddle二进制在运行时捕获了浮点数异常，只要出现浮点数异常(即训练过程中出现NaN或者Inf)，立刻退出。浮点异常通常的原因是浮点数溢出、除零等问题。 -主要原因包括两个方面:

        -
          -
        • 训练过程中参数或者训练过程中的梯度尺度过大，导致参数累加，乘除等时候，导致了浮点数溢出。
          • -
        • 模型一直不收敛，发散到了一个数值特别大的地方。
          • -
        • 训练数据有问题，导致参数收敛到了一些奇异的情况。或者输入数据尺度过大，有些特征的取值达到数百万，这时进行矩阵乘法运算就可能导致浮点数溢出。
          • -
        -

        主要的解决办法是减小学习律或者对数据进行归一化处理。

        -
      @@ -533,6 +226,8 @@ F1205 14:59:50.295174 14703 Traine
      - Folk me on Github + Fork me on Github
      English @@ -86,6 +86,7 @@
    • 进阶指南
    • API
    • FAQ
      • +
    • MOBILE
    @@ -109,9 +110,10 @@ @@ -187,28 +209,54 @@

    索引

    - D - | E + B + | C + | L + | M | P + | R | S + | T
    -

    D

    +

    B

    + + +
    + +

    C

    -

    E

    +

    L

    + +
    + +

    M

    +
    @@ -216,15 +264,23 @@

    P

    + +
    + +

    R

    + +
    @@ -232,7 +288,19 @@

    S

    + +
    + +

    T

    + +
    diff --git a/doc_cn/getstarted/basic_usage/index_cn.html b/doc_cn/getstarted/basic_usage/index_cn.html deleted file mode 100644 index 45ba93632ff18370891c9e5b6704b1b06b7b7fc2..0000000000000000000000000000000000000000 --- a/doc_cn/getstarted/basic_usage/index_cn.html +++ /dev/null @@ -1,336 +0,0 @@ - - - - - - - - - - - 经典的线性回归任务 — PaddlePaddle 文档 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    -
      - -
    • 经典的线性回归任务
      • -
    -
    - -
    -
    -
    -
    - -
    -

    经典的线性回归任务

    -

    PaddlePaddle是源于百度的一个深度学习平台。这份简短的介绍将向你展示如何利用PaddlePaddle来解决一个经典的线性回归问题。

    -
    -

    任务简介

    -

    我们展示如何用PaddlePaddle解决 单变量的线性回归 问题。线性回归的输入是一批点 (x, y) ，其中 y = wx + b + ε， 而 ε 是一个符合高斯分布的随机变量。线性回归的输出是从这批点估计出来的参数 w 和 b 。

    -

    一个例子是房产估值。我们假设房产的价格（y）是其大小（x）的一个线性函数，那么我们可以通过收集市场上房子的大小和价格，用来估计线性函数的参数w 和 b。

    -
    -
    -

    准备数据

    -

    假设变量 x 和 y 的真实关系为： y = 2x + 0.3 + ε，这里展示如何使用观测数据来拟合这一线性关系。首先，Python代码将随机产生2000个观测点，作为线性回归的输入。下面脚本符合PaddlePaddle期待的读取数据的Python程序的模式。

    -
    # dataprovider.py
-from paddle.trainer.PyDataProvider2 import *
-import random
-
-# 定义输入数据的类型: 2个浮点数
-@provider(input_types=[dense_vector(1), dense_vector(1)],use_seq=False)
-def process(settings, input_file):
-    for i in xrange(2000):
-        x = random.random()
-        yield [x], [2*x+0.3]
-
    -
    -
    -
    -

    训练模型

    -

    为了还原 y = 2x + 0.3，我们先从一条随机的直线 y’ = wx + b 开始，然后利用观测数据调整 w 和 b 使得 y’ 和 y 的差距不断减小，最终趋于接近。这个过程就是模型的训练过程，而 w 和 b 就是模型的参数，即我们的训练目标。

    -

    在PaddlePaddle里，该模型的网络配置如下。

    -
    # trainer_config.py
-from paddle.trainer_config_helpers import *
-
-# 1. 定义数据来源，调用上面的process函数获得观测数据
-data_file = 'empty.list'
-with open(data_file, 'w') as f: f.writelines(' ')
-define_py_data_sources2(train_list=data_file, test_list=None,
-                        module='dataprovider', obj='process',args={})
-
-# 2. 学习算法。控制如何改变模型参数 w 和 b
-settings(batch_size=12, learning_rate=1e-3, learning_method=MomentumOptimizer())
-
-# 3. 神经网络配置
-x = data_layer(name='x', size=1)
-y = data_layer(name='y', size=1)
-# 线性计算网络层: ȳ = wx + b
-ȳ = fc_layer(input=x, param_attr=ParamAttr(name='w'), size=1, act=LinearActivation(), bias_attr=ParamAttr(name='b'))
-# 计算误差函数，即  ȳ 和真实 y 之间的距离
-cost = mse_cost(input= ȳ, label=y)
-outputs(cost)
-
    -
    -

    这段简短的配置展示了PaddlePaddle的基本用法：

    -
      -

    • 第一部分定义了数据输入。一般情况下，PaddlePaddle先从一个文件列表里获得数据文件地址，然后交给用户自定义的函数（例如上面的 process`函数）进行读入和预处理从而得到真实输入。本文中由于输入数据是随机生成的不需要读输入文件，所以放一个空列表（`empty.list）即可。

      -
      • -

    • 第二部分主要是选择学习算法，它定义了模型参数改变的规则。PaddlePaddle提供了很多优秀的学习算法，这里使用一个基于momentum的随机梯度下降(SGD)算法，该算法每批量(batch)读取12个采样数据进行随机梯度计算来更新更新。

      -
      • -

    • 最后一部分是神经网络的配置。由于PaddlePaddle已经实现了丰富的网络层，所以很多时候你需要做的只是定义正确的网络层并把它们连接起来。这里使用了三种网络单元：

      -
      -
        -
      • 数据层：数据层 data_layer 是神经网络的入口，它读入数据并将它们传输到接下来的网络层。这里数据层有两个，分别对应于变量 x 和 y。
        • -
      • 全连接层：全连接层 fc_layer 是基础的计算单元，这里利用它建模变量之间的线性关系。计算单元是神经网络的核心，PaddlePaddle支持大量的计算单元和任意深度的网络连接，从而可以拟合任意的函数来学习复杂的数据关系。
        • -
      • 回归误差代价层：回归误差代价层 mse_cost 是众多误差代价函数层的一种，它们在训练过程作为网络的出口，用来计算模型的误差，是模型参数优化的目标函数。
        • -
      -
      -
      • -
    -

    定义了网络结构并保存为 trainer_config.py 之后，运行以下训练命令：

    -
    paddle train --config=trainer_config.py --save_dir=./output --num_passes=30
-
    -
    -

    PaddlePaddle将在观测数据集上迭代训练30轮，并将每轮的模型结果存放在 ./output 路径下。从输出日志可以看到，随着轮数增加误差代价函数的输出在不断的减小，这意味着模型在训练数据上不断的改进，直到逼近真实解：` y = 2x + 0.3 `

    -
    -
    -

    模型检验

    -

    训练完成后，我们希望能够检验模型的好坏。一种常用的做法是用学习的模型对另外一组测试数据进行预测，评价预测的效果。在这个例子中，由于已经知道了真实答案，我们可以直接观察模型的参数是否符合预期来进行检验。

    -

    PaddlePaddle将每个模型参数作为一个numpy数组单独存为一个文件，所以可以利用如下方法读取模型的参数。

    -
    import numpy as np
-import os
-
-def load(file_name):
-    with open(file_name, 'rb') as f:
-        f.read(16) # skip header for float type.
-        return np.fromfile(f, dtype=np.float32)
-
-print 'w=%.6f, b=%.6f' % (load('output/pass-00029/w'), load('output/pass-00029/b'))
-# w=1.999743, b=0.300137
-
    -
    -../../_images/parameters.png -

    从图中可以看到，虽然 w 和 b 都使用随机值初始化，但在起初的几轮训练中它们都在快速逼近真实值，并且后续仍在不断改进，使得最终得到的模型几乎与真实模型一致。

    -

    这样，我们用PaddlePaddle解决了单变量线性回归问题， 包括数据输入、模型训练和最后的结果验证。

    -
    -
    - - -
    -
    -
    - - -
    - -
    -

    - © Copyright 2016, PaddlePaddle developers. - -

    -
    - Built with Sphinx using a theme provided by Read the Docs. - -
    - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc_cn/getstarted/build_and_install/build_from_source_cn.html b/doc_cn/getstarted/build_and_install/build_from_source_cn.html new file mode 100644 index 0000000000000000000000000000000000000000..a4c933200f7337b26ba8ed3b9151bf8d6bf63e35 --- /dev/null +++ b/doc_cn/getstarted/build_and_install/build_from_source_cn.html @@ -0,0 +1,470 @@ + + + + + + + + + + + 从源码编译 — PaddlePaddle 文档 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + +
    +
    + Fork me on Github +
    + + English + + + + +
    + +
    +
    + + + + +
    +
    + + + +
    +
    +
    +
    +
    + +
    + + + + +
    + + + + + + + + + + + +
    + +
    + +
    +
    +
    +
    + +
    +

    从源码编译

    +
    +

    编译方法

    +

    PaddlePaddle主要使用 CMake 以及GCC, G++作为编译工具。 +我们推荐您使用PaddlePaddle Docker编译环境镜像完成编译，这样可以免去单独安装编译依赖的步骤，可选的不同编译环境Docker镜像 +可以在 这里 找到。

    +

    如果您选择不使用Docker镜像，则需要在本机安装下面章节列出的 编译依赖 之后才能开始编译的步骤。

    +

    编译PaddlePaddle，需要执行：

    +
    git clone https://github.com/PaddlePaddle/Paddle.git
+cd Paddle
+# 如果使用Docker编译环境，执行下面的命令编译CPU-Only的二进制
+docker run -it -v $PWD:/paddle -e "WITH_GPU=OFF" -e "WITH_TESTING=OFF" paddlepaddle/paddle_manylinux_devel:cuda8.0_cudnn5 bash -x /paddle/paddle/scripts/docker/build.sh
+# 如果不使用Docker编译环境，执行下面的命令
+mkdir build
+cd build
+cmake -DWITH_GPU=OFF -DWITH_TESTING=OFF ..
+make
+
    +
    +

    编译完成后会在build/python/dist目录下生成输出的whl包，可以选在在当前机器安装也可以拷贝到目标机器安装：

    +
    pip install build/python/dist/*.whl
+
    +
    +
    +
    +

    执行单元测试

    +

    如果您期望在编译完成后立即执行所有的单元测试，可以按照下面的方法：

    +

    使用Docker的情况下，设置 RUN_TEST=ON 和 WITH_TESTING=ON 就会在完成编译之后，立即执行单元测试。 +开启 WITH_GPU=ON 可以指定同时执行GPU上的单元测试。

    +
    docker run -it -v $PWD:/paddle -e "WITH_GPU=OFF" -e "WITH_TESTING=ON" -e "RUN_TEST=ON" paddlepaddle/paddle_manylinux_devel:cuda8.0_cudnn5 bash -x /paddle/paddle/scripts/docker/build.sh
+
    +
    +

    如果不使用Docker，可以执行ctest命令即可：

    +
    mkdir build
+cd build
+cmake -DWITH_GPU=OFF -DWITH_TESTING=OFF ..
+make
+ctest
+# 指定执行其中一个单元测试 test_mul_op
+ctest -R test_mul_op
+
    +
    +
    +
    +

    编译依赖

    +

    PaddlePaddle编译需要使用到下面的依赖（包含但不限于），其他的依赖软件，会自动在编译时下载。

    + + +++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    PaddlePaddle编译依赖
    依赖版本说明
    CMake>=3.5 
    GCC4.8.2推荐使用CentOS的devtools2
    Python2.7.x依赖libpython2.7.so
    pip>=9.0
    numpy 
    SWIG>=2.0 
    Go>=1.8可选
    +
    +
    +

    编译选项

    +

    PaddlePaddle的编译选项，包括生成CPU/GPU二进制文件、链接何种BLAS库等。 +用户可在调用cmake的时候设置它们，详细的cmake使用方法可以参考 +官方文档 。

    +

    在cmake的命令行中，通过使用 -D 命令设置该类编译选项，例如：

    +
    cmake .. -DWITH_GPU=OFF
+
    +
    + + +++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    编译选项说明
    选项说明默认值
    WITH_GPU是否支持GPUON
    WITH_C_API是否仅编译CAPIOFF
    WITH_DOUBLE是否使用双精度浮点数OFF
    WITH_DSO是否运行时动态加载CUDA动态库，而非静态加载CUDA动态库。ON
    WITH_AVX是否编译含有AVX指令集的PaddlePaddle二进制文件ON
    WITH_PYTHON是否内嵌PYTHON解释器ON
    WITH_STYLE_CHECK是否编译时进行代码风格检查ON
    WITH_TESTING是否开启单元测试ON
    WITH_DOC是否编译中英文文档OFF
    WITH_SWIG_PY是否编译PYTHON的SWIG接口，该接口可用于预测和定制化训练Auto
    WITH_GOLANG是否编译go语言的可容错parameter serverON
    WITH_MKL是否使用MKL数学库，如果为否则是用OpenBLASON
    +
    +

    BLAS

    +

    PaddlePaddle支持 MKL 和 +OpenBlAS 两种BLAS库。默认使用MKL。如果使用MKL并且机器含有AVX2指令集， +还会下载MKL-DNN数学库，详细参考 这里 。

    +

    如果关闭MKL，则会使用OpenBLAS作为BLAS库。

    +
    +
    +

    CUDA/cuDNN

    +

    PaddlePaddle在编译时/运行时会自动找到系统中安装的CUDA和cuDNN库进行编译和执行。 +使用参数 -DCUDA_ARCH_NAME=Auto 可以指定开启自动检测SM架构，加速编译。

    +

    PaddlePaddle可以使用cuDNN v5.1之后的任何一个版本来编译运行，但尽量请保持编译和运行使用的cuDNN是同一个版本。 +我们推荐使用最新版本的cuDNN。

    +
    +
    +

    编译选项的设置

    +

    PaddePaddle通过编译时指定路径来实现引用各种BLAS/CUDA/cuDNN库。cmake编译时，首先在系统路径（ /usr/lib:/usr/local/lib ）中搜索这几个库，同时也会读取相关路径变量来进行搜索。 通过使用 -D 命令可以设置，例如

    +
    cmake .. -DWITH_GPU=ON -DWITH_TESTING=OFF -DCUDNN_ROOT=/opt/cudnnv5
+
    +
    +

    注意：这几个编译选项的设置，只在第一次cmake的时候有效。如果之后想要重新设置，推荐清理整个编译目录（ rm -rf ）**后，再指定。**

    +
    +
    +
    + + +
    +
    + + +
    +
    + +
    + +
    + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc_cn/getstarted/build_and_install/cmake/build_from_source_cn.html b/doc_cn/getstarted/build_and_install/cmake/build_from_source_cn.html deleted file mode 100644 index 3603966b1ae5eae6aab4e3bb6652a6801b49ee41..0000000000000000000000000000000000000000 --- a/doc_cn/getstarted/build_and_install/cmake/build_from_source_cn.html +++ /dev/null @@ -1,377 +0,0 @@ - - - - - - - - - - - PaddlePaddle的编译选项 — PaddlePaddle 文档 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    - -
    - -
    -
    -
    -
    - -
    -

    PaddlePaddle的编译选项

    -

    PaddlePaddle的编译选项，包括生成CPU/GPU二进制文件、链接何种BLAS库等。用户可在调用cmake的时候设置它们，详细的cmake使用方法可以参考 官方文档 。

    -
    -

    Bool型的编译选项

    -

    用户可在cmake的命令行中，通过使用 -D 命令设置该类编译选项，例如

    -
    cmake .. -DWITH_GPU=OFF
-
    -
    - - ----- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Bool型的编译选项
    选项说明默认值
    WITH_GPU是否支持GPU。取决于是否寻找到CUDA工具链
    WITH_DOUBLE是否使用双精度浮点数。否
    WITH_DSO是否运行时动态加载CUDA动态库，而非静态加载CUDA动态库。是
    WITH_AVX是否编译含有AVX指令集的PaddlePaddle二进制文件是
    WITH_PYTHON是否内嵌PYTHON解释器。方便今后的嵌入式移植工作。是
    WITH_STYLE_CHECK是否编译时进行代码风格检查是
    WITH_RDMA是否开启RDMA否
    WITH_TIMER是否开启计时功能。如果开启会导致运行略慢，打印的日志变多，但是方便调试和测Benchmark否
    WITH_TESTING是否开启单元测试取决于是否寻找到GTEST
    WITH_DOC是否编译中英文文档否
    WITH_SWIG_PY是否编译PYTHON的SWIG接口，该接口可用于预测和定制化训练取决于是否寻找到SWIG
    -
    -
    -

    BLAS/CUDA/Cudnn的编译选项

    -
    -

    BLAS

    -

    PaddlePaddle支持以下任意一种BLAS库：MKL ，ATLAS ，OpenBlAS 和 REFERENCE BLAS 。

    - - ----- - - - - - - - - - - - - - - - - - - - - - - -
    BLAS路径相关的编译选项
    编译选项描述注意
    MKL_ROOTMKL的路径${MKL_ROOT}/include下需要包含mkl.h，${MKL_ROOT}/lib目录下需要包含mkl_core，mkl_sequential和mkl_intel_lp64三个库。
    ATLAS_ROOTATLAS的路径${ATLAS_ROOT}/include下需要包含cblas.h，${ATLAS_ROOT}/lib下需要包含cblas和atlas两个库。
    OPENBLAS_ROOTOpenBLAS的路径${OPENBLAS_ROOT}/include下需要包含cblas.h，${OPENBLAS_ROOT}/lib下需要包含openblas库。
    REFERENCE_CBLAS_ROOTREFERENCE BLAS的路径${REFERENCE_CBLAS_ROOT}/include下需要包含cblas.h，${REFERENCE_CBLAS_ROOT}/lib下需要包含cblas库。
    -
    -
    -

    CUDA/Cudnn

    -

    PaddlePaddle可以使用cudnn v2之后的任何一个版本来编译运行，但尽量请保持编译和运行使用的cudnn是同一个版本。 我们推荐使用最新版本的cudnn v5.1。

    -
    -
    -

    编译选项的设置

    -

    PaddePaddle通过编译时指定路径来实现引用各种BLAS/CUDA/Cudnn库。cmake编译时，首先在系统路径(/usr/lib:/usr/local/lib)中搜索这几个库，同时也会读取相关路径变量来进行搜索。 通过使用 -D 命令可以设置，例如

    -
    cmake .. -DMKL_ROOT=/opt/mkl/ -DCUDNN_ROOT=/opt/cudnnv5
-
    -
    -

    注意：这几个编译选项的设置，只在第一次cmake的时候有效。如果之后想要重新设置，推荐清理整个编译目录（rm -rf）后，再指定。

    -
    -
    -
    - - -
    -
    - - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc_cn/getstarted/build_and_install/docker_install_cn.html b/doc_cn/getstarted/build_and_install/docker_install_cn.html index 2725e55a742fe5b961fece6ba325f466728ecf69..a808c5d4a58f62ce7b0d902c1018e08c8c99fef8 100644 --- a/doc_cn/getstarted/build_and_install/docker_install_cn.html +++ b/doc_cn/getstarted/build_and_install/docker_install_cn.html @@ -8,7 +8,7 @@ - PaddlePaddle的Docker容器使用方式 — PaddlePaddle 文档 + 使用Docker安装运行 — PaddlePaddle 文档 @@ -34,8 +34,8 @@ - - + + @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -180,7 +202,7 @@
  • 安装与编译 >
    • -
  • PaddlePaddle的Docker容器使用方式
    • +
  • 使用Docker安装运行
    • @@ -189,146 +211,122 @@
    -
    -

    PaddlePaddle的Docker容器使用方式

    -

    PaddlePaddle目前唯一官方支持的运行的方式是Docker容器。因为Docker能在所有主要操作系统（包括Linux，Mac OS X和Windows）上运行。 请注意，您需要更改 Dockers设置 才能充分利用Mac OS X和Windows上的硬件资源。

    -
    -

    PaddlePaddle发布的Docker镜像使用说明

    -

    我们把PaddlePaddle的编译环境打包成一个镜像，称为开发镜像，里面涵盖了 -PaddlePaddle需要的所有编译工具。把编译出来的PaddlePaddle也打包成一个镜 -像，称为生产镜像，里面涵盖了PaddlePaddle运行所需的所有环境。每次 -PaddlePaddle发布新版本的时候都会发布对应版本的生产镜像以及开发镜像。运 -行镜像包括纯CPU版本和GPU版本以及其对应的非AVX版本。我们会在 -dockerhub.com 提供最新 -的Docker镜像，可以在”tags”标签下找到最新的Paddle镜像版本。为了方便在国 -内的开发者下载Docker镜像，我们提供了国内的镜像服务器供大家使用。如果您 -在国内，请把文档里命令中的paddlepaddle/paddle替换成 -docker.paddlepaddle.org/paddle。

    -
      -

    1. 开发镜像：paddlepaddle/paddle:<version>-dev

      -

      这个镜像包含了Paddle相关的开发工具以及编译和运行环境。用户可以使用开发镜像代替配置本地环境，完成开发，编译，发布， -文档编写等工作。由于不同的Paddle的版本可能需要不同的依赖和工具，所以如果需要自行配置开发环境需要考虑版本的因素。 -开发镜像包含了以下工具：

      -
        -
      • gcc/clang
        • -
      • nvcc
        • -
      • Python
        • -
      • sphinx
        • -
      • woboq
        • -
      • sshd
        • -
      -

      很多开发者会使用远程的安装有GPU的服务器工作，用户可以使用ssh登录到这台服务器上并执行 :code:`docker exec`进入开发镜像并开始工作， -也可以在开发镜像中启动一个SSHD服务，方便开发者直接登录到镜像中进行开发:

      -

      以交互容器方式运行开发镜像：

      -
      docker run -it --rm paddlepaddle/paddle:<version>-dev /bin/bash
+  
      
+
      使用Docker安装运行
      
+
      使用Docker安装和运行PaddlePaddle可以无需考虑依赖环境即可运行。并且也可以在Windows的docker中运行。
+您可以在 Docker官网 获得基本的Docker安装和使用方法。
      
+
      如果您在使用Windows，可以参考
+这篇
+教程，完成在Windows上安装和使用Docker。
      
+
      在了解Docker的基本使用方法之后，即可开始下面的步骤：
      
+
      
+
      获取PaddlePaddle的Docker镜像
      
+
      执行下面的命令获取最新的PaddlePaddle Docker镜像
      
+
      
+
      docker pull paddlepaddle/paddle
 
      
 
      
-
      或者，可以以后台进程方式运行容器：
      
-
      docker run -d -p 2202:22 -p 8888:8888 paddledev/paddle:<version>-dev
+
      
+
      对于国内用户，我们提供了加速访问的镜像源：
      
+
      
+
      docker pull docker.paddlepaddle.org/paddle
 
      
 
      
-
      然后用密码 root SSH进入容器：
      
-
      ssh -p 2202 root@localhost
+
      
+
      下载GPU版本的Docker镜像：
      
+
      
+
      docker pull paddlepaddle/paddle:latest-gpu
+docker pull docker.paddlepaddle.org/paddle:latest-gpu
 
      
 
      
-
      SSH方式的一个优点是我们可以从多个终端进入容器。比如，一个终端运行vi，另一个终端运行Python。另一个好处是我们可以把PaddlePaddle容器运行在远程服务器上，并在笔记本上通过SSH与其连接。
      
-
      2. -

    2. 生产镜像：根据CPU、GPU和非AVX区分了如下4个镜像：

      -
        -
      • GPU/AVX：paddlepaddle/paddle:<version>-gpu
        • -
      • GPU/no-AVX：paddlepaddle/paddle:<version>-gpu-noavx
        • -
      • CPU/AVX：paddlepaddle/paddle:<version>
        • -
      • CPU/no-AVX：paddlepaddle/paddle:<version>-noavx
        • -
      -

      纯CPU镜像以及GPU镜像都会用到AVX指令集，但是2008年之前生产的旧电脑不支持AVX。以下指令能检查Linux电脑是否支持AVX：

      -
      if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
+
      +

      选择下载使用不同的BLAS库的Docker镜像：

      +
      +
      # 默认是使用MKL的镜像
+docker pull paddlepaddle/paddle
+# 使用OpenBLAS的镜像
+docker pull paddlepaddle/paddle:latest-openblas
      -

      如果输出是No，就需要选择使用no-AVX的镜像

      -

      以上方法在GPU镜像里也能用，只是请不要忘记提前在物理机上安装GPU最新驱动。 -为了保证GPU驱动能够在镜像里面正常运行，我们推荐使用[nvidia-docker](https://github.com/NVIDIA/nvidia-docker)来运行镜像。

      -
      nvidia-docker run -it --rm paddledev/paddle:0.10.0rc1-gpu /bin/bash
+
      +

      下载指定版本的Docker镜像，可以从 DockerHub网站 获取可选的tag，并执行下面的命令：

      +
      +
      docker pull paddlepaddle/paddle:[tag]
+# 比如：
+docker pull docker.paddlepaddle.org/paddle:0.10.0-gpu
      -

      注意: 如果使用nvidia-docker存在问题，你也许可以尝试更老的方法，具体如下，但是我们并不推荐这种方法。：

      -
      export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
-export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
-docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:<version>-gpu
-
      +
      -
      3. -

    3. 运行以及发布您的AI程序

      -

      假设您已经完成了一个AI训练的python程序 a.py，这个程序是您在开发机上使用开发镜像完成开发。此时您可以运行这个命令在开发机上进行测试运行：

      -
      docker run -it -v $PWD:/work paddle /work/a.py
-
      -
      -

      如果要使用GPU，请运行：

      -
      nvidia-docker run -it -v $PWD:/work paddle /work/a.py
+
      
+
      在Docker中执行PaddlePaddle训练程序
      
+
      假设您已经在当前目录（比如在/home/work）编写了一个PaddlePaddle的程序 train.py （可以参考
+PaddlePaddleBook
+编写），就可以使用下面的命令开始执行训练：
      
+
      
+
      cd /home/work
+docker run -it -v $PWD:/work paddlepaddle/paddle /work/train.py
 
      
 
      
-
      这里`a.py`包含的所有依赖假设都可以在Paddle的运行容器中。如果需要包含更多的依赖、或者需要发布您的应用的镜像，可以编写`Dockerfile`使用`FROM paddledev/paddle:<version>`
-创建和发布自己的AI程序镜像。
      
-
      4. -
    +
    +

    上述命令中， -it 参数说明容器已交互式运行； -v $PWD:/work +指定将当前路径（Linux中$PWD变量会展开为当前路径的绝对路径）挂载到容器内部的 /work +目录； paddlepaddle/paddle 指定需要使用的容器； 最后 /work/train.py +为容器内执行的命令，即运行训练程序。

    +

    当然，您也可以进入到Docker容器中，以交互式的方式执行或调试您的代码：

    +
    +
    +

    注：PaddlePaddle Docker镜像为了减小体积，默认没有安装vim，您可以在容器中执行 apt-get install -y vim 安装后，在容器中编辑代码。

    -
    -

    运行PaddlePaddle Book

    -

    Jupyter Notebook是一个开源的web程序，大家可以通过它制作和分享带有代码、公式、图表、文字的交互式文档。用户可以通过网页浏览文档。

    -

    PaddlePaddle Book是为用户和开发者制作的一个交互式的Jupyter Nodebook。 -如果您想要更深入了解deep learning，PaddlePaddle Book一定是您最好的选择。

    +
    +

    使用Docker启动PaddlePaddle Book教程

    +

    使用Docker可以快速在本地启动一个包含了PaddlePaddle官方Book教程的Jupyter Notebook，可以通过网页浏览。 +PaddlePaddle Book是为用户和开发者制作的一个交互式的Jupyter Notebook。 +如果您想要更深入了解deep learning，PaddlePaddle Book一定是您最好的选择。 +大家可以通过它阅读教程，或者制作和分享带有代码、公式、图表、文字的交互式文档。

    我们提供可以直接运行PaddlePaddle Book的Docker镜像，直接运行：

    -
    docker run -p 8888:8888 paddlepaddle/book
+
    
+
    docker run -p 8888:8888 paddlepaddle/book
 
    
 
    
+
    
 
    然后在浏览器中输入以下网址：
    
-
    http://localhost:8888/
+
    
+
    http://localhost:8888/
 
    
 
    
+
    
 
    就这么简单，享受您的旅程！
    
 
    
-
    
-
    通过Docker容器开发PaddlePaddle
    
-
    开发人员可以在Docker开发镜像中开发PaddlePaddle。这样开发人员可以以一致的方式在不同的平台上工作 - Linux，Mac OS X和Windows。
    
-
      
-
    1. 制作PaddlePaddle开发镜像
      
-
      PaddlePaddle每次发布新版本都会发布对应的开发镜像供开发者直接使用。这里介绍如生成造这个开发镜像。
-生成Docker镜像的方式有两个，一个是直接把一个容器转换成镜像，另一个是创建Dockerfile并运行docker build指令按照Dockerfile生成镜像。第一个方法的好处是简单快捷，适合自己实验，可以快速迭代。第二个方法的好处是Dockerfile可以把整个生成流程描述很清楚，其他人很容易看懂镜像生成过程，持续集成系统也可以简单地复现这个过程。我们采用第二个方法。Dockerfile位于PaddlePaddle repo的根目录。生成生产镜像只需要运行：
      
-
      git clone https://github.com/PaddlePaddle/Paddle.git
-cd Paddle
-docker build -t paddle:dev .
-
      
-
      
-
      docker build这个命令的-t指定了生成的镜像的名字，这里我们用paddle:dev。到此，PaddlePaddle开发镜像就被构建完毕了。
      
-
      2. 
-
    2. 制作PaddlePaddle生产镜像
      
-
      生产镜像的生成分为两步，第一步是运行：
      
-
      docker run -v $(pwd):/paddle -e "WITH_GPU=OFF" -e "WITH_AVX=OFF" -e "WITH_TEST=ON" paddle:dev
+
      
+
      使用Docker执行GPU训练
      
+
      为了保证GPU驱动能够在镜像里面正常运行，我们推荐使用
+nvidia-docker 来运行镜像。
+请不要忘记提前在物理机上安装GPU最新驱动。
      
+
      
+
      nvidia-docker run -it -v $PWD:/work paddledev/paddle:latest-gpu /bin/bash
 
      
 
      
-
      以上命令会编译PaddlePaddle，生成运行程序，以及生成创建生产镜像的Dockerfile。所有生成的的文件都在build目录下。“WITH_GPU”控制生成的生产镜像是否支持GPU，“WITH_AVX”控制生成的生产镜像是否支持AVX，”WITH_TEST“控制是否生成单元测试。
      
-
      第二步是运行：
      
-
      docker build -t paddle:prod -f build/Dockerfile ./build
+
      
+
      注: 如果没有安装nvidia-docker，可以尝试以下的方法，将CUDA库和Linux设备挂载到Docker容器内：
      
+
      
+
      export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
+export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
+docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:latest-gpu
 
      
 
      
-
      以上命令会按照生成的Dockerfile把生成的程序拷贝到生产镜像中并做相应的配置，最终生成名为paddle:prod的生产镜像。
      
-
      3. 
-
    3. 运行单元测试
      
-
      运行以下指令：
      
-
      docker run -it -v $(pwd):/paddle paddle:dev bash -c "cd /paddle/build && ctest"
+
      
+
      关于AVX：
      
+
      AVX是一种CPU指令集，可以加速PaddlePaddle的计算。最新的PaddlePaddle Docker镜像默认
+是开启AVX编译的，所以，如果您的电脑不支持AVX，需要单独
+编译 PaddlePaddle为no-avx版本。
      
+
      以下指令能检查Linux电脑是否支持AVX：
      
+
      
+
      if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
 
      
 
      
-
      4. 
-
    
-
    
-
    
-
    文档
    
-
    Paddle的Docker开发镜像带有一个通过 woboq code browser 生成的HTML版本的C++源代码，便于用户浏览C++源码。
    
-
    只要在Docker里启动PaddlePaddle的时候给它一个名字，就可以再运行另一个Nginx Docker镜像来服务HTML代码：
    
-
    docker run -d --name paddle-cpu-doc paddle:<version>-dev
-docker run -d --volumes-from paddle-cpu-doc -p 8088:80 nginx
-
    
-
    
-
    接着我们就能够打开浏览器在 http://localhost:8088/paddle/ 浏览代码。
    
+
    
+
    如果输出是No，就需要选择使用no-AVX的镜像
    @@ -339,10 +337,10 @@ docker run -d --volumes-from paddle-cpu-doc -p 8088:80 ng diff --git a/doc_cn/getstarted/build_and_install/index_cn.html b/doc_cn/getstarted/build_and_install/index_cn.html index 6ea1e19f489d034b2d35ca3a74828d0379a467a7..bef3413af79742e00fdc30a886865991cc656ca8 100644 --- a/doc_cn/getstarted/build_and_install/index_cn.html +++ b/doc_cn/getstarted/build_and_install/index_cn.html @@ -34,7 +34,7 @@ - + @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -191,11 +213,12 @@

    安装与编译

    安装流程

    -

    PaddlePaddle提供数个预编译的二进制来进行安装，包括Docker镜像，ubuntu的deb安装包等。我们推荐使用Docker镜像来部署环境，同时欢迎贡献更多的安装包。

    +

    PaddlePaddle提供pip和Docker的安装方式：

    @@ -203,14 +226,18 @@

    编译流程

    警告

    -

    编译流程主要推荐高级用户查看，普通用户请走安装流程。

    +

    建议直接使用上述安装流程，方便快速安装。只有在遇到需要独立定制的二进制时才需要编译。

    +
    +

    常见问题解答

    +

    常见问题解答

    +
    @@ -220,7 +247,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -274,10 +296,10 @@ paddle.init(use_gpu
    y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear())
-cost = paddle.layer.mse_cost(input=y_predict, label=y)
+cost = paddle.layer.square_error_cost(input=y_predict, label=y)
    -

    其中，x与y为之前描述的输入层；而y_predict是接收x作为输入，接上一个全连接层；cost接收y_predict与y作为输入，接上均方误差层。

    +

    其中，x与y为之前描述的输入层；而y_predict是接收x作为输入，接上一个全连接层；cost接收y_predict与y作为输入，接上平方误差层。

    最后一层cost中记录了神经网络的所有拓扑结构，通过组合不同的layer，我们即可完成神经网络的搭建。

    @@ -297,7 +319,7 @@ paddle.init(use_gpu
    # define training dataset reader
 def train_reader():
     train_x = np.array([[1, 1], [1, 2], [3, 4], [5, 2]])
-    train_y = np.array([-2, -3, -7, -7])
+    train_y = np.array([[-2], [-3], [-7], [-7]])
     def reader():
         for i in xrange(train_y.shape[0]):
             yield train_x[i], train_y[i]
@@ -388,7 +410,7 @@ trainer.train(
 x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(2))
 y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear())
 y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1))
-cost = paddle.layer.mse_cost(input=y_predict, label=y)
+cost = paddle.layer.square_error_cost(input=y_predict, label=y)
 
 # create parameters
 parameters = paddle.parameters.create(cost)
@@ -411,7 +433,7 @@ trainer.train(
 # define training dataset reader
 def train_reader():
     train_x = np.array([[1, 1], [1, 2], [3, 4], [5, 2]])
-    train_y = np.array([-2, -3, -7, -7])
+    train_y = np.array([[-2], [-3], [-7], [-7]])
 
     def reader():
         for i in xrange(train_y.shape[0]):
@@ -446,7 +468,7 @@ trainer.train(
         Next 
       
       
-         Previous
+         Previous
    diff --git a/doc_cn/getstarted/index_cn.html b/doc_cn/getstarted/index_cn.html index a68ca1600fa566712bc915be888edbb19e9a532b..26e7503621e54e0988f0ed7d0f3eb49badda1957 100644 --- a/doc_cn/getstarted/index_cn.html +++ b/doc_cn/getstarted/index_cn.html @@ -64,7 +64,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -87,6 +87,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -110,9 +111,10 @@ @@ -186,15 +208,59 @@

    新手入门

    +
    +

    快速安装

    +

    PaddlePaddle支持使用pip快速安装，目前支持CentOS 6以上, Ubuntu 14.04以及MacOS 10.12，并安装有Python2.7。 +执行下面的命令完成快速安装：

    +
    +
    pip install paddlepaddle
+
    +
    +
    +

    如果需要安装支持GPU的版本，需要执行：

    +
    +
    pip install paddlepaddle-gpu
+
    +
    +
    +

    更详细的安装和编译方法参考：

    -
    +
    +

    快速开始

    +

    创建一个 housing.py 并粘贴此Python代码：

    +
    +
    import paddle.v2 as paddle
+
+# Initialize PaddlePaddle.
+paddle.init(use_gpu=False, trainer_count=1)
+
+# Configure the neural network.
+x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13))
+y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear())
+
+# Infer using provided test data.
+probs = paddle.infer(
+    output_layer=y_predict,
+    parameters=paddle.dataset.uci_housing.model(),
+    input=[item for item in paddle.dataset.uci_housing.test()()])
+
+for i in xrange(len(probs)):
+    print 'Predicted price: ${:,.2f}'.format(probs[i][0] * 1000)
+
    +
    +
    +

    执行 python housing.py 瞧！ 它应该打印出预测住房数据的清单。

    +
    + +
    +
    diff --git a/doc_cn/howto/deep_model/rnn/hierarchical_layer_cn.html b/doc_cn/howto/deep_model/rnn/hierarchical_layer_cn.html index edcfdafd7e6ccb21c160c9ee6eb9fe791fbef0bc..aaee3be764ea370781a210bb2229d840971a6449 100644 --- a/doc_cn/howto/deep_model/rnn/hierarchical_layer_cn.html +++ b/doc_cn/howto/deep_model/rnn/hierarchical_layer_cn.html @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -220,18 +242,18 @@

    pooling 的使用示例如下，详细见 pooling 配置API。

    seq_pool = pooling(input=layer,
                    pooling_type=pooling.Max(),
-                   agg_level=AggregateLevel.EACH_SEQUENCE)
+                   agg_level=AggregateLevel.TO_SEQUENCE)
    • pooling_type 目前支持两种，分别是：pooling.Max()和pooling.Avg()。
      • -
    • agg_level=AggregateLevel.EACH_TIMESTEP 时（默认值）：
        +
      • agg_level=AggregateLevel.TO_NO_SEQUENCE 时（默认值）：
        • 作用：双层序列经过运算变成一个0层序列，或单层序列经过运算变成一个0层序列
        • 输入：一个双层序列，或一个单层序列
        • 输出：一个0层序列，即整个输入序列（单层或双层）的平均值（或最大值）
        • -
      • agg_level=AggregateLevel.EACH_SEQUENCE 时：
          +
        • agg_level=AggregateLevel.TO_SEQUENCE 时：
          • 作用：一个双层序列经过运算变成一个单层序列
          • 输入：必须是一个双层序列
          • 输出：一个单层序列，序列的每个元素是原来双层序列每个subseq元素的平均值（或最大值）
            • @@ -243,17 +265,17 @@

            last_seq 和 first_seq

            last_seq 的使用示例如下（ first_seq 类似），详细见 last_seq 配置API。

            last = last_seq(input=layer,
-                agg_level=AggregateLevel.EACH_SEQUENCE)
+                agg_level=AggregateLevel.TO_SEQUENCE)
              -
            • agg_level=AggregateLevel.EACH_TIMESTEP 时（默认值）：
                +
              • agg_level=AggregateLevel.TO_NO_SEQUENCE 时（默认值）：
                • 作用：一个双层序列经过运算变成一个0层序列，或一个单层序列经过运算变成一个0层序列
                • 输入：一个双层序列或一个单层序列
                • 输出：一个0层序列，即整个输入序列（双层或者单层）最后一个，或第一个元素。
                • -
              • agg_level=AggregateLevel.EACH_SEQUENCE 时： +
              • agg_level=AggregateLevel.TO_SEQUENCE 时： - 作用：一个双层序列经过运算变成一个单层序列 - 输入：必须是一个双层序列 - 输出：一个单层序列，其中每个元素是双层序列中每个subseq最后一个（或第一个）元素。
                • @@ -261,14 +283,14 @@

    expand

    -

    expand 的使用示例如下，详细见 expand 配置API。

    +

    expand 的使用示例如下，详细见 ExpandLevel 配置API。

    ex = expand(input=layer1,
             expand_as=layer2,
-            expand_level=ExpandLevel.FROM_TIMESTEP)
+            expand_level=ExpandLevel.FROM_NO_SEQUENCE)
      -
    • expand_level=ExpandLevel.FROM_TIMESTEP 时（默认值）：
        +
      • expand_level=ExpandLevel.FROM_NO_SEQUENCE 时（默认值）：
        • 作用：一个0层序列经过运算扩展成一个单层序列，或者一个双层序列
        • 输入：layer1必须是一个0层序列，是待扩展的数据；layer2 可以是一个单层序列，或者是一个双层序列，提供扩展的长度信息
        • 输出：一个单层序列或一个双层序列，输出序列的类型（双层序列或单层序列）和序列中含有元素的数目同 layer2 一致。若输出是单层序列，单层序列的每个元素（0层序列），都是对layer1元素的拷贝；若输出是双层序列，双层序列每个subseq中每个元素（0层序列），都是对layer1元素的拷贝
          • diff --git a/doc_cn/howto/deep_model/rnn/hrnn_rnn_api_compare_cn.html b/doc_cn/howto/deep_model/rnn/hrnn_rnn_api_compare_cn.html index 01fd80308f39d65292e12492af5b8de089d38e5f..fe381e991ed9ab869667f742e94beab3c0de4ef0 100644 --- a/doc_cn/howto/deep_model/rnn/hrnn_rnn_api_compare_cn.html +++ b/doc_cn/howto/deep_model/rnn/hrnn_rnn_api_compare_cn.html @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -418,7 +440,7 @@
  • 在本例中，我们将原始数据的每一组，通过recurrent_group进行拆解，拆解成的每一句话再通过一个LSTM网络。这和单层RNN的配置是等价的。
    • -
  • 与单层RNN的配置类似，我们只需要使用LSTM encode成的最后一个向量。所以对recurrent_group进行了last_seq操作。但和单层RNN不同，我们是对每一个子序列取最后一个元素，因此agg_level=AggregateLevel.EACH_SEQUENCE。
    • +
  • 与单层RNN的配置类似，我们只需要使用LSTM encode成的最后一个向量。所以对recurrent_group进行了last_seq操作。但和单层RNN不同，我们是对每一个子序列取最后一个元素，因此agg_level=AggregateLevel.TO_SEQUENCE。
  • 至此，lstm_last便和单层RNN配置中的lstm_last具有相同的结果了。
    • - + diff --git a/doc_cn/howto/usage/cmd_parameter/detail_introduction_cn.html b/doc_cn/howto/usage/cmd_parameter/detail_introduction_cn.html index 4cc71ff0dd6faa95d87fddad9c627e711fa415ce..e59621de8933fb22fb6cda69d7a12298455b4bb6 100644 --- a/doc_cn/howto/usage/cmd_parameter/detail_introduction_cn.html +++ b/doc_cn/howto/usage/cmd_parameter/detail_introduction_cn.html @@ -34,7 +34,7 @@ - + @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -635,7 +657,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ diff --git a/doc_cn/howto/usage/cmd_parameter/use_case_cn.html b/doc_cn/howto/usage/cmd_parameter/use_case_cn.html index 787872588f35219a6977006f1462afd4314e3a7b..aded93d428541fd6c1926304bc48d3a329dd2cbf 100644 --- a/doc_cn/howto/usage/cmd_parameter/use_case_cn.html +++ b/doc_cn/howto/usage/cmd_parameter/use_case_cn.html @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ diff --git a/doc_cn/howto/usage/k8s/k8s_basis_cn.html b/doc_cn/howto/usage/k8s/k8s_basis_cn.html index 0273c0f81b8b8bc59119a0b26b540f2f1b4aacf5..b1025a706359a07efe8ca1b2b68129f0ff691540 100644 --- a/doc_cn/howto/usage/k8s/k8s_basis_cn.html +++ b/doc_cn/howto/usage/k8s/k8s_basis_cn.html @@ -35,7 +35,7 @@ - + @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -273,7 +295,7 @@ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s htt Next - Previous + Previous
    diff --git a/doc_cn/howto/usage/k8s/k8s_cn.html b/doc_cn/howto/usage/k8s/k8s_cn.html index b6cb63f660974d33e726a3cf531c1483c249e6df..4052613ca29d3cc5473151d3d71b89e60f83a351 100644 --- a/doc_cn/howto/usage/k8s/k8s_cn.html +++ b/doc_cn/howto/usage/k8s/k8s_cn.html @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ diff --git a/doc_cn/howto/usage/k8s/k8s_distributed_cn.html b/doc_cn/howto/usage/k8s/k8s_distributed_cn.html index 91541e9b048279e8d5e88746a94ea5bf192562d4..990f01c8bde0cfc06f8c7478807698cf884e7cff 100644 --- a/doc_cn/howto/usage/k8s/k8s_distributed_cn.html +++ b/doc_cn/howto/usage/k8s/k8s_distributed_cn.html @@ -34,7 +34,7 @@ - + @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -374,7 +396,7 @@ I1116 09:10:17.123440 50 Util.c I1116 09:10:17.123764 50 Util.cpp:143] Call runInitFunctions done. [WARNING 2016-11-16 09:10:17,227 default_decorators.py:40] please use keyword arguments in paddle config. [INFO 2016-11-16 09:10:17,239 networks.py:1282] The input order is [movie_id, title, genres, user_id, gender, age, occupation, rating] -[INFO 2016-11-16 09:10:17,239 networks.py:1289] The output order is [__mse_cost_0__] +[INFO 2016-11-16 09:10:17,239 networks.py:1289] The output order is [__square_error_cost_0__] I1116 09:10:17.392917 50 Trainer.cpp:170] trainer mode: Normal I1116 09:10:17.613910 50 PyDataProvider2.cpp:257] loading dataprovider dataprovider::process I1116 09:10:17.680917 50 PyDataProvider2.cpp:257] loading dataprovider dataprovider::process @@ -482,7 +504,7 @@ I1116 09:10:18.019836 50 Parame
    - Folk me on Github + Fork me on Github
    English @@ -85,6 +85,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -108,9 +109,10 @@ diff --git a/doc_cn/howto/usage/k8s/src/k8s_train/README.html b/doc_cn/howto/usage/k8s/src/k8s_train/README.html index c765eda4f16c972f8fe09d2482dd225749eec0da..32bcff05d831bfc1ffa3b7a8d11a17f4e4cd557a 100644 --- a/doc_cn/howto/usage/k8s/src/k8s_train/README.html +++ b/doc_cn/howto/usage/k8s/src/k8s_train/README.html @@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -85,6 +85,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -108,9 +109,10 @@ diff --git a/doc_cn/index_cn.html b/doc_cn/index_cn.html index c72de7230f7dac3335a7e40c09fc2295394ca37b..e27b42505bab2cd42541505d36452eb9a31d06af 100644 --- a/doc_cn/index_cn.html +++ b/doc_cn/index_cn.html @@ -63,7 +63,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -86,6 +86,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -109,9 +110,10 @@ @@ -191,6 +213,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • diff --git a/doc_cn/mobile/cross_compiling_for_android_cn.html b/doc_cn/mobile/cross_compiling_for_android_cn.html new file mode 100644 index 0000000000000000000000000000000000000000..eb1793e0ccfd14a2e0d697354afdd7014e1da5ec --- /dev/null +++ b/doc_cn/mobile/cross_compiling_for_android_cn.html @@ -0,0 +1,449 @@ + + + + + + + + + + + Android平台编译指南 — PaddlePaddle 文档 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + +
    +
    + Fork me on Github +
    + + English + + + + +
    + +
    +
    + + + + +
    +
    + + + + +
    +
    +
    +
    + +
    + + + + +
    + + + + + + + + + + + +
    +
      + +
    • MOBILE >
      • + +
    • Android平台编译指南
      • +
    +
    + +
    +
    +
    +
    + +
    +

    Android平台编译指南

    +

    用户可通过如下两种方式，交叉编译Android平台上适用的PaddlePaddle库：

    +
      +
    • 基于Docker容器的编译方式
      • +
    • 基于Linux交叉编译环境的编译方式
      • +
    +
    +

    基于Docker容器的编译方式

    +

    Docker能在所有主要操作系统（包括Linux，Mac OS X和Windows）上运行，因此，使用基于Docker容器的编译方式，用户可在自己熟悉的开发平台上编译Android平台上适用的PaddlePaddle库。

    +
    +

    构建PaddlePaddle的Android开发镜像

    +

    我们把PaddlePaddle的交叉编译环境打包成一个镜像，称为开发镜像，里面涵盖了交叉编译Android版PaddlePaddle库需要的所有编译工具。

    +
    $ git clone https://github.com/PaddlePaddle/Paddle.git
+$ cd Paddle
+$ docker build -t username/paddle-android:dev . -f Dockerfile.android
+
    +
    +
    +
    +

    编译PaddlePaddle C-API库

    +

    构建好开发镜像后，即可使用开发镜像来编译Android版PaddlePaddle C-API库。 +Android的Docker开发镜像向用户提供两个可配置的参数：

    +
     1
@@ -471,7 +493,7 @@
     input=SubsequenceInput(emb_group), step=lstm_group, name="lstm_nest_group")
 # hasSubseq ->(seqlastins) seq
 lstm_last = last_seq(
-    input=lstm_nest_group, agg_level=AggregateLevel.EACH_SEQUENCE)
+    input=lstm_nest_group, agg_level=AggregateLevel.TO_SEQUENCE)
 
 # seq ->(expand) hasSubseq
    diff --git a/doc_cn/howto/deep_model/rnn/index_cn.html b/doc_cn/howto/deep_model/rnn/index_cn.html index 70d7ef981cfe010f0f9a5b723d7753e3674b3d76..3562666f501c524ef11e5d3dabf3ddd301ff7e47 100644 --- a/doc_cn/howto/deep_model/rnn/index_cn.html +++ b/doc_cn/howto/deep_model/rnn/index_cn.html @@ -34,8 +34,8 @@ - - + + @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -191,6 +213,7 @@

    RNN相关模型

    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -299,7 +321,7 @@ Next - Previous + Previous
    diff --git a/doc_cn/howto/deep_model/rnn/rnn_config_cn.html b/doc_cn/howto/deep_model/rnn/rnn_config_cn.html index b121cba392af0083f80ef69b614459e39cd1c8cc..5ae3b76502d227ef4cf477619c6b5d4ecf3970d9 100644 --- a/doc_cn/howto/deep_model/rnn/rnn_config_cn.html +++ b/doc_cn/howto/deep_model/rnn/rnn_config_cn.html @@ -32,7 +32,10 @@ - + + + + @@ -62,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -80,11 +83,12 @@
    -
      + @@ -105,31 +109,33 @@
    -
    -

    生成序列

    +
    +

    生成序列

    训练模型后，我们可以使用它来生成序列。通常的做法是使用beam search 生成序列。以下代码片段定义 beam search 算法。注意，beam_search 函数假设 step 的输出函数返回的是下一个时刻输出词的 softmax @@ -387,42 +413,33 @@ attention，门控循环单元单步函数和输出函数：

  • max_length: 生成序列的最大长度。
    • -
  • 使用 seqtext_printer_evaluator -根据索引矩阵和字典打印文本。这个函数需要设置：
      -
    • id_input: 数据的整数ID，用于标识生成的文件中的相应输出。
      • -
    • dict_file: 用于将词ID转换为词的字典文件。
      • -
    • result_file: 生成结果文件的路径。
      • -
    -

    • 代码如下：

    -
    group_inputs=[StaticInput(input=encoded_vector,is_seq=True),
-              StaticInput(input=encoded_proj,is_seq=True)]
+
    group_input1 = paddle.layer.StaticInput(input=encoded_vector, is_seq=True)
+group_input2 = paddle.layer.StaticInput(input=encoded_proj, is_seq=True)
+group_inputs = [group_input1, group_input2]
 # 在生成时，解码器基于编码源序列和最后生成的目标词预测下一目标词。
 # 编码源序列（编码器输出）必须由只读Memory的 StaticInput 指定。
 # 这里， GeneratedInputs 自动获取上一个生成的词，并在最开始初始化为起始词，如 <s>。
-trg_embedding = GeneratedInput(
-    size=target_dict_dim,
-    embedding_name='_target_language_embedding',
-    embedding_size=word_vector_dim)
+trg_embedding = paddle.layer.GeneratedInput(
+        size=target_dict_dim,
+        embedding_name='_target_language_embedding',
+        embedding_size=word_vector_dim)
 group_inputs.append(trg_embedding)
-beam_gen = beam_search(name=decoder_group_name,
-                       step=gru_decoder_with_attention,
-                       input=group_inputs,
-                       bos_id=0, # Beginnning token.
-                       eos_id=1, # End of sentence token.
-                       beam_size=beam_size,
-                       max_length=max_length)
-
-seqtext_printer_evaluator(input=beam_gen,
-                          id_input=data_layer(name="sent_id", size=1),
-                          dict_file=trg_dict_path,
-                          result_file=gen_trans_file)
-outputs(beam_gen)
+beam_gen = paddle.layer.beam_search(
+        name=decoder_group_name,
+        step=gru_decoder_with_attention,
+        input=group_inputs,
+        bos_id=0, # Beginnning token.
+        eos_id=1, # End of sentence token.
+        beam_size=beam_size,
+        max_length=max_length)
+
+return beam_gen
 
    
 
    
-
    注意，这种生成技术只用于类似解码器的生成过程。如果你正在处理序列标记任务，请参阅 semantic_role_labeling 了解更多详细信息。
    
-
    完整的配置文件在demo/seqToseq/seqToseq_net.py。
    
+
    注意，这种生成技术只用于类似解码器的生成过程。如果你正在处理序列标记任务，请参阅 book/06.understand_sentiment 了解更多详细信息。
    
+
    完整的配置文件在 book/08.machine_translation/train.py 。
    @@ -431,6 +448,15 @@ attention，门控循环单元单步函数和输出函数：

    + +
    diff --git a/doc_cn/howto/dev/build_cn.html b/doc_cn/howto/dev/build_cn.html new file mode 100644 index 0000000000000000000000000000000000000000..e71ce726397c64996a5a0fd3d9809eb0ed169077 --- /dev/null +++ b/doc_cn/howto/dev/build_cn.html @@ -0,0 +1,387 @@ + + + + + + + + + + + 用Docker编译和测试PaddlePaddle — PaddlePaddle 文档 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + +
    +
    + Fork me on Github +
    + + English + + + + +
    + +
    +
    + + + + +
    +
    + + + +
    +
    +
    +
    +
    + +
    + + + + +
    + + + + + + + + + + + +
    + +
    + +
    +
    +
    +
    + +
    +

    用Docker编译和测试PaddlePaddle

    +
    +

    需要的软硬件

    +

    为了开发PaddlePaddle，我们需要

    +
      +
    1. 一台电脑，可以装的是 Linux, BSD, Windows 或者 MacOS 操作系统，以及
      2. +
    2. Docker。
      3. +
    +

    不需要依赖其他任何软件了。即便是 Python 和 GCC 都不需要，因为我们会把所有编译工具都安装进一个 Docker image 里。

    +
    +
    +

    总体流程

    +
      +

    1. 获取源码

      +
      git clone https://github.com/paddlepaddle/paddle
+
      +
      +
      2. +

    2. 安装开发工具到 Docker image 里

      +
      cd paddle; docker build -t paddle:dev .
+
      +
      +

      请注意这个命令结尾处的 .；它表示 docker build 应该读取当前目录下的 Dockerfile文件，按照其内容创建一个名为 paddle:dev 的 Docker image，并且把各种开发工具安装进去。

      +
      3. +

    3. 编译

      +

      以下命令启动一个 Docker container 来执行 paddle:dev 这个 Docker image，同时把当前目录（源码树根目录）映射为 container 里的 /paddle 目录，并且运行 Dockerfile 描述的默认入口程序 build.sh。这个脚本调用 cmake 和 make 来编译 /paddle 里的源码，结果输出到 /paddle/build，也就是本地的源码树根目录里的 build 子目录。

      +
      docker run --rm -v $PWD:/paddle paddle:dev
+
      +
      +

      上述命令编译出一个 CUDA-enabled 版本。如果我们只需要编译一个只支持 CPU 的版本，可以用

      +
      docker run --rm -e WITH_GPU=OFF -v $PWD:/paddle paddle:dev
+
      +
      +
      4. +

    4. 运行单元测试

      +

      用本机的第一个 GPU 来运行包括 GPU 单元测试在内的所有单元测试：

      +
      NV_GPU=0 nvidia-docker run --rm -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest"
+
      +
      +

      如果编译的时候我们用了 WITH_GPU=OFF 选项，那么编译过程只会产生 CPU-based 单元测试，那么我们也就不需要 nvidia-docker 来运行单元测试了。我们只需要：

      +
      docker run --rm -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest"
+
      +
      +

      有时候我们只想运行一个特定的单元测试，比如 memory_test，我们可以

      +
      nvidia-docker run --rm -v $PWD:/paddle paddle:dev bash -c "cd /paddle/build; ctest -V -R memory_test"
+
      +
      +
      5. +

    5. 清理

      +

      有时候我们会希望清理掉已经下载的第三方依赖以及已经编译的二进制文件。此时只需要：

      +
      rm -rf build
+
      +
      +
      6. +
    +
    +
    +

    为什么要 Docker 呀？

    +
      +

    • 什么是 Docker?

      +

      如果您没有听说 Docker，可以把它想象为一个类似 virtualenv 的系统，但是虚拟的不仅仅是 Python 的运行环境。

      +
      • +

    • Docker 还是虚拟机？

      +

      有人用虚拟机来类比 Docker。需要强调的是：Docker 不会虚拟任何硬件，Docker container 里运行的编译工具实际上都是在本机的 CPU 和操作系统上直接运行的，性能和把编译工具安装在本机运行一样。

      +
      • +

    • 为什么用 Docker?

      +

      把工具和配置都安装在一个 Docker image 里可以标准化编译环境。这样如果遇到问题，其他人可以复现问题以便帮助。

      +

      另外，对于习惯使用Windows和MacOS的开发者来说，使用Docker就不用配置交叉编译环境了。

      +
      • +

    • 我可以选择不用Docker吗？

      +

      当然可以。大家可以用把开发工具安装进入 Docker image 一样的方式，把这些工具安装到本机。这篇文档介绍基于 Docker 的开发流程，是因为这个流程比其他方法都更简便。

      +
      • +

    • 学习 Docker 有多难？

      +

      理解 Docker 并不难，大概花十分钟看一下这篇文章。这可以帮您省掉花一小时安装和配置各种开发工具，以及切换机器时需要新安装的辛苦。别忘了 PaddlePaddle 更新可能导致需要新的开发工具。更别提简化问题复现带来的好处了。

      +
      • +

    • 我可以用 IDE 吗？

      +

      当然可以，因为源码就在本机上。IDE 默认调用 make 之类的程序来编译源码，我们只需要配置 IDE 来调用 Docker 命令编译源码即可。

      +

      很多 PaddlePaddle 开发者使用 Emacs。他们在自己的 ~/.emacs 配置文件里加两行

      +
      (global-set-key "\C-cc" 'compile)
+(setq compile-command
+ "docker run --rm -it -v $(git rev-parse --show-toplevel):/paddle paddle:dev")
+
      +
      +

      就可以按 Ctrl-C 和 c 键来启动编译了。

      +
      • +

    • 可以并行编译吗？

      +

      是的。我们的 Docker image 运行一个 Bash 脚本。这个脚本调用 make -j$(nproc) 来启动和 CPU 核一样多的进程来并行编译。

      +
      • +
    +
    +
    +

    可能碰到的问题

    +
      +

    • Docker 需要 sudo

      +

      如果用自己的电脑开发，自然也就有管理员权限（sudo）了。如果用公用的电脑开发，需要请管理员安装和配置好 Docker。此外，PaddlePaddle 项目在努力开始支持其他不需要 sudo 的集装箱技术，比如 rkt。

      +
      • +

    • 在 Windows/MacOS 上编译很慢

      +

      Docker 在 Windows 和 MacOS 都可以运行。不过实际上是运行在一个 Linux 虚拟机上。可能需要注意给这个虚拟机多分配一些 CPU 和内存，以保证编译高效。具体做法请参考这个issue。

      +
      • +

    • 磁盘不够

      +

      本文中的例子里，docker run 命令里都用了 --rm 参数，这样保证运行结束之后的 containers 不会保留在磁盘上。可以用 docker ps -a 命令看到停止后但是没有删除的 containers。docker build 命令有时候会产生一些中间结果，是没有名字的 images，也会占用磁盘。可以参考这篇文章来清理这些内容。

      +
      • +
    +
    +
    + + +
    +
    + + +
    +
    + +
    + +
    + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc_cn/howto/dev/contribute_to_paddle_cn.html b/doc_cn/howto/dev/contribute_to_paddle_cn.html index b2094fe30cea059f61fe4c86f582bf3ddc213a79..bc5fcc10ce574abf7ef19f107e2581730e5ecb4c 100644 --- a/doc_cn/howto/dev/contribute_to_paddle_cn.html +++ b/doc_cn/howto/dev/contribute_to_paddle_cn.html @@ -34,8 +34,8 @@ - - + + @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -197,6 +219,7 @@
  • 确保编译器选项 WITH_STYLE_CHECK 已打开，并且编译能通过代码样式检查。
  • 所有代码必须具有单元测试。
  • 通过所有单元测试。
    • +
  • 请遵守提交代码的一些约定。

    • 以下教程将指导您提交代码。

    @@ -260,7 +283,7 @@ no changes added to commit (use "gi 
    ➜  docker build -t paddle:dev .
    -

    随后可以用这个开发镜像开build PaddlePaddle的源码。比如如果要build一个不依赖GPU，但是支持AVX指令集，并且包括unit tests的PaddlePaddle，可以：

    +

    随后可以用这个开发镜像开始build PaddlePaddle的源码。比如如果要build一个不依赖GPU，但是支持AVX指令集，并且包括unit tests的PaddlePaddle，可以：

    ➜  docker run -v $(pwd):/paddle -e "WITH_GPU=OFF" -e "WITH_AVX=ON" -e "WITH_TEST=ON" paddle:dev
    @@ -368,6 +391,36 @@ upstream

    至此，我们就完成了一次代码贡献的过程。

    +
    +

    提交代码的一些约定

    +

    为了使评审人在评审代码时更好地专注于代码本身，请您每次提交代码时，遵守以下约定：

    +
      +
    1. 请保证Travis-CI 中单元测试能顺利通过。如果没过，说明提交的代码存在问题，评审人一般不做评审。
      2. +
    2. 提交PUll Request前：
        +
      • 请注意commit的数量：
          +
        • 原因：如果仅仅修改一个文件但提交了十几个commit，每个commit只做了少量的修改，这会给评审人带来很大困扰。评审人需要逐一查看每个commit才能知道做了哪些修改，且不排除commit之间的修改存在相互覆盖的情况。
          • +
        • 建议：每次提交时，保持尽量少的commit，可以通过git commit --amend补充上次的commit。对已经Push到远程仓库的多个commit，可以参考squash commits after push。
          • +
        +
        • +
      • 请注意每个commit的名称：应能反映当前commit的内容，不能太随意。
        • +
      +
      3. +
    3. 如果解决了某个Issue的问题，请在该PUll Request的第一个评论框中加上：fix #issue_number，这样当该PUll Request被合并后，会自动关闭对应的Issue。关键词包括：close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved，请选择合适的词汇。详细可参考Closing issues via commit messages。
      4. +
    +

    此外，在回复评审人意见时，请您遵守以下约定：

    +
      +
    1. 评审人的每个意见都必须回复（这是开源社区的基本礼貌，别人帮了忙，应该说谢谢）：
        +
      • 对评审意见同意且按其修改完的，给个简单的Done即可；
        • +
      • 对评审意见不同意的，请给出您自己的反驳理由。
        • +
      +
      2. +
    2. 如果评审意见比较多：
        +
      • 请给出总体的修改情况。
        • +
      • 请采用start a review进行回复，而非直接回复的方式。原因是每个回复都会发送一封邮件，会造成邮件灾难。
        • +
      +
      3. +
    +
    @@ -377,10 +430,10 @@ upstream diff --git a/doc_cn/howto/dev/new_layer_cn.html b/doc_cn/howto/dev/new_layer_cn.html index 3b0b4d7bdd2783700f5ae80fa28a9f63a42eac75..03a3737a1c0145af10146ceded034452f439a5bb 100644 --- a/doc_cn/howto/dev/new_layer_cn.html +++ b/doc_cn/howto/dev/new_layer_cn.html @@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -85,6 +85,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -108,9 +109,10 @@ @@ -205,7 +227,7 @@

    假设损失函数是 \(c(y)\) ，那么

    \[\frac{\partial c(y)}{\partial x} = \frac{\partial c(y)}{\partial y} \frac{\partial y}{\partial x}\]
    -

    假设 \(z = f(W^T x + b)\) ，那么

    +

    假设 \(z = W^T x + b\) ，那么

    \[\frac{\partial y}{\partial z} = \frac{\partial f(z)}{\partial z}\]

    PaddlePaddle的base layer类可以自动计算上面的导数。

    diff --git a/doc_cn/howto/dev/new_op_cn.html b/doc_cn/howto/dev/new_op_cn.html new file mode 100644 index 0000000000000000000000000000000000000000..af3b491c803f91fee802e4b48cda7027aabe692a --- /dev/null +++ b/doc_cn/howto/dev/new_op_cn.html @@ -0,0 +1,574 @@ + + + + + + + + + + + 如何写新的Operator — PaddlePaddle 文档 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + +
    +
    + Fork me on Github +
    + + English + + + + +
    + +
    +
    + + + + +
    +
    + + + +
    +
    +
    +
    +
    + +
    + + + + +
    + + + + + + + + + + + +
    +
      + +
    • 如何写新的Operator
      • +
    +
    + +
    +
    +
    +
    + +
    +

    如何写新的Operator

    + +
    +

    概念简介

    +

    简单介绍需要用到基类，详细介绍请参考设计文档。

    +
      +
    • framework::OperatorBase: Operator(简写，Op)基类。
      • +
    • framework::OpKernel: Op计算函数的基类，称作Kernel。
      • +
    • framework::OperatorWithKernel：继承自OperatorBase，Op有计算函数，称作有Kernel。
      • +
    • class OpProtoAndCheckerMaker：描述该Op的输入、输出、属性、注释,主要用于Python API接口生成
      • +
    +

    依据是否包含kernel，可以将Op分为两种：包含Kernel的Op和不包含kernel的Op，前者Op的定义继承自OperatorBase，后者继承自OperatorWithKernel。本教程主要介绍带Kernel的Op如何写，简单总结Op需要包含的内容如下：

    +

    内容 | 定义位置 +————– | :———————- +OpProtoMake定义 | .cc文件，Backward Op不需要定义OpProtoMake +Op定义 | .cc文件 +Kernel实现 | CPU、GPU共享Kernel实现在.h文件中，否则，CPU 实现在.cc文件中，GPU 实现在.cu文件中。 +注册Op | Op注册实现在.cc文件；Kernel注册CPU实现在.cc文件中，GPU实现在.cu文件中

    +

    实现新的op都添加至目录paddle/operators下，文件命名以*_op.h（如有） 、 *_op.cc 、*_op.cu（如有）结尾。系统会根据文件名自动构建op和其对应的Python扩展。

    +

    下面以矩阵乘操作，即MulOp为例来介绍如何写带Kernel的Operator。

    +
    +
    +

    实现C++类

    +
    +

    1. 定义ProtoMaker类

    +

    矩阵乘法的公式：$Out = X * Y$, 可见该计算由两个输入，一个输出组成。

    +

    首先定义ProtoMaker来描述该Op的输入、输出，并添加注释：

    +
    class MulOpMaker : public framework::OpProtoAndCheckerMaker {
+ public:
+  MulOpMaker(framework::OpProto *proto, framework::OpAttrChecker *op_checker)
+      : OpProtoAndCheckerMaker(proto, op_checker) {
+    AddInput("X", "(Tensor), 2D tensor of size (M x K)");
+    AddInput("Y", "(Tensor), 2D tensor of size (K x N)");
+    AddOutput("Out", "(Tensor), 2D tensor of size (M x N)");
+    AddComment(R"DOC(
+Two Element Mul Operator.
+The equation is: Out = X * Y
+)DOC");
+  }
+};
+
    +
    +

    MulOpMaker继承自framework::OpProtoAndCheckerMaker，构造函数含有2个参数：

    +
      +
    • framework::OpProto ： 前者存储Op的输入输出和参数属性，将用于Python API接口的生成。
      • +
    • framework::OpAttrChecker ：后者用于检查参数属性的合法性。
      • +
    +

    构造函数里通过AddInput添加输入参数，通过AddOutput添加输出参数，通过AddComment添加Op的注释。这些函数会将对应内容添加到OpProto中。

    +

    上面的代码在MulOp中添加两个输入X和Y，添加了一个输出Out，并解释了各自含义，命名请遵守命名规范。

    +

    再以ScaleOp为例：

    +
    template <typename AttrType>
+class ScaleOpMaker : public framework::OpProtoAndCheckerMaker {
+ public:
+  ScaleOpMaker(framework::OpProto *proto, framework::OpAttrChecker *op_checker)
+      : OpProtoAndCheckerMaker(proto, op_checker) {
+    AddInput("X", "The input tensor of scale operator.").NotInGradient();
+    AddOutput("Out", "The output tensor of scale operator.").NotInGradient();
+    AddComment(R"DOC(Scale operator
+The equation is: Out = scale*X
+)DOC");
+    AddAttr<AttrType>("scale", "scale of scale operator.").SetDefault(1.0);
+  }
+};
+
    +
    +

    这个例子有两处不同：

    +
      +
    • AddInput("X","...").NotInGradient() : 表示X这个输入不参与ScaleOp对应的梯度Op计算之中，如果Op的某个输入不参与反向梯度的计算，请显示地调用.NotInGradient()进行设置。
      • +
    • AddAttr<AttrType>("scale", "...").SetDefault(1.0); : 增加scale系数，作为参数属性，并且设置默认值为1.0。
      • +
    +
    +
    +

    2. 定义Operator类

    +

    下面的点实现了MulOp的定义：

    +
    class MulOp : public framework::OperatorWithKernel {
+ public:
+  using framework::OperatorWithKernel::OperatorWithKernel;
+
+ protected:
+  void InferShape(const framework::InferShapeContext &ctx) const override {
+    auto dim0 = ctx.Input<Tensor>("X")->dims();
+    auto dim1 = ctx.Input<Tensor>("Y")->dims();
+    PADDLE_ENFORCE_EQ(dim0.size(), 2,
+                      "input X(%s) should be a tensor with 2 dims, a matrix",
+                      ctx.op_.Input("X"));
+    PADDLE_ENFORCE_EQ(dim1.size(), 2,
+                      "input Y(%s) should be a tensor with 2 dims, a matrix",
+                      ctx.op_.Input("Y"));
+    PADDLE_ENFORCE_EQ(
+        dim0[1], dim1[0],
+        "First matrix's width must be equal with second matrix's height.");
+    ctx.Output<Tensor>("Out")->Resize({dim0[0], dim1[1]});
+  }
+};
+
    +
    +

    MulOp继承自OperatorWithKernel。public成员：

    +
    using framework::OperatorWithKernel::OperatorWithKernel;
+
    +
    +

    这句表示使用基类OperatorWithKernel的构造函数，也可写成：

    +
    MulOp(const std::string &type, const framework::VariableNameMap &inputs,
+      const framework::VariableNameMap &outputs,
+      const framework::AttributeMap &attrs)
+  : OperatorWithKernel(type, inputs, outputs, attrs) {}
+
    +
    +

    还需要重写InferShape接口。InferShape为const函数，不能修改Op的成员变量，参数为const framework::InferShapeContext &ctx，通过该参数可获取到输入输出以及属性。它的功能是：

    +
      +
    • 1). 做检查， 尽早报错：检查输入数据维度、类型等是否合法。
      • +
    • 2). 设置输出Tensor的形状。
      • +
    +

    通常OpProtoMaker和Op类的定义写在.cc文件中，和下面将要介绍的注册函数一起放在.cc中

    +
    +
    +

    3. 定义OpKernel类

    +

    MulKernel继承自framework::OpKernel，带有下面两个模板参数:

    +
      +
    • typename Place: 表示设备类型，不同设备(CPU、GPU)共享同一个Kernel时，需加该模板参数，不共享则不加，一个不共享的例子是OnehotCrossEntropyOpKernel。
      • +
    • typename T : 表示数据类型，如float, double等。
      • +
    +

    需要为MulKernel类重写Compute接口。

    +
      +
    • Compute接受一个输入参数：const framework::ExecutionContext& context。
      • +
    • 与InferShapeContext相比，ExecutionContext增加了设备类型，同样可获取到输入输出和属性参数。
      • +
    • Compute函数里实现OpKernel的具体计算逻辑。
      • +
    +

    下面是 MulKernel Compute的实现：

    +
    template <typename Place, typename T>
+class MulKernel : public framework::OpKernel {
+public:
+void Compute(const framework::ExecutionContext& context) const override {
+  auto* X = context.Input<Tensor>("X");
+  auto* Y = context.Input<Tensor>("Y");
+  auto* Z = context.Output<Tensor>("Out");
+  Z->mutable_data<T>(context.GetPlace());
+  auto* device_context =
+      const_cast<platform::DeviceContext*>(context.device_context_);
+  math::matmul<Place, T>(*X, false, *Y, false, 1, Z, 0, device_context);
+}
+};
+
    +
    +

    需要注意：不同设备(CPU、GPU)共享一个Op定义，是否则共享同一个OpKernel，取决于Compute调用的函数是否支持不同设备。

    +

    MulOp的CPU、GPU实现共享同一个Kernel。OpKernel不共享的例子可以参考：OnehotCrossEntropyOpKernel。

    +

    为了使OpKernel的计算过程书写更加简单，并且CPU、GPU的代码可以复用，我们通常借助 Eigen unsupported Tensor模块来实现Compute接口。关于在PaddlePaddle中如何使用Eigen库，请参考使用文档。

    +

    到此，前向Op实现完成。接下来，需要在.cc文件中注册该op和kernel。 +反向Op类的定义，反向OpKernel的定义与前向Op类似，这里不再赘述。但需注意反向Op没有ProtoMaker。

    +
    +
    +

    4. 注册Operator

    +
      +

    • 在.cc文件中注册前向、反向Op类，注册CPU Kernel。

      +
      namespace ops = paddle::operators;
+REGISTER_OP(mul, ops::MulOp, ops::MulOpMaker, mul_grad, ops::MulOpGrad);
+REGISTER_OP_CPU_KERNEL(mul, ops::MulKernel<paddle::platform::CPUPlace, float>);
+REGISTER_OP_CPU_KERNEL(mul_grad,
+              ops::MulGradKernel<paddle::platform::CPUPlace, float>);
+
      +
      +

      在上面的代码中：

      +
        +
      • REGISTER_OP ： 注册ops::MulOp类，类型名为mul，该类的ProtoMaker为ops::MulOpMaker，注册ops::MulOpGrad，类型名为mul_grad。
        • +
      • REGISTER_OP_WITHOUT_GRADIENT ： 用于注册没有反向的Op。
        • +
      • REGISTER_OP_CPU_KERNEL ：注册ops::MulKernel类，并特化模板参数为paddle::platform::CPUPlace和float类型，同理，注册ops::MulGradKernel类。
        • +
      +
      • +
    +
      +

    • 在 .cu文件中注册GPU Kernel。

      +
        +
      • 请注意，如果GPU Kernel的实现基于Eigen unsupported模块，那么在 .cu的开始请加上宏定义 #define EIGEN_USE_GPU，代码示例如下：
        • +
      +
      // if use Eigen unsupported module before include head files
+// #define EIGEN_USE_GPU
+
+namespace ops = paddle::operators;
+REGISTER_OP_GPU_KERNEL(mul, ops::MulKernel<paddle::platform::GPUPlace, float>);
+REGISTER_OP_GPU_KERNEL(mul_grad,
+                       ops::MulGradKernel<paddle::platform::GPUPlace, float>);
+
      +
      +
      • +
    +
    +
    +

    5. 编译

    +

    运行下面命令可以进行编译：

    +
    make mul_op
+
    +
    +
    +
    +
    +

    绑定Python

    +

    系统会对新增的op自动绑定Python，并链接到生成的lib库中。

    +
    +
    +

    实现单元测试

    +

    单测包括对比前向Op不同设备(CPU、GPU)的实现、对比反向OP不同设备(CPU、GPU)的实现、反向Op的梯度测试。下面介绍介绍MulOp的单元测试。

    +
    +

    前向Operator单元测试

    +

    前向Op单元测试继承自unittest.TestCase，并定义元类__metaclass__ = OpTestMeta。各项更加具体的单元测试在OpTestMeta里完成。测试前向Operator，需要：

    +
      +
    1. 在setUp函数定义输入、输出，以及相关的属性参数。
      2. +
    2. 生成随机的输入数据。
      3. +
    3. 在Python脚本中实现与前向operator相同的计算逻辑，得到输出值，与operator前向计算的输出进行对比。
      4. +
    +
    import unittest
+import numpy as np
+from gradient_checker import GradientChecker, create_op
+from op_test_util import OpTestMeta
+
+class TestMulOp(unittest.TestCase):
+    __metaclass__ = OpTestMeta
+
+    def setUp(self):
+        self.type = "mul"
+        self.inputs = {
+            'X': np.random.random((32, 84)).astype("float32"),
+            'Y': np.random.random((84, 100)).astype("float32")
+        }
+        self.outputs = {'Out': np.dot(self.inputs['X'], self.inputs['Y'])}
+
    +
    +

    上面的代码首先导入依赖的包，下面是对setUp函数中操作的重要变量的详细解释：

    +
      +
    • self.type = "mul" : 定义类型，与operator注册时注册的类型一致。
      • +
    • self.inputs : 定义输入，类型为numpy.array，并初始化。
      • +
    • self.outputs : 定义输出，并在Python脚本中完成与operator同样的计算逻辑，返回Python端的计算结果。
      • +
    +
    +
    +

    反向Operator单元测试

    +

    反向Op单元测试继承自GradientChecker，而GradientChecker继承自unittest.TestCase，因此，反向单元测试函数需要以test_开头。

    +
    class TestMulGradOp(GradientChecker):
+    def setUp(self):
+        self.op = create_op("mul")
+        self.inputs = {
+            'X': np.random.random((32, 84)).astype("float32"),
+            'Y': np.random.random((84, 100)).astype("float32")
+        }
+
+    def test_check_grad_normal(self):
+        # mul op will enlarge the relative error
+        self.check_grad(['X', 'Y'], 'Out', max_relative_error=0.5)
+
+    def test_check_grad_ingore_x(self):
+        self.check_grad(
+            ['Y'], 'Out', max_relative_error=0.5, no_grad_set=set("X"))
+
+    def test_check_grad_ingore_y(self):
+        self.check_grad(
+            ['X'], 'Out', max_relative_error=0.5, no_grad_set=set('Y'))
+
    +
    +

    下面解释代码中一些关键的地方:

    +
      +
    • 调用create_op("mul")创建反向Op对应的前向Op。
      • +
    • test_check_grad_normal中调用check_grad使用数值法检测梯度正确性和稳定性。
        +
      • 第一个参数["X", "Y"] : 指定对输入变量X、Y做梯度检测。
        • +
      • 第二个参数"Out" : 指定前向网络最终的输出目标变量Out。
        • +
      • 第三个参数max_relative_error：指定检测梯度时能容忍的最大错误值。
        • +
      +
      • +
    • test_check_grad_ingore_x和test_check_grad_ingore_y分支用来测试只需要计算一个输入梯度的情况。
      • +
    +
    +
    +

    编译和执行单元测试

    +

    python/paddle/v2/framework/tests 目录下新增的 test_*.py 单元测试会被自动加入工程进行编译。

    +

    请注意，不同于Op的编译测试，运行单元测试测时需要编译整个工程，并且编译时需要打开WITH_TESTING, 即cmake paddle_dir -DWITH_TESTING=ON。编译成功后，执行下面的命令来运行单元测试：

    +
    make test ARGS="-R test_mul_op -V"
+
    +
    +

    或者:

    +
    ctest -R test_mul_op
+
    +
    +
    +
    +
    +

    注意事项

    +
      +
    • 为每个Op创建单独的*_op.h（如有）、*_op.cc和*_op.cu（如有）。不允许一个文件中包含多个Op，这将会导致编译出错。
      • +
    • 注册Op时的类型名，需要和该Op的名字一样。即不允许在A_op.cc里面，注册REGISTER_OP(B, ...)等，这将会导致单元测试出错。
      • +
    • 如果Op没有实现GPU Kernel，请不要创建空的*_op.cu，这将会导致单元测试出错。
      • +
    • 如果多个Op依赖一些共用的函数，可以创建非*_op.*格式的文件来存放，如gather.h文件。
      • +
    +
    +
    + + +
    +
    +
    + + +
    + +
    +

    + © Copyright 2016, PaddlePaddle developers. + +

    +
    + Built with Sphinx using a theme provided by Read the Docs. + +
    + +
    +
    + +
    + +
    + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc_cn/howto/dev/use_eigen_cn.html b/doc_cn/howto/dev/use_eigen_cn.html new file mode 100644 index 0000000000000000000000000000000000000000..0e686cbd62559e4485aa2fef7013be442d321253 --- /dev/null +++ b/doc_cn/howto/dev/use_eigen_cn.html @@ -0,0 +1,393 @@ + + + + + + + + + + + 在Paddle中如何使用Eigen — PaddlePaddle 文档 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + +
    +
    + Fork me on Github +
    + + English + + + + +
    + +
    +
    + + + + +
    +
    + + + +
    +
    +
    +
    +
    + +
    + + + + +
    + + + + + + + + + + + +
    +
      + +
    • 在Paddle中如何使用Eigen
      • +
    +
    + +
    +
    +
    +
    + +
    +

    在Paddle中如何使用Eigen

    +

    神经网络本质上是一个计算图，计算需要的数据存放在Tensor中，而计算过程是由Operartor来描述的。在执行时，Operator调用对应OpKernel中的Compute接口，实现对Tensor的操作。

    +
    +

    Eigen Tensor模块

    +

    Eigen Tensor模块对element-wise计算提供了强大的支持，并且书写一份代码，可以同时在CPU、GPU执行。但Eigen Tensor是一个正在开发中的模块，因此可能测试不够完备，文档较少。

    +

    关于Eigen Tensor模块的详细介绍请参考文档1 和文档2

    +
    +
    +

    paddle::framework::Tensor

    +

    Paddle Tensor定义在framework目录下，其主要接口如下：

    +
    class Tensor {
+ public:
+  /*! Return a pointer to mutable memory block. */
+  template <typename T>
+  inline T* data();
+  
+  /**
+   * @brief   Return a pointer to mutable memory block.
+   * @note    If not exist, then allocation.
+   */
+  template <typename T>
+  inline T* mutable_data(platform::Place place);
+  
+  /**
+   * @brief     Return a pointer to mutable memory block.
+   *
+   * @param[in] dims    The dimensions of the memory block.
+   * @param[in] place   The place of the memory block.
+   *
+   * @note      If not exist, then allocation.
+   */
+  template <typename T>
+  inline T* mutable_data(DDim dims, platform::Place place);
+  
+  /*! Resize the dimensions of the memory block. */
+  inline Tensor& Resize(const DDim& dims);
+  
+  /*! Return the dimensions of the memory block. */
+  inline const DDim& dims() const;
+
+ private:  
+  /*! holds the memory block if allocated. */
+  std::shared_ptr<Placeholder> holder_;
+  
+  /*! points to dimensions of memory block. */
+  DDim dim_;
+};
+
    +
    +

    Placeholder的作用是延迟分配内存，即我们可以先定义一个Tensor，然后使用Resize接口设置Tensor的大小，最后再调用mutable_data接口分配实际的内存。

    +
    paddle::framework::Tensor t;
+paddle::platform::CPUPlace place;
+// set size first
+t.Resize({2, 3});
+// allocate memory on CPU later
+t.mutable_data(place);
+
    +
    +
    +
    +

    paddle::framework::Tensor使用样例

    +

    下面以AddOp为例说明Tensor的使用过程：

    +
      +
    • InferShape
      • +
    +

    在运行神经网络计算图时，我们先调用每个Operator的InferShape接口，根据输入Tensor的大小来设置输出Tensor的大小，Resize接口会被调用。

    +
    void InferShape(const framework::InferShapeContext &ctx) const override {
+  PADDLE_ENFORCE_EQ(ctx.Input<Tensor>("X")->dims(),
+                    ctx.Input<Tensor>("Y")->dims(),
+                    "Two input of Add Op's dimension must be same.");
+  ctx.Output<Tensor>("Out")->Resize(ctx.Input<Tensor>("X")->dims());
+}
+
    +
    +
      +
    • Run
      • +
    +

    Operator的Run接口最终会调用对应OpKernel的Compute接口，在这时真正的分配内存，mutable_data接口会被调用。

    +
    void Compute(const framework::ExecutionContext& context) const override {
+  auto* input0 = context.Input<Tensor>("X");
+  auto* input1 = context.Input<Tensor>("Y");
+  auto* output = context.Output<Tensor>("Out");
+
+  output->mutable_data<T>(context.GetPlace());
+
+  auto x = EigenVector<T>::Flatten(*input0);
+  auto y = EigenVector<T>::Flatten(*input1);
+  auto z = EigenVector<T>::Flatten(*output);
+
+  auto place = context.GetEigenDevice<Place>();
+
+  z.device(place) = x + y;
+}
+
    +
    +
    +
    +

    paddle::framework::Tensor到EigenTensor的转换

    +

    如上一小节所示，在具体的计算中，我们需要先把输入Tensor和输出Tensor转换为Eigen支持的格式。我们在eigen.h中提供了一些全局函数用来实现paddle::framework::Tensor到EigenTensor/EigenMatrix/EigenVector/EigenScalar的转换。

    +

    以EigenTensor为例，做一个介绍

    +
    Tensor t;
+float* p = t.mutable_data<float>(make_ddim({1, 2, 3}), platform::CPUPlace());
+for (int i = 0; i < 1 * 2 * 3; i++) {
+  p[i] = static_cast<float>(i);
+}
+
+EigenTensor<float, 3>::Type et = EigenTensor<float, 3>::From(t);
+
    +
    +

    From是EigenTensor模板提供的一个接口，可以实现从paddle::framework::Tensor到对EigenTensor的转换。由于Tensor的rank是模板参数，因此在转换时需要显示的指定。

    +

    在Eigen中，不同rank的Tensor是不同类型，Vector是rank为1的Tensor。需要额外注意的是，EigenVector::From方法是把paddle中的一维Tensor转为Eigen的一维Tensor，在这里用EigenVector来表示；而EigenVector::Flatten方法是把paddle中的一个Tensor进行reshape操作，压扁成为Eigen的一维Tensor，类型仍然为EigenVector。

    +

    更多的转换方法请参考eigen_test.cc中的单元测试。

    +
    +
    +

    实现计算

    +

    当需要完成计算时，我们需要等式左边的EigenTensor调用device接口。在这里需要注意的是，这里的EigenTensor之间的运算只是改变了原有Tensor中的数据，而不会改变原有Tensor的shape信息。

    +
    auto x = EigenVector<T>::Flatten(*input0);
+auto y = EigenVector<T>::Flatten(*input1);
+auto z = EigenVector<T>::Flatten(*output);
+auto place = context.GetEigenDevice<Place>();
+z.device(place) = x + y;
+
    +
    +

    在这段代码中，input0/input1/output可以是任意维度的Tensor。我们调用了EigenVector的Flatten接口，把任意维度的Tensor转为了一维的EigenVector。而在计算结束之后，input0/input1/output的原有shape信息不变。如果想改变原有Tensor的shape信息，可以调用Resize接口进行改变。

    +

    由于Eigen Tensor模块的文档较少，我们可以参考TensorFlow的kernels模块下的相关OpKernel的计算代码。

    +
    +
    + + +
    +
    +
    + + +
    + +
    +

    + © Copyright 2016, PaddlePaddle developers. + +

    +
    + Built with Sphinx using a theme provided by Read the Docs. + +
    + +
    +
    + +
    + +
    + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc_cn/howto/dev/write_docs_cn.html b/doc_cn/howto/dev/write_docs_cn.html index 6955daa48fbe8ad0f939ee56ca36afe7840643eb..794c28aa4434723bc982d251139c6184898f1175 100644 --- a/doc_cn/howto/dev/write_docs_cn.html +++ b/doc_cn/howto/dev/write_docs_cn.html @@ -34,8 +34,8 @@ - - + + @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -189,40 +211,90 @@

    如何贡献/修改文档

    -

    PaddlePaddle的文档包括英文文档 doc 和中文文档 doc_cn 两个部分。文档都是通过 cmake 驱动 sphinx 编译生成，生成后的文档分别存储在编译目录的 doc 和 doc_cn 两个子目录下。

    -
    -

    如何构建PaddlePaddle的文档

    -

    PaddlePaddle的文档构建有直接构建和基于Docker构建两种方式。构建PaddlePaddle文档需要准备的环境相对较复杂，所以我们推荐使用基于Docker来构建PaddlePaddle的文档。

    -
    -

    使用Docker构建PaddlePaddle的文档

    +

    PaddlePaddle的文档包括英文文档 doc 和中文文档 doc_cn 两个部分。文档都是通过 cmake 驱动 sphinx 编译生成，生成后的文档分别存储在编译目录的 doc 和 doc_cn 两个子目录下。 +也可以利用PaddlePaddle 工具来编译文档，这个情况下所有的文件会存在整理过的的文件目录 .ppo_workspace/content 下

    +
    +

    如何构建文档

    +

    PaddlePaddle的文档构建有三种方式。

    +
    +

    使用PaddlePaddle.org工具

    +

    这个是目前推荐的使用方法。除了可以自动编译文档，也可以直接在网页预览文档。

    +

    文件工具是使用Docker，需要在系统里先安装好Docker工具包。Docker安装请参考Docker的官网。安装好Docker之后及可用以下命令启动工具

    +
    mkdir paddlepaddle # Create paddlepaddle working directory
+cd paddlepaddle
+
+# Clone the content repositories
+git clone https://github.com/PaddlePaddle/Paddle.git
+git clone https://github.com/PaddlePaddle/book.git
+git clone https://github.com/PaddlePaddle/models.git
+git clone https://github.com/PaddlePaddle/Mobile.git
+
+# Please specify the working directory through -v
+docker run -it -p 8000:8000 -v `pwd`:/var/content paddlepaddle/paddlepaddle.org:latest
+
    +
    +

    注意: PaddlePaddle.org 会在 -v (volume) 指定的内容存储库运行命令 +之后再用网页连到http://localhost:8000就可以在网页上生成需要的文档 +编译后的文件将被存储在工作目录 <paddlepaddle working directory>/.ppo_workspace/content。

    +

    如果不想使用 Docker，你还可以通过运行Django框架直接激活工具的服务器。使用下面的命令来运行它。

    +
    mkdir paddlepaddle # Create paddlepaddle working directory
+cd paddlepaddle
+
+# Clone the content repositories and PaddlePaddle.org
+git clone https://github.com/PaddlePaddle/Paddle.git
+git clone https://github.com/PaddlePaddle/book.git
+git clone https://github.com/PaddlePaddle/models.git
+git clone https://github.com/PaddlePaddle/Mobile.git
+git clone https://github.com/PaddlePaddle/PaddlePaddle.org.git
+
+# Please specify the PaddlePaddle working directory. In the current setting, it should be pwd
+export CONTENT_DIR=<path_to_paddlepaddle_working_directory>
+export ENV=''
+cd PaddlePaddle.org/portal/
+pip install -r requirements.txt
+python manage.py runserver
+
    +
    +

    工具服务器将读取环境变量 CONTENT_DIR 搜索代码库。请指定的PaddlePaddle工作目录给环境变量 CONTENT_DIR。 +之后再用网页连到http://localhost:8000就可以在网页上生成需要的文档。 +编译后的文件将被存储在工作目录 <paddlepaddle working directory>/.ppo_workspace/content。

    +

    想了解更多PaddlePaddle.org工具的详细信息，可以 点击这里 。

    +
    +
    +

    使用Docker构建

    使用Docker构建PaddlePaddle的文档，需要在系统里先安装好Docker工具包。Docker安装请参考 Docker的官网 。安装好Docker之后可以使用源码目录下的脚本构建文档，即

    cd TO_YOUR_PADDLE_CLONE_PATH
 cd paddle/scripts/tools/build_docs
-bash build_docs.sh
+sh build_docs.sh
    -

    编译完成后，该目录下会生成如下两个子目录:

    -
      -
    • doc 英文文档目录
      • -
    • doc_cn 中文文档目录
      • -
    -

    打开浏览器访问对应目录下的index.html即可访问本地文档。

    -
    open doc_cn/index.html
-
    +

    编译完成之后，会在当前目录生成两个子目录: doc(英文文档目录)和 doc_cn(中文文档目录)。 +打开浏览器访问对应目录下的index.html即可访问本地文档。

    +
    +

    直接构建

    +

    如果提示正确，可以执行以下命令编译生成文档，即

    +
    cd TO_YOUR_PADDLE_CLONE_PATH
+mkdir -p build
+cd build
+cmake .. -DCMAKE_BUILD_TYPE=Debug -DWITH_GPU=OFF -DWITH_MKL=OFF -DWITH_DOC=ON
+make gen_proto_py
+make paddle_docs paddle_docs_cn
+
    -
    -

    直接构建PaddlePaddle的文档

    -

    TBD

    +

    编译完成之后，会在当前目录生成两个子目录: doc(英文文档目录)和 doc_cn(中文文档目录)。 +打开浏览器访问对应目录下的index.html即可访问本地文档。

    -
    -

    如何书写PaddlePaddle的文档

    -

    TBD

    +
    +

    如何书写文档

    +

    PaddlePaddle文档使用 sphinx 自动生成，用户可以参考sphinx教程进行书写。

    -

    如何更新www.paddlepaddle.org文档

    -

    TBD

    +

    如何更新www.paddlepaddle.org

    +

    更新的文档以PR的形式提交到github中，提交方式参见 贡献文档 。 +目前PaddlePaddle的develop分支的文档是自动触发更新的，用户可以分别查看最新的 中文文档 和 +英文文档 。

    @@ -233,10 +305,10 @@ bash build_docs.sh diff --git a/doc_cn/howto/index_cn.html b/doc_cn/howto/index_cn.html index 13892205cd6a646dbde2391acf76a788b3e45ac3..c94abff6fdf72e847d7da1d3f1d65b2f0cd28a8e 100644 --- a/doc_cn/howto/index_cn.html +++ b/doc_cn/howto/index_cn.html @@ -64,7 +64,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -87,6 +87,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -110,9 +111,10 @@ @@ -191,7 +213,7 @@
    diff --git a/doc_cn/howto/optimization/cpu_profiling.html b/doc_cn/howto/optimization/cpu_profiling.html new file mode 100644 index 0000000000000000000000000000000000000000..a882941390d7889816c143e52e17d1686f720847 --- /dev/null +++ b/doc_cn/howto/optimization/cpu_profiling.html @@ -0,0 +1,434 @@ + + + + + + + + + + + Profiling the Python Code — PaddlePaddle 文档 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + +
    +
    + Fork me on Github +
    + + English + + + + +
    + +
    +
    + + + + +
    +
    + + + +
    +
    +
    +
    +
    + +
    + + + + +
    + + + + + + + + + + + +
    +
      + +
    • Profiling the Python Code
      • +
    +
    + +
    +
    +
    +
    + +

    This tutorial introduces techniques we use to profile and tune the +CPU performance of PaddlePaddle. We will use Python packages +cProfile and yep, and Google’s perftools.

    +

    Profiling is the process that reveals performance bottlenecks, +which could be very different from what’s in the developers’ mind. +Performance tuning is done to fix these bottlenecks. Performance optimization +repeats the steps of profiling and tuning alternatively.

    +

    PaddlePaddle users program AI applications by calling the Python API, which calls +into libpaddle.so. written in C++. In this tutorial, we focus on +the profiling and tuning of

    +
      +
    1. the Python code and
      2. +
    2. the mixture of Python and C++ code.
      3. +
    +
    +

    Profiling the Python Code

    +
    +

    Generate the Performance Profiling File

    +

    We can use Python standard +package, cProfile, +to generate Python profiling file. For example:

    +
    python -m cProfile -o profile.out main.py
+
    +
    +

    where main.py is the program we are going to profile, -o specifies +the output file. Without -o, cProfile would outputs to standard +output.

    +
    +
    +

    Look into the Profiling File

    +

    cProfile generates profile.out after main.py completes. We can +use cprofilev to look into +the details:

    +
    cprofilev -a 0.0.0.0 -p 3214 -f profile.out main.py
+
    +
    +

    where -a specifies the HTTP IP, -p specifies the port, -f +specifies the profiling file, and main.py is the source file.

    +

    Open the Web browser and points to the local IP and the specifies +port, we will see the output like the following:

    +
       ncalls  tottime  percall  cumtime  percall filename:lineno(function)
+        1    0.284    0.284   29.514   29.514 main.py:1(<module>)
+     4696    0.128    0.000   15.748    0.003 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/executor.py:20(run)
+     4696   12.040    0.003   12.040    0.003 {built-in method run}
+        1    0.144    0.144    6.534    6.534 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/__init__.py:14(<module>)
+
    +
    +

    where each line corresponds to Python function, and the meaning of +each column is as follows:

    +

    | column | meaning | +| — | — | +| ncalls | the number of calls into a function | +| tottime | the total execution time of the function, not including the +execution time of other functions called by the function | +| percall | tottime divided by ncalls | +| cumtime | the total execution time of the function, including the execution time of other functions being called | +| percall | cumtime divided by ncalls | +| filename:lineno(function) | where the function is defined |

    +
    +
    +

    Identify Performance Bottlenecks

    +

    Usually, tottime and the related percall time is what we want to +focus on. We can sort above profiling file by tottime:

    +
         4696   12.040    0.003   12.040    0.003 {built-in method run}
+   300005    0.874    0.000    1.681    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/dataset/mnist.py:38(reader)
+   107991    0.676    0.000    1.519    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:219(__init__)
+     4697    0.626    0.000    2.291    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp)
+        1    0.618    0.618    0.618    0.618 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/__init__.py:1(<module>)
+
    +
    +

    We can see that the most time-consuming function is the built-in method run, which is a C++ function in libpaddle.so. We will +explain how to profile C++ code in the next section. At this +moment, let’s look into the third function sync_with_cpp, which is a +Python function. We can click it to understand more about it:

    +
    Called By:
+
+   Ordered by: internal time
+   List reduced from 4497 to 2 due to restriction <'sync_with_cpp'>
+
+Function                                                                                                 was called by...
+                                                                                                             ncalls  tottime  cumtime
+/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp)  <-    4697    0.626    2.291  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp)
+/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp)  <-    4696    0.019    2.316  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:487(clone)
+                                                                                                                  1    0.000    0.001  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:534(append_backward)
+
+
+Called:
+
+   Ordered by: internal time
+   List reduced from 4497 to 2 due to restriction <'sync_with_cpp'>
+
    +
    +

    The lists of the callers of sync_with_cpp might help us understand +how to improve the function definition.

    +
    +
    +
    +

    Profiling Python and C++ Code

    +
    +

    Generate the Profiling File

    +

    To profile a mixture of Python and C++ code, we can use a Python +package, yep, that can work with Google’s perftools, which is a +commonly-used profiler for C/C++ code.

    +

    In Ubuntu systems, we can install yep and perftools by running the +following commands:

    +
    apt update
+apt install libgoogle-perftools-dev
+pip install yep
+
    +
    +

    Then we can run the following command

    +
    python -m yep -v main.py
+
    +
    +

    to generate the profiling file. The default filename is +main.py.prof.

    +

    Please be aware of the -v command line option, which prints the +analysis results after generating the profiling file. By examining the +the print result, we’d know that if we stripped debug +information from libpaddle.so at build time. The following hints +help make sure that the analysis results are readable:

    +
      +
    1. Use GCC command line option -g when building libpaddle.so so to +include the debug information. The standard building system of +PaddlePaddle is CMake, so you might want to set +CMAKE_BUILD_TYPE=RelWithDebInfo.
      2. +
    2. Use GCC command line option -O2 or -O3 to generate optimized +binary code. It doesn’t make sense to profile libpaddle.so +without optimization, because it would anyway run slowly.
      3. +
    3. Profiling the single-threaded binary file before the +multi-threading version, because the latter often generates tangled +profiling analysis result. You might want to set environment +variable OMP_NUM_THREADS=1 to prevents OpenMP from automatically +starting multiple threads.
      4. +
    +
    +
    +

    Examining the Profiling File

    +

    The tool we used to examine the profiling file generated by +perftools is pprof, which +provides a Web-based GUI like cprofilev.

    +

    We can rely on the standard Go toolchain to retrieve the source code +of pprof and build it:

    +
    go get github.com/google/pprof
+
    +
    +

    Then we can use it to profile main.py.prof generated in the previous +section:

    +
    pprof -http=0.0.0.0:3213 `which python`  ./main.py.prof
+
    +
    +

    Where -http specifies the IP and port of the HTTP service. +Directing our Web browser to the service, we would see something like +the following:

    +

    result

    +
    +
    +

    Identifying the Performance Bottlenecks

    +

    Similar to how we work with cprofilev, we’d focus on tottime and +cumtime.

    +

    kernel_perf

    +

    We can see that the execution time of multiplication and the computing +of the gradient of multiplication takes 2% to 4% of the total running +time, and MomentumOp takes about 17%. Obviously, we’d want to +optimize MomentumOp.

    +

    pprof would mark performance critical parts of the program in +red. It’s a good idea to follow the hints.

    +
    +
    + + +
    +
    +
    + + +
    + +
    +

    + © Copyright 2016, PaddlePaddle developers. + +

    +
    + Built with Sphinx using a theme provided by Read the Docs. + +
    + +
    +
    + +
    + +
    + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc_cn/howto/optimization/cpu_profiling_cn.html b/doc_cn/howto/optimization/cpu_profiling_cn.html new file mode 100644 index 0000000000000000000000000000000000000000..091b53797a84f4ad233aa6982be243acd2ece348 --- /dev/null +++ b/doc_cn/howto/optimization/cpu_profiling_cn.html @@ -0,0 +1,387 @@ + + + + + + + + + + + Python代码的性能分析 — PaddlePaddle 文档 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + +
    +
    + Fork me on Github +
    + + English + + + + +
    + +
    +
    + + + + +
    +
    + + + +
    +
    +
    +
    +
    + +
    + + + + +
    + + + + + + + + + + + +
    +
      + +
    • Python代码的性能分析
      • +
    +
    + +
    +
    +
    +
    + +

    此教程会介绍如何使用Python的cProfile包、Python库yep、Google perftools来进行性能分析 (profiling) 与调优（performance tuning）。

    +

    Profling 指发现性能瓶颈。系统中的瓶颈可能和程序员开发过程中想象的瓶颈相去甚远。Tuning 指消除瓶颈。性能优化的过程通常是不断重复地 profiling 和 tuning。

    +

    PaddlePaddle 用户一般通过调用 Python API 编写深度学习程序。大部分 Python API 调用用 C++ 写的 libpaddle.so。所以 PaddlePaddle 的性能分析与调优分为两个部分:

    +
      +
    • Python 代码的性能分析
      • +
    • Python 与 C++ 混合代码的性能分析
      • +
    +
    +

    Python代码的性能分析

    +
    +

    生成性能分析文件

    +

    Python标准库中提供了性能分析的工具包，cProfile。生成Python性能分析的命令如下:

    +
    python -m cProfile -o profile.out main.py
+
    +
    +

    其中 main.py 是我们要分析的程序，-o标识了一个输出的文件名，用来存储本次性能分析的结果。如果不指定这个文件，cProfile会打印到标准输出。

    +
    +
    +

    查看性能分析文件

    +

    cProfile 在main.py 运行完毕后输出profile.out。我们可以使用cprofilev来查看性能分析结果。cprofilev是一个Python的第三方库。使用它会开启一个HTTP服务，将性能分析结果以网页的形式展示出来：

    +
    cprofilev -a 0.0.0.0 -p 3214 -f profile.out main.py
+
    +
    +

    其中-a标识HTTP服务绑定的IP。使用0.0.0.0允许外网访问这个HTTP服务。-p标识HTTP服务的端口。-f标识性能分析的结果文件。main.py标识被性能分析的源文件。

    +

    用Web浏览器访问对应网址，即可显示性能分析的结果：

    +
       ncalls  tottime  percall  cumtime  percall filename:lineno(function)
+        1    0.284    0.284   29.514   29.514 main.py:1(<module>)
+     4696    0.128    0.000   15.748    0.003 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/executor.py:20(run)
+     4696   12.040    0.003   12.040    0.003 {built-in method run}
+        1    0.144    0.144    6.534    6.534 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/__init__.py:14(<module>)
+
    +
    +

    每一列的含义是:

    +

    | 列名 | 含义 | +| — | — | +| ncalls | 函数的调用次数 | +| tottime | 函数实际使用的总时间。该时间去除掉本函数调用其他函数的时间 | +| percall | tottime的每次调用平均时间 | +| cumtime | 函数总时间。包含这个函数调用其他函数的时间 | +| percall | cumtime的每次调用平均时间 | +| filename:lineno(function) | 文件名, 行号，函数名 |

    +
    +
    +

    寻找性能瓶颈

    +

    通常tottime和cumtime是寻找瓶颈的关键指标。这两个指标代表了某一个函数真实的运行时间。

    +

    将性能分析结果按照tottime排序，效果如下:

    +
         4696   12.040    0.003   12.040    0.003 {built-in method run}
+   300005    0.874    0.000    1.681    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/dataset/mnist.py:38(reader)
+   107991    0.676    0.000    1.519    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:219(__init__)
+     4697    0.626    0.000    2.291    0.000 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp)
+        1    0.618    0.618    0.618    0.618 /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/__init__.py:1(<module>)
+
    +
    +

    可以看到最耗时的函数是C++端的run函数。这需要联合我们第二节Python与C++混合代码的性能分析来进行调优。而sync_with_cpp函数的总共耗时很长，每次调用的耗时也很长。于是我们可以点击sync_with_cpp的详细信息，了解其调用关系。

    +
    Called By:
+
+   Ordered by: internal time
+   List reduced from 4497 to 2 due to restriction <'sync_with_cpp'>
+
+Function                                                                                                 was called by...
+                                                                                                             ncalls  tottime  cumtime
+/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:428(sync_with_cpp)  <-    4697    0.626    2.291  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp)
+/home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:562(sync_with_cpp)  <-    4696    0.019    2.316  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:487(clone)
+                                                                                                                  1    0.000    0.001  /home/yuyang/perf_test/.env/lib/python2.7/site-packages/paddle/v2/fluid/framework.py:534(append_backward)
+
+
+Called:
+
+   Ordered by: internal time
+   List reduced from 4497 to 2 due to restriction <'sync_with_cpp'>
+
    +
    +

    通常观察热点函数间的调用关系，和对应行的代码，就可以了解到问题代码在哪里。当我们做出性能修正后，再次进行性能分析(profiling)即可检查我们调优后的修正是否能够改善程序的性能。

    +
    +
    +
    +

    Python与C++混合代码的性能分析

    +
    +

    生成性能分析文件

    +

    C++的性能分析工具非常多。常见的包括gprof, valgrind, google-perftools。但是调试Python中使用的动态链接库与直接调试原始二进制相比增加了很多复杂度。幸而Python的一个第三方库yep提供了方便的和google-perftools交互的方法。于是这里使用yep进行Python与C++混合代码的性能分析

    +

    使用yep前需要安装google-perftools与yep包。ubuntu下安装命令为

    +
    apt update
+apt install libgoogle-perftools-dev
+pip install yep
+
    +
    +

    安装完毕后，我们可以通过

    +
    python -m yep -v main.py
+
    +
    +

    生成性能分析文件。生成的性能分析文件为main.py.prof。

    +

    命令行中的-v指定在生成性能分析文件之后，在命令行显示分析结果。我们可以在命令行中简单的看一下生成效果。因为C++与Python不同，编译时可能会去掉调试信息，运行时也可能因为多线程产生混乱不可读的性能分析结果。为了生成更可读的性能分析结果，可以采取下面几点措施:

    +
      +
    1. 编译时指定-g生成调试信息。使用cmake的话，可以将CMAKE_BUILD_TYPE指定为RelWithDebInfo。
      2. +
    2. 编译时一定要开启优化。单纯的Debug编译性能会和-O2或者-O3有非常大的差别。Debug模式下的性能测试是没有意义的。
      3. +
    3. 运行性能分析的时候，先从单线程开始，再开启多线程，进而多机。毕竟单线程调试更容易。可以设置OMP_NUM_THREADS=1这个环境变量关闭openmp优化。
      4. +
    +
    +
    +

    查看性能分析文件

    +

    在运行完性能分析后，会生成性能分析结果文件。我们可以使用pprof来显示性能分析结果。注意，这里使用了用Go语言重构后的pprof，因为这个工具具有web服务界面，且展示效果更好。

    +

    安装pprof的命令和一般的Go程序是一样的，其命令如下:

    +
    go get github.com/google/pprof
+
    +
    +

    进而我们可以使用如下命令开启一个HTTP服务:

    +
    pprof -http=0.0.0.0:3213 `which python`  ./main.py.prof
+
    +
    +

    这行命令中，-http指开启HTTP服务。which python会产生当前Python二进制的完整路径，进而指定了Python可执行文件的路径。./main.py.prof输入了性能分析结果。

    +

    访问对应的网址，我们可以查看性能分析的结果。结果如下图所示:

    +

    result

    +
    +
    +

    寻找性能瓶颈

    +

    与寻找Python代码的性能瓶颈类似，寻找Python与C++混合代码的性能瓶颈也是要看tottime和cumtime。而pprof展示的调用图也可以帮助我们发现性能中的问题。

    +

    例如下图中，

    +

    kernel_perf

    +

    在一次训练中，乘法和乘法梯度的计算占用2%-4%左右的计算时间。而MomentumOp占用了17%左右的计算时间。显然，MomentumOp的性能有问题。

    +

    在pprof中，对于性能的关键路径都做出了红色标记。先检查关键路径的性能问题，再检查其他部分的性能问题，可以更有次序的完成性能的优化。

    +
    +
    + + +
    +
    +
    + + +
    + +
    +

    + © Copyright 2016, PaddlePaddle developers. + +

    +
    + Built with Sphinx using a theme provided by Read the Docs. + +
    + +
    +
    + +
    + +
    + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc_cn/howto/optimization/gpu_profiling_cn.html b/doc_cn/howto/optimization/gpu_profiling_cn.html index c6fd8bd952b0f5bc1b8bf55485cf346bc78d6f52..b2ecfdbe19e9ca5136f6941ae231ac2dbe6a9a8c 100644 --- a/doc_cn/howto/optimization/gpu_profiling_cn.html +++ b/doc_cn/howto/optimization/gpu_profiling_cn.html @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ diff --git a/doc_cn/howto/usage/cluster/cluster_train_cn.html b/doc_cn/howto/usage/cluster/cluster_train_cn.html index f9b804e5e9f2506e5e07738ead7e1f0083034208..a4a6ead06c0bdc894902e868e95f8f364e6a1938 100644 --- a/doc_cn/howto/usage/cluster/cluster_train_cn.html +++ b/doc_cn/howto/usage/cluster/cluster_train_cn.html @@ -8,7 +8,7 @@ - 运行分布式训练 — PaddlePaddle 文档 + PaddlePaddle分布式训练 — PaddlePaddle 文档 @@ -65,7 +65,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -88,6 +88,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -111,9 +112,10 @@ @@ -178,7 +200,7 @@
  • 进阶指南 >
    • -
  • 运行分布式训练
    • +
  • PaddlePaddle分布式训练
    • @@ -187,139 +209,290 @@
    -
    -

    运行分布式训练

    -

    在本文中，我们将阐释如何在集群上运行分布式 Paddle 训练作业。我们将以推荐系统为例创建分布式的单进程训练。

    -

    在本文中使用的脚本通过 SSH 运行分布式作业。 它们还可以供那些运行更复杂的集群管理系统（如 MPI 和 Kubernetes ）的用户参考。

    -
    -

    前提条件

    -
      -

    1. 上述脚本使用 Python 库 fabric 来运行 SSH 命令。 我们使用 pip 来安装 fabric:

      -
      pip install fabric
-
      -
      +
      +

      PaddlePaddle分布式训练

      + +
      +

      概述

      +

      本文将介绍如何使用PaddlePaddle在不同的集群框架下完成分布式训练。分布式训练架构如下图所示：

      +

      +
        +
      • 数据分片（Data shard): 用于训练神经网络的数据，被切分成多个部分，每个部分分别给每个trainer使用。
        • +
      • 计算节点（Trainer）: 每个trainer启动后读取切分好的一部分数据，开始神经网络的“前馈”和“后馈”计算，并和参数服务器通信。在完成一定量数据的训练后，上传计算得出的梯度（gradients），然后下载优化更新后的神经网络参数（parameters）。
        • +
      • 参数服务器（Parameter server）:每个参数服务器只保存整个神经网络所有参数的一部分。参数服务器接收从计算节点上传的梯度，并完成参数优化更新，再将更新后的参数下发到每个计算节点。
        • +
      +

      这样，通过计算节点和参数服务器的分布式协作，可以完成神经网络的SGD方法的训练。PaddlePaddle可以同时支持同步随机梯度下降（SGD）和异步随机梯度下降。

      +

      在使用同步SGD训练神经网络时，PaddlePaddle使用同步屏障（barrier），使梯度的提交和参数的更新按照顺序方式执行。在异步SGD中，则并不会等待所有trainer提交梯度才更新参数，这样极大地提高了计算的并行性：参数服务器之间不相互依赖，并行地接收梯度和更新参数，参数服务器也不会等待计算节点全部都提交梯度之后才开始下一步，计算节点之间也不会相互依赖，并行地执行模型的训练。可以看出，虽然异步SGD方式会提高参数更新并行度, 但是并不能保证参数同步更新，在任意时间某一台参数服务器上保存的参数可能比另一台要更新，与同步SGD相比，梯度会有噪声。

      +
      +
      +

      环境准备

      +
        +
      1. 准备您的计算集群。计算集群通常由一组（几台到几千台规模）的Linux服务器组成。服务器之间可以通过局域网（LAN）联通，每台服务器具有集群中唯一的IP地址（或者可被DNS解析的主机名）。集群中的每台计算机通常被成为一个“节点”。
        2. +
      2. 我们需要在集群的所有节点上安装 PaddlePaddle。 如果要启用GPU，还需要在节点上安装对应的GPU驱动以及CUDA。PaddlePaddle的安装可以参考build_and_install的多种安装方式。我们推荐使用Docker安装方式来快速安装PaddlePaddle。
      +

      安装完成之后，执行下面的命令可以查看已经安装的版本（docker安装方式可以进入docker容器执行：docker run -it paddlepaddle/paddle:[tag] /bin/bash）：

      +
      $ paddle version
+PaddlePaddle 0.10.0, compiled with
+    with_avx: ON
+    with_gpu: OFF
+    with_double: OFF
+    with_python: ON
+    with_rdma: OFF
+    with_timer: OFF
+
      +
      +

      下面以doc/howto/usage/cluster/src/word2vec中的代码作为实例，介绍使用PaddlePaddle v2 API完成分布式训练。

      -

      准备工作空间

      -

      我们将放置依赖库、配置等文件的目录视为 工作空间（workspace）。

      -

      这些 train/test 数据应该在启动集群作业之前准备好。 为了满足训练/测试数据放置在工作空间中不同目录的要求，PADDLE 根据在模型配置文件中使用的名为 train.list/test.list 的索引文件引用训练/测试数据，所以训练/测试数据也包含 train.list/test.list 两个列表文件。所有本地训练 demo 已经提供了脚本来帮助您创建这两个文件，并且集群作业中的所有节点将在正常情况下处理具有相同逻辑代码的文件。

      -

      通常，你可以使用本地训练中的相同模型文件进行集群训练。请记住，在模型文件的 setting函数中设置的 batch_size 表示在集群作业每个节点中的 batch 大小，而不是使用同步 SGD 的总 batch 大小。

      -

      以下步骤基于 demo 目录中的 demo/recommendation。

      -

      你只需完成 demo/recommendation 教程文档到 Train 的部分，之后你会得到训练/测试数据和模型配置文件。最后，只需使用 demo/recommendation 作为集群训练的工作空间。

      -

      最后，你的工作空间应如下所示：

      -
      .
-|-- common_utils.py
-|-- data
-|   |-- config.json
-|   |-- config_generator.py
-|   |-- meta.bin
-|   |-- meta_config.json
-|   |-- meta_generator.py
-|   |-- ml-1m
-|   |-- ml_data.sh
-|   |-- ratings.dat.test
-|   |-- ratings.dat.train
-|   |-- split.py
-|   |-- test.list
-|   `-- train.list
-|-- dataprovider.py
-|-- evaluate.sh
-|-- prediction.py
-|-- preprocess.sh
-|-- requirements.txt
-|-- run.sh
-`-- trainer_config.py
+
      启动参数说明
      
+
      
+
      启动参数服务器
      
+
      执行以下的命令启动一个参数服务器并等待和计算节点的数据交互
      
+
      $ paddle pserver --port=7164 --ports_num=1 --ports_num_for_sparse=1 --num_gradient_servers=1
 
      
 
      
-
      虽然这些文件并非都需要集群训练，但是也没有必要删除无用的文件。
      
-
      trainer_config.py
-表示模型配置文件。
      
-
      train.list 和 test.list
-文件索引。它存储当前节点所有训练/测试数据的所有相对或绝对文件路径。
      
-
      dataprovider.py
-用于读取训练/测试样本。这与本地训练相同。
      
-
      data
-数据目录中的所有文件被 train.list/test.list 引用。
      
+
      如果希望可以在后台运行pserver程序，并保存输出到一个日志文件，可以运行：
      
+
      $ stdbuf -oL /usr/bin/nohup paddle pserver --port=7164 --ports_num=1 --ports_num_for_sparse=1 --num_gradient_servers=1 &> pserver.log
+
      
+
      
+
      | 参数  | 是否必选 | 默认值 | 说明 |
+| ————- | ————- | ————- | ————- |
+| port  | 必选 | 7164 | pserver监听的起始端口，根据ports_num决定
      总端口个数，从起始端口监听多个端口用于通信  |
+| ports_num  | 必选 | 1 | 监听的端口个数  |
+| ports_num_for_sparse  | 必选 | 1 | 用于稀疏类型参数通信的端口个数  |
+| num_gradient_servers  | 必选 | 1 | 当前训练任务pserver总数 |
      
 
      
 
      
-
      准备集群作业配置
      
-
      以下选项必须在 cluster_train/conf.py 中认真设置
      
-
      HOSTS  所有节点运行集群作业的主机名或 IP 。你还可以将用户和 ssh 端口附加到主机名上，例如 root@192.168.100.17:9090。
      
-
      ROOT_DIR 用于放置 JOB 工作空间目录的工作空间 ROOT 目录
      
-
      PADDLE_NIC 集群通信通道的 NIC(Network Interface Card, 网络接口卡) 接口名称，例如以太网的 eth0，infiniband 的 ib0。
      
-
      PADDLE_PORT 集群通信通道的端口号
      
-
      PADDLE_PORTS_NUM 用于集群通信通道的端口数。 如果集群节点数量少（少于5〜6个节点），建议将其设置为较大，如2〜8，以获得更好的网络性能。
      
-
      PADDLE_PORTS_NUM_FOR_SPARSE 用于 sparse remote updater 集群通信信道的端口数。如果使用 sparse remote update，则可以像 PADDLE_PORTS_NUM 一样设置。
      
-
      LD_LIBRARY_PATH 为集群作业设置额外的 LD_LIBRARY_PATH。你可以使用它来设置 CUDA 库的路径。
      
-
      默认配置如下：
      
-
      HOSTS = [
-        "root@192.168.100.17",
-        "root@192.168.100.18",
-        ]
-
-'''
-工作空间配置
-'''
-
-#工作空间根目录
-ROOT_DIR = "/home/paddle"
-
-'''
-网络配置
-'''
-#pserver NIC
-PADDLE_NIC = "eth0"
-#pserver 端口
-PADDLE_PORT = 7164
-#pserver 端口数
-PADDLE_PORTS_NUM = 2
-#pserver sparse ports num
-PADDLE_PORTS_NUM_FOR_SPARSE = 2
-
-#集群作业中所有进程的环境设置
-LD_LIBRARY_PATH="/usr/local/cuda/lib64:/usr/lib64"
+
      启动计算节点
      
+
      执行以下命令启动使用python编写的trainer程序（文件名为任意文件名，如train.py）
      
+
      $ python train.py
+
      
+
      
+
      trainer需要和pserver保持网络联通以完成训练。trainer启动需要传入端口、pserver地址等参数使trainer可以正确连接到pserver。这些参数可以通过环境变量（https://zh.wikipedia.org/wiki/环境变量 ）或编写程序时paddle.init()中传入参数。如果同时使用paddle.init()参数和环境变量，将会优先使用paddle.init()中传入的参数。
      
+
      使用环境变量：
      
+
      export PADDLE_INIT_USE_GPU=False
+export PADDLE_INIT_TRAINER_COUNT=1
+export PADDLE_INIT_PORT=7164
+export PADDLE_INIT_PORTS_NUM=1
+export PADDLE_INIT_PORTS_NUM_FOR_SPARSE=1
+export PADDLE_INIT_NUM_GRADIENT_SERVERS=1
+export PADDLE_INIT_TRAINER_ID=0
+export PADDLE_INIT_PSERVERS=127.0.0.1
 
      
 
      
+
      使用参数：
      
+
      paddle.init(
+        use_gpu=False,
+        trainer_count=1,
+        port=7164,
+        ports_num=1,
+        ports_num_for_sparse=1,
+        num_gradient_servers=1,
+        trainer_id=0,
+        pservers="127.0.0.1")
+
      
+
      
+
      | 参数  | 是否必选 | 默认 | 说明 |
+| ————- | ————- | ————- | ————- |
+| use_gpu  | 可选 | False | 是否启用GPU训练 |
+| trainer_count  | 必选 | 1 | 当前训练任务trainer总个数 |
+| port  | 必选 | 7164 | 连接到pserver的端口  |
+| ports_num  | 必选 | 1 | 连接到pserver的端口个数  |
+| ports_num_for_sparse  | 必选 | 1 | 和pserver之间用于稀疏类型参数通信的端口个数  |
+| num_gradient_servers  | 必选 | 1 | 当前训练任务pserver总数 |
+| trainer_id  | 必选 | 0 | 每个trainer的唯一ID，从0开始的整数 |
+| pservers  | 必选 | 127.0.0.1 | 当前训练任务启动的pserver的IP列表，多个IP使用“,”隔开 |
      
+
      
 
      
-
      启动集群作业
      
-
      paddle.py 提供了自动化脚本来启动不同节点中的所有 PaddlePaddle 集群进程。默认情况下，所有命令行选项可以设置为paddle.py 命令选项并且 paddle.py 将透明、自动地将这些选项应用到 PaddlePaddle 底层进程。
      
+
      准备数据集
      
+
      参考样例数据准备脚本prepare.py，准备训练数据和验证数据集，我们使用paddle.dataset.imikolov数据集，并根据分布式训练并发数（trainer节点个数），在prepare.py开头部分指定SPLIT_COUNT将数据切分成多份。
      
+
      在线上系统中，通常会使用MapReduce任务的输出结果作为训练结果，这样训练文件的个数会比较多，而且个数并不确定。在trainer中可以使用下面取模的方法为每个trainer分配训练数据文件：
      
+
      import os
+train_list = []
+flist = os.listdir("/train_data/")
+for f in flist:
+  suffix = int(f.split("-")[1])
+  if suffix % TRAINER_COUNT == TRAINER_ID:
+    train_list.append(f)
+
      
+
      
+
      示例程序prepare.py会把训练集和测试集分别分割成多个文件（例子中为3个，后缀为-00000、-00001和-00002）:
      
+
      train.txt
+train.txt-00000
+train.txt-00001
+train.txt-00002
+test.txt
+test.txt-00000
+test.txt-00001
+test.txt-00002
+
      
+
      
+
      在进行分布式训练时，每个trainer进程需要能够读取属于自己的一份数据。在一些分布式系统中，系统会提供一个分布式存储服务，这样保存在分布式存储中的数据可以被集群中的每个节点读取到。如果不使用分布式存储，则需要手动拷贝属于每个trainer节点的训练数据到对应的节点上。
      
+
      对于不同的训练任务，训练数据格式和训练程序的reader()会大不相同，所以开发者需要根据自己训练任务的实际场景完成训练数据的分割和reader()的编写。
      
+
      
+
      
+
      准备训练程序
      
+
      我们会对每个训练任务都会在每个节点上创建一个工作空间（workspace），其中包含了用户的训练程序、程序依赖、挂载或下载的训练数据分片。
      
+
      最后，工作空间应如下所示：
      
+
      .
+|-- my_lib.py
+|-- word_dict.pickle
+|-- train.py
+|-- train_data_dir/
+|   |-- train.txt-00000
+|   |-- train.txt-00001
+|   |-- train.txt-00002
+`-- test_data_dir/
+    |-- test.txt-00000
+    |-- test.txt-00001
+    `-- test.txt-00002
+
      
+
      
+
        
+
      • my_lib.py：会被train.py调用的一些用户定义的库函数，比如PIL库等。
        
+
        • 
+
      • word_dict.pickle：在train.py中会使用到的字典数据文件。
        
+
        • 
+
      • train.py：训练程序，代码参考api_train_v2_cluster.py。注意： 对于本样例代码，在使用不同的分布式计算平台时，您可能需要修改train.py开头的部分（如下），以便获得训练数据的位置和获取环境变量配置：
        
+
        cluster_train_file = "./train_data_dir/train/train.txt"
+cluster_test_file = "./test_data_dir/test/test.txt"
+node_id = os.getenv("OMPI_COMM_WORLD_RANK")
+if not node_id:
+    raise EnvironmentError("must provied OMPI_COMM_WORLD_RANK")
+
        
+
        
+
        • 
+
      • train_data_dir：包含训练数据的目录，可以是从分布式存储挂载过来的，也可以是在任务启动前下载到本地的。
        
+
        • 
+
      • test_data_dir：包含测试数据集的目录。
        
+
        • 
+
      
+
      
+
      
+
      
+
      使用分布式计算平台或工具
      
+
      PaddlePaddle可以使用多种分布式计算平台构建分布式计算任务，包括：
      
+
        
+
      • Kubernetes Google开源的容器集群的调度框架，支持大规模集群生产环境的完整集群方案。
        • 
+
      • OpenMPI 成熟的高性能并行计算框架。
        • 
+
      • Fabric 集群管理工具。可以使用Fabric编写集群任务提交和管理脚本。
        • 
+
      
+
      对于不同的集群平台，会分别介绍集群作业的启动和停止方法。这些例子都可以在cluster_train_v2找到。
      
+
      在使用分布式计算平台进行训练时，任务被调度在集群中时，分布式计算平台通常会通过API或者环境变量提供任务运行需要的参数，比如节点的ID、IP和任务节点个数等。
      
+
      
+
      使用Fabric启动集群作业
      
+
      
+
      准备一个Linux集群
      
+
      可以在paddle/scripts/cluster_train_v2/fabric/docker_cluster目录下，执行kubectl -f ssh_servers.yaml启动一个测试集群，并使用kubectl get po -o wide获得这些节点的IP地址。
      
+
      
+
      
+
      启动集群作业
      
+
      paddle.py 提供了自动化脚本来启动不同节点中的所有 PaddlePaddle 集群进程。默认情况下，所有命令行选项可以设置为 paddle.py 命令选项并且 paddle.py 将透明、自动地将这些选项应用到 PaddlePaddle 底层进程。
      
 
      paddle.py 为方便作业启动提供了两个独特的命令选项。
      
-
      job_dispatch_package  设为本地 workspace 目录，它将被分发到 conf.py 中设置的所有节点。  它有助于帮助频繁修改和访问工作区文件的用户减少负担，否则频繁的多节点工作空间部署可能会很麻烦。
-job_workspace  设为已部署的工作空间目录，paddle.py 将跳过分发阶段直接启动所有节点的集群作业。它可以帮助减少分发延迟。
      
-
      cluster_train/run.sh 提供了命令样例来运行 demo/recommendation 集群工作，只需用你定义的目录修改 job_dispatch_package 和 job_workspace，然后：
      
+
        
+
      • job_dispatch_package  设为本地 workspace 目录，它将被分发到 conf.py 中设置的所有节点。它有助于帮助频繁修改和访问工作区文件的用户减少负担，否则频繁的多节点工作空间部署可能会很麻烦。
        • 
+
      • job_workspace  设为已部署的工作空间目录，paddle.py 将跳过分发阶段直接启动所有节点的集群作业。它可以帮助减少分发延迟。
        • 
+
      
+
      cluster_train/run.sh 提供了命令样例来运行 doc/howto/usage/cluster/src/word2vec 集群任务，只需用您定义的目录修改 job_dispatch_package 和 job_workspace，然后：
      
 
      sh run.sh
 
      
 
      
 
      集群作业将会在几秒后启动。
      
 
      
 
      
-
      终止集群作业
      
+
      终止集群作业
      
 
      paddle.py能获取Ctrl + C SIGINT 信号来自动终止它启动的所有进程。只需中断 paddle.py 任务来终止集群作业。如果程序崩溃你也可以手动终止。
      
 
      
 
      
-
      检查集群训练结果
      
+
      检查集群训练结果
      
 
      详细信息请检查 $workspace/log 里的日志，每一个节点都有相同的日志结构。
      
 
      paddle_trainer.INFO
 提供几乎所有训练的内部输出日志，与本地训练相同。这里检验运行时间模型的收敛。
      
 
      paddle_pserver2.INFO
 提供 pserver 运行日志，有助于诊断分布式错误。
      
 
      server.log
-提供 pserver 进程的 stderr 和 stdout。训练失败时可以检查错误日志。
      
+提供 parameter server 进程的 stderr 和 stdout。训练失败时可以检查错误日志。
      
 
      train.log
 提供训练过程的 stderr 和 stdout。训练失败时可以检查错误日志。
      
 
      
 
      
-
      检查模型输出
      
+
      检查模型输出
      
 
      运行完成后，模型文件将被写入节点 0 的 output 目录中。
 工作空间中的 nodefile 表示当前集群作业的节点 ID。
      
 
      
 
      
+
      
+
      在OpenMPI集群中提交训练作业
      
+
      
+
      准备OpenMPI集群
      
+
      执行下面的命令以启动3个节点的OpenMPI集群和一个”head”节点：
      
+
      paddle/scripts/cluster_train_v2/openmpi/docker_cluster
+kubectl create -f head.yaml
+kubectl create -f mpi-nodes.yaml
+
      
+
      
+
      然后可以从head节点ssh无密码登录到OpenMPI的每个节点上。
      
+
      
+
      
+
      启动集群作业
      
+
      您可以按照下面的步骤在OpenMPI集群中提交paddle训练任务：
      
+
      # 获得head和node节点的IP地址
+kubectl get po -o wide
+# 将node节点的IP地址保存到machines文件中
+kubectl get po -o wide | grep nodes | awk '{print $6}' > machines
+# 拷贝必要的文件到head节点
+scp -i ssh/id_rsa.mpi.pub machines prepare.py train.py start_mpi_train.sh tutorial@[headIP]:~
+# ssh 登录到head节点
+ssh -i ssh/id_rsa.mpi.pub tutorial@[headIP]
+# --------------- 以下操作均在head节点中执行 ---------------
+# 准备训练数据
+python prepare.py
+# 拷贝训练程序和字典文件到每台MPI节点
+cat machines | xargs -i scp word_dict.pickle train.py start_mpi_train.sh machines {}:/home/tutorial
+# 创建日志目录
+mpirun -hostfile machines -n 3 mkdir /home/tutorial/logs
+# 拷贝训练数据到各自的节点
+scp train.txt-00000 test.txt-00000 [node1IP]:/home/tutorial
+scp train.txt-00001 test.txt-00001 [node2IP]:/home/tutorial
+scp train.txt-00002 test.txt-00002 [node3IP]:/home/tutorial
+# 启动训练任务
+mpirun -hostfile machines -n 3  /home/tutorial/start_mpi_train.sh
+
      
+
      
+
      
+
      
+
      
+
      在Kubernetes集群中提交训练作业
      
+
      此部分的使用方法可以参考here。
      
+
      
+
      
 
      
 
 
diff --git a/doc_cn/howto/usage/cmd_parameter/arguments_cn.html b/doc_cn/howto/usage/cmd_parameter/arguments_cn.html
index 158a6dcfd807af61d419614129e666fe62d1c87c..c87f6b5bad6d3c2f8e7f6c12a4e023c468291de4 100644
--- a/doc_cn/howto/usage/cmd_parameter/arguments_cn.html
+++ b/doc_cn/howto/usage/cmd_parameter/arguments_cn.html
@@ -65,7 +65,7 @@
      - Folk me on Github + Fork me on Github
      English @@ -88,6 +88,7 @@
    2. 进阶指南
    3. API
    4. FAQ
      5. +
    5. MOBILE
      6. @@ -111,9 +112,10 @@ @@ -233,7 +255,7 @@
    		show_layer_stat √√√√
    训练dot_period训练dot_period √√
    test_period
    ++ + + + + + + + + + + + + + + + + + + + + + + +
    ArgumentOptional ValuesDefault
    ANDROID_ABIarmeabi-v7a, arm64-v8aarmeabi-v7a
    ANDROID_API>= 2121
      +

    • 编译armeabi-v7a，Android API 21的PaddlePaddle库

      +
      $ docker run -it --rm -v $PWD:/paddle -e "ANDROID_ABI=armeabi-v7a" -e "ANDROID_API=21" username/paddle-android:dev
+
      +
      +
      • +

    • 编译arm64-v8a，Android API 21的PaddlePaddle库

      +
      $ docker run -it --rm -v $PWD:/paddle -e "ANDROID_ABI=arm64-v8a" -e "ANDROID_API=21" username/paddle-android:dev
+
      +
      +
      • +
    +

    执行上述docker run命令时，容器默认执行paddle/scripts/docker/build_android.sh脚本。该脚本中记录了交叉编译Android版PaddlePaddle库常用的CMake配置，并且会根据ANDROID_ABI和ANDROID_API自动构建独立工具链、进行编译和安装。由于arm64架构要求Android API不小于21。因此当ANDROID_ABI=arm64-v8a，ANDROID_API<21时，Docker容器中将默认使用Android API 21的编译工具链。用户可以参考下文配置交叉编译参数章节，根据个人的需求修改定制Docker容器所执行的脚本。编译安装结束之后，PaddlePaddle的C-API库将被安装到$PWD/install_android目录，所依赖的第三方库同时也被安装到$PWD/install_android/third_party目录。

    +
    +
    +
    +

    基于Linux交叉编译环境的编译方式

    +

    本文档将以Linux x86-64平台为例，介绍交叉编译Android平台上适用的PaddlePaddle库的方法和步骤。

    +
    +

    准备交叉编译环境

    +

    从源码交叉编译PaddlePaddle，用户需要提前准备好交叉编译环境。Android平台上使用的C/C++交叉编译工具链为Android NDK，用户可自行前往下载预编译好的版本，也可通过以下命令获取：

    +
    wget -q https://dl.google.com/android/repository/android-ndk-r14b-linux-x86_64.zip
+unzip -q android-ndk-r14b-linux-x86_64.zip
+
    +
    +

    Android NDK中包含了所有Android API级别、所有架构（arm/arm64/x86/mips）需要用到的编译工具和系统库。用户可根据自己的编译目标架构、所需支持的最低Android API级别，构建独立工具链。

    +
      +
    • 构建armeabi-v7a、 Android API 21的独立工具链：
      • +
    +
    your/path/to/android-ndk-r14b-linux-x86_64/build/tools/make-standalone-toolchain.sh \
+        --arch=arm --platform=android-21 --install-dir=your/path/to/arm_standalone_toolchain
+
    +
    +

    此命令将在your/path/to/arm_standalone_toolchain目录生成一套独立编译工具链，面向架构为32位ARM架构，支持的最小的Android API级别为21，支持编译器arm-linux-androideabi-gcc (GCC) 4.9和clang 3.8。

    +
      +
    • 构建arm64-v8a、 Android API 21的独立工具链：
      • +
    +
    your/path/to/android-ndk-r14b-linux-x86_64/build/tools/make-standalone-toolchain.sh \
+        --arch=arm64 --platform=android-21 --install-dir=your/path/to/arm64_standalone_toolchain
+
    +
    +

    此命令将在your/path/to/arm64_standalone_toolchain目录生成一套独立编译工具链，面向架构为64位ARM64架构，支持的最小Android API级别为21，支持编译器arm-linux-androideabi-gcc (GCC) 4.9和clang 3.8。

    +

    注意：PaddlePaddle要求使用的编译工具链所支持的Android API级别不小于21。

    +
    +
    +

    配置交叉编译参数

    +

    CMake系统对交叉编译提供了支持cmake-toolchains。为了简化cmake配置，PaddlePaddle为交叉编译提供了工具链配置文档cmake/cross_compiling/android.cmake，以提供一些默认的编译器和编译参数相关配置。注意，从CMake 3.7版本开始，CMake官方对Android平台的交叉编译提供了通用的支持。PaddlePaddle若检测到用户使用的CMake版本不低于3.7时，将会将用户传进来的配置参数传递CMake系统，交由CMake系统本身来处理。有关参数配置的详细说明见cmake-toolchains。

    +

    交叉编译Android版本的PaddlePaddle库时，有一些必须配置的参数：

    +
      +
    • CMAKE_SYSTEM_NAME，CMake编译的目标平台，必须设置为Android。在设置CMAKE_SYSTEM_NAME=Android后，PaddlePaddle的CMake系统才认为是在交叉编译Android系统的版本，并自动编译宿主机版protoc可执行文件、目标机版protobuf库、以及Android所需arm_soft_fp_abi分支的目标机版OpenBLAS库。此外，还会强制设置一些PaddlePaddle参数的值（WITH_GPU=OFF、WITH_AVX=OFF、WITH_PYTHON=OFF、WITH_RDMA=OFF）。
      • +
    • WITH_C_API，必须设置为ON。在Android平台上只支持使用C-API来预测。
      • +
    • WITH_SWIG_PY，必须设置为OFF。在Android平台上不支持通过swig调用来训练或者预测。
      • +
    +

    Android平台可选配置参数：

    +
      +
    • ANDROID_STANDALONE_TOOLCHAIN，独立工具链所在的绝对路径，或者相对于构建目录的相对路径。PaddlePaddle的CMake系统将根据该值自动推导和设置需要使用的交叉编译器、sysroot、以及Android API级别；否则，用户需要在cmake时手动设置这些值。无默认值。
      • +
    • ANDROID_TOOLCHAIN，目标工具链。可设置gcc/clang，默认值为clang。
        +
      • CMake 3.7以上，将会始终使用clang工具链；CMake 3.7以下，可设置ANDROID_TOOLCHAIN=gcc以使用gcc工具链。
        • +
      • Android官方提供的clang编译器要求系统支持GLIBC 2.15以上。
        • +
      +
      • +
    • ANDROID_ABI，目标架构ABI。目前支持armeabi-v7a和arm64-v8a，默认值为armeabi-v7a。
      • +
    • ANDROID_NATIVE_API_LEVEL，工具链的Android API级别。若没有显式设置，PaddlePaddle将根据ANDROID_STANDALONE_TOOLCHAIN的值自动推导得到。
      • +
    • ANROID_ARM_MODE，是否使用ARM模式。
        +
      • ANDROID_ABI=armeabi-v7a时，可设置ON/OFF，默认值为ON；
        • +
      • ANDROID_ABI=arm64-v8a时，不需要设置。
        • +
      +
      • +
    • ANDROID_ARM_NEON，是否使用NEON指令。
        +
      • ANDROID_ABI=armeabi-v7a时，可设置ON/OFF，默认值为ON；
        • +
      • ANDROID_ABI=arm64-v8a时，不需要设置。
        • +
      +
      • +
    +

    其他配置参数：

    +
      +
    • USE_EIGEN_FOR_BLAS，是否使用Eigen库进行矩阵计算。可设置ON/OFF，默认值为OFF。
      • +
    • HOST_C/CXX_COMPILER，宿主机的C/C++编译器。在编译宿主机版protoc可执行文件和目标机版OpenBLAS库时需要用到。默认设置成环境变量CC的值；若环境变量CC没有设置，则设置成cc编译器。
      • +
    +

    常用的cmake配置如下：

    +
    cmake -DCMAKE_SYSTEM_NAME=Android \
+      -DANDROID_STANDALONE_TOOLCHAIN=your/path/to/arm_standalone_toolchain \
+      -DANDROID_ABI=armeabi-v7a \
+      -DANDROID_ARM_NEON=ON \
+      -DANDROID_ARM_MODE=ON \
+      -DUSE_EIGEN_FOR_BLAS=ON \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_C_API=ON \
+      -DWITH_SWIG_PY=OFF \
+      ..
+
    +
    +
    cmake -DCMAKE_SYSTEM_NAME=Android \
+      -DANDROID_STANDALONE_TOOLCHAIN=your/path/to/arm64_standalone_toolchain \
+      -DANDROID_ABI=arm64-v8a \
+      -DUSE_EIGEN_FOR_BLAS=OFF \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_C_API=ON \
+      -DWITH_SWIG_PY=OFF \
+      ..
+
    +
    +

    用户还可根据自己的需求设置其他编译参数。比如希望最小化生成的库的大小，可以设置CMAKE_BUILD_TYPE为MinSizeRel；若希望最快的执行速度，则可设置CMAKE_BUILD_TYPE为Release。亦可以通过手动设置CMAKE_C/CXX_FLAGS_MINSIZEREL/RELEASE来影响PaddlePaddle的编译过程。

    +

    性能TIPS，为了达到最快的计算速度，在CMake参数配置上，有以下建议：

    +
      +
    • 设置CMAKE_BUILD_TYPE为Release
      • +
    • 使用clang编译工具链
      • +
    • armeabi-v7a时，设置USE_EIGEN_BLAS=ON，使用Eigen进行矩阵计算；arm64-v8a时，设置USE_EIGEN_FOR_BLAS=OFF，使用OpenBLAS进行矩阵计算
      • +
    +
    +
    +

    编译和安装

    +

    CMake配置完成后，执行以下命令，PaddlePaddle将自动下载和编译所有第三方依赖库、编译和安装PaddlePaddle预测库。

    +
    make
+make install
+
    +
    +

    注意：如果你曾经在源码目录下编译过其他平台的PaddlePaddle库，请先使用rm -rf命令删除third_party目录和build目录，以确保所有的第三方依赖库和PaddlePaddle代码都是针对新的CMake配置重新编译的。

    +

    执行完安装命令后，your/path/to/install目录中会包含include、lib和third_party目录，其中include中包含C-API的头文件，lib中包含若干个不同Android ABI的PaddlePaddle库，third_party中包含所依赖的所有第三方库。自此，PaddlePaddle的已经安装完成，用户可将your/path/to/install目录下的生成文件用于深度学习相关Android App中，调用方法见C-API文档。

    +
    +
    +
    + + +
    +
    + + +
    +
    + + + +
    + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc_cn/mobile/cross_compiling_for_ios_cn.html b/doc_cn/mobile/cross_compiling_for_ios_cn.html new file mode 100644 index 0000000000000000000000000000000000000000..ce62ad6224c778695cafe7892d4f7c7406c173ad --- /dev/null +++ b/doc_cn/mobile/cross_compiling_for_ios_cn.html @@ -0,0 +1,396 @@ + + + + + + + + + + + iOS平台编译指南 — PaddlePaddle 文档 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + +
    +
    + Fork me on Github +
    + + English + + + + +
    + +
    +
    + + + + +
    +
    + + + +
    +
    +
    +
    +
    + +
    + + + + +
    + + + + + + + + + + + +
    +
      + +
    • MOBILE >
      • + +
    • iOS平台编译指南
      • +
    +
    + +
    +
    +
    +
    + +
    +

    iOS平台编译指南

    +

    交叉编译iOS平台上适用的PaddlePaddle库，需要在MacOS系统上进行。本文的将介绍在MacOS上，从源码交叉编译iOS平台上适用的PaddlePaddle库。

    +
    +

    准备交叉编译环境

    +

    Apple官方为iOS开发提供了完整的交叉编译工具和集成开发环境，用户从App Store下载安装Xcode即可。也可自行前往官网下载，Xcode。安装完成之后，可在命令行执行xcodebuild -version，判断是否安装成功。

    +
    $ xcodebuild -version
+Xcode 9.0
+Build version 9A235
+
    +
    +
    +
    +

    配置交叉编译参数

    +

    PaddlePaddle为交叉编译提供了工具链配置文档cmake/cross_compiling/ios.cmake，以提供一些默认的编译器和编译参数配置。

    +

    交叉编译iOS版本的PaddlePaddle库时，有一些必须配置的参数：

    +
      +
    • CMAKE_SYSTEM_NAME，CMake编译的目标平台，必须设置为iOS。在设置CMAKE_SYSTEM_NAME=iOS后，PaddlePaddle的CMake系统会自动编译所有的第三方依赖库，并且强制设置一些PaddlePaddle参数的值（WITH_C_API=ON、WITH_GPU=OFF、WITH_AVX=OFF、WITH_PYTHON=OFF、WITH_RDMA=OFF）。
      • +
    • WITH_C_API，是否编译C-API预测库，必须设置为ON。在iOS平台上只支持使用C-API来预测。
      • +
    • WITH_SWIG_PY，必须设置为ON。在iOS平台上不支持通过swig调用来训练或者预测。
      • +
    +

    iOS平台可选配置参数：

    +
      +

    • IOS_PLATFORM，可设置为OS/SIMULATOR，默认值为OS。

      +
        +
      • OS，构建目标为arm架构的iPhone或者iPad等物理设备。
        • +
      • SIMULATOR，构建目标为x86架构的模拟器平台。
        • +
      +
      • +

    • IOS_ARCH，目标架构。针对不同的IOS_PLATFORM，可设置的目标架构如下表所示，默认编译所有架构：

      + + + + + + + + + + + + + + + + + + + + + +
      IOS_PLATFORMIOS_ARCH
      OSarmv7, armv7s, arm64
      SIMULATORi386, x86_64
      • +

    • IOS_DEPLOYMENT_TARGET，最小的iOS部署版本，默认值为7.0。

      +
      • +

    • IOS_ENABLE_BITCODE，是否使能Bitcode，可设置ON/OFF，默认值为ON。

      +
      • +

    • IOS_USE_VECLIB_FOR_BLAS，是否使用vecLib框架进行BLAS矩阵计算，可设置ON/OFF，默认值为OFF。

      +
      • +

    • IOS_DEVELOPMENT_ROOT，Developer目录，可显式指定为/path/to/platform/Developer。若未显式指定，PaddlePaddle将会根据IOS_PLATFORM自动选择Xcode对应platform的Developer目录。

      +
      • +

    • IOS_SDK_ROOT，所使用SDK的根目录，可显式指定为/path/to/platform/Developer/SDKs/SDK。若未显式指定，PaddlePaddle将会自动选择IOS_DEVELOPMENT_ROOT目录下最新的SDK版本。

      +
      • +
    +

    其他配置参数：

    +
      +
    • USE_EIGEN_FOR_BLAS，是否使用Eigen库进行矩阵计算，在IOS_USE_VECLIB_FOR_BLAS=OFF时有效。可设置ON/OFF，默认值为OFF。
      • +
    • HOST_C/CXX_COMPILER，宿主机的C/C++编译器。默认值为环境变量CC/CXX的值；若环境变量CC/CXX未设置，则使用cc/c++编译器。
      • +
    +

    常用的cmake配置如下：

    +
    cmake -DCMAKE_SYSTEM_NAME=iOS \
+      -DIOS_PLATFORM=OS \
+      -DIOS_ARCH="armv7;arm64" \
+      -DIOS_ENABLE_BITCODE=ON \
+      -DIOS_USE_VECLIB_FOR_BLAS=ON \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_C_API=ON \
+      -DWITH_TESTING=OFF \
+      -DWITH_SWIG_PY=OFF \
+      ..
+
    +
    +
    cmake -DCMAKE_SYSTEM_NAME=iOS \
+      -DIOS_PLATFORM=SIMULATOR \
+      -DIOS_ARCH="x86_64" \
+      -DIOS_USE_VECLIB_FOR_BLAS=ON \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_C_API=ON \
+      -DWITH_TESTING=OFF \
+      -DWITH_SWIG_PY=OFF \
+      ..
+
    +
    +

    用户还可根据自己的需求设置其他编译参数。比如希望最小化生成库的大小，可以设置CMAKE_BUILD_TYPE为MinSizeRel；若希望得到最快的执行速度，则可设置CMAKE_BUILD_TYPE为Release。亦可以通过手动设置CMAKE_C/CXX_FLAGS来影响PaddlePaddle的编译过程。

    +

    性能TIPS，为了达到最快的计算速度，在CMake参数配置上，有以下建议：

    +
      +
    • 设置CMAKE_BUILD_TYPE为Release
      • +
    • 设置IOS_USE_VECLIB_FOR_BLAS=ON，调用vecLib框架提供的BLAS函数进行矩阵计算。
      • +
    +
    +
    +

    编译和安装

    +

    CMake配置完成后，执行以下命令，PaddlePaddle将自动下载和编译所有第三方依赖库、编译和安装PaddlePaddle预测库。

    +
    $ make
+$ make install
+
    +
    +

    注意：如果你曾在源码目录下编译过其他平台的PaddlePaddle库，请先使用rm -rf命令删除third_party目录和build目录，以确保所有的第三方依赖库和PaddlePaddle代码都是针对新的CMake配置重新编译的。

    +

    执行完安装命令后，your/path/to/install目录中会包含以下内容：

    +
      +
    • include目录，其中包含所有C-API的头文件
      • +
    • lib目录，其中包含PaddlePaddle的C-API静态库
      • +
    • third_party目录，其中包含所依赖的所有第三方库
      • +
    +

    注意，如果PaddlePaddle库需要同时支持真机和模拟器，则需要分别编译真机和模拟器版本，然后使用lipo工具合并fat库。

    +

    自此，PaddlePaddle库已经安装完成，用户可将合成的fat库用于深度学习相关的iOS App中，调用方法见C-API文档。

    +
    +
    + + +
    +
    + + +
    +
    + +
    + +
    + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc_cn/mobile/cross_compiling_for_raspberry_cn.html b/doc_cn/mobile/cross_compiling_for_raspberry_cn.html new file mode 100644 index 0000000000000000000000000000000000000000..071ed3dcedb68299c1910bf3432011474376dd72 --- /dev/null +++ b/doc_cn/mobile/cross_compiling_for_raspberry_cn.html @@ -0,0 +1,327 @@ + + + + + + + + + + + Raspberry Pi平台编译指南 — PaddlePaddle 文档 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + +
    +
    + Fork me on Github +
    + + English + + + + +
    + +
    +
    + + + + +
    +
    + + + +
    +
    +
    +
    +
    + +
    + + + + +
    + + + + + + + + + + + +
    +
      + +
    • MOBILE >
      • + +
    • Raspberry Pi平台编译指南
      • +
    +
    + +
    +
    +
    +
    + +
    +

    Raspberry Pi平台编译指南

    +

    通常有两个方法来构建基于 Rasspberry Pi 的版本：

    +
      +
    1. 通过ssh等方式登录到Raspberry Pi系统上来构建。所需的开发工具和第三方库可以参考 /Dockerfile。
      2. +
    2. 另一个方法是交叉编译。这篇文档介绍在 Linux/x64 上交叉编译Raspberry Pi平台上适用的PaddlePaddle的方法和步骤。
      3. +
    +
    +

    安装交叉编译器

    +

    克隆下面 Github repo

    +
    git clone https://github.com/raspberrypi/tools.git
+
    +
    +

    即可在 ./tools/tree/master/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64 目录里找到交叉编译器 arm-linux-gnueabihf-gcc 4.8.3。运行该编译工具链需要一台 Linux x64 机器上以及 2.14版本以上的 glibc。

    +
    +
    +

    配置交叉编译参数

    +

    CMake支持交叉编译。PaddlePaddle for Raspberry Pi的配置信息在cmake/cross_compiling/raspberry_pi.cmake。

    +

    交叉编译Raspberry Pi版本PaddlePaddle库时，有一些必须配置的参数：

    +
      +
    • CMAKE_SYSTEM_NAME：CMake编译的目标平台，必须配置为RPi。在设置CMAKE_SYSTEM_NAME=RPi后，PaddlePaddle的CMake系统才认为在是在交叉编译Raspberry Pi系统的版本，并自动编译宿主机版protoc可执行文件、目标机版protobuf库、以及目标机版OpenBLAS库。
      • +
    • RPI_TOOLCHAIN：编译工具链所在的绝对路径，或者相对于构建目录的相对路径。PaddlePaddle的CMake系统将根据该值自动设置需要使用的交叉编译器；否则，用户需要在cmake时手动设置这些值。无默认值。
      • +
    • RPI_ARM_NEON：是否使用NEON指令。目前必须设置成ON，默认值为ON。
      • +
    • HOST_C/CXX_COMPILER，宿主机的C/C++编译器。在编译宿主机版protoc可执行文件和目标机版OpenBLAS库时需要用到。默认设置成环境变量CC的值；若环境变量CC没有设置，则设置成cc编译器。
      • +
    +

    一个常用的CMake配置如下：

    +
    cmake -DCMAKE_SYSTEM_NAME=RPi \
+      -DRPI_TOOLCHAIN=your/path/to/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64 \
+      -DRPI_ARM_NEON=ON \
+      -DCMAKE_INSTALL_PREFIX=your/path/to/install \
+      -DWITH_GPU=OFF \
+      -DWITH_C_API=ON \
+      -DWITH_PYTHON=OFF \
+      -DWITH_SWIG_PY=OFF \
+      ..
+
    +
    +

    其中WITH_C_API=ON表示需要构建推理库。

    +

    用户还可根据自己的需求设置其他编译参数。比如希望最小化生成的库的大小，可以设置CMAKE_BUILD_TYPE为MinSizeRel；若希望最快的执行速度，则可设置CMAKE_BUILD_TYPE为Release。

    +
    +
    +

    编译和安装

    +

    CMake配置完成后，执行以下命令，PaddlePaddle将自动下载和编译所有第三方依赖库、编译和安装PaddlePaddle。

    +
    make
+make install
+
    +
    +

    注意：如果你曾经在源码目录下编译过其他平台的PaddlePaddle库，请先使用rm -rf命令删除third_party目录和build目录，以确保所有的第三方依赖库和PaddlePaddle代码都是针对新的CMake配置重新编译的。

    +

    执行完安装命令后，your/path/to/install目录中会包含include和lib目录，其中include中包含C-API的头文件，lib中包含一个Raspberry Pi版本的库。

    +
    +
    + + +
    +
    + + +
    +
    + +
    + +
    + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc_cn/about/index_cn.html b/doc_cn/mobile/index_cn.html similarity index 70% rename from doc_cn/about/index_cn.html rename to doc_cn/mobile/index_cn.html index f9e521649db039bb244c19f893878c5c494c7ac9..85fa14abf8294c2c1bd6b3cbd06d48d530c954cf 100644 --- a/doc_cn/about/index_cn.html +++ b/doc_cn/mobile/index_cn.html @@ -8,7 +8,7 @@ - 关于PaddlePaddle — PaddlePaddle 文档 + MOBILE — PaddlePaddle 文档 @@ -32,7 +32,9 @@ - + + + @@ -62,7 +64,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -80,11 +82,12 @@
    -
    - Folk me on Github + Fork me on Github
    English @@ -85,6 +85,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -108,9 +109,10 @@ diff --git a/doc_cn/searchindex.js b/doc_cn/searchindex.js index c0f62ab2a9722987dce63ac076817516f71015f6..94851689536841c22ae715474a3ff87194e54dbf 100644 --- a/doc_cn/searchindex.js +++ b/doc_cn/searchindex.js @@ -1 +1 @@ -Search.setIndex({docnames:["about/index_cn","api/index_cn","api/v1/data_provider/dataprovider_cn","api/v1/data_provider/pydataprovider2_cn","api/v1/index_cn","api/v1/predict/swig_py_paddle_cn","api/v1/trainer_config_helpers/activations","api/v1/trainer_config_helpers/attrs","api/v1/trainer_config_helpers/data_sources","api/v1/trainer_config_helpers/evaluators","api/v1/trainer_config_helpers/layers","api/v1/trainer_config_helpers/networks","api/v1/trainer_config_helpers/optimizers","api/v1/trainer_config_helpers/poolings","api/v2/config/activation","api/v2/config/attr","api/v2/config/layer","api/v2/config/networks","api/v2/config/optimizer","api/v2/config/pooling","api/v2/data","api/v2/model_configs","api/v2/run_logic","design/api","design/dist/README","design/multi_language_interface/why_plain_c","design/reader/README","faq/index_cn","getstarted/basic_usage/index_cn","getstarted/build_and_install/cmake/build_from_source_cn","getstarted/build_and_install/docker_install_cn","getstarted/build_and_install/index_cn","getstarted/build_and_install/ubuntu_install_cn","getstarted/concepts/use_concepts_cn","getstarted/index_cn","howto/deep_model/rnn/hierarchical_layer_cn","howto/deep_model/rnn/hrnn_rnn_api_compare_cn","howto/deep_model/rnn/index_cn","howto/deep_model/rnn/recurrent_group_cn","howto/deep_model/rnn/rnn_config_cn","howto/dev/contribute_to_paddle_cn","howto/dev/new_layer_cn","howto/dev/write_docs_cn","howto/index_cn","howto/optimization/gpu_profiling_cn","howto/usage/cluster/cluster_train_cn","howto/usage/cmd_parameter/arguments_cn","howto/usage/cmd_parameter/detail_introduction_cn","howto/usage/cmd_parameter/index_cn","howto/usage/cmd_parameter/use_case_cn","howto/usage/k8s/k8s_basis_cn","howto/usage/k8s/k8s_cn","howto/usage/k8s/k8s_distributed_cn","howto/usage/k8s/src/k8s_data/README","howto/usage/k8s/src/k8s_train/README","index_cn","tutorials/embedding_model/index_cn","tutorials/image_classification/index_cn","tutorials/imagenet_model/resnet_model_cn","tutorials/index_cn","tutorials/quick_start/index_cn","tutorials/rec/ml_dataset_cn","tutorials/rec/ml_regression_cn","tutorials/semantic_role_labeling/index_cn","tutorials/sentiment_analysis/index_cn","tutorials/text_generation/index_cn"],envversion:50,filenames:["about/index_cn.md","api/index_cn.rst","api/v1/data_provider/dataprovider_cn.rst","api/v1/data_provider/pydataprovider2_cn.rst","api/v1/index_cn.rst","api/v1/predict/swig_py_paddle_cn.rst","api/v1/trainer_config_helpers/activations.rst","api/v1/trainer_config_helpers/attrs.rst","api/v1/trainer_config_helpers/data_sources.rst","api/v1/trainer_config_helpers/evaluators.rst","api/v1/trainer_config_helpers/layers.rst","api/v1/trainer_config_helpers/networks.rst","api/v1/trainer_config_helpers/optimizers.rst","api/v1/trainer_config_helpers/poolings.rst","api/v2/config/activation.rst","api/v2/config/attr.rst","api/v2/config/layer.rst","api/v2/config/networks.rst","api/v2/config/optimizer.rst","api/v2/config/pooling.rst","api/v2/data.rst","api/v2/model_configs.rst","api/v2/run_logic.rst","design/api.md","design/dist/README.md","design/multi_language_interface/why_plain_c.md","design/reader/README.md","faq/index_cn.rst","getstarted/basic_usage/index_cn.rst","getstarted/build_and_install/cmake/build_from_source_cn.rst","getstarted/build_and_install/docker_install_cn.rst","getstarted/build_and_install/index_cn.rst","getstarted/build_and_install/ubuntu_install_cn.rst","getstarted/concepts/use_concepts_cn.rst","getstarted/index_cn.rst","howto/deep_model/rnn/hierarchical_layer_cn.rst","howto/deep_model/rnn/hrnn_rnn_api_compare_cn.rst","howto/deep_model/rnn/index_cn.rst","howto/deep_model/rnn/recurrent_group_cn.md","howto/deep_model/rnn/rnn_config_cn.rst","howto/dev/contribute_to_paddle_cn.md","howto/dev/new_layer_cn.rst","howto/dev/write_docs_cn.rst","howto/index_cn.rst","howto/optimization/gpu_profiling_cn.rst","howto/usage/cluster/cluster_train_cn.md","howto/usage/cmd_parameter/arguments_cn.md","howto/usage/cmd_parameter/detail_introduction_cn.md","howto/usage/cmd_parameter/index_cn.rst","howto/usage/cmd_parameter/use_case_cn.md","howto/usage/k8s/k8s_basis_cn.md","howto/usage/k8s/k8s_cn.md","howto/usage/k8s/k8s_distributed_cn.md","howto/usage/k8s/src/k8s_data/README.md","howto/usage/k8s/src/k8s_train/README.md","index_cn.rst","tutorials/embedding_model/index_cn.md","tutorials/image_classification/index_cn.md","tutorials/imagenet_model/resnet_model_cn.md","tutorials/index_cn.md","tutorials/quick_start/index_cn.rst","tutorials/rec/ml_dataset_cn.md","tutorials/rec/ml_regression_cn.rst","tutorials/semantic_role_labeling/index_cn.md","tutorials/sentiment_analysis/index_cn.md","tutorials/text_generation/index_cn.md"],objects:{"paddle.trainer_config_helpers":{attrs:[7,0,0,"-"],data_sources:[8,0,0,"-"]},"paddle.trainer_config_helpers.attrs":{ExtraAttr:[7,1,1,""],ExtraLayerAttribute:[7,2,1,""],ParamAttr:[7,1,1,""],ParameterAttribute:[7,2,1,""]},"paddle.trainer_config_helpers.attrs.ParameterAttribute":{set_default_parameter_name:[7,3,1,""]},"paddle.trainer_config_helpers.data_sources":{define_py_data_sources2:[8,4,1,""]}},objnames:{"0":["py","module","Python \u6a21\u5757"],"1":["py","attribute","Python \u5c5e\u6027"],"2":["py","class","Python \u7c7b"],"3":["py","method","Python \u65b9\u6cd5"],"4":["py","function","Python \u51fd\u6570"]},objtypes:{"0":"py:module","1":"py:attribute","2":"py:class","3":"py:method","4":"py:function"},terms:{"00012\u7684\u6a21\u578b\u6709\u7740\u6700\u9ad8\u7684bleu\u503c27":65,"0005\u4e58\u4ee5batch":57,"000\u4e2a\u5df2\u6807\u6ce8\u8fc7\u7684\u9ad8\u6781\u6027\u7535\u5f71\u8bc4\u8bba\u7528\u4e8e\u8bad\u7ec3":64,"000\u4e2a\u7528\u4e8e\u6d4b\u8bd5":64,"000\u4e2atxt\u6587\u4ef6":64,"000\u4f4d\u7528\u6237\u5bf94":61,"000\u5e45\u56fe\u50cf\u4e0a\u6d4b\u8bd5\u4e86\u6a21\u578b\u7684\u5206\u7c7b\u9519\u8bef\u7387":58,"000\u5f20\u7070\u5ea6\u56fe\u7247\u7684\u6570\u5b57\u5206\u7c7b\u6570\u636e\u96c6":3,"000\u6761\u8bc4\u4ef7":61,"000\u90e8\u7535\u5f71\u76841":61,"00186201e":5,"00m":44,"02595v1":[10,16],"03m":44,"0424m":44,"0473v3":[11,17],"05d":57,"0630u":44,"06u":44,"0810u":44,"08823112e":5,"0957m":44,"0\u53f7\u8bad\u7ec3\u8282\u70b9\u662f\u4e3b\u8bad\u7ec3\u8282\u70b9":47,"0\u5c42\u5e8f\u5217":35,"0\u8868\u793a\u7b2c\u4e00\u6b21\u7ecf\u8fc7\u8bad\u7ec3\u96c6":64,"0ab":[10,16],"0b1":32,"0rc1":30,"10000\u5f20\u4f5c\u4e3a\u6d4b\u8bd5\u96c6":57,"10007_10":64,"10014_7":64,"100m":27,"101\u5c42\u548c152\u5c42\u7684\u7f51\u7edc\u7ed3\u6784\u4e2d":58,"101\u5c42\u548c152\u5c42\u7684\u7f51\u7edc\u914d\u7f6e\u6587\u4ef6\u53ef\u53c2\u7167":58,"101\u5c42\u7f51\u7edc\u6a21\u578b":58,"10\u4e2d\u7684\u8bad\u7ec3\u96c6\u548c\u6d4b\u8bd5\u96c6":57,"10\u6570\u636e\u96c6":57,"10\u6570\u636e\u96c6\u5305\u542b60000\u5f2032x32\u7684\u5f69\u8272\u56fe\u7247":57,"10\u6570\u636e\u96c6\u7684\u5b98\u65b9\u7f51\u5740":57,"10\u6570\u636e\u96c6\u8bad\u7ec3\u4e00\u4e2a\u5377\u79ef\u795e\u7ecf\u7f51\u7edc":57,"1150u":44,"11e6":51,"12194102e":5,"124n":44,"128\u7ef4\u548c256\u7ef4":56,"13m":51,"1490u":44,"14\u6570\u636e\u96c6":65,"14\u6570\u636e\u96c6\u4e0a\u5f97\u5230\u826f\u597d\u8868\u73b0\u7684\u8bad\u7ec3\u8fc7\u7a0b":65,"14\u8fd9\u79cd\u5199\u6cd5\u5c06\u4f1a\u6d4b\u8bd5\u6a21\u578b":49,"152\u5c42\u7f51\u7edc\u6a21\u578b":58,"15501715e":5,"1550u":44,"15\u884c":36,"1636k":65,"16u":44,"173m":58,"173n":44,"1770u":44,"18\u5c81\u4ee5\u4e0b":61,"18e457ce3d362ff5f3febf8e7f85ffec852f70f3b629add10aed84f930a68750":51,"197u":44,"1\u7684\u5c42\u4e4b\u5916":49,"1\u7a00\u758f\u6570\u636e":41,"1\u8f6e\u5b58\u50a8\u7684\u6240\u6709\u6a21\u578b":49,"1\u9664\u4ee5batch":57,"1m\u6570\u636e\u96c6\u4e2d":62,"1m\u7684\u5b57\u6bb5\u914d\u7f6e\u6587\u4ef6\u5728\u76ee\u5f55":62,"200\u6570\u636e\u96c6\u4e0a\u4f7f\u7528vgg\u6a21\u578b\u8bad\u7ec3\u4e00\u4e2a\u9e1f\u7c7b\u56fe\u7247\u5206\u7c7b\u6a21\u578b":57,"210u":44,"211839e770f7b538e2d8":[11,17],"215n":44,"228u":44,"234m":58,"24\u5c81":61,"2520u":44,"25639710e":5,"25k":60,"2680u":44,"26\u884c":36,"27787406e":5,"279n":44,"27m":44,"285m":44,"2863m":44,"28\u7684\u56fe\u7247\u50cf\u7d20\u7070\u5ea6\u503c":3,"28\u7ef4\u7684\u7a20\u5bc6\u6d6e\u70b9\u6570\u5411\u91cf\u548c\u4e00\u4e2a":3,"28m":44,"2977m":44,"29997\u4e2a\u6700\u9ad8\u9891\u5355\u8bcd\u548c3\u4e2a\u7279\u6b8a\u7b26\u53f7":65,"2\u4e09\u7c7b\u7684\u6bd4\u4f8b\u4e3a":27,"2\u4e2a\u6d6e\u70b9\u6570":28,"2\u5206\u522b\u4ee3\u88683\u4e2a\u8282\u70b9\u7684trainer":52,"2\u610f\u5473\u77400\u53f7\u548c1\u53f7gpu\u5c06\u4f1a\u4f7f\u7528\u6570\u636e\u5e76\u884c\u6765\u8ba1\u7b97fc1\u548cfc2\u5c42":49,"2\u8fd9\u51e0\u4e2a\u76ee\u5f55\u8868\u793apaddlepaddle\u8282\u70b9\u4e0etrain":52,"2nd":[10,16],"302n":44,"30u":44,"3206326\u4e2a\u8bcd\u548c4\u4e2a\u7279\u6b8a\u6807\u8bb0":56,"32777140e":5,"328n":44,"32\u7ef4":56,"32u":44,"32x32":20,"331n":44,"3320u":44,"34\u5c81":61,"35\u65f6":65,"36540484e":5,"36u":44,"3710m":44,"3768m":44,"387u":44,"38u":44,"3920u":44,"39u":44,"3\u4e2a\u7279\u6b8a\u7b26\u53f7":65,"3\u53f7gpu":27,"4035m":44,"4090u":44,"4096mb":47,"4279m":44,"43630644e":5,"43u":44,"448a5b355b84":51,"44\u5c81":61,"4560u":44,"4563m":44,"45u":44,"4650u":44,"4726m":44,"473m":51,"48565123e":5,"48684503e":5,"49316648e":5,"49\u5c81":61,"4gb":47,"500\u4e2atxt\u6587\u4ef6":64,"500m":27,"50\u5c42":58,"50\u5c42\u7f51\u7edc\u6a21\u578b":58,"51111044e":5,"514u":44,"525n":44,"526u":44,"53018653e":5,"536u":44,"5460u":44,"5470u":44,"54u":44,"55\u5c81":61,"55g":65,"5690m":44,"573u":44,"578n":44,"5798m":44,"586u":44,"58s":51,"5969m":44,"5\u4e2a\u6d4b\u8bd5\u6837\u4f8b\u548c2\u4e2a\u751f\u6210\u5f0f\u6837\u4f8b":56,"5\u5230\u672c\u5730\u73af\u5883\u4e2d":32,"6080u":44,"6082v4":[10,16],"6140u":44,"6305m":44,"639u":44,"64\u7ef4":56,"655u":44,"6780u":44,"6810u":44,"682u":44,"6970u":44,"6\u4e07\u4ebf\u6b21\u6d6e\u70b9\u8fd0\u7b97\u6bcf\u79d2":44,"6\u4e2a\u8282\u70b9":45,"6\u5143\u4e0a\u4e0b\u6587\u4f5c\u4e3a\u8f93\u5165\u5c42":56,"704u":44,"70634608e":5,"7090u":44,"72296313e":5,"72u":44,"73u":44,"75u":44,"760u":44,"767u":44,"783n":44,"784u":44,"78m":44,"7eamaa":20,"7kb":51,"8250u":44,"8300u":44,"830n":44,"849m":44,"85625684e":5,"861u":44,"864k":65,"8661m":44,"877\u4e2a\u88ab\u5411\u91cf\u5316\u7684\u8bcd":56,"877\u884c":56,"892m":44,"901n":44,"90u":44,"918u":44,"9247m":44,"924n":44,"9261m":44,"93137714e":5,"9330m":44,"94u":44,"9530m":44,"96644767e":5,"983m":44,"988u":44,"997u":44,"99982715e":5,"99m":58,"99u":44,"9\u4e2d\u7684\u4e00\u4e2a\u6570\u5b57":3,"9f18":51,"\u0233":28,"\u03b5":28,"\u4e00":36,"\u4e00\u4e2a0\u5c42\u5e8f\u5217":35,"\u4e00\u4e2a0\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u6269\u5c55\u6210\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":35,"\u4e00\u4e2a\u4f8b\u5b50\u662f\u623f\u4ea7\u4f30\u503c":28,"\u4e00\u4e2a\u5178\u578b\u7684\u795e\u7ecf\u7f51\u7edc\u5982\u4e0b\u56fe\u6240\u793a":57,"\u4e00\u4e2a\u5206\u5e03\u5f0f\u7684\u5b58\u50a8\u7cfb\u7edf":50,"\u4e00\u4e2a\u5206\u5e03\u5f0fpaddle\u8bad\u7ec3\u4efb\u52a1\u4e2d\u7684\u6bcf\u4e2a\u8fdb\u7a0b\u90fd\u53ef\u4ee5\u4ececeph\u8bfb\u53d6\u6570\u636e":51,"\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":35,"\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217\u6216\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":35,"\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u6269\u5c55\u6210\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":35,"\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217\u8fdb\u5165":38,"\u4e00\u4e2a\u5377\u79ef\u795e\u7ecf\u7f51\u7edc\u5305\u542b\u5982\u4e0b\u5c42":57,"\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":35,"\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217\u6216\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":35,"\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u53d8\u6210\u4e00\u4e2a0\u5c42\u5e8f\u5217":35,"\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u53d8\u6210\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":35,"\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217\u8fdb\u5165":38,"\u4e00\u4e2a\u53cc\u5c42rnn\u7531\u591a\u4e2a\u5355\u5c42rnn\u7ec4\u6210":38,"\u4e00\u4e2a\u53ef\u8c03\u7528\u7684\u51fd\u6570":38,"\u4e00\u4e2a\u57fa\u672c\u7684\u5e94\u7528\u573a\u666f\u662f\u533a\u5206\u7ed9\u5b9a\u6587\u672c\u7684\u8912\u8d2c\u4e24\u6781\u6027":64,"\u4e00\u4e2a\u6216\u591a\u4e2a":50,"\u4e00\u4e2a\u6570\u636e\u96c6\u5927\u90e8\u5206\u5e8f\u5217\u957f\u5ea6\u662f100":27,"\u4e00\u4e2a\u6587\u4ef6":3,"\u4e00\u4e2a\u662f\u6d6e\u70b9\u8ba1\u7b97\u91cf":44,"\u4e00\u4e2a\u662f\u76f4\u63a5\u628a\u4e00\u4e2a\u5bb9\u5668\u8f6c\u6362\u6210\u955c\u50cf":30,"\u4e00\u4e2a\u72ec\u7acb\u7684\u5143\u7d20":35,"\u4e00\u4e2a\u72ec\u7acb\u7684\u8bcd\u8bed":35,"\u4e00\u4e2a\u7b80\u5355\u7684\u4f8b\u5b50\u5982":64,"\u4e00\u4e2a\u7ec8\u7aef\u8fd0\u884cvi":30,"\u4e00\u4e2a\u7f51\u7edc\u5c42\u7684\u524d\u5411\u4f20\u64ad\u90e8\u5206\u628a\u8f93\u5165\u8f6c\u5316\u4e3a\u76f8\u5e94\u7684\u8f93\u51fa":41,"\u4e00\u4e2a\u7f51\u7edc\u5c42\u7684\u53c2\u6570\u662f\u5728":41,"\u4e00\u4e2a\u7f51\u7edc\u5c42\u7684c":41,"\u4e00\u4e2a\u91cd\u8981\u7684\u95ee\u9898\u662f\u9009\u62e9\u6b63\u786e\u7684learning_r":27,"\u4e00\u4e2agpu\u8bbe\u5907\u4e0a\u4e0d\u5141\u8bb8\u914d\u7f6e\u591a\u4e2a\u6a21\u578b":47,"\u4e00\u4e2alabel":36,"\u4e00\u4e2alogging\u5bf9\u8c61":3,"\u4e00\u4e2amemory\u5305\u542b":39,"\u4e00\u4e2apass\u610f\u5473\u7740paddlepaddle\u8bad\u7ec3\u6570\u636e\u96c6\u4e2d\u7684\u6240\u6709\u6837\u672c\u88ab\u904d\u5386\u4e00\u6b21":63,"\u4e00\u4e2apass\u8868\u793a\u8fc7\u4e00\u904d\u6240\u6709\u8bad\u7ec3\u6837\u672c":60,"\u4e00\u4e2apod\u4e2d\u7684\u6240\u6709\u5bb9\u5668\u4f1a\u88ab\u8c03\u5ea6\u5230\u540c\u4e00\u4e2anode\u4e0a":50,"\u4e00\u4eba":36,"\u4e00\u53e5\u8bdd\u662f\u7531\u8bcd\u8bed\u6784\u6210\u7684\u5e8f\u5217":38,"\u4e00\u53f0\u673a\u5668\u4e0a\u9762\u7684\u7ebf\u7a0b\u6570\u91cf":62,"\u4e00\u65e9":36,"\u4e00\u662fbatch":27,"\u4e00\u6761\u6837\u672c":3,"\u4e00\u6837\u8bbe\u7f6e":45,"\u4e00\u6b21\u4f5c\u4e1a\u79f0\u4e3a\u4e00\u4e2ajob":50,"\u4e00\u6b21\u6027\u676f\u5b50":36,"\u4e00\u6b21yield\u8c03\u7528":3,"\u4e00\u79cd\u5e38\u7528\u7684\u505a\u6cd5\u662f\u7528\u5b66\u4e60\u7684\u6a21\u578b\u5bf9\u53e6\u5916\u4e00\u7ec4\u6d4b\u8bd5\u6570\u636e\u8fdb\u884c\u9884\u6d4b":28,"\u4e00\u7bc7\u8bba\u6587":65,"\u4e00\u7ea7\u76ee\u5f55":[64,65],"\u4e00\u81f4":[35,36],"\u4e00\u822c\u4ece":40,"\u4e00\u822c\u5728paddlepaddle\u4e2d":36,"\u4e00\u822c\u60c5\u51b5\u4e0b":[2,28],"\u4e00\u822c\u63a8\u8350\u8bbe\u7f6e\u6210true":3,"\u4e00\u822c\u662f\u7531\u4e8e\u76f4\u63a5\u4f20\u9012\u5927\u5b57\u5178\u5bfc\u81f4\u7684":27,"\u4e00\u822c\u6765\u8bf4":39,"\u4e00\u822c\u800c\u8a00":65,"\u4e00\u822c\u8868\u793a":36,"\u4e00\u884c\u4e3a\u4e00\u4e2a\u6837\u672c":60,"\u4e09\u79cd\u5e8f\u5217\u6a21\u5f0f":[3,33],"\u4e09\u7ea7\u76ee\u5f55":[64,65],"\u4e0a":40,"\u4e0a\u4e0b\u6587\u5927\u5c0f\u8bbe\u7f6e\u4e3a1\u7684\u4e00\u4e2a\u6837\u672c\u7684\u7279\u5f81\u5982\u4e0b":63,"\u4e0a\u53d1\u8868\u7684\u8bc4\u8bba\u5206\u6210\u6b63\u9762\u8bc4\u8bba\u548c\u8d1f\u9762\u8bc4\u8bba\u4e24\u7c7b":64,"\u4e0a\u56fe\u4e2d\u865a\u7ebf\u7684\u8fde\u63a5":36,"\u4e0a\u56fe\u63cf\u8ff0\u4e86\u4e00\u4e2a3\u8282\u70b9\u7684\u5206\u5e03\u5f0f\u8bad\u7ec3\u573a\u666f":52,"\u4e0a\u7f51":36,"\u4e0a\u8fd0\u884c":30,"\u4e0a\u8ff0\u4ee3\u7801\u5c06bias\u5168\u90e8\u521d\u59cb\u5316\u4e3a1":27,"\u4e0a\u8ff0\u7684\u4ee3\u7801\u7247\u6bb5\u5305\u542b\u4e86\u4e24\u79cd\u65b9\u6cd5":44,"\u4e0a\u8ff0\u811a\u672c\u4f7f\u7528":45,"\u4e0b":57,"\u4e0b\u56fe\u4e2d\u5c31\u5c55\u793a\u4e86\u4e00\u4e9b\u5173\u4e8e\u5185\u5b58\u6570\u636e\u8fc1\u5f99\u548c\u8ba1\u7b97\u8d44\u6e90\u5229\u7528\u7387\u7684\u5efa\u8bae":44,"\u4e0b\u56fe\u5c55\u793a\u4e86\u6240\u6709\u7684\u56fe\u7247\u7c7b\u522b":57,"\u4e0b\u56fe\u5c55\u793a\u4e86\u65f6\u95f4\u6269\u5c55\u76842\u5c42":63,"\u4e0b\u56fe\u5c55\u793a\u7684\u662f\u57fa\u4e8e\u6b8b\u5dee\u7684\u8fde\u63a5\u65b9\u5f0f":58,"\u4e0b\u56fe\u662f\u4e00\u4e2a\u5168\u8fde\u63a5\u5c42\u7684\u793a\u610f\u56fe":41,"\u4e0b\u6587\u4ee5nlp\u4efb\u52a1\u4e3a\u4f8b":38,"\u4e0b\u6587\u4f7f\u7528":52,"\u4e0b\u6587\u5c31\u662f\u7528job\u7c7b\u578b\u7684\u8d44\u6e90\u6765\u8fdb\u884c\u8bad\u7ec3":51,"\u4e0b\u6b21":36,"\u4e0b\u7684":52,"\u4e0b\u8868\u5c55\u793a\u4e86batch":58,"\u4e0b\u8f7d\u5b8c\u6570\u636e\u540e":51,"\u4e0b\u8f7d\u5b8c\u76f8\u5173\u5b89\u88c5\u5305\u540e":32,"\u4e0b\u8f7d\u6570\u636e\u96c6":57,"\u4e0b\u8f7dwmt":65,"\u4e0b\u9762\u4e3e\u4e2a\u7b80\u5355\u7684\u4f8b\u5b50":44,"\u4e0b\u9762\u4ecb\u7ecd\u9884\u5904\u7406\u8fc7\u7a0b\u5177\u4f53\u7684\u6b65\u9aa4":62,"\u4e0b\u9762\u5148\u7b80\u8981\u4ecb\u7ecd\u4e00\u4e0b\u672c\u6587\u7528\u5230\u7684\u51e0\u4e2akubernetes\u6982\u5ff5":50,"\u4e0b\u9762\u5217\u51fa\u4e86":39,"\u4e0b\u9762\u5217\u51fa\u4e86\u5168\u8fde\u63a5\u5c42\u7684\u68af\u5ea6\u68c0\u67e5\u5355\u5143\u6d4b\u8bd5":41,"\u4e0b\u9762\u5c31\u6839\u636e\u8fd9\u51e0\u4e2a\u6b65\u9aa4\u5206\u522b\u4ecb\u7ecd":52,"\u4e0b\u9762\u6211\u4eec\u7ed9\u51fa\u4e86\u4e00\u4e2a\u914d\u7f6e\u793a\u4f8b":57,"\u4e0b\u9762\u662f\u4e00\u4e2a\u8bef\u5dee\u66f2\u7ebf\u56fe\u7684\u793a\u4f8b":57,"\u4e0b\u9762\u662fcifar":57,"\u4e0b\u9762\u7684\u4ee3\u7801\u7247\u6bb5\u5b9e\u73b0\u4e86":41,"\u4e0b\u9762\u7684\u4f8b\u5b50\u4f7f\u7528\u4e86":58,"\u4e0b\u9762\u7684\u4f8b\u5b50\u540c\u6837\u4f7f\u7528\u4e86":58,"\u4e0b\u9762\u7ed9\u51fa\u4e86\u4e00\u4e2a\u4f8b\u5b50":41,"\u4e0b\u9762\u7ed9\u51fa\u5728\u4e09\u7ef4\u7a7a\u95f4\u4e2d\u4f7f\u7528\u7ebf\u6027\u56de\u5f52\u62df\u5408\u4e00\u6761\u76f4\u7ebf\u7684\u4f8b\u5b50":33,"\u4e0b\u9762\u811a\u672c\u7b26\u5408paddlepaddle\u671f\u5f85\u7684\u8bfb\u53d6\u6570\u636e\u7684python\u7a0b\u5e8f\u7684\u6a21\u5f0f":28,"\u4e0b\u9762\u8fd9\u4e9blayer\u80fd\u591f\u63a5\u53d7\u53cc\u5c42\u5e8f\u5217\u4f5c\u4e3a\u8f93\u5165":35,"\u4e0d":36,"\u4e0d\u4e00\u5b9a\u548c\u65f6\u95f4\u6709\u5173\u7cfb":3,"\u4e0d\u4f1a\u518d\u4ece":27,"\u4e0d\u4f7f\u7528\u9759\u6001\u5e93":25,"\u4e0d\u4f7f\u7528\u989d\u5916\u7a7a\u95f4":41,"\u4e0d\u4f7f\u7528c":25,"\u4e0d\u4f7f\u7528swig":25,"\u4e0d\u5305\u542b\u5728\u5b57\u5178\u4e2d\u7684\u5355\u8bcd":65,"\u4e0d\u540c":63,"\u4e0d\u540c\u4e3b\u673a":50,"\u4e0d\u540c\u4ea7\u54c1":64,"\u4e0d\u540c\u53c2\u6570\u670d\u52a1\u5668\u4e0a\u6570\u636e\u5927\u5c0f\u7684\u6700\u5927\u503c\u4e0e\u6700\u5c0f\u503c\u7684\u6bd4\u7387":47,"\u4e0d\u540c\u5c42\u7684\u7279\u5f81\u7531\u5206\u53f7":58,"\u4e0d\u540c\u65f6\u95f4\u6b65\u7684\u8f93\u5165\u662f\u4e0d\u540c\u7684":39,"\u4e0d\u540c\u7248\u672c\u7684\u7f16\u8bd1\u5668\u4e4b\u95f4":25,"\u4e0d\u540c\u7684\u4f18\u5316\u7b97\u6cd5\u9700\u8981\u4f7f\u7528\u4e0d\u540c\u5927\u5c0f\u7684\u5185\u5b58":27,"\u4e0d\u540c\u7684\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf":52,"\u4e0d\u540c\u7684\u6570\u636e\u7c7b\u578b\u548c\u5e8f\u5217\u6a21\u5f0f\u8fd4\u56de\u7684\u683c\u5f0f\u4e0d\u540c":[3,33],"\u4e0d\u540c\u7a7a\u95f4\u7684\u8d44\u6e90\u540d\u53ef\u4ee5\u91cd\u590d":50,"\u4e0d\u540c\u8bed\u8a00\u7684\u63a5\u53e3\u9002\u5e94\u4e0d\u540c\u8bed\u8a00\u7684\u7279\u6027":25,"\u4e0d\u540c\u8f93\u5165\u542b\u6709\u7684\u5b50\u53e5":38,"\u4e0d\u540c\u8f93\u5165\u5e8f\u5217\u542b\u6709\u7684\u8bcd\u8bed\u6570\u5fc5\u987b\u4e25\u683c\u76f8\u7b49":38,"\u4e0d\u540cdataprovider\u5bf9\u6bd4\u5982\u4e0b":36,"\u4e0d\u540cpod\u4e4b\u95f4\u53ef\u4ee5\u901a\u8fc7ip\u5730\u5740\u8bbf\u95ee":50,"\u4e0d\u5c11":36,"\u4e0d\u5d4c\u5165\u5176\u4ed6\u8bed\u8a00\u89e3\u91ca\u5668":25,"\u4e0d\u5d4c\u5165python\u89e3\u91ca\u5668":25,"\u4e0d\u5e94\u8be5\u88ab\u62c6\u89e3":38,"\u4e0d\u5fc5\u518d\u5c06\u4efb\u610f\u957f\u5ea6\u6e90\u8bed\u53e5\u4e2d\u7684\u6240\u6709\u4fe1\u606f\u538b\u7f29\u81f3\u4e00\u4e2a\u5b9a\u957f\u7684\u5411\u91cf\u4e2d":65,"\u4e0d\u6307\u5b9a\u65f6":38,"\u4e0d\u63d0\u4f9b\u5206\u5e03\u5f0f\u5b58\u50a8":50,"\u4e0d\u652f\u6301":63,"\u4e0d\u662f\u4e00\u6761\u5e8f\u5217":[3,33],"\u4e0d\u663e\u793a\u7684\u5199\u6bcf\u4e2a\u7c7b\u5177\u4f53\u5305\u542b\u4ec0\u4e48":25,"\u4e0d\u6ee1\u8db3\u94a9\u5b50\u7684":40,"\u4e0d\u7f13\u5b58\u4efb\u4f55\u6570\u636e":3,"\u4e0d\u80fd\u88ab\u63d0\u4ea4\u5230":40,"\u4e0d\u8fc7":36,"\u4e0d\u8fdc":36,"\u4e0d\u9519":36,"\u4e0d\u9700\u8981\u5bf9\u5e8f\u5217\u6570\u636e\u8fdb\u884c\u4efb\u4f55\u9884\u5904\u7406":39,"\u4e0e":[52,56,65],"\u4e0e\u5355\u5c42rnn\u7684\u914d\u7f6e\u7c7b\u4f3c":36,"\u4e0e\u5728":63,"\u4e0e\u672c\u5730\u8bad\u7ec3\u76f8\u540c":45,"\u4e0e\u6b64\u4e0d\u540c\u7684\u662f":52,"\u4e0e\u7ffb\u8bd1":65,"\u4e0e\u8bad\u7ec3\u4e0d\u540c":64,"\u4e0e\u8bad\u7ec3\u6a21\u578b\u4e0d\u540c\u7684\u662f":65,"\u4e0e\u8fd9\u4e2a\u8bad\u7ec3\u6570\u636e\u4ea4\u4e92\u7684layer":27,"\u4e0eimdb\u7f51\u7ad9\u63d0\u4f9b\u7684\u4e00\u81f4":61,"\u4e0ejob":52,"\u4e0etime":61,"\u4e14":36,"\u4e14\u589e\u52a0\u4e00\u4e2a\u7b2c\u4e09\u65b9\u8bed\u8a00":25,"\u4e14\u5e8f\u5217\u7684\u6bcf\u4e00\u4e2a\u5143\u7d20\u8fd8\u662f\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217":[3,33],"\u4e14\u652f\u6301\u90e8\u7f72\u5230":50,"\u4e14\u6bcf\u4e2a\u53e5\u5b50\u8868\u793a\u4e3a\u5bf9\u5e94\u7684\u8bcd\u8868\u7d22\u5f15\u6570\u7ec4":36,"\u4e14\u9ed8\u8ba4\u5728\u8bad\u7ec3\u96c6\u4e0a\u6784\u5efa\u5b57\u5178":64,"\u4e14c99\u652f\u6301bool\u7c7b\u578b\u548c\u5b9a\u957f\u6574\u6570":25,"\u4e14c99\u76f8\u5bf9\u4e8ec11\u4f7f\u7528\u66f4\u52a0\u5e7f\u6cdb":25,"\u4e24":36,"\u4e24\u4e2a\u5217\u8868\u6587\u4ef6":45,"\u4e24\u4e2a\u5b50\u76ee\u5f55\u4e0b":42,"\u4e24\u4e2a\u5d4c\u5957\u7684":38,"\u4e24\u4e2a\u64cd\u4f5c":44,"\u4e24\u4e2a\u6587\u4ef6\u5939":57,"\u4e24\u4e2a\u8f93\u5165\u7279\u5f81\u5728\u8fd9\u4e2a\u6d41\u7a0b\u4e2d\u8d77\u7740\u81f3\u5173\u91cd\u8981\u7684\u4f5c\u7528":63,"\u4e24\u4e2a\u8f93\u5165\u7684\u5b50\u5e8f\u5217\u957f\u5ea6\u4e5f\u5e76\u4e0d\u76f8\u540c":36,"\u4e24\u4e2a\u90e8\u5206":42,"\u4e24\u79cd\u7c7b\u522b":60,"\u4e24\u8005\u5747\u4e3a\u7eaf\u6587\u672c\u6587\u4ef6":2,"\u4e2a":60,"\u4e2a\u5185\u5b58\u6c60\u5b9e\u9645\u4e0a\u51b3\u5b9a\u4e86shuffle\u7684\u7c92\u5ea6":27,"\u4e2a\u5355\u8bcd":65,"\u4e2a\u6027\u5316\u63a8\u8350":59,"\u4e2a\u6279\u6b21\u540e\u6253\u5370\u4e00\u4e2a":62,"\u4e2a\u6279\u6b21\u7684\u53c2\u6570\u5e73\u5747\u503c\u8fdb\u884c\u6d4b\u8bd5":47,"\u4e2a\u6a21\u578b\u6d4b\u8bd5\u6570\u636e":47,"\u4e2d":[25,27,41,52,57,60,62,63,64],"\u4e2d\u4e0d\u8981\u6dfb\u52a0\u5927\u6587\u4ef6\u7b49":40,"\u4e2d\u4ecb\u7ecd\u7684\u65b9\u6cd5":56,"\u4e2d\u4efb\u610f\u7b2ci\u884c\u7684\u53e5\u5b50\u4e4b\u95f4\u90fd\u5fc5\u987b\u6709\u7740\u4e00\u4e00\u5bf9\u5e94\u7684\u5173\u7cfb":65,"\u4e2d\u4efb\u610f\u7b2ci\u884c\u7684\u53e5\u5b50\u4e4b\u95f4\u90fd\u6709\u7740\u4e00\u4e00\u5bf9\u5e94\u7684\u5173\u7cfb":65,"\u4e2d\u5173\u4e8e\u65f6\u95f4\u9012\u5f52\u795e\u7ecf\u7f51\u7edc\u7684\u4ecb\u7ecd":36,"\u4e2d\u5305\u542b\u4e86\u8bad\u7ec3\u6a21\u578b\u7684\u57fa\u672c\u547d\u4ee4":60,"\u4e2d\u5305\u542b\u5982\u4e0b\u8868\u6240\u793a\u76843\u4e2a\u6587\u4ef6\u5939":65,"\u4e2d\u5355\u5143\u6d4b\u8bd5\u7684\u4e00\u90e8\u5206":40,"\u4e2d\u5b89\u88c5":45,"\u4e2d\u5b8c\u5168\u4e00\u81f4":25,"\u4e2d\u5b8c\u6210":64,"\u4e2d\u5b9a\u4e49":39,"\u4e2d\u5b9a\u4e49\u4f7f\u7528\u54ea\u79cddataprovid":2,"\u4e2d\u5b9a\u4e49\u548c\u4f7f\u7528":38,"\u4e2d\u5bfc\u51fa\u9884\u5b9a\u4e49\u7684\u7f51\u7edc":64,"\u4e2d\u5c55\u793a\u4e86\u5982\u4f55\u4f7f\u7528python\u6765\u63d0\u53d6\u7279\u5f81":58,"\u4e2d\u6307\u5b9a":47,"\u4e2d\u6307\u5b9a\u7684\u540d\u5b57":49,"\u4e2d\u6307\u5b9a\u7684\u5c42\u987a\u5e8f\u4e00\u81f4":58,"\u4e2d\u63d0\u51fa\u7684resnet\u7f51\u7edc\u7ed3\u6784\u57282015\u5e74imagenet\u5927\u89c4\u6a21\u89c6\u89c9\u8bc6\u522b\u7ade\u8d5b":58,"\u4e2d\u641c\u7d22\u8fd9\u51e0\u4e2a\u5e93":29,"\u4e2d\u6587\u6587\u6863\u76ee\u5f55":42,"\u4e2d\u6587\u7ef4\u57fa\u767e\u79d1\u9875\u9762":36,"\u4e2d\u6709\u8bb8\u591a\u7684\u7279\u5f81":61,"\u4e2d\u6bcf\u4e2apod\u7684ip\u5730\u5740":52,"\u4e2d\u6bcf\u5c42\u7684\u6570\u503c\u7edf\u8ba1":47,"\u4e2d\u7684":58,"\u4e2d\u7684\u4e00\u884c":[3,40],"\u4e2d\u7684\u5185\u5bb9":63,"\u4e2d\u7684\u5bf9\u5e94\u5206\u652f\u5373\u53ef":40,"\u4e2d\u7684\u63a5\u53e3":62,"\u4e2d\u7684\u6570\u636e":58,"\u4e2d\u7684\u6570\u636e\u662f\u5426\u4e3a\u5e8f\u5217\u6a21\u5f0f":62,"\u4e2d\u7684\u6570\u636e\u8fdb\u884c\u9884\u6d4b":58,"\u4e2d\u7684\u6570\u6910\u96c6\u7684\u7ed3\u6784\u5982\u4e0b":64,"\u4e2d\u7684\u6bcf\u4e00\u884c\u547d\u4ee4":62,"\u4e2d\u7684\u751f\u6210\u7ed3\u679c\u5982\u4e0b\u6240\u793a":65,"\u4e2d\u7684\u7528\u6237\u8bc1\u4e66":50,"\u4e2d\u7684\u7b2ci\u884c":65,"\u4e2d\u7684\u8bf4\u660e":3,"\u4e2d\u7684\u8fd9\u4e9b\u6570\u636e\u6587\u4ef6":61,"\u4e2d\u770b\u5230\u4e0b\u9762\u7684\u6587\u4ef6":64,"\u4e2d\u83b7\u53d6":52,"\u4e2d\u8ba4\u771f\u8bbe\u7f6e":45,"\u4e2d\u8bbe\u7f6e":45,"\u4e2d\u8bbe\u7f6e\u7684\u6240\u6709\u8282\u70b9":45,"\u4e2d\u8be6\u7ec6\u4ecb\u7ecd":41,"\u4e2d\u8bfb\u53d6":3,"\u4e2d\u914d\u7f6e\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":39,"\u4e2d\u914d\u7f6e\u7684\u6548\u679c\u4e00\u81f4":3,"\u4e34\u65f6\u53d8\u91cf\u7b49\u7b49":27,"\u4e3a":[3,39],"\u4e3a0":3,"\u4e3a\u4e86\u4f7f\u7528\u63d0\u524d\u7f16\u5199\u7684\u811a\u672c":64,"\u4e3a\u4e86\u4fdd\u8bc1\u6548\u7387":41,"\u4e3a\u4e86\u4fdd\u8bc1gpu\u9a71\u52a8\u80fd\u591f\u5728\u955c\u50cf\u91cc\u9762\u6b63\u5e38\u8fd0\u884c":30,"\u4e3a\u4e86\u5145\u5206\u7684\u968f\u673a\u6253\u4e71\u8bad\u7ec3\u96c6":64,"\u4e3a\u4e86\u5b8c\u6210\u5206\u5e03\u5f0f\u673a\u5668\u5b66\u4e60\u8bad\u7ec3\u4efb\u52a1":50,"\u4e3a\u4e86\u5c01\u88c5\u80fd\u591f\u6b63\u786e\u5de5\u4f5c":41,"\u4e3a\u4e86\u63cf\u8ff0\u65b9\u4fbf":38,"\u4e3a\u4e86\u65b9\u4fbf\u5728\u56fd":30,"\u4e3a\u4e86\u65b9\u4fbf\u5927\u5bb6":40,"\u4e3a\u4e86\u65b9\u4fbf\u8d77\u89c1":45,"\u4e3a\u4e86\u6ee1\u8db3\u8bad\u7ec3":45,"\u4e3a\u4e86\u8fbe\u5230\u6027\u80fd\u6700\u4f18":44,"\u4e3a\u4e86\u8fd8\u539f":28,"\u4e3a\u4f8b":60,"\u4e3a\u4f8b\u521b\u5efa\u5206\u5e03\u5f0f\u7684\u5355\u8fdb\u7a0b\u8bad\u7ec3":45,"\u4e3a\u4f8b\u8fdb\u884c\u9884\u6d4b":60,"\u4e3a\u53c2\u6570\u77e9\u9635\u7684\u5bbd\u5ea6":27,"\u4e3a\u60a8\u505a\u6027\u80fd\u8c03\u4f18\u63d0\u4f9b\u4e86\u65b9\u5411":44,"\u4e3a\u60f3\u4fee\u6b63\u8bcd\u5411\u91cf\u6a21\u578b\u7684\u7528\u6237\u63d0\u4f9b\u4e86\u5c06\u6587\u672c\u8bcd\u5411\u91cf\u6a21\u578b\u8f6c\u6362\u4e3a\u4e8c\u8fdb\u5236\u6a21\u578b\u7684\u547d\u4ee4":56,"\u4e3a\u65b9\u4fbf\u4f5c\u4e1a\u542f\u52a8\u63d0\u4f9b\u4e86\u4e24\u4e2a\u72ec\u7279\u7684\u547d\u4ee4\u9009\u9879":45,"\u4e3a\u6b64":51,"\u4e3a\u8f93\u51fa\u5206\u914d\u5185\u5b58":41,"\u4e3a\u96c6\u7fa4\u4f5c\u4e1a\u8bbe\u7f6e\u989d\u5916\u7684":45,"\u4e3ajson\u6216yaml\u683c\u5f0f":62,"\u4e3aoutput_\u7533\u8bf7\u5185\u5b58":41,"\u4e3b\u8981\u4e3a\u5f00\u53d1\u8005\u4f7f\u7528":47,"\u4e3b\u8981\u5305\u62ec\u4ee5\u4e0b\u4e94\u4e2a\u6b65\u9aa4":5,"\u4e3b\u8981\u5305\u62ec\u56db\u79cd\u7c7b\u578b":33,"\u4e3b\u8981\u539f\u56e0":36,"\u4e3b\u8981\u539f\u56e0\u5305\u62ec\u4e24\u4e2a\u65b9\u9762":27,"\u4e3b\u8981\u539f\u56e0\u662f\u589e\u52a0\u4e86\u521d\u59cb\u5316\u673a\u5236":3,"\u4e3b\u8981\u6765\u81ea\u5317\u7f8e\u6d32":57,"\u4e3b\u8981\u7684\u89e3\u51b3\u529e\u6cd5\u662f\u51cf\u5c0f\u5b66\u4e60\u5f8b\u6216\u8005\u5bf9\u6570\u636e\u8fdb\u884c\u5f52\u4e00\u5316\u5904\u7406":27,"\u4e3b\u8981\u804c\u8d23\u5728\u4e8e\u5c06\u8bad\u7ec3\u6570\u636e\u4f20\u5165\u5185\u5b58\u6216\u8005\u663e\u5b58":60,"\u4e3e\u4e00\u4e2a\u4f8b\u5b50":27,"\u4e3e\u4f8b":27,"\u4e3e\u4f8b\u8bf4\u660e":36,"\u4e4b\u524d":40,"\u4e4b\u524d\u914d\u7f6e\u6587\u4ef6\u4e2d":60,"\u4e4b\u540e":[28,33,41],"\u4e4b\u540e\u4f60\u4f1a\u5f97\u5230\u8bad\u7ec3":45,"\u4e4b\u540e\u4f7f\u7528":41,"\u4e4b\u540e\u4f7f\u7528\u77e9\u9635\u8fd0\u7b97\u51fd\u6570\u6765\u8ba1\u7b97":41,"\u4e4b\u540e\u521d\u59cb\u5316\u6240\u6709\u7684\u6743\u91cd\u77e9\u9635":41,"\u4e4b\u540e\u5b9a\u4e49\u7684":57,"\u4e4b\u95f4\u7684\u8ddd\u79bb":28,"\u4e4b\u95f4\u7684\u8fd0\u7b97\u662f\u72ec\u7acb\u7684":38,"\u4e58\u4e0a\u8f93\u51fa\u7684\u68af\u5ea6":41,"\u4e58\u9664\u7b49\u65f6\u5019":27,"\u4e5d\u4e2a":63,"\u4e5f":36,"\u4e5f\u4e0d\u4f7f\u7528\u5176\u4ed6\u52a8\u6001\u5e93":25,"\u4e5f\u4e0d\u5b58\u5728\u4e00\u4e2asubseq\u76f4\u63a5\u751f\u6210\u4e0b\u4e00\u4e2asubseq\u7684\u60c5\u51b5":38,"\u4e5f\u53ef\u4ee5\u4f7f\u7528":40,"\u4e5f\u53ef\u4ee5\u53bb\u6389\u8fd9\u4e9b\u8bc1\u4e66\u7684\u914d\u7f6e":50,"\u4e5f\u53ef\u4ee5\u5728\u5f00\u53d1\u955c\u50cf\u4e2d\u542f\u52a8\u4e00\u4e2asshd\u670d\u52a1":30,"\u4e5f\u53ef\u4ee5\u662f\u4e00\u4e2a\u8bcd\u8bed":38,"\u4e5f\u53ef\u4ee5\u8bf4\u662f\u67d0\u4e9b\u7279\u5b9a\u6307\u4ee4\u7684\u4f7f\u7528\u60c5\u51b5":44,"\u4e5f\u53ef\u4ee5\u901a\u8fc7\u4fee\u6539":52,"\u4e5f\u53ef\u4ee5\u901a\u8fc7saving_period_by_batches\u8bbe\u7f6e\u6bcf\u9694\u591a\u5c11batch\u4fdd\u5b58\u4e00\u6b21\u6a21\u578b":60,"\u4e5f\u53ef\u4ee5\u914d\u7f6e\u4e0d\u540c\u7684\u91cd\u8bd5\u673a\u5236":50,"\u4e5f\u5c31\u662f":40,"\u4e5f\u5c31\u662f\u5c06\u8bcd\u5411\u91cf\u6a21\u578b\u8fdb\u4e00\u6b65\u6f14\u5316\u4e3a\u4e09\u4e2a\u65b0\u6b65\u9aa4":60,"\u4e5f\u5c31\u662f\u81ea\u5df1\u7528\u6237\u540d\u4e0b\u7684":40,"\u4e5f\u5c31\u662f\u8bf4":[47,49,56],"\u4e5f\u5f97\u5230\u4e00\u4e2a\u7528\u6237\u7279\u5f81":62,"\u4e5f\u63cf\u8ff0\u4e86\u5bb9\u5668\u9700\u8981\u4f7f\u7528\u7684\u5b58\u50a8\u5377\u6302\u8f7d\u7684\u60c5\u51b5":52,"\u4e5f\u652f\u6301cpu\u7684\u6027\u80fd\u5206\u6790":44,"\u4e5f\u662f\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217":36,"\u4e5f\u662f\u5bb9\u5668\u4e0enode\u4e4b\u95f4\u5171\u4eab\u6587\u4ef6\u7684\u65b9\u5f0f":50,"\u4e5f\u662fdecoder\u5faa\u73af\u5c55\u5f00\u7684\u4f9d\u636e":38,"\u4e5f\u662fpaddlepaddle\u6240\u80fd\u591f\u4fdd\u8bc1\u7684shuffle\u7c92\u5ea6":3,"\u4e5f\u6ca1\u7528":27,"\u4e5f\u79f0\u4e3arnn\u6a21\u578b":60,"\u4e5f\u8bb8\u662f\u56e0\u4e3a\u9700\u8981\u5b89\u88c5":57,"\u4e5f\u9700\u8981\u4e24\u6b21\u968f\u673a\u9009\u62e9\u5230\u76f8\u540cgenerator\u7684\u65f6\u5019":3,"\u4e66\u5199":25,"\u4e7e":36,"\u4e86":36,"\u4e86\u89e3\u60a8\u7684\u786c\u4ef6":44,"\u4e86\u89e3\u66f4\u591a\u7ec6\u8282":39,"\u4e86\u89e3\u66f4\u591a\u8be6\u7ec6\u4fe1\u606f":39,"\u4e8c\u7ea7\u76ee\u5f55":[64,65],"\u4e8c\u7ef4\u77e9\u9635":58,"\u4e8c\u8005\u8bed\u610f\u4e0a\u5b8c\u5168\u4e00\u81f4":36,"\u4e8c\u8fdb\u5236":56,"\u4e92\u76f8\u901a\u4fe1":50,"\u4e92\u8054\u7f51\u7535\u5f71\u6570\u636e\u5e93":64,"\u4e94\u661f\u7ea7":36,"\u4e9a\u9a6c\u900a":64,"\u4ea4\u901a":36,"\u4ea4\u901a\u4fbf\u5229":36,"\u4eab\u53d7\u60a8\u7684\u65c5\u7a0b":30,"\u4ec0\u4e48":62,"\u4ec5\u4ec5\u4f7f\u7528":25,"\u4ec5\u4ec5\u662f\u4e00\u4e9b\u5173\u952e\u8bcd":64,"\u4ec5\u4ec5\u662f\u4e24\u4e2a\u5168\u8fde\u63a5\u5c42":62,"\u4ec5\u4ec5\u662f\u7b80\u5355\u7684\u5d4c\u5165":62,"\u4ec5\u5305\u542b\u8bad\u7ec3\u548c\u6d4b\u8bd5\u6570\u6910\u96c6":64,"\u4ec5\u5728\u8fdc\u7a0b\u7a00\u758f\u8bad\u7ec3\u65f6\u6709\u6548":41,"\u4ec5\u5bf9\u7a00\u758f\u6570\u636e\u6709\u6548":41,"\u4ec5\u9700\u8981\u77e5\u9053\u5982\u4f55\u4ece":3,"\u4ecb\u7ecd\u4e86\u4e00\u79cd\u901a\u8fc7ssh\u8fdc\u7a0b\u5206\u53d1\u4efb\u52a1":52,"\u4ecb\u7ecd\u5206\u5e03\u5f0f\u8bad\u7ec3\u4e4b\u524d":50,"\u4ecb\u7ecdpaddlepaddle\u7684\u57fa\u672c\u4f7f\u7528\u65b9\u6cd5":60,"\u4ece":[44,63],"\u4ece0\u5230num":47,"\u4ece\u4e00\u4e2aword\u751f\u6210\u4e0b\u4e00\u4e2aword":38,"\u4ece\u5185\u6838\u51fd\u6570\u7684\u89d2\u5ea6":44,"\u4ece\u56fe\u4e2d\u53ef\u4ee5\u770b\u5230":28,"\u4ece\u5916\u90e8\u7f51\u7ad9\u4e0a\u4e0b\u8f7d\u7684\u539f\u59cb\u6570\u6910\u96c6":64,"\u4ece\u5927\u5230\u5c0f":65,"\u4ece\u6570\u636e\u63d0\u4f9b\u7a0b\u5e8f\u52a0\u8f7d\u5b9e\u4f8b":63,"\u4ece\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65\u6765\u770b":36,"\u4ece\u6bcf\u4e2a\u5355\u8bcd\u5de6\u53f3\u4e24\u7aef\u5206\u522b\u83b7\u53d6k\u4e2a\u76f8\u90bb\u7684\u5355\u8bcd":60,"\u4ece\u7b2c0\u4e2a\u8bc4\u4f30\u5230\u5f53\u524d\u8bc4\u4f30\u4e2d":65,"\u4ece\u7b2c0\u4e2abatch\u5230\u5f53\u524dbatch\u7684\u5e73\u5747\u635f\u5931":64,"\u4ece\u7b2c0\u4e2abatch\u5230\u5f53\u524dbatch\u7684\u5e73\u5747cost":65,"\u4ece\u800c\u53ef\u4ee5\u505a\u4e00\u4e9b\u4e0e\u8ba1\u7b97\u91cd\u53e0\u7684\u5de5\u4f5c":41,"\u4ece\u800c\u53ef\u4ee5\u62df\u5408\u4efb\u610f\u7684\u51fd\u6570\u6765\u5b66\u4e60\u590d\u6742\u7684\u6570\u636e\u5173\u7cfb":28,"\u4ece\u800c\u751f\u6210\u591a\u4e2agener":3,"\u4ece\u800c\u80fd\u591f\u88abpaddlepaddl":60,"\u4ece\u800c\u9632\u6b62\u8fc7\u62df\u5408":2,"\u4ece\u8be5\u94fe\u63a5":65,"\u4ece\u8bed\u4e49\u4e0a\u770b":38,"\u4ece\u8f93\u5165\u6570\u636e\u4e0a\u770b":36,"\u4ece\u8f93\u51fa\u65e5\u5fd7\u53ef\u4ee5\u770b\u5230":28,"\u4ece\u9884\u8bad\u7ec3\u6a21\u578b\u4e2d":56,"\u4ecestart":47,"\u4ecetest":65,"\u4ed3\u5e93\u7684\u8fdc\u7a0b\u4e3b\u673a":40,"\u4ed4\u7ec6\u89c2\u5bdf":58,"\u4ed6\u4eec\u5206\u522b\u662f":36,"\u4ed6\u4eec\u5728paddle\u7684\u6587\u6863\u548capi\u4e2d\u662f\u4e00\u4e2a\u6982\u5ff5":36,"\u4ed6\u4eec\u63d0\u51fa\u6b8b\u5dee\u5b66\u4e60\u7684\u6846\u67b6\u6765\u7b80\u5316\u7f51\u7edc\u7684\u8bad\u7ec3":58,"\u4ed6\u7684\u76ee\u6807\u662f\u4f7f\u7528c":25,"\u4ee3\u66ff":52,"\u4ee3\u7801":62,"\u4ee3\u7801\u4e2d9":36,"\u4ee3\u7801\u5982\u4e0b":39,"\u4ee3\u7801\u6ce8\u91ca\u8bf7\u9075\u5b88":40,"\u4ee3\u7801\u751f\u6210\u7684\u7b26\u53f7\u53ef\u80fd\u4e0d\u4e00\u81f4":25,"\u4ee3\u8868\u5bbf\u4e3b\u673a\u76ee\u5f55":52,"\u4ee3\u8868\u7f16\u53f7":62,"\u4ee5\u4e0a":40,"\u4ee5\u4e0a\u547d\u4ee4\u4f1a\u6309\u7167\u751f\u6210\u7684dockerfile\u628a\u751f\u6210\u7684\u7a0b\u5e8f\u62f7\u8d1d\u5230\u751f\u4ea7\u955c\u50cf\u4e2d\u5e76\u505a\u76f8\u5e94\u7684\u914d\u7f6e":30,"\u4ee5\u4e0a\u547d\u4ee4\u4f1a\u7f16\u8bd1paddlepaddl":30,"\u4ee5\u4e0a\u65b9\u6cd5\u5728gpu\u955c\u50cf\u91cc\u4e5f\u80fd\u7528":30,"\u4ee5\u4e0b":62,"\u4ee5\u4e0b\u4ee3\u7801\u6bb5\u5b9a\u4e49\u4e86\u4e09\u4e2a\u8f93\u5165":39,"\u4ee5\u4e0b\u4ee3\u7801\u7247\u6bb5\u5b9a\u4e49":39,"\u4ee5\u4e0b\u6211\u4eec\u7ffb\u8bd1\u6570\u636e\u96c6\u7f51\u7ad9\u4e2dreadme\u6587\u4ef6\u7684\u63cf\u8ff0":61,"\u4ee5\u4e0b\u6307\u4ee4\u80fd\u68c0\u67e5linux\u7535\u8111\u662f\u5426\u652f\u6301avx":30,"\u4ee5\u4e0b\u6559\u7a0b\u5c06\u6307\u5bfc\u60a8\u63d0\u4ea4\u4ee3\u7801":40,"\u4ee5\u4e0b\u662f\u5bf9\u4e0a\u8ff0\u6570\u636e\u52a0\u8f7d\u7684\u89e3\u91ca":60,"\u4ee5\u4e0b\u6b65\u9aa4\u57fa\u4e8e":45,"\u4ee5\u4e0b\u793a\u8303\u5982\u4f55\u4f7f\u7528\u9884\u8bad\u7ec3\u7684\u4e2d\u6587\u5b57\u5178\u548c\u8bcd\u5411\u91cf\u8fdb\u884c\u77ed\u8bed\u6539\u5199":56,"\u4ee5\u4e0b\u9009\u9879\u5fc5\u987b\u5728":45,"\u4ee5\u4ea4\u4e92\u5bb9\u5668\u65b9\u5f0f\u8fd0\u884c\u5f00\u53d1\u955c\u50cf":30,"\u4ee5\u4fbf\u7528\u6237":45,"\u4ee5\u4fdd\u8bc1\u68af\u5ea6\u7684\u6b63\u786e\u8ba1\u7b97":41,"\u4ee5\u4fdd\u8bc1\u68af\u5ea6\u8ba1\u7b97\u7684\u6b63\u786e\u6027":41,"\u4ee5\u5206\u7c7b\u6765\u81ea":64,"\u4ee5\u53ca":41,"\u4ee5\u53ca\u4f7f\u7528\u5b50\u5e8f\u5217\u6765\u5b9a\u4e49\u5206\u7ea7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u67b6\u6784":39,"\u4ee5\u53ca\u53cc\u5c42\u5e8f\u5217":35,"\u4ee5\u53ca\u5728wmt":65,"\u4ee5\u53ca\u5982\u4f55\u5728\u5c42\u4e4b\u95f4\u8fdb\u884c\u8fde\u63a5":57,"\u4ee5\u53ca\u751f\u6210\u521b\u5efa\u751f\u4ea7\u955c\u50cf\u7684dockerfil":30,"\u4ee5\u53ca\u8ba1\u7b97\u903b\u8f91\u5728\u5e8f\u5217\u4e0a\u7684\u5faa\u73af\u5c55\u5f00":38,"\u4ee5\u53ca\u8f93\u5165\u7684\u68af\u5ea6":41,"\u4ee5\u53capaddle\u5982\u4f55\u5904\u7406\u591a\u79cd\u7c7b\u578b\u7684\u8f93\u5165":62,"\u4ee5\u53carelu":41,"\u4ee5\u76f8\u5bf9\u8def\u5f84\u5f15\u7528":2,"\u4ee5\u83b7\u5f97\u66f4\u597d\u7684\u7f51\u7edc\u6027\u80fd":45,"\u4ee5\u9017\u53f7":56,"\u4ee5\u9017\u53f7\u95f4\u9694":47,"\u4ef7\u683c":36,"\u4efb\u52a1":62,"\u4efb\u52a1\u6765\u7ec8\u6b62\u96c6\u7fa4\u4f5c\u4e1a":45,"\u4efb\u610f\u5c06\u4e00\u4e9b\u6570\u636e\u7ec4\u5408\u6210\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217":36,"\u4f18\u5316":64,"\u4f18\u5316\u5668\u5219\u7528\u94fe\u5f0f\u6cd5\u5219\u6765\u5bf9\u6bcf\u4e2a\u53c2\u6570\u8ba1\u7b97\u635f\u5931\u51fd\u6570\u7684\u68af\u5ea6":41,"\u4f1a\u5148\u8fdb\u884c\u53c2\u6570\u7684\u521d\u59cb\u5316\u4e0e\u89e3\u6790":52,"\u4f1a\u5171\u4eab\u53c2\u6570":27,"\u4f1a\u52a0\u8f7d\u4e0a\u4e00\u8f6e\u7684\u53c2\u6570":47,"\u4f1a\u53d8\u6210\u8bcd\u8868\u4e2d\u7684\u4f4d\u7f6e":36,"\u4f1a\u542f\u52a8pserver\u4e0etrainer\u8fdb\u7a0b":52,"\u4f1a\u5bf9\u6bcf\u4e00\u4e2a\u6fc0\u6d3b\u6682\u5b58\u4e00\u4e9b\u6570\u636e":27,"\u4f1a\u5bf9\u8fd9\u7c7b\u8f93\u5165\u8fdb\u884c\u62c6\u89e3":38,"\u4f1a\u5bfc\u81f4\u4e0d\u540c\u7248\u672cpython\u5728\u4e00\u4e2a\u8fdb\u7a0b\u91cc\u7684bug":25,"\u4f1a\u5c06\u6bcf\u4e2a\u65f6\u95f4\u6b65\u7684\u8f93\u51fa\u62fc\u63a5":38,"\u4f1a\u5c06\u7b2c\u4e00\u4e2a":27,"\u4f1a\u6210\u4e3astep\u51fd\u6570\u7684\u8f93\u5165":38,"\u4f1a\u6254\u5230\u8fd9\u6761\u6570\u636e":3,"\u4f1a\u62a5\u9519":38,"\u4f1a\u6839\u636e\u547d\u4ee4\u884c\u53c2\u6570\u6307\u5b9a\u7684\u6d4b\u8bd5\u65b9\u5f0f":2,"\u4f1a\u6839\u636einput_types\u68c0\u67e5\u6570\u636e\u7684\u5408\u6cd5\u6027":3,"\u4f1a\u76f4\u63a5\u62a5\u9519\u9000\u51fa":25,"\u4f1a\u76f8\u5e94\u5730\u6539\u53d8\u8f93\u51fa\u7684\u5c3a\u5bf8":41,"\u4f1a\u81ea\u9002\u5e94\u5730\u4ece\u8fd9\u4e9b\u5411\u91cf\u4e2d\u9009\u62e9\u4e00\u4e2a\u5b50\u96c6\u51fa\u6765":65,"\u4f1a\u83b7\u53d6\u5f53\u524dnamespace\u4e0b\u7684\u6240\u6709pod":52,"\u4f1a\u88ab\u62c6\u89e3\u4e3a\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":38,"\u4f1a\u88ab\u62c6\u89e3\u4e3a\u975e\u5e8f\u5217":38,"\u4f20\u5165":3,"\u4f20\u5165\u4e0a\u4e00\u6b65\u89e3\u6790\u51fa\u6765\u7684\u6a21\u578b\u914d\u7f6e\u5c31\u53ef\u4ee5\u521b\u5efa\u4e00\u4e2a":5,"\u4f20\u5165\u9884\u6d4b\u6570\u636e":5,"\u4f20\u7ed9dataprovider\u7684\u67d0\u4e00\u4e2aargs\u8fc7\u5927":27,"\u4f20\u9012\u7ed9\u914d\u7f6e\u6587\u4ef6\u7684\u53c2\u6570":47,"\u4f46\u4e0d\u7528\u4e8e\u8ba1\u7b97\u68af\u5ea6":41,"\u4f46\u4e0d\u9700\u8981\u63d0\u524d\u521b\u5efa":47,"\u4f46\u4e8e\u53cc\u5c42\u5e8f\u5217\u7684lstm\u6765\u8bf4":36,"\u4f46\u548c\u5355\u5c42rnn\u4e0d\u540c":36,"\u4f46\u5728\u8d77\u521d\u7684\u51e0\u8f6e\u8bad\u7ec3\u4e2d\u5b83\u4eec\u90fd\u5728\u5feb\u901f\u903c\u8fd1\u771f\u5b9e\u503c":28,"\u4f46\u5b50\u53e5\u542b\u6709\u7684\u8bcd\u8bed\u6570\u53ef\u4ee5\u4e0d\u76f8\u7b49":38,"\u4f46\u5c3d\u91cf\u8bf7\u4fdd\u6301\u7f16\u8bd1\u548c\u8fd0\u884c\u4f7f\u7528\u7684cudnn\u662f\u540c\u4e00\u4e2a\u7248\u672c":29,"\u4f46\u5e8f\u5217\u8f93\u51fa\u65f6":36,"\u4f46\u5f53\u8c03\u7528\u8fc7\u4e00\u6b21\u540e":3,"\u4f46\u662f":[27,36],"\u4f46\u662f2008\u5e74\u4e4b\u524d\u751f\u4ea7\u7684\u65e7\u7535\u8111\u4e0d\u652f\u6301avx":30,"\u4f46\u662f\u4e5f\u6ca1\u6709\u5fc5\u8981\u5220\u9664\u65e0\u7528\u7684\u6587\u4ef6":45,"\u4f46\u662f\u5927\u90e8\u5206\u53c2\u6570\u662f\u4e3a\u5f00\u53d1\u8005\u63d0\u4f9b\u7684":46,"\u4f46\u662f\u5b50\u5e8f\u5217\u7684\u6570\u76ee\u5fc5\u987b\u4e00\u6837":36,"\u4f46\u662f\u6211\u4eec\u5e76\u4e0d\u63a8\u8350\u8fd9\u79cd\u65b9\u6cd5":30,"\u4f46\u662f\u652f\u6301avx\u6307\u4ee4\u96c6":40,"\u4f46\u662f\u65b9\u4fbf\u8c03\u8bd5\u548c\u6d4bbenchmark":29,"\u4f46\u662f\u6bcf\u4e2a\u6837\u672c\u4ec5\u5305\u542b\u51e0\u4e2a\u8bcd":49,"\u4f46\u662f\u7a81\u7136\u6709\u4e00\u4e2a10000\u957f\u7684\u5e8f\u5217":27,"\u4f46\u662f\u89e3\u91ca\u6027\u8bed\u8a00":25,"\u4f46\u662f\u8fd9\u79cd\u65b9\u6cd5\u5728\u6bcf\u5c42\u53ea\u4fdd\u5b58\u9884\u8bbe\u6570\u91cf\u7684\u6700\u4f18\u72b6\u6001":65,"\u4f46\u662f\u8fdc\u672a\u5b8c\u5584":0,"\u4f46\u662f\u9690\u85cf\u5c42\u4e2d\u7684\u6bcf\u4e2a\u666e\u901a\u8282\u70b9\u88ab\u4e00\u4e2a\u8bb0\u5fc6\u5355\u5143\u66ff\u6362":64,"\u4f46\u662fbatch":27,"\u4f46\u6709\u503c\u7684\u5730\u65b9\u5fc5\u987b\u4e3a1":[3,33],"\u4f46\u6709\u503c\u7684\u90e8\u5206\u53ef\u4ee5\u662f\u4efb\u4f55\u6d6e\u70b9\u6570":[3,33],"\u4f46\u8fd9\u4e2a\u5173\u7cfb\u53ef\u80fd\u4e0d\u6b63\u786e":3,"\u4f4d\u7f6e":36,"\u4f4f":36,"\u4f53\u88c1\u5b57\u5178":62,"\u4f53\u88c1\u5b57\u6bb5":62,"\u4f59\u5f26\u76f8\u4f3c\u5ea6\u56de\u5f52":62,"\u4f59\u5f26\u76f8\u4f3c\u5ea6\u5c42":62,"\u4f5c\u4e3a\u4e0b\u4e00\u4e2a\u5b50\u53e5memory\u7684\u521d\u59cb\u72b6\u6001":36,"\u4f5c\u4e3a\u4f8b\u5b50\u6f14\u793a\u5982\u4f55\u914d\u7f6e\u590d\u6742\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u6a21\u578b":39,"\u4f5c\u4e3a\u53c2\u6570\u7684id":27,"\u4f5c\u4e3a\u5f53\u524d\u65f6\u523b\u8f93\u5165":38,"\u4f5c\u4e3a\u793a\u4f8b\u6570\u636e":61,"\u4f5c\u4e3a\u7c7b\u53e5\u67c4":25,"\u4f5c\u4e3a\u7ebf\u6027\u56de\u5f52\u7684\u8f93\u5165":28,"\u4f5c\u4e3a\u8f93\u51fa":39,"\u4f5c\u4e3a\u96c6\u7fa4\u8bad\u7ec3\u7684\u5de5\u4f5c\u7a7a\u95f4":45,"\u4f5c\u4e3aboot_layer\u4f20\u7ed9\u4e0b\u4e00\u4e2a\u5b50\u53e5\u7684memori":36,"\u4f5c\u5bb6":61,"\u4f5c\u7528":35,"\u4f60\u4e5f\u53ef\u4ee5\u4f7f\u7528\u8fd9\u4e09\u4e2a\u503c":58,"\u4f60\u4e5f\u53ef\u4ee5\u5148\u8df3\u8fc7\u672c\u6587\u7684\u89e3\u91ca\u73af\u8282":60,"\u4f60\u4e5f\u53ef\u4ee5\u7b80\u5355\u7684\u8fd0\u884c\u4ee5\u4e0b\u7684\u547d\u4ee4":56,"\u4f60\u4e5f\u53ef\u4ee5\u901a\u8fc7\u5728\u547d\u4ee4\u884c\u53c2\u6570\u4e2d\u589e\u52a0\u4e00\u4e2a\u53c2\u6570\u5982":58,"\u4f60\u4e5f\u8bb8\u53ef\u4ee5\u5c1d\u8bd5\u66f4\u8001\u7684\u65b9\u6cd5":30,"\u4f60\u53ea\u9700\u5b8c\u6210":45,"\u4f60\u53ea\u9700\u8981\u5728\u547d\u4ee4\u884c\u8f93\u5165\u4ee5\u4e0b\u547d\u4ee4":60,"\u4f60\u53ea\u9700\u8981\u6309\u7167\u5982\u4e0b\u65b9\u5f0f\u7ec4\u7ec7\u6570\u636e":65,"\u4f60\u53ef\u4ee5\u4f7f\u7528":58,"\u4f60\u53ef\u4ee5\u4f7f\u7528\u4e0b\u9762\u7684\u811a\u672c\u4e0b\u8f7d":64,"\u4f60\u53ef\u4ee5\u4f7f\u7528\u5b83\u6765\u8bbe\u7f6e":45,"\u4f60\u53ef\u4ee5\u4f7f\u7528\u672c\u5730\u8bad\u7ec3\u4e2d\u7684\u76f8\u540c\u6a21\u578b\u6587\u4ef6\u8fdb\u884c\u96c6\u7fa4\u8bad\u7ec3":45,"\u4f60\u53ef\u4ee5\u5728\u4efb\u4f55\u65f6\u5019\u7528":62,"\u4f60\u53ef\u4ee5\u5bf9\u6a21\u578b\u8fdb\u884c\u8bc4\u4f30":62,"\u4f60\u53ef\u4ee5\u5c06\u7f51\u7edc\u914d\u7f6e\u6210\u67d0\u4e9b\u5c42\u4f7f\u7528gpu\u8ba1\u7b97":49,"\u4f60\u53ef\u4ee5\u6267\u884c\u4e0a\u8ff0\u547d\u4ee4\u6765\u4e0b\u8f7d\u6240\u6709\u7684\u6a21\u578b\u548c\u5747\u503c\u6587\u4ef6":58,"\u4f60\u53ef\u4ee5\u901a\u8fc7\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u6765\u5f97\u5230resnet\u7f51\u7edc\u7684\u7ed3\u6784\u53ef\u89c6\u5316\u56fe":58,"\u4f60\u53ef\u4ee5\u9884\u6d4b\u4efb\u4f55\u7528\u6237\u5bf9\u4e8e\u4efb\u4f55\u4e00\u90e8\u7535\u5f71\u7684\u8bc4\u4ef7":62,"\u4f60\u5c06\u4f1a\u770b\u5230\u4ee5\u4e0b\u7684\u6a21\u578b\u7ed3\u6784":56,"\u4f60\u5c06\u4f1a\u770b\u5230\u5982\u4e0b\u6d88\u606f":65,"\u4f60\u5c06\u4f1a\u770b\u5230\u5982\u4e0b\u7ed3\u679c":58,"\u4f60\u5c06\u4f1a\u770b\u5230\u7279\u5f81\u5b58\u50a8\u5728":58,"\u4f60\u5c06\u4f1a\u770b\u5230\u8fd9\u6837\u7684\u6d88\u606f":65,"\u4f60\u5c06\u5728\u76ee\u5f55":64,"\u4f60\u5c06\u770b\u5230\u5982\u4e0b\u7684\u4fe1\u606f":62,"\u4f60\u7684\u5de5\u4f5c\u7a7a\u95f4\u5e94\u5982\u4e0b\u6240\u793a":45,"\u4f60\u8fd8\u53ef\u4ee5\u5c06\u7528\u6237\u548c":45,"\u4f60\u9700\u8981\u4e00\u4e9b\u66f4\u590d\u6742\u7684\u5355\u5143\u6d4b\u8bd5\u6765\u4fdd\u8bc1\u4f60\u5b9e\u73b0\u7684\u7f51\u7edc\u5c42\u662f\u6b63\u786e\u7684":41,"\u4f60\u9700\u8981\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u6307\u5b9a\u8bbe\u5907\u7684id\u53f7":49,"\u4f60\u9700\u8981\u5728\u914d\u7f6ecmake\u65f6\u5c06":41,"\u4f60\u9700\u8981\u5b89\u88c5python\u7684\u7b2c\u4e09\u65b9\u5e93":62,"\u4f60\u9700\u8981\u628a\u8be5\u6587\u4ef6\u52a0\u5165":41,"\u4f7f\u5176\u8f6c\u53d8\u4e3a\u7ef4\u5ea6\u4e3ahidden_dim\u7684\u65b0\u5411\u91cf":60,"\u4f7f\u5f97":28,"\u4f7f\u5f97\u4e24\u4e2a\u5b57\u5178\u6709\u76f8\u540c\u7684\u4e0a\u4e0b\u6587":65,"\u4f7f\u5f97\u5355\u5143\u6d4b\u8bd5\u6709\u4e00\u4e2a\u5e72\u51c0\u7684\u73af\u5883":27,"\u4f7f\u5f97\u642d\u6a21\u578b\u65f6\u66f4\u65b9\u4fbf":41,"\u4f7f\u5f97\u6700\u7ec8\u5f97\u5230\u7684\u6a21\u578b\u51e0\u4e4e\u4e0e\u771f\u5b9e\u6a21\u578b\u4e00\u81f4":28,"\u4f7f\u7528":[27,30,36,38,39,41,44,47,60,63,64],"\u4f7f\u75280\u53f7\u548c1\u53f7gpu\u8ba1\u7b97fc2\u5c42":49,"\u4f7f\u75280\u53f7gpu\u8ba1\u7b97fc2\u5c42":49,"\u4f7f\u752810\u4e2a\u88c1\u526a\u56fe\u50cf\u5757":58,"\u4f7f\u75281\u53f7gpu\u8ba1\u7b97fc3\u5c42":49,"\u4f7f\u75282\u53f7\u548c3\u53f7gpu\u8ba1\u7b97fc3\u5c42":49,"\u4f7f\u7528\u4e00\u4e2a\u5c3a\u5ea6\u4e3a":41,"\u4f7f\u7528\u4e00\u4e2a\u8bcd\u524d\u4e24\u4e2a\u8bcd\u548c\u540e\u4e24\u4e2a\u8bcd":27,"\u4f7f\u7528\u4e0a\u6587\u521b\u5efa\u7684yaml\u6587\u4ef6\u521b\u5efakubernet":51,"\u4f7f\u7528\u4e86\u540c\u6837\u7684parameter\u548cbia":27,"\u4f7f\u7528\u4e86\u57fa\u4e8e\u53e5\u6cd5\u7ed3\u6784\u7684\u9884\u5b9a\u4e49\u7279\u5f81\u6a21\u677f":63,"\u4f7f\u7528\u4e86avx\u6307\u4ee4\u96c6":32,"\u4f7f\u7528\u4ee5\u4e0b\u547d\u4ee4\u8fdb\u884c\u6a21\u578b\u8bad\u7ec3":56,"\u4f7f\u7528\u5177\u6709softmax\u6fc0\u6d3b\u7684\u5168\u8fde\u63a5\u524d\u9988\u5c42\u6765\u6267\u884c\u5206\u7c7b\u4efb\u52a1":64,"\u4f7f\u7528\u52a8\u6001\u5e93":25,"\u4f7f\u7528\u591a\u5757\u663e\u5361\u8bad\u7ec3":27,"\u4f7f\u7528\u591a\u7ebf\u7a0b\u8bad\u7ec3":27,"\u4f7f\u7528\u5982\u4e0b\u53c2\u6570":57,"\u4f7f\u7528\u5982\u4e0b\u547d\u4ee4":56,"\u4f7f\u7528\u5b66\u4e60\u5b8c\u6210\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u6a21\u578b\u751f\u6210\u5e8f\u5217":39,"\u4f7f\u7528\u5bb9\u5668\u65b9\u5f0f\u8fd0\u884c\u8bad\u7ec3\u4efb\u52a1\u7684kubernet":52,"\u4f7f\u7528\u6211\u4eec\u4e4b\u524d\u6784\u9020\u7684\u955c\u50cf":51,"\u4f7f\u7528\u663e\u5361\u8bad\u7ec3":27,"\u4f7f\u7528\u6848\u4f8b":48,"\u4f7f\u7528\u7684":27,"\u4f7f\u7528\u8005\u4e0d\u9700\u8981\u5173\u5fc3":47,"\u4f7f\u7528\u8005\u53ea\u9700\u8981\u5173\u6ce8\u4e8e\u8bbe\u8ba1rnn\u5728\u4e00\u4e2a\u65f6\u95f4\u6b65\u4e4b\u5185\u5b8c\u6210\u7684\u8ba1\u7b97":38,"\u4f7f\u7528\u8005\u53ef\u4ee5\u4f7f\u7528\u4e0b\u9762\u7684python\u811a\u672c\u6765\u8bfb\u53d6\u53c2\u6570\u503c":58,"\u4f7f\u7528\u8005\u65e0\u9700\u5173\u5fc3\u8fd9\u4e2a\u53c2\u6570":47,"\u4f7f\u7528\u8005\u901a\u5e38\u65e0\u9700\u5173\u5fc3":47,"\u4f7f\u7528\u81ea\u52a8\u7684\u66ff\u8865\u6765\u66ff\u4ee3\u7ecf\u9a8c\u4e30\u5bcc\u7684\u4eba\u5de5\u8bc4\u5224":65,"\u4f7f\u7528\u8c13\u8bcd\u4e0a\u4e0b\u6587":63,"\u4f7f\u7528\u8fd9\u4e2a\u811a\u672c\u524d\u8bf7\u786e\u8ba4\u5df2\u7ecf\u5b89\u88c5\u4e86pillow\u53ca\u76f8\u5173\u4f9d\u8d56\u6a21\u5757":57,"\u4f7f\u7528\u8fd9\u79cd\u65b9\u5f0f":36,"\u4f7f\u7528\u8fdc\u7a0b\u7a00\u758f\u65b9\u5f0f\u8bad\u7ec3\u65f6":41,"\u4f7f\u7528\u968f\u673a\u68af\u5ea6\u4e0b\u964d":64,"\u4f7f\u7528\u9759\u6001\u5e93\u548c\u52a8\u6001\u5e93\u96be\u5ea6\u5dee\u4e0d\u591a":25,"\u4f7f\u7528\u9884\u8bad\u7ec3\u7684\u6807\u51c6\u683c\u5f0f\u8bcd\u5411\u91cf\u6a21\u578b":56,"\u4f7f\u7528args\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u8bbe\u7f6e":3,"\u4f7f\u7528c99\u505a\u63a5\u53e3":25,"\u4f7f\u7528c99\u800c\u4e0d\u4f7f\u7528c11\u7684\u539f\u56e0\u662f":25,"\u4f7f\u7528c99\u800c\u4e0d\u4f7f\u7528c89":25,"\u4f7f\u7528checkgrad\u6a21\u5f0f\u65f6\u7684\u53c2\u6570\u53d8\u5316\u5927\u5c0f":47,"\u4f7f\u7528cpu\u4e24\u7ebf\u7a0b\u8ba1\u7b97fc4\u5c42":49,"\u4f7f\u7528cpu\u8ba1\u7b97fc4\u5c42":49,"\u4f7f\u7528cpu\u8bad\u7ec3":64,"\u4f7f\u7528init":49,"\u4f7f\u7528lstm\u4f5c\u4e3aencod":36,"\u4f7f\u7528max":57,"\u4f7f\u7528memory\u7684rnn\u5b9e\u73b0\u4fbf\u5982\u4e0b\u56fe\u6240\u793a":36,"\u4f7f\u7528model":49,"\u4f7f\u7528paddlepaddl":60,"\u4f7f\u7528python\u6570\u636e\u63d0\u4f9b\u5668":57,"\u4f7f\u7528rdma\u8fd8\u662ftcp\u4f20\u8f93\u534f\u8bae":47,"\u4f7f\u7528swig\u53ea\u652f\u6301cpython\u89e3\u91ca\u5668":25,"\u4f7f\u7528swig\u9700\u8981\u591a\u8bed\u8a00\u7ed1\u5b9a\u7684\u5f00\u53d1\u4eba\u5458\u719f\u7ec3\u638c\u63e1swig\u914d\u7f6e":25,"\u4f7f\u7528void":25,"\u4f7f\u8f93\u5165\u5c42\u5230\u9690\u85cf\u5c42\u7684\u795e\u7ecf\u5143\u662f\u5168\u90e8\u8fde\u63a5\u7684":57,"\u4f86":36,"\u4f8b\u5982":[3,25,27,29,33,36,39,41,44,45,46,47,49,52,58,60,62,64],"\u4f8b\u5982\u4e0a\u6587\u7684pod":50,"\u4f8b\u5982\u4e0a\u9762\u7684":28,"\u4f8b\u5982\u4ee5\u592a\u7f51\u7684":45,"\u4f8b\u5982\u4f7f\u7528":27,"\u4f8b\u5982\u586b\u5145":39,"\u4f8b\u5982\u5bf9\u4e8ejava\u6216\u8005python":25,"\u4f8b\u5982\u5bf9\u4e8ejava\u6765\u8bf4":25,"\u4f8b\u5982\u5bf9\u4e8epython":25,"\u4f8b\u5982\u5c06\u7b2c\u4e00\u6761\u6570\u636e\u8f6c\u5316\u4e3a":36,"\u4f8b\u5982\u6587\u672c\u5206\u7c7b\u4e2d":36,"\u4f8b\u5982\u672c\u4f8b\u4e2d\u7684\u4e24\u4e2a\u7279\u5f81":36,"\u4f8b\u5982\u673a\u5668\u4e0a\u67094\u5757gpu":27,"\u4f8b\u5982\u7b2c300\u4e2apass\u7684\u6a21\u578b\u4f1a\u88ab\u4fdd\u5b58\u5728":57,"\u4f8b\u5982c":25,"\u4f8b\u5982hostpath":50,"\u4f8b\u5982java\u4e0epython\u7684\u9519\u8bef\u5904\u7406\u662f\u76f4\u63a5\u6254\u51fa\u6765except":25,"\u4f8b\u5982output\u76ee\u5f55\u4e0b\u5c31\u5b58\u653e\u4e86\u8f93\u51fa\u7ed3\u679c":52,"\u4f8b\u5982python\u53ef\u4ee5\u4f7f\u7528":25,"\u4f8b\u5982python\u7684":25,"\u4f8b\u5982sigmoid":41,"\u4f8b\u5982sigmoid\u53d8\u6362":60,"\u4f8b\u5b50\u4e2d\u662f":41,"\u4f8b\u5b50\u4e2d\u662f0":41,"\u4f8b\u5b50\u4e2d\u662f100":41,"\u4f8b\u5b50\u4e2d\u662f4096":41,"\u4f8b\u5b50\u4e2d\u662f8192":41,"\u4f8b\u5b50\u4e2d\u662ffc":41,"\u4f8b\u5b50\u4e2d\u662fsoftmax":41,"\u4f8b\u5b50\u4f7f\u7528":50,"\u4f9bpaddlepaddle\u52a0\u8f7d":47,"\u4f9d\u636e\u5206\u7c7b\u9519\u8bef\u7387\u83b7\u5f97\u6700\u4f73\u6a21\u578b\u8fdb\u884c\u6d4b\u8bd5":64,"\u4f9d\u8d56\u4e8epython\u7684":57,"\u4fbf\u4e8e\u5b58\u50a8\u8d44\u6e90\u7ba1\u7406\u548cpod\u5f15\u7528":50,"\u4fbf\u4e8e\u672c\u5730\u9a8c\u8bc1\u548c\u6d4b\u8bd5":50,"\u4fbf\u4e8e\u7528\u6237\u6d4f\u89c8c":30,"\u4fbf\u5229":36,"\u4fbf\u548c\u5355\u5c42rnn\u914d\u7f6e\u4e2d\u7684":36,"\u4fbf\u5b9c":36,"\u4fbf\u662f\u5c06\u9759\u6001\u5e93\u52a0\u5165jvm\u4e2d":25,"\u4fdd\u5b58\u6a21\u578b\u53c2\u6570\u7684\u76ee\u5f55":47,"\u4fdd\u5b58\u751f\u6210\u7ed3\u679c\u7684\u6587\u4ef6":65,"\u4fdd\u5b58\u7f51\u7edc\u5c42\u8f93\u51fa\u7ed3\u679c\u7684\u76ee\u5f55":47,"\u4fdd\u5b58\u9884\u6d4b\u7ed3\u679c\u7684\u6587\u4ef6\u540d":47,"\u4fdd\u6301\u5bbd\u9ad8\u6bd4\u7f29\u653e\u5230\u77ed\u8fb9\u4e3a256":58,"\u4fe1\u53f7\u6765\u81ea\u52a8\u7ec8\u6b62\u5b83\u542f\u52a8\u7684\u6240\u6709\u8fdb\u7a0b":45,"\u4fee\u6539":[50,51],"\u4fee\u6539\u542f\u52a8\u811a\u672c\u540e":51,"\u4fee\u6539\u6210\u66f4\u5feb\u7684\u7248\u672c":44,"\u4fee\u6539\u6587\u6863":43,"\u503c\u5f97\u6ce8\u610f\u7684\u662f":[36,40],"\u503c\u5f97\u6df1\u5165\u5206\u6790":44,"\u503c\u7c7b\u578b":49,"\u5047\u5982\u6211\u4eec\u662f\u4e09\u5206\u7c7b\u95ee\u9898":27,"\u5047\u8bbe":41,"\u5047\u8bbe\u53d8\u91cf":28,"\u5047\u8bbe\u60a8\u5df2\u7ecf\u5b8c\u6210\u4e86\u4e00\u4e2aai\u8bad\u7ec3\u7684python\u7a0b\u5e8f":30,"\u5047\u8bbe\u635f\u5931\u51fd\u6570\u662f":41,"\u5047\u8bbe\u8bcd\u5411\u91cf\u7ef4\u5ea6\u4e3a32":56,"\u504f\u7f6e\u53c2\u6570":58,"\u504f\u7f6e\u53c2\u6570\u7684\u5927\u5c0f":41,"\u505a\u63a5\u53e3":25,"\u505c\u6b62\u52a0\u8f7d\u6570\u636e":47,"\u505c\u7535":36,"\u50cf":30,"\u513f\u7ae5\u7247":61,"\u5143\u7d20":35,"\u5143\u7d20\u4e4b\u95f4\u7684\u987a\u5e8f\u662f\u91cd\u8981\u7684\u8f93\u5165\u4fe1\u606f":35,"\u5148\u8c03\u7528initializer\u51fd\u6570":60,"\u5168\u5bb6":36,"\u5168\u8fde\u63a5\u5c42":[28,56,57,62],"\u5168\u8fde\u63a5\u5c42\u4ee5\u4e00\u4e2a\u7ef4\u5ea6\u4e3a":41,"\u5168\u8fde\u63a5\u5c42\u5c06\u7535\u5f71\u7684\u6bcf\u4e2a\u7279\u5f81\u7ed3\u5408\u6210\u4e00\u4e2a\u7535\u5f71\u7279\u5f81":62,"\u5168\u8fde\u63a5\u5c42\u6743\u91cd":58,"\u5168\u8fde\u63a5\u5c42\u6ca1\u6709\u7f51\u7edc\u5c42\u914d\u7f6e\u7684\u8d85\u53c2\u6570":41,"\u5168\u8fde\u63a5\u5c42\u7684\u5b9e\u73b0\u4f4d\u4e8e":41,"\u5168\u8fde\u63a5\u5c42\u7684\u6bcf\u4e2a\u8f93\u51fa\u90fd\u8fde\u63a5\u5230\u4e0a\u4e00\u5c42\u7684\u6240\u6709\u7684\u795e\u7ecf\u5143\u4e0a":41,"\u5168\u8fde\u63a5\u5c42python\u5c01\u88c5\u7684\u4f8b\u5b50\u4e2d\u5305\u542b\u4e0b\u9762\u51e0\u6b65":41,"\u516b\u4e2a\u7279\u5f81\u5206\u522b\u8f6c\u6362\u4e3a\u5411\u91cf":63,"\u516c\u5f0f":30,"\u516c\u94a5\u5199\u5165":45,"\u516d\u4e2a\u7279\u5f81\u548c\u6807\u7b7e\u90fd\u662f\u7d22\u5f15\u69fd":63,"\u5171\u4eab\u4efb\u52a1\u4e2d\u8bbe\u7f6e\u7684\u6570\u636e\u8fdb\u884c\u8bad\u7ec3\u548c\u6d4b\u8bd5":63,"\u5171\u4eab\u5b58\u50a8\u6302\u5728\u7684\u8def\u5f84":52,"\u5171\u670932":56,"\u5173\u4e8e\u5982\u4f55\u5b9a\u4e49\u7f51\u7edc\u4e2d\u7684\u5c42":57,"\u5173\u4e8e\u65f6\u95f4\u5e8f\u5217":36,"\u5173\u4e8e\u6784\u5efa\u548c\u6d4b\u8bd5\u7684\u66f4\u591a\u4fe1\u606f":40,"\u5173\u4e8epaddlepaddle\u7684\u5206\u5e03\u5f0f\u8bad\u7ec3":52,"\u5173\u4e8epaddlepaddle\u7684\u66f4\u591a\u4f7f\u7528\u65b9\u6cd5\u8bf7\u53c2\u8003":33,"\u5173\u4e8eunbound":38,"\u5173\u4e8evgg\u5377\u79ef\u795e\u7ecf\u7f51\u7edc\u7684\u63cf\u8ff0\u53ef\u4ee5\u53c2\u8003":57,"\u5176\u4e0b\u5b50\u6587\u4ef6\u5939\u7684\u7ed3\u6784\u5982\u4e0b":57,"\u5176\u4e2d":[3,25,27,28,33,39,41,56,57,58],"\u5176\u4e2d156\u548c285\u662f\u8fd9\u4e9b\u56fe\u50cf\u7684\u5206\u7c7b\u6807\u7b7e":58,"\u5176\u4e2d50000\u5f20\u56fe\u7247\u4f5c\u4e3a\u8bad\u7ec3\u96c6":57,"\u5176\u4e2d\u5206\u522b\u5305\u542b\u4e86cifar":57,"\u5176\u4e2d\u5305\u542b6":61,"\u5176\u4e2d\u5305\u542b\u4e86200\u79cd\u9e1f\u7c7b\u7684\u7167\u7247":57,"\u5176\u4e2d\u5305\u542b\u7b97\u6cd5\u548c\u7f51\u7edc\u914d\u7f6e":64,"\u5176\u4e2d\u5305\u62ec\u51fd\u6570":63,"\u5176\u4e2d\u5b9a\u4e49\u4e86\u6a21\u578b\u67b6\u6784\u548csolver\u914d\u7f6e":65,"\u5176\u4e2d\u6587\u672c\u8f93\u5165\u7c7b\u578b\u5b9a\u4e49\u4e3a\u6574\u6570\u65f6\u5e8f\u7c7b\u578binteger_value_sequ":60,"\u5176\u4e2d\u6bcf\u4e00\u884c\u5bf9\u5e94\u4e00\u4e2a\u6570\u636e\u6587\u4ef6\u5730\u5740":2,"\u5176\u4e2d\u6bcf\u4e2a\u5143\u7d20\u662f\u53cc\u5c42\u5e8f\u5217\u4e2d\u6bcf\u4e2asubseq\u6700\u540e\u4e00\u4e2a":35,"\u5176\u4e2d\u6bcf\u4e2a\u5411\u91cf\u5bf9\u5e94\u8f93\u5165\u8bed\u53e5\u4e2d\u7684\u4e00\u4e2a\u5143\u7d20":65,"\u5176\u4e2d\u6bcf\u6761pass\u82b1\u8d39\u4e867\u4e2a\u5c0f\u65f6":65,"\u5176\u4e2d\u6bcf\u884c\u6570\u636e\u4ee3\u8868\u4e00\u5f20\u56fe\u7247":3,"\u5176\u4e2d\u8be6\u7ec6\u8bf4\u660e\u4e86\u6a21\u578b\u67b6\u6784":65,"\u5176\u4e2d\u8f93\u5165\u56fe\u50cf\u7684\u989c\u8272\u901a\u9053\u987a\u5e8f\u4e3a":58,"\u5176\u4e2dbeam":65,"\u5176\u4e2dcheckgrad\u4e3b\u8981\u4e3a\u5f00\u53d1\u8005\u4f7f\u7528":47,"\u5176\u4e2dmean\u548cstd\u662f\u8bad\u7ec3\u914d\u7f6e\u4e2d\u7684\u53c2\u6570":47,"\u5176\u4e2dvalue\u5373\u4e3asoftmax\u5c42\u7684\u8f93\u51fa":5,"\u5176\u4e2dx\u8868\u793a\u8f93\u5165\u6570\u636e\u662f\u4e00\u4e2a\u7ef4\u5ea6\u4e3a2\u7684\u7a20\u5bc6\u5411\u91cf":33,"\u5176\u4ed6":61,"\u5176\u4ed6\u4eba\u5f88\u5bb9\u6613\u770b\u61c2\u955c\u50cf\u751f\u6210\u8fc7\u7a0b":30,"\u5176\u4ed6\u516d\u884c\u5217\u51fa\u4e86\u96c6\u675f\u641c\u7d22\u7684\u7ed3\u679c":65,"\u5176\u4ed6\u5185\u5b58\u6742\u9879":27,"\u5176\u4ed6\u5185\u5b58\u6742\u9879\u662f\u6307paddlepaddle\u672c\u8eab\u6240\u7528\u7684\u4e00\u4e9b\u5185\u5b58":27,"\u5176\u4ed6\u53c2\u6570\u4f7f\u7528":3,"\u5176\u4ed6\u53c2\u6570\u8bf7\u53c2\u8003":60,"\u5176\u4ed6\u6240\u6709\u5c42\u90fd\u4f1a\u4f7f\u7528gpu\u8ba1\u7b97":49,"\u5176\u4ed6\u9ad8\u7ea7\u529f\u80fd\u5305\u62ec\u5b9a\u4e49\u591a\u4e2amemori":39,"\u5176\u4f1a\u81ea\u52a8\u88ab\u52a0\u5165\u7f16\u8bd1\u5217\u8868":41,"\u5176\u4f59\u884c\u662f":56,"\u5176\u4f5c\u7528\u662f\u5c06\u6570\u636e\u4f20\u5165\u5185\u5b58\u6216\u663e\u5b58":2,"\u5176\u5177\u4f53\u8bf4\u660e\u4e86\u5b57\u6bb5\u7c7b\u578b\u548c\u6587\u4ef6\u540d\u79f0":62,"\u5176\u5185\u90e8\u7684\u6587\u4ef6\u4e5f\u4f1a\u968f\u4e4b\u6d88\u5931":50,"\u5176\u5305\u62ec\u4e24\u4e2a\u51fd\u6570":60,"\u5176\u53c2\u6570\u5982\u4e0b":3,"\u5176\u5b83\u90e8\u5206\u548c\u903b\u8f91\u56de\u5f52\u7f51\u7edc\u7ed3\u6784\u4e00\u81f4":60,"\u5176\u5b83layer\u7684\u8f93\u51fa":38,"\u5176\u5b9e\u4e5f\u662f\u548c\u6bcf\u4e2amini":27,"\u5176\u63d0\u4f9b\u5e94\u7528\u90e8\u7f72":50,"\u5176\u6b21":[3,36,60],"\u5176\u76ee\u7684\u662f\u5728\u7ed9\u5b9a\u7684\u8f93\u5165\u53e5\u5b50\u4e2d\u53d1\u73b0\u6bcf\u4e2a\u8c13\u8bcd\u7684\u8c13\u8bcd\u8bba\u5143\u7ed3\u6784":63,"\u5176\u8bf4\u660e\u5982\u4e0b":36,"\u5176\u8f93\u5165\u53c2\u6570\u5982\u4e0b":65,"\u5176\u8f93\u51fa\u88ab\u7528\u4f5cmemory\u7684\u521d\u59cb\u503c":39,"\u5177\u4f53\u4f7f\u7528\u65b9\u6cd5\u4e3a":27,"\u5177\u4f53\u53ef\u4ee5\u53c2\u8003":[3,41],"\u5177\u4f53\u53ef\u53c2\u8003\u6587\u6863":38,"\u5177\u4f53\u5982\u4e0b":30,"\u5177\u4f53\u60c5\u51b5\u56e0\u4eba\u800c\u5f02":44,"\u5177\u4f53\u64cd\u4f5c\u5982\u4e0b":27,"\u5177\u4f53\u6d41\u7a0b\u5982\u4e0b":60,"\u5177\u4f53\u7684\u683c\u5f0f\u8bf4\u660e":3,"\u5177\u4f53\u7684\u89e3\u51b3\u65b9\u6cd5\u662f":27,"\u5177\u4f53\u8bf7\u53c2\u7167\u793a\u4f8b":58,"\u5177\u4f53\u8bf7\u53c2\u8003":[3,40],"\u5177\u4f53\u8bf7\u89c1":40,"\u5177\u6709\u76f8\u540c\u7684\u7ed3\u679c\u4e86":36,"\u5177\u6709\u81ea\u5faa\u73af\u8fde\u63a5\u7684\u795e\u7ecf\u5143":64,"\u517c\u5907\u6613\u7528\u6027":0,"\u5185":39,"\u5185\u5b58":44,"\u5185\u5b58\u5bb9\u9650\u9608\u503c":47,"\u5185\u5bb9":60,"\u5185\u5bb9\u5982\u4e0b":51,"\u5185\u5c42inner_step\u7684recurrent_group\u548c\u5355\u5c42\u5e8f\u5217\u7684\u51e0\u4e4e\u4e00\u6837":36,"\u5185\u5df2\u7ecf\u5305\u542bpaddlepaddle\u7684\u6267\u884c\u7a0b\u5e8f\u4f46\u662f\u8fd8\u6ca1\u4e0a\u8ff0\u529f\u80fd":52,"\u5185\u7684\u5f00\u53d1\u8005\u4e0b\u8f7ddocker\u955c\u50cf":30,"\u5185\u90e8":52,"\u5185\u90e8\u9a71\u52a8python\u89e3\u91ca\u5668\u8fdb\u884c\u6a21\u578b\u914d\u7f6e\u89e3\u6790\u548c\u6570\u636e\u8bfb\u53d6":25,"\u518d\u4e3apaddle\u7684\u8bad\u7ec3\u8fc7\u7a0b\u63d0\u4f9b\u6587\u4ef6\u5217\u8868":62,"\u518d\u5728\u6bcf\u4e00\u4e2aapi\u4e2d\u81ea\u5df1\u68c0\u67e5\u7c7b\u578b":25,"\u518d\u5bf9\u6bcf\u4e00\u4e2a\u5355\u5c42\u65f6\u95f4\u5e8f\u5217\u8fdb\u884c\u5904\u7406":36,"\u518d\u5bf9\u6bcf\u4e00\u53e5\u8bdd\u7684\u7f16\u7801\u5411\u91cf\u7528lstm\u7f16\u7801\u6210\u4e00\u4e2a\u6bb5\u843d\u7684\u5411\u91cf":36,"\u518d\u5bf9\u8fd9\u4e2a\u6bb5\u843d\u5411\u91cf\u8fdb\u884c\u5206\u7c7b":36,"\u518d\u6307\u5b9a":29,"\u518d\u6b21\u5bf9\u4ee3\u7801\u8fdb\u884c\u6027\u80fd\u5206\u6790":44,"\u518d\u7528\u8fd9\u4e2a\u68af\u5ea6\u53bb\u548c":41,"\u518d\u901a\u8fc7\u51fd\u6570":52,"\u5192\u9669\u7247":61,"\u5197\u4f59\u7b49\u529f\u80fd":50,"\u5199\u4ee3\u7801":25,"\u5199\u5b8c\u6a21\u578b\u914d\u7f6e\u4e4b\u540e":65,"\u5199\u68af\u5ea6\u68c0\u67e5\u5355\u5143\u6d4b\u8bd5\u662f\u4e00\u4e2a\u9a8c\u8bc1\u65b0\u5b9e\u73b0\u7684\u5c42\u662f\u5426\u6b63\u786e\u7684\u76f8\u5bf9\u7b80\u5355\u7684\u529e\u6cd5":41,"\u519c\u6c11":61,"\u51c6\u5907":36,"\u51c6\u5907\u597d\u6570\u636e":62,"\u51c6\u5907\u7528\u6765\u5b66\u4e60\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u5e8f\u5217\u6570\u636e":39,"\u51c6\u5907\u9884\u6d4b\u6570\u636e":5,"\u51cf\u5c0f\u5e8f\u5217\u7684\u957f\u5ea6":27,"\u51cf\u5c0f\u8fd9\u4e2a\u5185\u5b58\u6c60\u5373\u53ef\u51cf\u5c0f\u5185\u5b58\u5360\u7528":27,"\u51cf\u5c0fbatch":27,"\u51fa\u53bb\u73a9":36,"\u51fa\u5dee":36,"\u51fa\u6765":36,"\u51fa\u73b0\u4ee5\u4e0b\u9519\u8bef":27,"\u51fa\u73b0\u8fd9\u4e2a\u95ee\u9898\u7684\u4e3b\u8981\u539f\u56e0\u662f":27,"\u51fd\u6570":[3,28,39,41,44,63,64],"\u51fd\u6570\u4e2d":39,"\u51fd\u6570\u4e2d\u4f7f\u7528":3,"\u51fd\u6570\u4e2d\u8bbe\u7f6e\u7684":45,"\u51fd\u6570\u5047\u8bbe":39,"\u51fd\u6570\u52a0\u5230\u4ee3\u7801\u4e2d":44,"\u51fd\u6570\u53ea\u5173\u6ce8\u4e8ernn\u4e00\u4e2a\u65f6\u95f4\u6b65\u4e4b\u5185\u7684\u8ba1\u7b97":38,"\u51fd\u6570\u547d\u540d":25,"\u51fd\u6570\u5c06\u8fd4\u56de\u4e09\u4e2a\u6574\u6570\u5217\u8868":39,"\u51fd\u6570\u5c31\u662f\u6839\u636e\u8be5\u673a\u5236\u914d\u7f6e\u7684":3,"\u51fd\u6570\u5f97\u5230\u7684\u68af\u5ea6\u53bb\u5bf9\u6bd4":41,"\u51fd\u6570\u5fc5\u987b\u5148\u8c03\u7528\u57fa\u7c7b\u4e2d\u7684\u51fd\u6570":41,"\u51fd\u6570\u5fc5\u987b\u8fd4\u56de\u4e00\u4e2a\u6216\u591a\u4e2alayer\u7684\u8f93\u51fa":38,"\u51fd\u6570\u6307\u51fa\u4e86\u5728\u8bad\u7ec3\u65f6\u9700\u8981\u4ece\u53c2\u6570\u670d\u52a1\u5668\u53d6\u51fa\u7684\u884c":41,"\u51fd\u6570\u6765\u5c06\u4fe1\u606f\u8f93\u51fa\u5230\u754c\u9762\u4e2d":44,"\u51fd\u6570\u67e5\u8be2\u8f6f\u4ef6\u5305\u76f8\u5173api\u8bf4\u660e":5,"\u51fd\u6570\u7684":3,"\u51fd\u6570\u7684\u5b9e\u73b0\u662f\u6b63\u786e\u7684":41,"\u51fd\u6570\u7684\u5f00\u5934\u5fc5\u987b\u8c03\u7528":41,"\u5206\u4e3a\u597d\u8bc4":60,"\u5206\u522b\u4e3a":56,"\u5206\u522b\u4e3atrain":65,"\u5206\u522b\u4ece\u8bcd\u8bed\u548c\u53e5\u5b50\u7ea7\u522b\u7f16\u7801\u8f93\u5165\u6570\u636e":38,"\u5206\u522b\u4f7f\u7528\u5355\u53cc\u5c42rnn\u4f5c\u4e3a\u7f51\u7edc\u914d\u7f6e\u7684\u6a21\u578b":36,"\u5206\u522b\u5305\u542b\u4e86\u6cd5\u8bed\u5230\u82f1\u8bed\u7684\u5e73\u884c\u8bed\u6599\u5e93\u7684\u8bad\u7ec3\u6570\u636e":65,"\u5206\u522b\u5b9a\u4e49\u5b50\u53e5\u7ea7\u522b\u548c\u8bcd\u8bed\u7ea7\u522b\u4e0a\u9700\u8981\u5b8c\u6210\u7684\u8fd0\u7b97":38,"\u5206\u522b\u5bf9\u5e94\u4e8e\u53d8\u91cf":28,"\u5206\u522b\u662f":35,"\u5206\u522b\u662frnn\u72b6\u6001\u548c\u8f93\u5165\u7684\u53d8\u6362\u77e9\u9635":39,"\u5206\u522b\u662fsentences\u548clabel":36,"\u5206\u522b\u662fwords\u548clabel":36,"\u5206\u522b\u8ba1\u7b97\u6bcf\u4e2a\u53c2\u6570\u7684\u68af\u5ea6":41,"\u5206\u522b\u8fdb\u884c\u5e8f\u5217\u64cd\u4f5c":36,"\u5206\u5272":[61,63],"\u5206\u5272\u6587\u4ef6\u7684\u65b9\u6cd5\u662f":62,"\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf":50,"\u5206\u6210\u4e24\u90e8\u5206":3,"\u5206\u652f":40,"\u5206\u652f\u4e0a":40,"\u5206\u652f\u4e0a\u521b\u5efa\u65b0\u5206\u652f":40,"\u5206\u652f\u540d":40,"\u5206\u652f\u89c4\u8303":40,"\u5206\u6790\u5f97\u5230\u7684\u4fe1\u606f\u7528\u4e8e\u534f\u52a9\u8fdb\u884c\u7a0b\u5e8f\u7684\u4f18\u5316":44,"\u5206\u7c7b\u6210\u6b63\u9762\u60c5\u7eea\u548c\u8d1f\u9762\u60c5\u7eea\u4e24\u7c7b":3,"\u5206\u7c7b\u8bef\u5dee\u662f0":64,"\u5206\u7c7b\u9519\u8bef\u7387\u548c\u6a21\u578b\u5927\u5c0f\u7531\u4e0b\u8868\u7ed9\u51fa":58,"\u5206\u8bcd\u5e8f\u5217\u7684\u5f00\u59cb":56,"\u5206\u8bcd\u5e8f\u5217\u7684\u7ed3\u675f":56,"\u5206\u8bcd\u98ce\u683c\u5982\u4e0b":56,"\u5206\u914d\u5230\u5f53\u524d\u6570\u636e\u5757\u6837\u672c\u6570\u7684\u56db\u5206\u4e4b\u4e00":47,"\u5206\u9694":[56,62],"\u5206\u9694\u7b26\u4e3a":61,"\u5207\u6362\u5230":40,"\u5207\u6362\u5230\u6240\u5efa\u5206\u652f":40,"\u5217\u8868":62,"\u5217\u8868\u5982\u4e0b":[3,33],"\u5219\u4e0d\u5728\u4e4e\u5185\u5b58\u6682\u5b58\u591a\u5c11\u6761\u6570\u636e":3,"\u5219\u4e0d\u9700\u8981\u91cd\u5199\u8be5\u51fd\u6570":41,"\u5219\u4f1a\u9884\u5148\u8bfb\u53d6\u5168\u90e8\u6570\u636e\u5230\u5185\u5b58\u4e2d":3,"\u5219\u4f7f\u7528\u533a\u57df\u6807\u8bb0":63,"\u5219\u4f7f\u7528\u540c\u6b65\u8bad\u7ec3":47,"\u5219\u4f7f\u7528\u8be5\u53c2\u6570\u4f5c\u4e3a\u9ed8\u8ba4\u503c":47,"\u5219\u5148\u505a\u5d4c\u5165":62,"\u5219\u53ef\u4ee5\u50cf":45,"\u5219\u5b57\u4e0e\u5b57\u4e4b\u95f4\u7528\u7a7a\u683c\u5206\u9694":60,"\u5219\u603b\u4f1a\u663e\u793a\u963b\u9694\u6458\u8981\u4fe1\u606f":47,"\u5219\u63a8\u8350\u5927\u4e8e\u8bad\u7ec3\u65f6batch":3,"\u5219\u662f\u5e26gui\u7684nvidia\u53ef\u89c6\u5316\u6027\u80fd\u5206\u6790\u5de5\u5177":44,"\u5219\u663e\u793a\u963b\u9694\u6027\u80fd\u7684\u6458\u8981\u4fe1\u606f":47,"\u5219\u9700\u8981\u4f7f\u7528\u7b49\u4e8e\u6743\u91cd\u53c2\u6570\u89c4\u6a21\u5927\u7ea65\u500d\u7684\u5185\u5b58":27,"\u5219\u9700\u8981\u914d\u7f6e":50,"\u521b\u5efa\u4e00\u4e2akubernet":52,"\u521b\u5efa\u548c\u53d1\u5e03\u81ea\u5df1\u7684ai\u7a0b\u5e8f\u955c\u50cf":30,"\u521b\u5efa\u5e76\u5207\u6362\u5230\u65b0\u5206\u652f":40,"\u521b\u5efa\u6210\u529f\u540e":52,"\u521b\u5efa\u8bad\u7ec3\u6570\u636e\u7684":65,"\u521b\u5efagener":3,"\u521d\u59cb\u5316\u4e4b\u540e":5,"\u521d\u59cb\u5316\u504f\u7f6e\u5411\u91cf":41,"\u521d\u59cb\u5316\u65f6\u8c03\u7528\u7684\u51fd\u6570":3,"\u521d\u59cb\u5316\u6743\u91cd\u8868":41,"\u521d\u59cb\u5316\u6a21\u578b\u7684\u8def\u5f84":47,"\u521d\u59cb\u5316\u6a21\u578b\u7684\u8def\u5f84\u914d\u7f6e\u4e3a":56,"\u521d\u59cb\u5316\u7236\u7c7b":41,"\u521d\u59cb\u5316biases_":41,"\u521d\u59cb\u5316paddlepaddle\u73af\u5883":5,"\u521d\u59cb\u72b6\u6001":38,"\u5220\u9664":40,"\u5229\u7528\u5206\u5e03\u5f0f\u8bad\u7ec3\u9a7e\u9a6d\u66f4\u591a\u7684\u8ba1\u7b97\u8d44\u6e90":27,"\u5229\u7528\u5355\u8bcdid\u67e5\u627e\u8be5\u5355\u8bcd\u5bf9\u5e94\u7684\u8fde\u7eed\u5411\u91cf":60,"\u5229\u7528\u66f4\u591a\u7684\u8ba1\u7b97\u8d44\u6e90\u53ef\u4ee5\u5206\u4e3a\u4e00\u4e0b\u51e0\u4e2a\u65b9\u5f0f\u6765\u8fdb\u884c":27,"\u5229\u7528\u6b63\u5219\u8868\u8fbe\u5f0f\u6765\u89e3\u6790\u8be5\u7279\u5f81":62,"\u5229\u7528\u8bad\u7ec3\u96c6\u751f\u6210\u7684\u5b57\u5178":64,"\u5229\u7528\u8fd9\u79cd\u7279\u6027":38,"\u5229\u7528\u903b\u8f91\u56de\u5f52\u6a21\u578b\u5bf9\u8be5\u5411\u91cf\u8fdb\u884c\u5206\u7c7b":60,"\u5229\u7528kubernetes\u80fd\u65b9\u4fbf\u5730\u7ba1\u7406\u8de8\u673a\u5668\u8fd0\u884c\u5bb9\u5668\u5316\u7684\u5e94\u7528":50,"\u5229\u843d":36,"\u5230":[27,39],"\u5230\u6240\u6709\u8282\u70b9\u800c\u4e0d\u7528\u5bc6\u7801":45,"\u5230\u672c\u5730":40,"\u5230\u6b64":30,"\u5230\u76ee\u524d\u4e3a\u6b62":63,"\u5236\u4f5c\u65b0\u955c\u50cf\u6765\u5b8c\u6210\u4ee5\u4e0a\u7684\u5de5\u4f5c":52,"\u5236\u4f5cpaddlepaddle\u5f00\u53d1\u955c\u50cf":30,"\u5236\u4f5cpaddlepaddle\u751f\u4ea7\u955c\u50cf":30,"\u5236\u4f5cpaddlepaddle\u955c\u50cf":52,"\u5237\u7259":36,"\u524d\u4e00\u7bc7\u6587\u7ae0\u4ecb\u7ecd\u4e86\u5982\u4f55\u5728kubernetes\u96c6\u7fa4\u4e0a\u542f\u52a8\u4e00\u4e2a\u5355\u673apaddlepaddle\u8bad\u7ec3\u4f5c\u4e1a":52,"\u524d\u4e09\u884cimport\u4e86\u5b9a\u4e49network":65,"\u524d\u53f0":36,"\u524d\u5411\u4f20\u64ad":41,"\u524d\u5411\u4f20\u64ad\u7ed9\u5b9a\u8f93\u5165":41,"\u524d\u5411\u548c\u540e\u5411":41,"\u524d\u81ea\u52a8\u68c0\u67e5\u4e00\u4e9b\u57fa\u672c\u4e8b\u5b9c":40,"\u5269\u4e0b\u7684pass\u4f1a\u76f4\u63a5\u4ece\u5185\u5b58\u91cc":3,"\u52a0\u4e0a\u504f\u7f6e\u5411\u91cf":41,"\u52a0\u4e86l2\u6b63\u5219\u548c\u68af\u5ea6\u622a\u65ad":60,"\u52a0\u5165":44,"\u52a0\u6743\u548c\u7528\u6765\u751f\u6210":39,"\u52a0\u6743\u7f16\u7801\u5411\u91cf":39,"\u52a0\u8f7d\u6570\u636e":63,"\u52a0\u8f7d\u6a21\u578b":63,"\u52a0\u8f7d\u6a21\u578b\u53c2\u6570":65,"\u52a0\u8f7dtest":47,"\u52a0\u901fpaddlepaddle\u8bad\u7ec3\u53ef\u4ee5\u8003\u8651\u4ece\u4ee5\u4e0b\u51e0\u4e2a\u65b9\u9762":27,"\u52a8\u4f5c\u7247":61,"\u52a8\u6001\u5e93":25,"\u52a8\u753b\u7247":61,"\u52a8\u8bcd":63,"\u52a9\u624b":41,"\u5305\u542b12":64,"\u5305\u542b20\u4e2a\u8bad\u7ec3\u6837\u4f8b":56,"\u5305\u542b3\u4e2a\u5c5e\u6027":56,"\u5305\u542b50":64,"\u5305\u542b\u7684\u6240\u6709\u4f9d\u8d56\u5047\u8bbe\u90fd\u53ef\u4ee5\u5728paddle\u7684\u8fd0\u884c\u5bb9\u5668\u4e2d":30,"\u5305\u5e76\u91cd\u65b0\u7f16\u8bd1paddlepaddl":27,"\u5305\u62ec":[47,60,63,65],"\u5305\u62ec\u4ee5\u4e0b\u4e24\u79cd":3,"\u5305\u62ec\u53d1\u884c\u65f6\u95f4":61,"\u5305\u62ec\u5b57\u7b26\u4e32\u5206\u914d":27,"\u5305\u62ec\u6570\u636e\u8f93\u5165":28,"\u5305\u62ec\u751f\u6210cpu":29,"\u5305\u62ec\u795e\u7ecf\u7f51\u7edc\u62d3\u6251\u7ed3\u6784":33,"\u5305\u62ec\u7b80\u5355\u7684":60,"\u5305\u62ecbool":49,"\u5305\u62ecdocker\u955c\u50cf":31,"\u5305\u62eclinux":30,"\u5305\u662f\u6700\u65b0\u7684":27,"\u5305\u6bd4\u8f83\u8001":27,"\u5305\u7684\u65b9\u6cd5\u662f":27,"\u533a\u522b\u662f\u540c\u65f6\u5904\u7406\u4e86\u4e24\u4e2a\u8f93\u5165":36,"\u533a\u522b\u662frnn\u4f7f\u7528\u4e24\u5c42\u5e8f\u5217\u6a21\u578b":36,"\u533b\u751f":61,"\u533b\u7597\u4fdd\u5065":61,"\u5341\u4e00":36,"\u5347\u5e8f\u6392\u5217":65,"\u534e\u6da6\u4e07\u5bb6":36,"\u5355\u4f4d\u662fmb":47,"\u5355\u5143\u6d4b\u8bd5\u4f1a\u5f15\u7528site":27,"\u5355\u5143\u6d4b\u8bd5checkgrad_ep":46,"\u5355\u53cc\u5c42\u5e8f\u5217\u7684\u53e5\u5b50\u662f\u4e00\u6837\u7684":36,"\u5355\u53cc\u5c42rnn":37,"\u5355\u53d8\u91cf\u7684\u7ebf\u6027\u56de\u5f52":28,"\u5355\u5c42":38,"\u5355\u5c42\u4e0d\u7b49\u957frnn":36,"\u5355\u5c42\u548c\u53cc\u5c42\u5e8f\u5217\u7684\u4f7f\u7528\u548c\u793a\u4f8b2\u4e2d\u7684\u793a\u4f8b\u7c7b\u4f3c":36,"\u5355\u5c42\u5e8f\u5217":35,"\u5355\u5c42\u5e8f\u5217\u7684\u6bcf\u4e2a\u5143\u7d20":35,"\u5355\u5c42\u5e8f\u5217\u7b2ci\u4e2a\u5143\u7d20":35,"\u5355\u5c42\u6216\u53cc\u5c42":35,"\u5355\u5c42\u65f6\u95f4\u5e8f\u5217":36,"\u5355\u5c42rnn":[36,38],"\u5355\u5c42rnn\u548c\u53cc\u5c42rnn\u7684\u7f51\u7edc\u914d\u7f6e":36,"\u5355\u673acpu\u8bad\u7ec3":27,"\u5355\u673agpu\u8bad\u7ec3":27,"\u5355\u6b65\u51fd\u6570":39,"\u5355\u6b65\u51fd\u6570\u548c\u8f93\u51fa\u51fd\u6570\u5728":39,"\u5355\u6b65\u51fd\u6570\u548c\u8f93\u51fa\u51fd\u6570\u90fd\u975e\u5e38\u7b80\u5355":39,"\u5355\u6b65\u51fd\u6570\u7684\u5b9e\u73b0\u5982\u4e0b\u6240\u793a":39,"\u5355\u8fdb\u5355\u51fa":38,"\u5360\u4f4d\u7b26":56,"\u536b\u751f":36,"\u5373":[27,28,42,52,60,64],"\u5373\u4e00\u4e2a\u5c06\u5355\u8bcd\u5b57\u7b26\u4e32\u6620\u5c04\u5230\u5355\u8bcdid\u7684\u5b57\u5178":3,"\u5373\u4e0a\u8ff0\u4ee3\u7801\u4e2d\u7684\u7b2c19\u884c":36,"\u5373\u4e0d\u8981\u5c06\u6bcf\u4e00\u4e2a\u6837\u672c\u90fd\u653e\u5165train":3,"\u5373\u4e0d\u9700\u8981\u4f7f\u7528memori":36,"\u5373\u4e3a\u4e00\u4e2a\u65f6\u95f4\u6b65":36,"\u5373\u4e3a\u5355\u5c42rnn\u5e8f\u5217\u7684\u4f7f\u7528\u4ee3\u7801":36,"\u5373\u4e3a\u65f6\u95f4\u5e8f\u5217\u7684\u8f93\u5165":36,"\u5373\u4e3a\u8fd9\u4e2a\u53cc\u5c42rnn\u7684\u7f51\u7edc\u7ed3\u6784":36,"\u5373\u4e3a\u8fd9\u4e2a\u6570\u636e\u6587\u4ef6\u7684\u540d\u5b57":3,"\u5373\u4e8c\u7ef4\u6570\u7ec4":36,"\u5373\u4f7f\u95f4\u9694\u5f88\u5c0f":47,"\u5373\u4f7fprocess\u51fd\u6570\u91cc\u9762\u53ea\u6709\u4e00\u4e2ayield":3,"\u5373\u4fbf\u8bbe\u7f6e":27,"\u5373\u521d\u59cb\u72b6\u6001\u4e3a0":38,"\u5373\u5305\u542b\u65f6\u95f4\u6b65\u4fe1\u606f":3,"\u5373\u5355\u65f6\u95f4\u6b65\u6267\u884c\u7684\u51fd\u6570":39,"\u5373\u53cc\u5411lstm\u548c\u4e09\u5c42\u5806\u53e0lstm":64,"\u5373\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217":36,"\u5373\u53cc\u5c42rnn\u7684\u6bcf\u4e2a\u72b6\u6001":38,"\u5373\u53ef":28,"\u5373\u53ef\u4ee5\u6781\u5927\u7684\u52a0\u901f\u6570\u636e\u8f7d\u5165\u6d41\u7a0b":27,"\u5373\u5728\u53cc\u5c42\u5e8f\u5217\u7684\u539f\u59cb\u6570\u636e\u4e2d":36,"\u5373\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d":27,"\u5373\u5927\u90e8\u5206\u503c\u4e3a0":[3,33],"\u5373\u5bf9\u7b2c\u4e09\u6b65\u8fdb\u884c\u66ff\u6362":60,"\u5373\u5c06\u4e00\u6bb5\u82f1\u6587\u6587\u672c\u6570\u636e":3,"\u5373\u5c06\u4e00\u6bb5\u8bdd\u8fdb\u884c\u5206\u7c7b":36,"\u5373\u5f53\u524d\u65f6\u95f4\u6b65\u4e0b\u7684\u795e\u7ecf\u7f51\u7edc\u4f9d\u8d56\u524d\u4e00\u4e2a\u65f6\u95f4\u6b65\u795e\u7ecf\u7f51\u7edc\u4e2d\u67d0\u4e00\u4e2a\u795e\u7ecf\u5143\u8f93\u51fa":36,"\u5373\u6211\u4eec\u7684\u8bad\u7ec3\u76ee\u6807":28,"\u5373\u628a\u5355\u5c42rnn\u751f\u6210\u540e\u7684subseq\u7ed9\u62fc\u63a5\u6210\u4e00\u4e2a\u65b0\u7684\u53cc\u5c42seq":38,"\u5373\u6574\u4e2a\u53cc\u5c42group\u662f\u5c06\u524d\u4e00\u4e2a\u5b50\u53e5\u7684\u6700\u540e\u4e00\u4e2a\u5411\u91cf":36,"\u5373\u6574\u4e2a\u8f93\u5165\u5e8f\u5217":35,"\u5373\u6574\u6570\u6570\u7ec4":36,"\u5373\u65f6\u95f4\u9012\u5f52\u795e\u7ecf\u7f51\u7edc":36,"\u5373\u662f\u8de8\u8d8a\u65f6\u95f4\u6b65\u7684\u7f51\u7edc\u8fde\u63a5":36,"\u5373\u6b63\u9762\u548c\u8d1f\u9762":64,"\u5373\u6b63\u9762\u8bc4\u4ef7\u6807\u7b7e\u548c\u8d1f\u9762\u8bc4\u4ef7\u6807\u7b7e":64,"\u5373\u7279\u5f81\u7684\u6570\u7ec4":36,"\u5373\u7f51\u5361\u540d":52,"\u5373\u82e5\u5e72\u6570\u636e\u6587\u4ef6\u8def\u5f84\u7684\u67d0\u4e00\u4e2a":3,"\u5373\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u51fa\u73b0nan\u6216\u8005inf":27,"\u5373\u8bbe\u7f6e":27,"\u5373\u8fd9\u4e2a\u52a8\u6001\u5e93\u662f\u4e0d\u4f9d\u8d56\u4e8e\u5176\u4ed6\u4efb\u4f55\u6587\u4ef6\u7684":25,"\u5373define_py_data_sources2\u5e94\u6539\u4e3a":27,"\u5373input":38,"\u5373rnn\u4e4b\u95f4\u6709\u4e00\u6b21\u5d4c\u5957\u5173\u7cfb":36,"\u5377\u79ef\u5c42":57,"\u5377\u79ef\u5c42\u6743\u91cd":58,"\u5377\u79ef\u795e\u7ecf\u7f51\u7edc\u53ef\u4ee5\u8fa8\u8bc6\u56fe\u7247\u4e2d\u7684\u4e3b\u4f53":57,"\u5377\u79ef\u795e\u7ecf\u7f51\u7edc\u5728\u56fe\u7247\u5206\u7c7b\u4e0a\u6709\u7740\u60ca\u4eba\u7684\u6027\u80fd":57,"\u5377\u79ef\u795e\u7ecf\u7f51\u7edc\u662f\u4e00\u79cd\u4f7f\u7528\u5377\u79ef\u5c42\u7684\u524d\u5411\u795e\u7ecf\u7f51\u7edc":57,"\u5377\u79ef\u795e\u7ecf\u7f51\u7edc\u80fd\u591f\u5f88\u597d\u7684\u8868\u793a\u8fd9\u4e24\u7c7b\u4fe1\u606f":57,"\u5377\u79ef\u7f51\u7edc\u662f\u4e00\u79cd\u7279\u6b8a\u7684\u4ece\u8bcd\u5411\u91cf\u8868\u793a\u5230\u53e5\u5b50\u8868\u793a\u7684\u65b9\u6cd5":60,"\u5378\u8f7dpaddlepaddle\u5305":27,"\u538b\u7f29\u6210\u4e00\u4e2a\u5411\u91cf":36,"\u539f\u56e0\u5728\u4e8e\u6ca1\u6709\u628a\u673a\u5668\u4e0acuda\u76f8\u5173\u7684\u9a71\u52a8\u548c\u5e93\u6620\u5c04\u5230\u5bb9\u5668\u5185\u90e8":27,"\u539f\u56e0\u662f\u672a\u8bbe\u7f6ecuda\u8fd0\u884c\u65f6\u73af\u5883\u53d8\u91cf":32,"\u53bb\u8fc7":36,"\u53c2\u6570":[3,7,8,9,10,11,12,15,16,17,18,19,20,22,25,41,46,52,56,58,64],"\u53c2\u6570\u5171\u4eab\u7684\u914d\u7f6e\u793a\u4f8b\u4e3a":27,"\u53c2\u6570\u521d\u59cb\u5316\u8def\u5f84":63,"\u53c2\u6570\u5373\u53ef":64,"\u53c2\u6570\u540d":58,"\u53c2\u6570\u6570\u91cf":60,"\u53c2\u6570\u670d\u52a1\u5668":46,"\u53c2\u6570\u670d\u52a1\u5668\u7684\u53c2\u6570\u5206\u5757\u5927\u5c0f":47,"\u53c2\u6570\u670d\u52a1\u5668\u7684\u76d1\u542c\u7aef\u53e3":47,"\u53c2\u6570\u670d\u52a1\u5668\u7684\u7f51\u7edc\u8bbe\u5907\u540d\u79f0":47,"\u53c2\u6570\u670d\u52a1\u5668\u7684ip\u5730\u5740":47,"\u53c2\u6570\u670d\u52a1\u5668\u7a00\u758f\u66f4\u65b0\u7684\u53c2\u6570\u5206\u5757\u5927\u5c0f":47,"\u53c2\u6570\u6765\u63a7\u5236\u7f13\u5b58\u65b9\u6cd5":27,"\u53c2\u6570\u6982\u8ff0":48,"\u53c2\u6570\u7684\u89e3\u6790":52,"\u53c2\u6570\u7ef4\u5ea6":56,"\u53c2\u6570\u884c":56,"\u53c2\u6570\u8bbe\u7f6e\u4e86\u5916\u5c42":36,"\u53c2\u6570\u9700\u8981\u5b9e\u73b0":39,"\u53c2\u7167\u4e0a\u8ff0\u6b65\u9aa4\u66f4\u65b0":40,"\u53c2\u8003":50,"\u53c2\u8003\u5f3a\u8c03\u90e8\u5206":44,"\u53c2\u8003\u6587\u732e":65,"\u53c2\u8003\u65f6\u95f4\u5e8f\u5217":36,"\u53c2\u8003\u955c\u50cf\u7684":52,"\u53c8":36,"\u53c8\u662f\u4e00\u4e2a\u5355\u5c42\u7684\u5e8f\u5217":35,"\u53c8\u8981\u4fdd\u8bc1\u6570\u636e\u662f\u968f\u673a\u7684":27,"\u53ca":41,"\u53cc\u5411\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u9690\u85cf\u72b6\u6001":39,"\u53cc\u5c42":38,"\u53cc\u5c42\u4e0d\u7b49\u957frnn":36,"\u53cc\u5c42\u5e8f\u5217":35,"\u53cc\u5c42\u5e8f\u5217\u6216\u5355\u5c42\u5e8f\u5217":35,"\u53cc\u5c42\u5e8f\u5217\u6570\u636e\u4e00\u5171\u67094\u4e2a\u6837\u672c":36,"\u53cc\u5c42\u5e8f\u5217\u662f\u4e00\u4e2a\u5d4c\u5957\u7684\u5e8f\u5217":35,"\u53cc\u5c42\u5e8f\u5217\u662fpaddlepaddle\u652f\u6301\u7684\u4e00\u79cd\u975e\u5e38\u7075\u6d3b\u7684\u6570\u636e\u7ec4\u7ec7\u65b9\u5f0f":38,"\u53cc\u5c42\u5e8f\u5217\u6bcf\u4e2asubseq\u4e2d\u6bcf\u4e2a\u5143\u7d20":35,"\u53cc\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u53d8\u6210\u4e00\u4e2a0\u5c42\u5e8f\u5217":35,"\u53cc\u5c42\u6216\u8005\u5355\u5c42":35,"\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217\u7684dataprovider\u7684\u4ee3\u7801":36,"\u53cc\u5c42rnn":38,"\u53cc\u5c42rnn\u6570\u636e\u968f\u610f\u52a0\u4e86\u4e00\u4e9b\u9694\u65ad":36,"\u53cc\u5c42rnn\u987e\u540d\u601d\u4e49":36,"\u53cc\u8fdb\u5355\u51fa":38,"\u53cc\u8fdb\u53cc\u51fa":38,"\u53cd\u4e4b\u5219":63,"\u53cd\u5411\u4f20\u64ad":41,"\u53cd\u5411\u4f20\u64ad\u6839\u636e\u8f93\u51fa\u7684\u68af\u5ea6":41,"\u53d1\u5e03":30,"\u53d1\u6563\u5230\u4e86\u4e00\u4e2a\u6570\u503c\u7279\u522b\u5927\u7684\u5730\u65b9":27,"\u53d1\u884c\u548c\u7ef4\u62a4":40,"\u53d1\u9001\u53c2\u6570\u7684\u7aef\u53e3\u53f7":47,"\u53d6\u51b3\u4e8e\u662f\u5426\u5bfb\u627e\u5230cuda\u5de5\u5177\u94fe":29,"\u53d6\u51b3\u4e8e\u662f\u5426\u5bfb\u627e\u5230gtest":29,"\u53d6\u51b3\u4e8e\u662f\u5426\u5bfb\u627e\u5230swig":29,"\u53d8\u6362\u77e9\u9635":41,"\u53d8\u91cf\u6765\u8bbe\u7f6e\u5185\u5b58\u4e2d\u6682\u5b58\u7684\u6570\u636e\u6761":3,"\u53e3\u5934":36,"\u53e3\u7edf\u8ba1\u5b66\u4fe1\u606f\u7684\u7528\u6237\u624d\u88ab\u5305\u542b\u5728\u6570\u636e\u96c6\u4e2d":61,"\u53e5\u5b50":64,"\u53e5\u5b50\u4e2d\u7684\u7ec4\u5757\u5c06\u4f1a\u626e\u6f14\u67d0\u4e9b\u8bed\u4e49\u89d2\u8272":63,"\u53e5\u5b50\u8868\u793a\u7684\u8ba1\u7b97\u66f4\u65b0\u4e3a\u4e24\u6b65":60,"\u53e6\u4e00\u4e2a\u4f8b\u5b50\u662f\u901a\u8fc7\u5206\u6790\u6bcf\u65e5twitter\u535a\u5ba2\u7684\u6587\u672c\u5185\u5bb9\u6765\u9884\u6d4b\u80a1\u7968\u53d8\u52a8":64,"\u53e6\u4e00\u4e2a\u597d\u5904\u662f\u6211\u4eec\u53ef\u4ee5\u628apaddlepaddle\u5bb9\u5668\u8fd0\u884c\u5728\u8fdc\u7a0b\u670d\u52a1\u5668\u4e0a":30,"\u53e6\u4e00\u4e2a\u662f\u5185\u5b58\u64cd\u4f5c\u91cf":44,"\u53e6\u4e00\u4e2a\u662f\u521b\u5efadockerfile\u5e76\u8fd0\u884cdock":30,"\u53e6\u4e00\u4e2a\u662f\u6bcf\u6761\u5e8f\u5217":27,"\u53e6\u4e00\u4e2a\u7ec8\u7aef\u8fd0\u884cpython":30,"\u53e6\u4e00\u65b9\u9762":64,"\u53e6\u4e00\u79cd\u65b9\u5f0f\u662f\u5c06\u7f51\u7edc\u5c42\u5212\u5206\u5230\u4e0d\u540c\u7684gpu\u4e0a\u53bb\u8ba1\u7b97":49,"\u53e6\u5916":36,"\u53e6\u5916\u4e24\u4e2a\u5206\u522b\u662f\u6ed1\u52a8\u5747\u503c\u548c\u65b9\u5dee":58,"\u53ea\u4f5c\u4e3aread":38,"\u53ea\u4fdd\u5b58\u6700\u540e\u4e00\u8f6e\u7684\u53c2\u6570":47,"\u53ea\u5141\u8bb8\u6574\u6570\u7684\u661f\u7ea7":61,"\u53ea\u5728\u7b2c\u4e00\u6b21cmake\u7684\u65f6\u5019\u6709\u6548":29,"\u53ea\u622a\u53d6\u4e2d\u5fc3\u65b9\u5f62\u7684\u56fe\u50cf\u533a\u57df":58,"\u53ea\u662f\u53cc\u5c42\u5e8f\u5217\u5c06\u5176\u53c8\u505a\u4e86\u5b50\u5e8f\u5217\u5212\u5206":36,"\u53ea\u662f\u5c06\u53e5\u5b50\u7528\u8fde\u7eed\u5411\u91cf\u8868\u793a\u66ff\u6362\u4e3a\u7528\u7a00\u758f\u5411\u91cf\u8868\u793a":60,"\u53ea\u662f\u8bf4\u660e\u6570\u636e\u7684\u987a\u5e8f\u662f\u91cd\u8981\u7684":3,"\u53ea\u662f\u8bf7\u4e0d\u8981\u5fd8\u8bb0\u63d0\u524d\u5728\u7269\u7406\u673a\u4e0a\u5b89\u88c5gpu\u6700\u65b0\u9a71\u52a8":30,"\u53ea\u6709":36,"\u53ea\u67092\u4e2a\u914d\u7f6e\u4e0d\u4e00\u6837":56,"\u53ea\u6709\u542b\u6709\u4eba":61,"\u53ea\u6709\u5f53\u8bbe\u7f6e\u4e86spars":47,"\u53ea\u7528\u4e8e\u5728\u5e8f\u5217\u751f\u6210\u4efb\u52a1\u4e2d\u6307\u5b9a\u8f93\u5165\u6570\u636e":38,"\u53ea\u80fd\u6d4b\u8bd5\u5355\u4e2a\u6a21\u578b":49,"\u53ea\u80fd\u8c03\u7528paddle\u7684\u52a8\u6001\u5e93":25,"\u53ea\u8981\u4e00\u7cfb\u5217\u7279\u5f81\u6570\u636e\u4e2d\u7684":36,"\u53ea\u8981\u51fa\u73b0\u6d6e\u70b9\u6570\u5f02\u5e38":27,"\u53ea\u8981\u5728docker\u91cc\u542f\u52a8paddlepaddle\u7684\u65f6\u5019\u7ed9\u5b83\u4e00\u4e2a\u540d\u5b57":30,"\u53ea\u8bfbmemory\u8f93\u5165":38,"\u53ea\u9488\u5bf9\u5185\u5b58":27,"\u53ea\u9700\u4e2d\u65ad":45,"\u53ea\u9700\u4f7f\u7528":45,"\u53ea\u9700\u5220\u9664\u6700\u540e\u4e00\u884c\u4e2d\u7684\u6ce8\u91ca\u5e76\u628a":64,"\u53ea\u9700\u5728linux\u4e0b\u8fd0\u884c\u5982\u4e0b\u547d\u4ee4":65,"\u53ea\u9700\u7528\u4f60\u5b9a\u4e49\u7684\u76ee\u5f55\u4fee\u6539":45,"\u53ea\u9700\u77e5\u9053\u8fd9\u662f\u4e00\u4e2a\u6807\u8bb0\u5c5e\u6027\u7684\u65b9\u6cd5\u5c31\u53ef\u4ee5\u4e86":3,"\u53ea\u9700\u8981":39,"\u53ea\u9700\u8981\u4e00\u884c\u4ee3\u7801\u5c31\u53ef\u4ee5\u8c03\u7528\u8fd9\u4e2apydataprovider2":3,"\u53ea\u9700\u8981\u5728\u51fd\u6570\u4e2d\u8c03\u7528\u591a\u6b21yield\u5373\u53ef":3,"\u53ea\u9700\u8981\u7b80\u5355\u5730\u8fd0\u884c\u4ee5\u4e0b\u547d\u4ee4":65,"\u53ea\u9700\u8981\u7b80\u5355\u7684\u8fd0\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u5373\u53ef":62,"\u53ea\u9700\u8981\u8fd0\u884c":62,"\u53ef\u4ee5":[36,40,45],"\u53ef\u4ee5\u4ee5\u540e\u53f0\u8fdb\u7a0b\u65b9\u5f0f\u8fd0\u884c\u5bb9\u5668":30,"\u53ef\u4ee5\u4f20\u5165\u4e00\u4e2a\u51fd\u6570":3,"\u53ef\u4ee5\u4f30\u8ba1\u51fa\u5982\u679c\u6a21\u578b\u91c7\u7528\u4e0d\u53d8\u7684\u8f93\u51fa\u6700\u5c0f\u7684cost0\u662f\u591a\u5c11":27,"\u53ef\u4ee5\u4f7f\u7528":27,"\u53ef\u4ee5\u4f7f\u7528\u547d\u4ee4":32,"\u53ef\u4ee5\u4f7f\u7528\u5982\u4e0b\u4ee3\u7801":27,"\u53ef\u4ee5\u4f7f\u7528\u8be5\u53c2\u6570":47,"\u53ef\u4ee5\u4f7f\u7528kubernetes\u7684\u547d\u4ee4\u884c\u5de5\u5177\u521b\u5efajob":52,"\u53ef\u4ee5\u4f7f\u7528python\u7684":5,"\u53ef\u4ee5\u51cf\u5c11\u7f13\u5b58\u6c60\u7684\u5927\u5c0f":27,"\u53ef\u4ee5\u521b\u5efa\u4e00\u4e2a":51,"\u53ef\u4ee5\u53c2\u7167\u4e0b\u9762\u7684\u547d\u4ee4\u8fdb\u884c\u5b89\u88c5":57,"\u53ef\u4ee5\u53c2\u8003":[36,39,50,52,65],"\u53ef\u4ee5\u53c2\u8003\u4fdd\u5b58\u5728":56,"\u53ef\u4ee5\u53c2\u8003paddlepaddl":33,"\u53ef\u4ee5\u544a\u8bc9\u60a8\u67d0\u4e2a\u64cd\u4f5c\u5230\u5e95\u82b1\u4e86\u591a\u957f\u65f6\u95f4":44,"\u53ef\u4ee5\u5728":30,"\u53ef\u4ee5\u5728\u4efb\u4f55\u673a\u5668\u4e0a\u6267\u884c\u7684":25,"\u53ef\u4ee5\u5728\u5171\u4eab\u5b58\u50a8\u4e0a\u67e5\u770b\u8f93\u51fa\u7684\u65e5\u5fd7\u548c\u6a21\u578b":52,"\u53ef\u4ee5\u5728\u5f88\u5927\u7a0b\u5ea6\u4e0a\u6d88\u9664\u6b67\u4e49":63,"\u53ef\u4ee5\u5728\u7f51\u7ad9\u4e0a\u627e\u5230":63,"\u53ef\u4ee5\u5728\u8fd9\u4e2a":40,"\u53ef\u4ee5\u5728kubernetes\u4e2d\u6309\u7167":50,"\u53ef\u4ee5\u5c06\u67d0\u4e00\u4e2a\u51fd\u6570\u6807\u8bb0\u6210\u4e00\u4e2apydataprovider2":3,"\u53ef\u4ee5\u5c06\u78c1\u76d8\u4e0a\u67d0\u4e2a\u76ee\u5f55\u5171\u4eab\u7ed9\u7f51\u7edc\u4e2d\u5176\u4ed6\u673a\u5668\u8bbf\u95ee":50,"\u53ef\u4ee5\u5c06memory\u7406\u89e3\u4e3a\u4e00\u4e2a\u65f6\u5ef6\u64cd\u4f5c":38,"\u53ef\u4ee5\u5e2e\u60a8\u63d0\u4f9b\u4e00\u4e9b\u5b9a\u4f4d\u6027\u80fd\u74f6\u9888\u7684\u5efa\u8bae":44,"\u53ef\u4ee5\u5feb\u901f\u8fed\u4ee3":30,"\u53ef\u4ee5\u6307\u5b9a\u54ea\u4e00\u4e2a\u8f93\u5165\u548c\u8f93\u51fa\u5e8f\u5217\u4fe1\u606f\u4e00\u81f4":36,"\u53ef\u4ee5\u6309\u5982\u4e0b\u7684\u7ed3\u6784\u6765\u51c6\u5907\u6570\u6910":64,"\u53ef\u4ee5\u662f\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":[35,38],"\u53ef\u4ee5\u662f\u4e00\u4e2a\u975e\u5e8f\u5217":38,"\u53ef\u4ee5\u662f\u4ee5\u4e0b\u51e0\u79cd":41,"\u53ef\u4ee5\u663e\u793a\u5730\u6307\u5b9a\u4e00\u4e2alayer\u7684\u8f93\u51fa\u7528\u4e8e\u521d\u59cb\u5316memori":38,"\u53ef\u4ee5\u6709\u4ee5\u4e0b\u4e24\u79cd":38,"\u53ef\u4ee5\u6709\u6548\u51cf\u5c0f\u7f51\u7edc\u7684\u963b\u585e":47,"\u53ef\u4ee5\u67e5\u770b":52,"\u53ef\u4ee5\u67e5\u770b\u6b64pod\u8fd0\u884c\u7684\u5bbf\u4e3b\u673a":51,"\u53ef\u4ee5\u6d4b\u8bd5\u591a\u4e2a\u6a21\u578b":49,"\u53ef\u4ee5\u7528\u4e8e\u4ece\u5b98\u65b9\u7f51\u7ad9\u4e0a\u4e0b\u8f7dcifar":57,"\u53ef\u4ee5\u7528\u4e8e\u5c0f\u91cf\u6570\u636e\u7684\u9a8c\u8bc1":50,"\u53ef\u4ee5\u7528\u4e8e\u63a5\u6536\u548cpydataprovider2\u4e00\u6837\u7684\u8f93\u5165\u6570\u636e\u5e76\u8f6c\u6362\u6210\u9884\u6d4b\u63a5\u53e3\u6240\u9700\u7684\u6570\u636e\u7c7b\u578b":5,"\u53ef\u4ee5\u7528\u5982\u4e0b\u547d\u4ee4":40,"\u53ef\u4ee5\u7528\u6765\u8ba1\u7b97cpu\u51fd\u6570\u6216cuda\u5185\u6838\u7684\u65f6\u95f4\u6d88\u8017":44,"\u53ef\u4ee5\u770b\u4f5c\u662f\u4e00\u4e2a\u975e\u5e8f\u5217\u8f93\u5165":35,"\u53ef\u4ee5\u7cbe\u786e\u8bf4\u660e\u4e00\u4e2a\u957f\u8017\u65f6\u64cd\u4f5c\u7684\u5177\u4f53\u539f\u56e0":44,"\u53ef\u4ee5\u7f16\u5199":30,"\u53ef\u4ee5\u8003\u8651\u4f7f\u7528\u4e00\u4e9b\u4f18\u5316\u7b97\u6cd5":27,"\u53ef\u4ee5\u8bbe\u7f6e":57,"\u53ef\u4ee5\u8f7b\u677e\u5730\u5b8c\u6210\u795e\u7ecf\u7f51\u7edc\u914d\u7f6e":33,"\u53ef\u4ee5\u8fd0\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u6765\u751f\u6210":62,"\u53ef\u4ee5\u8fd0\u884c\u811a\u672ctrain":57,"\u53ef\u4ee5\u9009\u62e9\u662f\u5426\u4f7f\u7528\u53c2\u6570":49,"\u53ef\u4ee5\u901a\u8fc7":50,"\u53ef\u4ee5\u901a\u8fc7\u4fee\u6539\u8fd9\u4e24\u4e2a\u51fd\u6570\u6765\u5b9e\u73b0\u590d\u6742\u7684\u7f51\u7edc\u914d\u7f6e":39,"\u53ef\u4ee5\u901a\u8fc7\u8c03\u7528":5,"\u53ef\u4ee5\u901a\u8fc7show_parameter_stats_period\u8bbe\u7f6e\u6253\u5370\u53c2\u6570\u4fe1\u606f\u7b49":60,"\u53ef\u7528\u4e8e\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u89e3\u6790\u8fd9\u4e9b\u53c2\u6570":49,"\u53ef\u7528\u5728\u6d4b\u8bd5\u6216\u8bad\u7ec3\u65f6\u6307\u5b9a\u521d\u59cb\u5316\u6a21\u578b":60,"\u53ef\u80fd\u4f1a\u5bfc\u81f4\u51fa\u9519":52,"\u53ef\u80fd\u7684\u4ee3\u7801\u4e3a":27,"\u53ef\u80fd\u7684\u539f\u56e0\u662f":27,"\u53ef\u80fd\u7684\u60c5\u51b5\u4e0b":44,"\u53ef\u9009":[3,41],"\u53f3\u8fb9\u662f":58,"\u5403":36,"\u5403\u996d":36,"\u5404\u65b9\u9762":36,"\u5404\u9879\u53c2\u6570\u7684\u8be6\u7ec6\u8bf4\u660e\u53ef\u4ee5\u5728\u547d\u4ee4\u884c\u53c2\u6570\u76f8\u5173\u6587\u6863\u4e2d\u627e\u5230":57,"\u5408":36,"\u5408\u5e76":65,"\u5408\u5e76\u6bcf\u4e2a":65,"\u5408\u7406":36,"\u540c\u65f6":[27,44],"\u540c\u65f6\u4e5f\u4f1a\u8bfb\u53d6\u76f8\u5173\u8def\u5f84\u53d8\u91cf\u6765\u8fdb\u884c\u641c\u7d22":29,"\u540c\u65f6\u4e5f\u53ef\u4ee5\u52a0\u901f\u5f00\u59cb\u8bad\u7ec3\u524d\u6570\u636e\u8f7d\u5165\u7684\u8fc7\u7a0b":27,"\u540c\u65f6\u4e5f\u53ef\u4ee5\u901a\u8fc7":40,"\u540c\u65f6\u4e5f\u80fd\u591f\u5f15\u5165\u66f4\u52a0\u590d\u6742\u7684\u8bb0\u5fc6\u673a\u5236":38,"\u540c\u65f6\u4f1a\u8ba1\u7b97\u5206\u7c7b\u51c6\u786e\u7387":60,"\u540c\u65f6\u4f60\u53ef\u4ee5\u4f7f\u7528":58,"\u540c\u65f6\u4f7f\u7528\u4e86l2\u6b63\u5219":60,"\u540c\u65f6\u5176\u5185\u90e8\u5b9e\u73b0\u53ef\u4ee5\u907f\u514d\u7eafcpu\u7248\u672cpaddlepaddle\u5728\u6267\u884c\u672c\u8bed\u53e5\u65f6\u53d1\u751f\u5d29\u6e83":44,"\u540c\u65f6\u53ef\u4ee5\u4f7f\u7528\u6237\u53ea\u5173\u6ce8\u5982\u4f55\u4ece\u6587\u4ef6\u4e2d\u8bfb\u53d6\u6bcf\u4e00\u6761\u6570\u636e":3,"\u540c\u65f6\u5728\u5185\u5b58\u91cc\u76f4\u63a5\u968f\u5373\u9009\u53d6\u6570\u636e\u6765\u505ashuffl":27,"\u540c\u65f6\u5c06\u53c2\u6570\u521d\u59cb\u5316\u4e3a":27,"\u540c\u65f6\u6211\u4eec\u5e0c\u671b\u5e7f\u5927\u5f00\u53d1\u8005\u79ef\u6781\u63d0\u4f9b\u53cd\u9988\u548c\u8d21\u732e\u6e90\u4ee3\u7801":0,"\u540c\u65f6\u6b22\u8fce\u8d21\u732e\u66f4\u591a\u7684\u5b89\u88c5\u5305":31,"\u540c\u65f6\u7528\u6237\u9700\u8981\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u6307\u5b9a":49,"\u540c\u65f6\u8bbe\u7f6e\u5185\u5b58\u7f13\u5b58\u529f\u80fd":27,"\u540c\u65f6\u8bbe\u7f6e\u5b83\u7684input_types\u5c5e\u6027":3,"\u540c\u65f6\u9884\u6d4b\u7f51\u7edc\u901a\u5e38\u76f4\u63a5\u8f93\u51fa\u6700\u540e\u4e00\u5c42\u7684\u7ed3\u679c\u800c\u4e0d\u662f\u50cf\u8bad\u7ec3\u7f51\u7edc\u4e00\u6837\u518d\u63a5\u4e00\u5c42cost":5,"\u540c\u6837":33,"\u540c\u6837\u4e5f\u53ef\u4ee5\u5728\u6d4b\u8bd5\u6a21\u5f0f\u4e2d\u6307\u5b9a\u6a21\u578b\u8def\u5f84":47,"\u540c\u6837\u53ef\u4ee5\u6269\u5c55\u5230\u53cc\u5c42\u5e8f\u5217\u7684\u5904\u7406\u4e0a":38,"\u540c\u6b65\u6267\u884c\u64cd\u4f5c\u7684\u7ebf\u7a0b\u6570":47,"\u540d\u5b57\u4fee\u9970":25,"\u540d\u79f0":60,"\u540e":[27,29,40,52,64],"\u540e\u5411\u4f20\u64ad":41,"\u540e\u5411\u4f20\u64ad\u7ed9\u5b9a\u8f93\u51fa\u7684\u68af\u5ea6":41,"\u540e\u9762\u8fde\u5168\u8fde\u63a5\u5c42\u548csoftmax\u5c42":64,"\u5411\u91cfenable_parallel_vector":46,"\u5426":29,"\u5426\u5219":[2,45,62],"\u5426\u5219\u4f1a\u628a":40,"\u5426\u5219\u4f60\u9700\u8981\u81ea\u5df1\u4e0b\u8f7d":65,"\u5426\u5219\u4f7f\u7528\u591a\u673a\u8bad\u7ec3":47,"\u5426\u5219\u4f7f\u7528cpu\u6a21\u5f0f":47,"\u5426\u5219\u4f7f\u7528gpu":49,"\u5426\u5219\u5b83\u4ee5\u4e00\u4e2a\u5e8f\u5217\u8f93\u5165":39,"\u5426\u5219\u5f97\u628apaddle\u9759\u6001\u5e93\u94fe\u63a5\u5230\u89e3\u91ca\u5668\u91cc":25,"\u5426\u5219\u9891\u7e41\u7684\u591a\u8282\u70b9\u5de5\u4f5c\u7a7a\u95f4\u90e8\u7f72\u53ef\u80fd\u4f1a\u5f88\u9ebb\u70e6":45,"\u5426\u5b9a":63,"\u542b\u4e49":[58,64],"\u542b\u6709":61,"\u542b\u6709\u5e8f\u5217\u4fe1\u606f\u548c\u5b50\u5e8f\u5217\u4fe1\u606f\u7684\u7a20\u5bc6\u5411\u91cf":41,"\u542b\u6709\u5e8f\u5217\u4fe1\u606f\u7684\u6574\u6570":41,"\u542b\u6709\u5e8f\u5217\u4fe1\u606f\u7684\u7a20\u5bc6\u5411\u91cf":41,"\u542f\u52a8\u5bb9\u5668\u5f00\u59cb\u8bad\u7ec3":52,"\u542f\u52a8\u5e76\u884c\u5411\u91cf\u7684\u9608\u503c":47,"\u542f\u52a8\u5feb\u901f\u5e94\u7b54":47,"\u542f\u7528\u68af\u5ea6\u53c2\u6570\u7684\u9608\u503c":47,"\u5440":36,"\u544a\u8bc9paddle\u54ea\u4e2a\u6587\u4ef6\u662f\u795e\u7ecf\u7f51\u7edc\u7684\u914d\u7f6e\u6587\u4ef6":62,"\u544a\u8bc9paddle\u5c06\u6a21\u578b\u4fdd\u5b58\u5728":62,"\u5468\u56f4":36,"\u547d\u4ee4":45,"\u547d\u4ee4\u4e3a":51,"\u547d\u4ee4\u521b\u5efa\u65b0\u955c\u50cf":51,"\u547d\u4ee4\u53ef\u4ee5\u8bbe\u7f6e":29,"\u547d\u4ee4\u6307\u5b9a\u7684\u53c2\u6570\u4f1a\u4f20\u5165\u7f51\u7edc\u914d\u7f6e\u4e2d":60,"\u547d\u4ee4\u884c\u53c2\u6570\u6587\u6863":60,"\u547d\u4ee4\u8bbe\u7f6e\u8be5\u7c7b\u7f16\u8bd1\u9009\u9879":29,"\u547d\u4ee4\u9009\u9879\u5e76\u4e14":45,"\u547d\u540d\u4e3a":40,"\u547d\u540d\u7a7a\u95f4":50,"\u547d\u540d\u7a7a\u95f4\u4e3b\u8981\u4e3a\u4e86\u5bf9\u8c61\u8fdb\u884c\u903b\u8f91\u4e0a\u7684\u5206\u7ec4\u4fbf\u4e8e\u7ba1\u7406":50,"\u548c":[25,27,28,29,36,39,40,41,42,44,45,49,50,56,57,60,62,65],"\u548c\u4e00\u4e2a\u5df2\u7ecf\u5206\u8bcd\u540e\u7684\u53e5\u5b50":36,"\u548c\u4e09\u79cd\u5e8f\u5217\u6a21\u5f0f":[3,33],"\u548c\u4e2d\u6587\u6587\u6863":42,"\u548c\u4e4b\u524d\u51cf\u5c0f\u901a\u8fc7\u51cf\u5c0f\u7f13\u5b58\u6c60\u6765\u51cf\u5c0f\u5185\u5b58\u5360\u7528\u7684\u539f\u7406\u4e00\u81f4":27,"\u548c\u504f\u7f6e\u5411\u91cf":41,"\u548c\u533a\u57df\u6807\u8bb0":63,"\u548c\u53cc\u5c42\u5e8f\u5217\u542b\u6709subseq":35,"\u548c\u5728":3,"\u548c\u5bf9\u8c61\u5b58\u50a8api":50,"\u548c\u5dee\u8bc4":60,"\u548c\u5e8f\u5217\u4e2d\u542b\u6709\u5143\u7d20\u7684\u6570\u76ee\u540c":35,"\u548c\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u8f93\u5165":39,"\u548c\u68af\u5ea6\u622a\u65ad":60,"\u548c\u6a21\u578b\u8def\u5f84":64,"\u548c\u771f\u5b9e":28,"\u548c\u793a\u4f8b2\u4e2d\u7684\u914d\u7f6e\u7c7b\u4f3c":36,"\u548c\u7b2c6\u884c\u7684":65,"\u548c\u90e8\u5206layer":38,"\u548cadam\u5b66\u4e60\u65b9\u6cd5":65,"\u548cargument":63,"\u548ccudnn":32,"\u548cpool":35,"\u548cpython\u63a5\u53e3\u6765\u63d0\u53d6\u7279\u5f81":58,"\u54c1\u8d28":36,"\u54ea\u4e9b\u4e0d\u662f":36,"\u552f\u4e00\u9700\u8981\u505a\u7684\u662f\u5c06\u76f8\u5e94\u7c7b\u578b\u8bbe\u7f6e\u4e3a\u8f93\u5165":39,"\u5546\u52a1":36,"\u554a":36,"\u559c\u5267\u7247":61,"\u5668":60,"\u56db\u79cd\u6570\u636e\u7c7b\u578b":[3,33],"\u56de\u5f52\u8bef\u5dee\u4ee3\u4ef7\u5c42":28,"\u56e0\u4e3a\u5168\u8fde\u63a5\u5c42\u7684\u6fc0\u6d3b\u53ef\u4ee5\u662fsoftmax":41,"\u56e0\u4e3a\u5355\u4e2a\u8c13\u8bcd\u4e0d\u80fd\u7cbe\u786e\u5730\u63cf\u8ff0\u8c13\u8bcd\u4fe1\u606f":63,"\u56e0\u4e3a\u53c2\u6570":49,"\u56e0\u4e3a\u5b83\u4eec\u7684\u8ba1\u7b97\u6548\u7387\u6bd4":39,"\u56e0\u4e3a\u5b83\u6bd4":39,"\u56e0\u4e3a\u5b98\u65b9\u955c\u50cf":52,"\u56e0\u4e3a\u5bb9\u5668\u5185\u7684\u6587\u4ef6\u90fd\u662f\u6682\u65f6\u5b58\u5728\u7684":50,"\u56e0\u4e3a\u8be5\u6587\u4ef6\u53ef\u9002\u7528\u4e8e\u9884\u6d4b":57,"\u56e0\u4e3adocker\u80fd\u5728\u6240\u6709\u4e3b\u8981\u64cd\u4f5c\u7cfb\u7edf":30,"\u56e0\u4e3apython\u7684\u641c\u7d22\u8def\u5f84\u662f\u4f18\u5148\u5df2\u7ecf\u5b89\u88c5\u7684python\u5305":27,"\u56e0\u4e3aswig\u5728\u7b2c\u4e09\u65b9\u8bed\u8a00\u4e2d\u66b4\u9732\u7684\u51fd\u6570\u540d":25,"\u56e0\u6b64":[2,3,36,38,41],"\u56e0\u6b64\u4f7f\u7528":3,"\u56e0\u6b64\u53cc\u5c42\u5e8f\u5217\u7684\u914d\u7f6e\u4e2d":36,"\u56e0\u6b64\u53ef\u4ee5\u4f7f\u7528\u8be5\u9009\u9879":56,"\u56e0\u6b64\u53ef\u80fd\u4f1a\u6709\u4e00\u4e9b\u9519\u8bef\u548c\u4e0d\u4e00\u81f4\u53d1\u751f":61,"\u56e0\u6b64\u5982\u679c\u8fd9\u4e2a\u811a\u672c\u8fd0\u884c\u5931\u8d25":57,"\u56e0\u6b64\u5b83\u662finteger_value_sub_sequ":36,"\u56e0\u6b64\u6211\u4eec\u91c7\u7528\u8f93\u51fa\u7684\u52a0\u6743\u548c":41,"\u56e0\u6b64\u6709\u4e24\u79cd\u89e3\u51b3\u65b9\u6848":3,"\u56e0\u6b64\u7528\u6237\u5e76\u4e0d\u9700\u8981\u5173\u5fc3\u5b83\u4eec":46,"\u56e0\u6b64\u8be5\u5c42\u4e2d\u6ca1\u6709\u504f\u7f6e":58,"\u56e0\u6b64\u9519\u8bef\u7684\u4f7f\u7528\u4e8c\u8fdb\u5236\u53d1\u884c\u7248\u53ef\u80fd\u4f1a\u5bfc\u81f4\u8fd9\u79cd\u9519\u8bef":27,"\u56e0\u6b64init_hook\u5c3d\u91cf\u4f7f\u7528":3,"\u56e2\u8d2d\u7f51\u7ad9":64,"\u56fe":[58,64],"\u56fe2\u662f\u53cc\u5411lstm\u7f51\u7edc":64,"\u56fe3\u662f\u4e09\u5c42lstm\u7ed3\u6784":64,"\u56fe\u50cf\u5206\u7c7b":59,"\u56fe\u50cf\u5927\u5c0f\u4e3a3":58,"\u56fe\u50cf\u63cf\u8ff0":65,"\u56fe\u7247\u5206\u4e3a10\u7c7b":57,"\u56fe\u7684\u5e95\u90e8\u662fword":64,"\u56fe\u8868":[30,64],"\u5728":[3,32,35,36,39,40,45,58,60,61,63],"\u5728\u4e00\u4e2a\u529f\u80fd\u9f50\u5168\u7684kubernetes\u673a\u7fa4\u91cc":51,"\u5728\u4e00\u4e2a\u53c2\u6570\u7684\u68af\u5ea6\u88ab\u66f4\u65b0\u540e":41,"\u5728\u4e00\u4e2a\u5468\u671f\u5185\u6d4b\u8bd5\u6240\u6709\u6570\u636e":63,"\u5728\u4e00\u8f6e\u4e2d\u6bcfsave":47,"\u5728\u4e0a\u9762\u4ee3\u7801\u4e2d":36,"\u5728\u4e0b\u4e00\u7bc7\u4e2d":51,"\u5728\u4e0b\u9762\u4f8b\u5b50\u91cc":60,"\u5728\u4e0b\u9762\u7684\u4f8b\u5b50\u4e2d":57,"\u5728\u4e0d\u540c\u64cd\u4f5c\u7cfb\u7edf":50,"\u5728\u4e4b\u540e\u7684":27,"\u5728\u4efb\u610f\u957f\u5ea6\u8bed\u53e5\u7ffb\u8bd1\u7684\u573a\u666f\u4e0b\u90fd\u53ef\u4ee5\u89c2\u5bdf\u5230\u5176\u6548\u679c\u7684\u63d0\u5347":65,"\u5728\u4f7f\u7528\u5b83\u4e4b\u524d\u8bf7\u5b89\u88c5paddlepaddle\u7684python":64,"\u5728\u4f7f\u7528\u8be5\u6587\u6863\u4e4b\u524d":33,"\u5728\u5168\u8fde\u63a5\u5c42\u4e2d":41,"\u5728\u51c6\u5907\u53d1\u8d77":40,"\u5728\u51fd\u6570":52,"\u5728\u5206\u5e03\u5f0f\u73af\u5883\u4e2d\u6d4b\u8bd5":47,"\u5728\u5206\u5e03\u5f0f\u8bad\u7ec3\u4e2d":47,"\u5728\u5355\u5c42\u6570\u636e\u7684\u57fa\u7840\u4e0a":36,"\u5728\u53c2\u6570\u670d\u52a1\u5668\u4e0a\u52a0\u8f7d\u548c\u4fdd\u5b58\u53c2\u6570":47,"\u5728\u53c2\u6570\u670d\u52a1\u5668\u7ec8\u7aef\u6bcflog":47,"\u5728\u53cc\u5c42rnn\u4e2d\u7684\u7ecf\u5178\u60c5\u51b5\u662f\u5c06\u5185\u5c42\u7684\u6bcf\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217\u6570\u636e":36,"\u5728\u53cd\u5411\u4f20\u9012\u7684\u65f6\u5019":27,"\u5728\u53d8\u6362\u65f6\u9700\u8981\u5c06\u8f93\u5165\u5e8f\u5217\u4f20\u5165":36,"\u5728\u540c\u4e00\u4e2a\u547d\u540d\u7a7a\u95f4\u4e2d":50,"\u5728\u542f\u52a8job\u4e4b\u524d":52,"\u5728\u56fd\u5185":30,"\u5728\u58f0\u660edataprovider\u7684\u65f6\u5019\u4f20\u5165dictionary\u4f5c\u4e3a\u53c2\u6570":3,"\u5728\u591acpu\u8bad\u7ec3\u65f6\u5171\u4eab\u8be5\u53c2\u6570":47,"\u5728\u5b8c\u6210\u795e\u7ecf\u7f51\u7edc\u7684\u642d\u5efa\u4e4b\u540e":33,"\u5728\u5b9a\u4e49\u8f93\u5165layer\u4e4b\u540e":33,"\u5728\u5bb9\u5668\u521b\u5efa\u540e":52,"\u5728\u5bf9\u5bb9\u5668\u7684\u63cf\u8ff0":52,"\u5728\u5c42\u4e2d\u6307\u5b9a":49,"\u5728\u5e8f\u5217\u751f\u6210\u4efb\u52a1\u4e2d":38,"\u5728\u5f00\u59cb\u8bad\u7ec3\u4e4b\u524d":57,"\u5728\u5f53\u524d\u7684\u5b9e\u73b0\u65b9\u5f0f\u4e0b":41,"\u5728\u5f97\u5230":52,"\u5728\u6211\u4eec\u7684\u4f8b\u5b50\u4e2d":39,"\u5728\u6211\u4eec\u7684\u6d4b\u8bd5\u4e2d":64,"\u5728\u63d0\u4ea4":40,"\u5728\u642d\u5efa\u795e\u7ecf\u7f51\u7edc\u7684\u8fc7\u7a0b\u4e2d":33,"\u5728\u6570\u636e\u52a0\u8f7d\u548c\u7f51\u7edc\u914d\u7f6e\u5b8c\u6210\u4e4b\u540e":60,"\u5728\u6587\u4ef6":62,"\u5728\u6709\u65b0\u7684\u5355\u8bcd\u6765\u4e34\u7684\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65\u9aa4\u5185":64,"\u5728\u672c\u4f8b\u4e2d":[36,40,49],"\u5728\u672c\u4f8b\u4e2d\u6ca1\u6709\u4f7f\u7528":3,"\u5728\u672c\u6559\u7a0b\u4e2d":[39,57],"\u5728\u672c\u6587\u4e2d":45,"\u5728\u672c\u6587\u4e2d\u4f7f\u7528\u7684":45,"\u5728\u672c\u6f14\u793a\u4e2d":64,"\u5728\u672c\u793a\u4f8b\u4e2d":[36,64],"\u5728\u672c\u8282\u4e2d":39,"\u5728\u6811\u7684\u6bcf\u4e00\u5c42\u4e0a":47,"\u5728\u6a21\u578b\u6587\u4ef6\u7684":45,"\u5728\u6a21\u578b\u914d\u7f6e\u4e2d\u901a\u8fc7":60,"\u5728\u6b64":[0,46,49],"\u5728\u6b64\u4e3a\u65b9\u4fbf\u5bf9\u6bd4\u4e0d\u540c\u7f51\u7edc\u7ed3\u6784":60,"\u5728\u6b64\u611f\u8c22":56,"\u5728\u6bcf\u4e2a\u65f6\u95f4\u6b65\u4e2d":39,"\u5728\u6bcf\u4e2a\u65f6\u95f4\u6b65\u7684\u5b50\u5e8f\u5217\u957f\u5ea6\u53ef\u4ee5\u4e0d\u76f8\u7b49":36,"\u5728\u6bcf\u4e2a\u65f6\u95f4\u6b65\u957f":39,"\u5728\u6bcf\u4e2apod\u4e0a\u90fd\u901a\u8fc7volume\u65b9\u5f0f\u6302\u8f7d\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf\u7684\u4e00\u4e2a\u76ee\u5f55\u7528\u4e8e\u4fdd\u5b58\u8bad\u7ec3\u6570\u636e\u548c\u8f93\u51fa\u7ed3\u679c":52,"\u5728\u6bcf\u8bad\u7ec3":62,"\u5728\u6d4b\u8bd5\u9636\u6bb5":47,"\u5728\u6d4b\u8bd5\u9636\u6bb5\u5b83\u4eec\u5c06\u4f1a\u88ab\u52a0\u8f7d\u5230\u6a21\u578b\u4e2d":58,"\u5728\u6e90\u7801\u76ee\u5f55\u6811\u7684\u6839\u76ee\u5f55\u4e2d\u8fd0\u884c":40,"\u5728\u6f14\u793a\u4e2d":63,"\u5728\u7269\u7406\u673a\u4e0a\u624b\u52a8\u90e8\u7f72":50,"\u5728\u751f\u6210\u65f6":39,"\u5728\u751f\u6210\u8fc7\u7a0b\u4e2d":65,"\u5728\u751f\u6210\u8fc7\u7a0b\u4e2d\u6211\u4eec\u4f7f\u7528sgd\u8bad\u7ec3\u7b97\u6cd5":65,"\u5728\u7528\u6237\u6587\u4ef6user":62,"\u5728\u7535\u5f71\u6587\u4ef6movi":62,"\u5728\u793a\u4f8b\u4e2d\u6211\u4eec\u4f7f\u7528attention\u7248\u672c\u7684gru\u7f16\u89e3\u7801\u7f51\u7edc":65,"\u5728\u793a\u4f8b\u4e2d\u6211\u4eec\u4f7f\u7528sgd\u8bad\u7ec3\u7b97\u6cd5":65,"\u5728\u793a\u4f8b\u4e2d\u6211\u4eec\u5b9a\u4e49\u4e86\u4e00\u4e2a\u5e8f\u5217\u5230\u5e8f\u5217\u7684\u751f\u6210\u6570\u636e":65,"\u5728\u793a\u4f8b\u4e2d\u6211\u4eec\u5b9a\u4e49\u4e86\u4e00\u4e2a\u5e8f\u5217\u5230\u5e8f\u5217\u7684\u8bad\u7ec3\u548c\u6d4b\u8bd5\u6570\u636e":65,"\u5728\u7a0b\u5e8f\u5f00\u59cb\u9636\u6bb5":5,"\u5728\u7b2c\u4e00\u884c\u4e2d\u6211\u4eec\u8f7d\u5165\u7528\u4e8e\u5b9a\u4e49\u7f51\u7edc\u7684\u51fd\u6570":57,"\u5728\u7ec4\u5408\u65f6":33,"\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d":41,"\u5728\u7f51\u7edc\u914d\u7f6e\u91cc":3,"\u5728\u7ffb\u8bd1\u6cd5\u8bed\u53e5\u5b50\u4e4b\u524d":65,"\u5728\u811a\u672c":62,"\u5728\u81ea\u7136\u8bed\u8a00\u5904\u7406\u4efb\u52a1\u4e2d":35,"\u5728\u8bad\u7ec3\u4e2d":39,"\u5728\u8bad\u7ec3\u4e4b\u524d":52,"\u5728\u8bad\u7ec3\u4e86":62,"\u5728\u8bad\u7ec3\u4e86\u51e0\u4e2a\u8f6e\u6b21\u4ee5\u540e":62,"\u5728\u8bad\u7ec3\u5b8c\u6210\u540e":57,"\u5728\u8bad\u7ec3\u6570\u96c6\u4e0a\u8bad\u7ec3\u751f\u6210\u8bcd\u5411\u91cf\u5b57\u5178":56,"\u5728\u8bad\u7ec3\u65f6":51,"\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d":[52,65],"\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u6bcfshow":47,"\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u8fdb\u884c\u6d4b\u8bd5":2,"\u5728\u8be5\u914d\u7f6e\u76847":36,"\u5728\u8bed\u8a00\u751f\u6210\u9886\u57df\u4e2d":65,"\u5728\u8d2d\u7269\u7f51\u7ad9\u4e0a":60,"\u5728\u8f6f\u4ef6\u5de5\u7a0b\u7684\u8303\u7574\u91cc":44,"\u5728\u8f93\u51fa\u7684\u8fc7\u7a0b\u4e2d":38,"\u5728\u8fd0\u884c":64,"\u5728\u8fd9\u4e2a\u4efb\u52a1\u4e2d":65,"\u5728\u8fd9\u4e2a\u4f8b\u5b50\u4e2d":[28,64],"\u5728\u8fd9\u4e2a\u4f8b\u5b50\u91cc":[41,51],"\u5728\u8fd9\u4e2a\u51fd\u6570\u4e2d":36,"\u5728\u8fd9\u4e2a\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165\u4efb\u4f55\u5176\u4ed6\u8bed\u8a00\u7684\u89e3\u91ca\u5668":25,"\u5728\u8fd9\u4e2a\u6559\u7a0b\u4e2d":44,"\u5728\u8fd9\u4e2a\u6a21\u578b\u4e2d":39,"\u5728\u8fd9\u4e2a\u793a\u4f8b\u4e2d":63,"\u5728\u8fd9\u4e9b\u7f51\u7edc\u4e2d":62,"\u5728\u8fd9\u4e9blayer\u4e2d":36,"\u5728\u8fd9\u6b65\u4efb\u52a1\u4e2d":64,"\u5728\u8fd9\u79cd\u60c5\u51b5\u4e0b":[39,41],"\u5728\u8fd9\u79cd\u7ed3\u6784\u4e2d":38,"\u5728\u8fd9\u7bc7\u6587\u6863\u91cc":51,"\u5728\u8fd9\u7bc7\u6587\u7ae0\u91cc":52,"\u5728\u8fd9\u91cc":38,"\u5728\u8fd9\u91cc\u6211\u4eec\u4f7f\u7528\u5168\u8fde\u63a5\u5c42\u4f5c\u4e3a\u4f8b\u5b50\u6765\u5c55\u793a\u5b9e\u73b0\u65b0\u7f51\u7edc\u5c42\u6240\u9700\u8981\u7684\u56db\u4e2a\u6b65\u9aa4":41,"\u5728\u8fdb\u884c\u7f51\u7edc\u914d\u7f6e\u4e4b\u524d":33,"\u5728\u914d\u7f6e\u4e2d\u9700\u8981\u8bfb\u53d6\u5916\u90e8\u5b57\u5178":3,"\u5728\u914d\u7f6e\u6587\u4ef6\u4e2d\u7684":58,"\u5728\u91c7\u7528sgd":27,"\u5728\u96c6\u7fa4\u4e0a\u8bad\u7ec3\u4e00\u4e2a\u7a00\u758f\u6a21\u578b\u9700\u8981\u52a0\u4e0a\u4e0b\u9762\u7684\u53c2\u6570":49,"\u5728\u9884\u5904\u7406\u542b\u6709\u591a\u884c\u6570\u6910\u7684\u6587\u672c\u6587\u4ef6\u65f6\u53c2\u6570\u8bbe\u7f6e\u7a0d\u6709\u4e0d\u540c":64,"\u5728\u9884\u6d4b\u5e8f\u5217\u6216\u6bb5\u843d\u7684\u60c5\u611f\u4e2d\u8d77\u4e3b\u8981\u4f5c\u7528":64,"\u5728aws\u4e0a\u5feb\u901f\u90e8\u7f72\u96c6\u7fa4":50,"\u5728c":25,"\u5728c\u7684\u5934\u6587\u4ef6":25,"\u5728cub":57,"\u5728generator\u7684\u4e0a\u4e0b\u6587\u4e2d\u5c3d\u91cf\u7559\u4e0b\u975e\u5e38\u5c11\u7684\u53d8\u91cf\u5f15\u7528":3,"\u5728kubernetes\u4e2d\u521b\u5efa\u7684\u6240\u6709\u8d44\u6e90\u5bf9\u8c61":50,"\u5728linux\u4e0b":65,"\u5728meta\u6587\u4ef6\u4e2d\u6709\u4e24\u79cd\u7279\u5f81":62,"\u5728movielen":62,"\u5728paddl":52,"\u5728paddle\u4e2d":49,"\u5728paddlepaddle\u4e2d":[33,38],"\u5728paddlepaddle\u7684\u6587\u6863\u4e2d":36,"\u5728paddlepaddle\u91cc":28,"\u5728step\u51fd\u6570\u4e2d\u5b9a\u4e49":38,"\u5728step\u51fd\u6570\u4e2d\u5b9a\u4e49memori":38,"\u5728trainer":49,"\u5730\u5740\u4e5f\u53ef\u4ee5\u4e3ahdfs\u6587\u4ef6\u8def\u5f84":2,"\u5730\u6bb5":36,"\u5730\u7406\u4f4d\u7f6e":36,"\u5730\u94c1\u7ad9":36,"\u5747\u503c\u56fe\u50cf\u6587\u4ef6":58,"\u5747\u5300\u5206\u5e03":27,"\u5747\u5300\u5206\u5e03\u7684\u8303\u56f4\u662f":47,"\u5747\u6709\u4e09\u4e2a\u5b50\u5e8f\u5217":36,"\u5747\u6709\u4e24\u7ec4\u7279\u5f81":36,"\u57fa\u4e8e\u53cc\u5c42\u5e8f\u5217\u8f93\u5165":38,"\u57fa\u4e8e\u5b57\u6bcd\u7684\u8bcd\u5d4c\u5165\u7279\u5f81":62,"\u57fa\u4e8epython\u7684\u6a21\u578b\u9884\u6d4b":5,"\u57fa\u4e8epython\u7684\u9884\u6d4b":[4,60],"\u57fa\u672c\u4e0a\u548cmnist\u6837\u4f8b\u4e00\u81f4":3,"\u57fa\u672c\u4f7f\u7528\u6982\u5ff5":34,"\u57fa\u672c\u76f8\u540c":56,"\u586b\u5199":40,"\u589e\u52a0\u4e86\u4e00\u6761cd\u547d\u4ee4":51,"\u589e\u52a0\u5982\u4e0b\u53c2\u6570":49,"\u589e\u52a0\u68af\u5ea6\u68c0\u6d4b\u7684\u5355\u5143\u6d4b\u8bd5":41,"\u58f0\u660epython\u6570\u636e\u6e90":62,"\u5904\u7406\u5668\u6709\u4e24\u4e2a\u5173\u952e\u6027\u80fd\u9650\u5236":44,"\u5904\u7406\u6570\u636e\u7684python\u811a\u672c\u6587\u4ef6":60,"\u5904\u7406\u7684\u8f93\u5165\u5e8f\u5217\u4e3b\u8981\u5206\u4e3a\u4ee5\u4e0b\u4e09\u79cd\u7c7b\u578b":38,"\u5904\u7406\u76f8\u4f3c\u5ea6\u56de\u5f52":62,"\u5904\u7406\u8fc7\u7a0b\u4e2d\u6570\u636e\u5b58\u50a8\u683c\u5f0f":57,"\u5907\u6ce8":44,"\u590d\u6742\u5ea6\u6216\u65f6\u95f4\u590d\u6742\u5ea6":44,"\u5916\u5c42memory\u662f\u4e00\u4e2a\u5143\u7d20":36,"\u5916\u5c42outer_step\u4e2d":36,"\u591a\u4e2ainput\u4ee5list\u65b9\u5f0f\u8f93\u5165":60,"\u591a\u53e5\u8bdd\u8fdb\u4e00\u6b65\u6784\u6210\u4e86\u6bb5\u843d":38,"\u591a\u673a\u8bad\u7ec3":27,"\u591a\u7ebf\u7a0b\u7684\u6570\u636e\u8bfb\u53d6":3,"\u591a\u8f6e\u5bf9\u8bdd\u7b49\u66f4\u4e3a\u590d\u6742\u7684\u8bed\u8a00\u6570\u636e":38,"\u5927\u578b\u7535\u5f71\u8bc4\u8bba\u6570\u636e\u96c6":64,"\u5927\u591a\u6570\u5c42\u4e0d\u9700\u8981\u8fdc\u7a0b\u7a00\u758f\u8bad\u7ec3\u51fd\u6570":41,"\u5927\u591a\u6570\u5c42\u9700\u8981\u8bbe\u7f6e\u4e3a":41,"\u5927\u591a\u6570\u6210\u529f\u7684srl\u7cfb\u7edf\u662f\u5efa\u7acb\u5728\u67d0\u79cd\u5f62\u5f0f\u7684\u53e5\u6cd5\u5206\u6790\u7ed3\u679c\u4e4b\u4e0a\u7684":63,"\u5927\u591a\u6570\u7f51\u7edc\u5c42\u4e0d\u9700\u8981\u652f\u6301\u8fdc\u7a0b\u7a00\u758f\u66f4\u65b0":41,"\u5927\u591a\u6570\u8bed\u8a00\u90fd\u652f\u6301\u4f7f\u7528c\u8bed\u8a00api":25,"\u5927\u5b66\u751f":61,"\u5927\u5bb6\u53ef\u4ee5\u901a\u8fc7\u5b83\u5236\u4f5c\u548c\u5206\u4eab\u5e26\u6709\u4ee3\u7801":30,"\u5927\u5c0f":45,"\u5929":36,"\u5929\u4e00\u5e7f\u573a":36,"\u5929\u4e00\u9601":36,"\u5929\u732b":64,"\u5934\u6587\u4ef6\u4e2d\u628a\u53c2\u6570\u5b9a\u4e49\u4e3a\u7c7b\u7684\u6210\u5458\u53d8\u91cf":41,"\u5934\u6587\u4ef6\u5982\u4e0b":41,"\u5947\u5e7b\u7247":61,"\u597d":36,"\u597d\u5403":36,"\u597d\u8bc4":60,"\u5982":[3,39,40,45,49],"\u59822":45,"\u5982\u4e0b":[3,62,64],"\u5982\u4e0b\u56fe\u6240\u793a":[36,44,57],"\u5982\u4e0b\u6240\u793a":[49,58,62],"\u5982\u4e0b\u662f\u4e00\u6bb5\u4f7f\u7528mnist":5,"\u5982\u4e0b\u8868\u683c":60,"\u5982\u4f55":62,"\u5982\u4f55\u5b58\u50a8\u7b49\u7b49":3,"\u5982\u4f55\u89e3\u6790\u8be5\u5730\u5740\u4e5f\u662f\u7528\u6237\u81ea\u5b9a\u4e49dataprovider\u65f6\u9700\u8981\u8003\u8651\u7684\u5730\u65b9":2,"\u5982\u4f55\u8d21\u732e":43,"\u5982\u4f55\u8d21\u732e\u4ee3\u7801":43,"\u5982\u4f55\u8fdb\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3":60,"\u5982\u4fe1\u606f\u63d0\u53d6":63,"\u5982\u56fe2\u6240\u793a":64,"\u5982\u5f62\u5bb9\u8bcd\u548c\u526f\u8bcd":64,"\u5982\u60f3\u4e86\u89e3\u66f4\u591a\u8be6\u7ec6\u7684\u89e3\u91ca":65,"\u5982\u672c\u4f8b\u4e2d":3,"\u5982\u672c\u4f8b\u7684":3,"\u5982\u679c\u4e00\u4e2a\u7f51\u7edc\u5c42\u9700\u8981\u914d\u7f6e\u7684\u8bdd":41,"\u5982\u679c\u4e0b\u8f7d\u6210\u529f":58,"\u5982\u679c\u4e0d\u4e3a0":47,"\u5982\u679c\u4e0d\u4e86\u89e3":3,"\u5982\u679c\u4e0d\u5207\u8bcd":60,"\u5982\u679c\u4e0d\u6536\u655b":27,"\u5982\u679c\u4e0d\u662f\u5e8f\u5217":62,"\u5982\u679c\u4e3a":3,"\u5982\u679c\u4e3a0":47,"\u5982\u679c\u4e3afals":47,"\u5982\u679c\u4e3atrue":[3,47],"\u5982\u679c\u4e4b\u540e\u60f3\u8981\u91cd\u65b0\u8bbe\u7f6e":29,"\u5982\u679c\u4ed4\u7ec6\u8bbe\u7f6e\u7684\u8bdd":47,"\u5982\u679c\u4f60\u4e0d\u9700\u8981\u8fd9\u4e2a\u64cd\u4f5c":64,"\u5982\u679c\u4f60\u53ea\u9700\u8981\u4f7f\u7528\u7b80\u5355\u7684rnn":39,"\u5982\u679c\u4f60\u5b89\u88c5gpu\u7248\u672c\u7684paddlepaddl":64,"\u5982\u679c\u4f60\u60f3\u4f7f\u7528\u8fd9\u4e9b\u7279\u6027":49,"\u5982\u679c\u4f60\u60f3\u8981\u4fdd\u5b58\u67d0\u4e9b\u5c42\u7684\u7279\u5f81\u56fe":47,"\u5982\u679c\u4f60\u60f3\u8fdb\u884c\u8bf8\u5982\u8bed\u4e49\u8f6c\u8ff0":65,"\u5982\u679c\u4f60\u6267\u884c\u5176\u5b83\u7684\u7528\u60c5\u611f\u5206\u6790\u6765\u5206\u7c7b\u6587\u672c\u7684\u4efb\u52a1":64,"\u5982\u679c\u4f60\u6b63\u5728\u5904\u7406\u5e8f\u5217\u6807\u8bb0\u4efb\u52a1":39,"\u5982\u679c\u4f60\u6ca1\u6709gpu\u73af\u5883":57,"\u5982\u679c\u4f60\u8981\u4e3a\u4e86\u6d4b\u8bd5\u800c\u589e\u52a0\u65b0\u7684\u6587\u4ef6":41,"\u5982\u679c\u4f7f\u7528":[45,56],"\u5982\u679c\u4f7f\u7528gpu\u7248\u672c\u7684paddlepaddl":32,"\u5982\u679c\u4f7f\u7528nvidia":30,"\u5982\u679c\u4f7f\u7528ssl\u8ba4\u8bc1":50,"\u5982\u679c\u4f7f\u7528swig\u6211\u4eec\u9700\u8981\u5c06\u5728interface\u6587\u4ef6\u91cc":25,"\u5982\u679c\u51fa\u73b0\u4ee5\u4e0bpython\u76f8\u5173\u7684\u5355\u5143\u6d4b\u8bd5\u90fd\u8fc7\u4e0d\u4e86\u7684\u60c5\u51b5":27,"\u5982\u679c\u53c2\u6570\u4fdd\u5b58\u4e0b\u6765\u7684\u6a21\u578b\u76ee\u5f55":27,"\u5982\u679c\u53c2\u6570\u6a21\u578b\u6587\u4ef6\u7f3a\u5931":56,"\u5982\u679c\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u672a\u8bbe\u7f6easync":47,"\u5982\u679c\u5728\u8bad\u7ec3\u671f\u95f4\u540c\u65f6\u53d1\u8d77\u53e6\u5916\u4e00\u4e2a\u8fdb\u7a0b\u8fdb\u884c\u6d4b\u8bd5":47,"\u5982\u679c\u5728\u8bad\u7ec3\u914d\u7f6e\u4e2d\u8bbe\u7f6ebatch":47,"\u5982\u679c\u5728\u8bad\u7ec3nlp\u76f8\u5173\u6a21\u578b\u65f6":27,"\u5982\u679c\u5b83\u4f4d\u4e8e\u8c13\u8bcd\u4e0a\u4e0b\u6587\u533a\u57df\u4e2d":63,"\u5982\u679c\u5c06\u8fd9\u4e2a\u5185\u5b58\u6c60\u51cf\u5c0f":27,"\u5982\u679c\u5df2\u5b89\u88c5":63,"\u5982\u679c\u5df2\u7ecf\u6709pod\u8fd0\u884c":52,"\u5982\u679c\u5f00\u542f\u4f1a\u5bfc\u81f4\u8fd0\u884c\u7565\u6162":29,"\u5982\u679c\u60a8":30,"\u5982\u679c\u60a8\u60f3\u8981\u66f4\u6df1\u5165\u4e86\u89e3deep":30,"\u5982\u679c\u60a8\u6709\u597d\u7684\u5efa\u8bae\u6765":62,"\u5982\u679c\u60a8\u7684gpu\u7406\u8bba\u53ef\u4ee5\u8fbe\u52306":44,"\u5982\u679c\u60f3\u4e3a\u4e00\u4e2a\u6570\u636e\u6587\u4ef6\u8fd4\u56de\u591a\u6761\u6837\u672c":3,"\u5982\u679c\u60f3\u4f7f\u7528\u53ef\u89c6\u5316\u7684\u5206\u6790\u5668":44,"\u5982\u679c\u60f3\u5f88\u597d\u7684\u7406\u89e3\u7a0b\u5e8f\u7684\u884c\u4e3a":44,"\u5982\u679c\u60f3\u8981\u4e86\u89e3\u53cc\u5c42rnn\u5728\u5177\u4f53\u95ee\u9898\u4e2d\u7684\u4f7f\u7528":36,"\u5982\u679c\u60f3\u8981\u542f\u7528paddlepaddle\u7684\u5185\u7f6e\u5b9a\u65f6\u5668":44,"\u5982\u679c\u6211\u77e5\u9053\u5185\u6838\u82b1\u4e8610ms\u6765\u79fb\u52a81gb\u6570\u636e":44,"\u5982\u679c\u6267\u884c\u5931\u8d25":50,"\u5982\u679c\u6267\u884c\u6210\u529f":58,"\u5982\u679c\u6570\u636e\u6587\u4ef6\u5b58\u4e8e\u672c\u5730\u78c1\u76d8":2,"\u5982\u679c\u6570\u6910\u83b7\u53d6\u6210\u529f":64,"\u5982\u679c\u662f\u4f7f\u7528\u975essl\u65b9\u5f0f\u8bbf\u95ee":50,"\u5982\u679c\u662f\u5e8f\u5217":62,"\u5982\u679c\u6709\u591a\u4e2a\u8f93\u5165":38,"\u5982\u679c\u6709\u591a\u4e2a\u8f93\u5165\u5e8f\u5217":38,"\u5982\u679c\u6709\u66f4\u590d\u6742\u7684\u4f7f\u7528":2,"\u5982\u679c\u6709\u9700\u8981\u4fee\u6539\u7684\u5730\u65b9":40,"\u5982\u679c\u672a\u8bbe\u7f6e":47,"\u5982\u679c\u672a\u8bbe\u7f6egpu":49,"\u5982\u679c\u67d0\u4e00\u5757\u6839\u672c\u5c31\u4e0d\u600e\u4e48\u8017\u65f6":44,"\u5982\u679c\u68c0\u67e5\u5230\u5206\u914d\u5728\u4e0d\u540c\u53c2\u6570\u670d\u52a1\u5668\u4e0a\u7684\u53c2\u6570\u7684\u5206\u5e03\u4e0d\u5747\u5300\u6b21\u6570\u5927\u4e8echeck":47,"\u5982\u679c\u6ca1\u6709\u5b9a\u4e49memori":38,"\u5982\u679c\u6ca1\u6709\u8bbe\u7f6e":65,"\u5982\u679c\u6ca1\u6709\u8bbe\u7f6etest":2,"\u5982\u679c\u6d88\u606f\u6570\u636e\u592a\u5c0f":47,"\u5982\u679c\u7528\u6237\u4e0d\u663e\u793a\u6307\u5b9a\u8fd4\u56de\u6570\u636e\u7684\u5bf9\u5e94\u5173\u7cfb":3,"\u5982\u679c\u7528\u6237\u60f3\u8981\u4e86\u89e3\u8be6\u7ec6\u7684\u6570\u636e\u96c6\u7684\u683c\u5f0f":56,"\u5982\u679c\u7528\u6237\u60f3\u8981\u81ea\u5b9a\u4e49\u521d\u59cb\u5316\u65b9\u5f0f":27,"\u5982\u679c\u7528\u6237\u8981\u628apaddle\u7684\u9759\u6001\u5e93":25,"\u5982\u679c\u771f\u60f3\u6316\u6398\u5185\u6838\u6df1\u5904\u7684\u67d0\u4e2a\u79d8\u5bc6":44,"\u5982\u679c\u7a0b\u5e8f\u5d29\u6e83\u4f60\u4e5f\u53ef\u4ee5\u624b\u52a8\u7ec8\u6b62":45,"\u5982\u679c\u7cfb\u7edf\u5b89\u88c5\u4e86\u591a\u4e2apython\u7248\u672c":27,"\u5982\u679c\u7f51\u7edc\u5c42\u4e0d\u9700\u8981\u8fdc\u7a0b\u7a00\u758f\u66f4\u65b0":41,"\u5982\u679c\u7f51\u7edc\u67b6\u6784\u7b80\u5355":39,"\u5982\u679c\u8981\u4f7f\u7528\u53cc\u5411lstm":64,"\u5982\u679c\u8981\u4f7f\u7528gpu":30,"\u5982\u679c\u8981\u542f\u7528gpu":45,"\u5982\u679c\u8981\u8fd0\u884c\u6240\u6709\u7684\u5355\u5143\u6d4b\u8bd5":40,"\u5982\u679c\u8bad\u7ec3\u4e00\u4e2apass":27,"\u5982\u679c\u8bad\u7ec3\u8fc7\u7a0b\u542f\u52a8\u6210\u529f\u7684\u8bdd":62,"\u5982\u679c\u8bad\u7ec3\u8fc7\u7a0b\u7684\u7684cost\u660e\u663e\u9ad8\u4e8e\u8fd9\u4e2a\u5e38\u6570\u8f93\u51fa\u7684cost":27,"\u5982\u679c\u8bbe\u7f6e":3,"\u5982\u679c\u8bbe\u7f6e\u8be5\u53c2\u6570":47,"\u5982\u679c\u8c03\u7528\u9759\u6001\u5e93\u53ea\u80fd\u5c06\u9759\u6001\u5e93\u4e0e\u89e3\u91ca\u5668\u94fe\u63a5":25,"\u5982\u679c\u8f93\u51fa\u662fno":30,"\u5982\u679c\u8fd0\u884c\u6210\u529f":[58,64],"\u5982\u679c\u96c6\u7fa4\u8282\u70b9\u6570\u91cf\u5c11":45,"\u5982\u679c\u9700\u8981\u5305\u542b\u66f4\u591a\u7684\u4f9d\u8d56":30,"\u5982\u679c\u9700\u8981\u6269\u5927\u77e9\u9635":41,"\u5982\u679c\u9700\u8981\u7f29\u51cf\u77e9\u9635":41,"\u5982\u679clearning_rate\u592a\u5927":27,"\u5982\u679clearning_rate\u592a\u5c0f":27,"\u5982\u679cpaddlepaddle\u5305\u5df2\u7ecf\u5728python\u7684sit":27,"\u5982\u6bcf\u4e2a\u6587\u4ef6\u53ea\u6709\u4e00\u4e2a":40,"\u5982\u795e\u7ecf\u5143\u6fc0\u6d3b\u503c\u7b49":27,"\u5982\u8981build\u8fd9\u4e2a\u5f00\u53d1\u955c\u50cf":40,"\u5982\u9ad8\u4eae\u90e8\u5206":44,"\u5b50":36,"\u5b50\u53e5":38,"\u5b50\u53e5\u7684\u5355\u8bcd\u6570\u548c\u6307\u5b9a\u7684\u4e00\u4e2a\u8f93\u5165\u5e8f\u5217\u4e00\u81f4":38,"\u5b57\u5178":65,"\u5b57\u5178\u4f1a\u5305\u542b\u8f93\u5165\u6570\u636e\u96c6\u4e2d\u7684\u6240\u6709\u5355\u8bcd":65,"\u5b57\u5178\u5171\u5305\u542b":56,"\u5b57\u5178\u6587\u4ef6":[63,64],"\u5b57\u5178\u91c7\u7528utf8\u7f16\u7801":56,"\u5b57\u5178imdb":64,"\u5b57\u6bb5\u4e2d":52,"\u5b57\u6bb5\u8868\u793a\u5bb9\u5668\u7684\u73af\u5883\u53d8\u91cf":52,"\u5b57\u6bb5\u8868\u793a\u8fd9\u4e2ajob\u4f1a\u540c\u65f6\u5f00\u542f3\u4e2apaddlepaddle\u8282\u70b9":52,"\u5b58\u50a8\u5377":50,"\u5b58\u50a8\u5728\u8bb0\u5fc6\u5355\u5143\u533a\u5757\u7684\u5386\u53f2\u4fe1\u606f\u88ab\u66f4\u65b0\u7528\u6765\u8fed\u4ee3\u7684\u5b66\u4e60\u5355\u8bcd\u4ee5\u5408\u7406\u7684\u5e8f\u5217\u7a0b\u73b0":64,"\u5b58\u50a8\u6a21\u578b\u7684\u8def\u5f84":65,"\u5b58\u50a8\u7740\u7535\u5f71\u6216\u7528\u6237\u4fe1\u606f":62,"\u5b58\u5165settings\u5bf9\u8c61":3,"\u5b58\u5728\u6216\u66f4\u6539\u4e3a\u5176\u5b83\u6a21\u578b\u8def\u5f84":64,"\u5b66\u4e60\u6210\u672c\u9ad8":25,"\u5b66\u4e60\u7b97\u6cd5":28,"\u5b66\u672f":61,"\u5b81\u6ce2":36,"\u5b83\u4e0d\u4ec5\u80fd\u591f\u5904\u7406imdb\u6570\u636e":64,"\u5b83\u4eec\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4f5c\u4e3a\u7f51\u7edc\u7684\u51fa\u53e3":28,"\u5b83\u4eec\u7684\u5927\u5c0f\u662f":39,"\u5b83\u4eec\u8fd8\u53ef\u4ee5\u4f9b\u90a3\u4e9b\u8fd0\u884c\u66f4\u590d\u6742\u7684\u96c6\u7fa4\u7ba1\u7406\u7cfb\u7edf":45,"\u5b83\u4eec\u90fd\u662f\u5e8f\u5217":39,"\u5b83\u4f1a\u5728dataprovider\u521b\u5efa\u7684\u65f6\u5019\u6267\u884c":3,"\u5b83\u4f7f\u752850\u5c42\u7684resnet\u6a21\u578b\u6765\u5bf9":58,"\u5b83\u5305\u542b\u4ee5\u4e0b\u51e0\u6b65":41,"\u5b83\u5305\u542b\u4ee5\u4e0b\u53c2\u6570":41,"\u5b83\u5305\u542b\u56db\u4e2a\u7248\u672c":32,"\u5b83\u5305\u542b\u7684\u5c5e\u6027\u53c2\u6570\u5982\u4e0b":3,"\u5b83\u5305\u62ec\u4e86\u4e00\u4e2a\u53cc\u5411\u7684gru\u4f5c\u4e3a\u7f16\u7801\u5668\u548c\u89e3\u7801\u5668":65,"\u5b83\u53eb\u505a":39,"\u5b83\u53ef\u4ee5\u5728\u53e5\u5b50\u7ea7\u522b\u5229\u7528\u53ef\u6269\u5c55\u7684\u4e0a\u4e0b\u6587":64,"\u5b83\u53ef\u4ee5\u5e2e\u52a9\u51cf\u5c11\u5206\u53d1\u5ef6\u8fdf":45,"\u5b83\u53ef\u4ee5\u5e2e\u52a9\u6211\u4eec\u683c\u5f0f\u5316\u6e90\u4ee3\u7801":40,"\u5b83\u53ef\u4ee5\u6307\u6d4b\u91cf\u4e00\u4e2a\u7a0b\u5e8f\u7684\u7a7a\u95f4":44,"\u5b83\u53ef\u4ee5\u88ab\u5e94\u7528\u4e8e\u8fdb\u884c\u673a\u5668\u7ffb\u8bd1":65,"\u5b83\u53ef\u80fd\u6709\u4e0d\u6b62\u4e00\u4e2a\u6743\u91cd":41,"\u5b83\u540c\u65f6\u5b66\u4e60\u6392\u5217":65,"\u5b83\u548c\u6570\u636e\u4f20\u5165\u51fd\u6570\u7684\u7b2c\u4e00\u4e2a\u53c2\u6570":3,"\u5b83\u5b58\u50a8\u5f53\u524d\u8282\u70b9\u6240\u6709\u8bad\u7ec3":45,"\u5b83\u5b9a\u4e49\u4e86":39,"\u5b83\u5b9a\u4e49\u4e86\u6a21\u578b\u53c2\u6570\u6539\u53d8\u7684\u89c4\u5219":28,"\u5b83\u5b9a\u4e49\u89e3\u7801\u7f51\u7edc\u7684":39,"\u5b83\u5c06\u88ab\u5206\u53d1\u5230":45,"\u5b83\u5c06\u8f93\u5165\u8bed\u53e5\u7f16\u7801\u4e3a\u5411\u91cf\u7684\u5e8f\u5217":65,"\u5b83\u5c06\u8fd4\u56de\u5982\u4e0b\u7684\u5b57\u5178":58,"\u5b83\u5c31\u4f1a\u5728\u6e90\u8bed\u53e5\u4e2d\u641c\u7d22\u51fa\u6700\u76f8\u5173\u4fe1\u606f\u7684\u4f4d\u7f6e\u7684\u96c6\u5408":65,"\u5b83\u652f\u6301\u591a\u7ebf\u7a0b\u66f4\u65b0":41,"\u5b83\u662finteger_value\u7c7b\u578b\u7684":36,"\u5b83\u662finteger_value_sequence\u7c7b\u578b\u7684":36,"\u5b83\u6709\u52a9\u4e8e\u5e2e\u52a9\u9891\u7e41\u4fee\u6539\u548c\u8bbf\u95ee\u5de5\u4f5c\u533a\u6587\u4ef6\u7684\u7528\u6237\u51cf\u5c11\u8d1f\u62c5":45,"\u5b83\u6a21\u62df\u4e86\u89e3\u7801\u7ffb\u8bd1\u8fc7\u7a0b\u4e2d\u5728\u6e90\u8bed\u53e5\u4e2d\u7684\u641c\u7d22":65,"\u5b83\u7684":39,"\u5b83\u7684\u6536\u655b\u901f\u5ea6\u6bd4":64,"\u5b83\u7684\u6bcf\u4e00\u4e2a\u5143\u7d20":35,"\u5b83\u7684\u76ee\u7684\u662f\u9884\u6d4b\u5728\u4e00\u4e2a\u5e8f\u5217\u4e2d\u8868\u8fbe\u7684\u60c5\u611f\u6001\u5ea6":64,"\u5b83\u7684\u8f93\u5165\u4e0e\u7ecf\u8fc7\u5b66\u4e60\u7684\u53c2\u6570\u505a\u5185\u79ef\u5e76\u52a0\u4e0a\u504f\u7f6e":41,"\u5b83\u76f4\u63a5\u5b66\u4e60\u6bb5\u843d\u8868\u793a":64,"\u5b83\u80fd\u591f\u4ece\u8bcd\u7ea7\u5230\u5177\u6709\u53ef\u53d8\u4e0a\u4e0b\u6587\u957f\u5ea6\u7684\u4e0a\u4e0b\u6587\u7ea7\u522b\u6765\u603b\u7ed3\u8868\u793a":64,"\u5b83\u8bfb\u5165\u6570\u636e\u5e76\u5c06\u5b83\u4eec\u4f20\u8f93\u5230\u63a5\u4e0b\u6765\u7684\u7f51\u7edc\u5c42":28,"\u5b83\u8fd4\u56degen":65,"\u5b83\u8fd4\u56detrain":65,"\u5b83\u9700\u8981\u5728\u8fd9\u91cc\u6307\u5b9a":64,"\u5b83\u9996\u5148\u8c03\u7528\u57fa\u6784\u9020\u51fd\u6570":41,"\u5b89\u6392":36,"\u5b89\u88c5\u4e0e\u7f16\u8bd1":34,"\u5b89\u88c5\u5305\u7684\u4e0b\u8f7d\u5730\u5740\u662f":32,"\u5b89\u88c5\u597ddocker\u4e4b\u540e\u53ef\u4ee5\u4f7f\u7528\u6e90\u7801\u76ee\u5f55\u4e0b\u7684\u811a\u672c\u6784\u5efa\u6587\u6863":42,"\u5b89\u88c5\u5b8c\u6210\u540e":32,"\u5b89\u88c5\u6587\u6863":33,"\u5b89\u88c5\u6d41\u7a0b":60,"\u5b89\u88c5\u8be5\u8f6f\u4ef6\u5305\u5c31\u53ef\u4ee5\u5728python\u73af\u5883\u4e0b\u5b9e\u73b0\u6a21\u578b\u9884\u6d4b":5,"\u5b89\u88c5paddlepaddl":60,"\u5b89\u88c5pillow":57,"\u5b89\u9759":36,"\u5b8c\u6210":40,"\u5b8c\u6210\u4efb\u610f\u7684\u8fd0\u7b97\u903b\u8f91":38,"\u5b8c\u6210\u540evolume\u4e2d\u7684\u6587\u4ef6\u5185\u5bb9\u5927\u81f4\u5982\u4e0b":52,"\u5b8c\u6210\u5f00\u53d1":30,"\u5b8c\u6210\u76f8\u5e94\u7684\u8ba1\u7b97":35,"\u5b8c\u6210paddlepaddle\u7684\u5b89\u88c5":33,"\u5b8c\u6574\u6e90\u7801\u53ef\u53c2\u8003":27,"\u5b8c\u6574\u7684\u4ee3\u7801\u89c1":5,"\u5b8c\u6574\u7684\u53c2\u6570\u77e9\u9635\u88ab\u5206\u5e03\u5728\u4e0d\u540c\u7684\u53c2\u6570\u670d\u52a1\u5668\u4e0a":41,"\u5b8c\u6574\u7684\u6570\u636e\u63d0\u4f9b\u6587\u4ef6\u5728":39,"\u5b8c\u6574\u7684\u914d\u7f6e\u6587\u4ef6\u5728":39,"\u5b98\u65b9\u6587\u6863":29,"\u5b9a\u4e49\u4e00\u4e2a\u65f6\u95f4\u6b65\u4e4b\u5185rnn\u5355\u5143\u5b8c\u6210\u7684\u8ba1\u7b97":38,"\u5b9a\u4e49\u4e00\u4e2apython\u7684":3,"\u5b9a\u4e49\u4e86\u4e00\u4e2a\u53ea\u8bfb\u7684memori":38,"\u5b9a\u4e49\u4e86\u7f51\u7edc\u7684\u6570\u636e\u69fd":63,"\u5b9a\u4e49\u4e86\u7f51\u7edc\u7ed3\u6784":57,"\u5b9a\u4e49\u4e86\u7f51\u7edc\u7ed3\u6784\u5e76\u4fdd\u5b58\u4e3a":28,"\u5b9a\u4e49\u5728\u5916\u5c42":38,"\u5b9a\u4e49\u5f02\u6b65\u8bad\u7ec3\u7684\u957f\u5ea6":47,"\u5b9a\u4e49\u6570\u636e\u6765\u6e90":28,"\u5b9a\u4e49\u6e90\u8bed\u53e5\u7684\u6570\u636e\u5c42":39,"\u5b9a\u4e49\u89e3\u7801\u5668\u7684memori":39,"\u5b9a\u4e49\u8bad\u7ec3\u6570\u6910\u548c\u6d4b\u8bd5\u6570\u6910\u63d0\u4f9b\u8005":64,"\u5b9a\u4e49\u8f93\u5165\u6570\u636e\u7684\u7c7b\u578b":28,"\u5b9a\u4e49\u8f93\u51fa\u51fd\u6570":39,"\u5b9a\u4e49\u95e8\u63a7\u5faa\u73af\u5355\u5143\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u5355\u6b65\u51fd\u6570":39,"\u5b9d\u5854\u7684\u5e95\u7aef\u9700\u8981\u575a\u5b9e\u7684\u57fa\u5ea7\u6765\u652f\u6491":33,"\u5b9e\u4f8b\u5982\u4e0b":63,"\u5b9e\u73b0\u4e24\u4e2a\u5b8c\u5168\u7b49\u4ef7\u7684\u5168\u8fde\u63a5rnn":36,"\u5b9e\u73b0\u524d\u5411\u4f20\u64ad\u7684\u90e8\u5206\u6709\u4e0b\u9762\u51e0\u4e2a\u6b65\u9aa4":41,"\u5b9e\u73b0\u5355\u6b65\u51fd\u6570":39,"\u5b9e\u73b0\u540e\u5411\u4f20\u64ad\u7684\u90e8\u5206\u6709\u4e0b\u9762\u51e0\u4e2a\u6b65\u9aa4":41,"\u5b9e\u73b0\u6570\u636e\u8f93\u5165\u51fd\u6570":3,"\u5b9e\u73b0\u6784\u9020\u51fd\u6570":41,"\u5b9e\u73b0\u7b80\u5355":25,"\u5b9e\u73b0\u7ec6\u8282":41,"\u5b9e\u73b0\u7f51\u7edc\u5c42\u7684\u524d\u5411\u4f20\u64ad":41,"\u5b9e\u73b0\u7f51\u7edc\u5c42\u7684\u540e\u5411\u4f20\u64ad":41,"\u5b9e\u73b0\u8bcd\u8bed\u548c\u53e5\u5b50\u4e24\u4e2a\u7ea7\u522b\u7684\u53cc\u5c42rnn\u7ed3\u6784":38,"\u5b9e\u73b0\u8be5\u5c42\u7684c":41,"\u5b9e\u9645\u4e0a\u53ea\u6709":58,"\u5b9e\u9645\u4e0a\u662fcsv\u6587\u4ef6":61,"\u5ba2\u6237":36,"\u5ba2\u6237\u670d\u52a1":61,"\u5bb6":36,"\u5bb9\u5668":50,"\u5bb9\u5668\u4e0d\u4f1a\u4fdd\u7559\u5728\u8fd0\u884c\u65f6\u751f\u6210\u7684\u6570\u636e":50,"\u5bb9\u5668\u8fd0\u884c\u90fd\u8fd0\u884c":52,"\u5bbf\u4e3b\u673a\u76ee\u5f55":50,"\u5bc4\u5b58\u5668\u4f7f\u7528\u60c5\u51b5\u548c\u5171\u4eab\u5185\u5b58\u4f7f\u7528\u60c5\u51b5\u80fd\u8ba9\u6211\u4eec\u5bf9gpu\u7684\u6574\u4f53\u4f7f\u7528\u6709\u66f4\u597d\u7684\u7406\u89e3":44,"\u5bf9":36,"\u5bf9\u4e00\u4e2a5\u7ef4\u975e\u5e8f\u5217\u7684\u7a00\u758f01\u5411\u91cf":[3,33],"\u5bf9\u4e00\u4e2a5\u7ef4\u975e\u5e8f\u5217\u7684\u7a00\u758f\u6d6e\u70b9\u5411\u91cf":[3,33],"\u5bf9\u4e8e":39,"\u5bf9\u4e8e\u4e0d\u540c\u8bed\u8a00":25,"\u5bf9\u4e8e\u4e24\u79cd\u4e0d\u540c\u7684\u8f93\u5165\u6570\u636e\u7c7b\u578b":36,"\u5bf9\u4e8e\u5185\u5b58\u8f83\u5c0f\u7684\u673a\u5668":3,"\u5bf9\u4e8e\u5355\u5c42rnn":36,"\u5bf9\u4e8e\u5355\u5c42rnn\u7684\u6570\u636e\u4e00\u5171\u6709\u4e24\u4e2a\u6837\u672c":36,"\u5bf9\u4e8e\u53cc\u5c42rnn":36,"\u5bf9\u4e8e\u540c\u4e00\u6bb5c":25,"\u5bf9\u4e8e\u540c\u6837\u7684\u6570\u636e":36,"\u5bf9\u4e8e\u591a\u8bed\u8a00\u63a5\u53e3":25,"\u5bf9\u4e8e\u5927\u591a\u6570\u8bed\u8a00":25,"\u5bf9\u4e8e\u6211\u4eec\u652f\u6301\u7684\u5168\u90e8\u77e9\u9635\u64cd\u4f5c":41,"\u5bf9\u4e8e\u6811\u7684\u6bcf\u4e00\u5c42":65,"\u5bf9\u4e8e\u6bb5\u843d\u7684\u6587\u672c\u5206\u7c7b":36,"\u5bf9\u4e8e\u6bcf\u4e00\u4e2a\u5355\u5c42rnn\u7684\u6570\u636e":36,"\u5bf9\u4e8e\u6bcf\u4f4d\u7528\u6237":62,"\u5bf9\u4e8e\u7ed9\u5b9a\u7684\u4e00\u6761\u6587\u672c":60,"\u5bf9\u4e8e\u914d\u5907\u6709\u6ce8\u610f\u529b\u673a\u5236\u7684\u89e3\u7801\u5668":39,"\u5bf9\u4e8eamazon":60,"\u5bf9\u4ee3\u7801\u8fdb\u884c\u6027\u80fd\u5206\u6790":44,"\u5bf9\u5168\u8fde\u63a5\u5c42\u6765\u8bf4":41,"\u5bf9\u56fe\u7247\u8fdb\u884c\u9884\u5904\u7406":57,"\u5bf9\u5e94\u4e00\u4e2a\u5b50\u53e5":38,"\u5bf9\u5e94\u4e00\u4e2a\u8bcd":38,"\u5bf9\u5e94\u4e8e\u5b57\u5178":56,"\u5bf9\u5e94\u7684":3,"\u5bf9\u6570\u636e\u96c6\u8fdb\u884c\u9884\u5904\u7406\u7684\u57fa\u672c\u547d\u4ee4\u662f":65,"\u5bf9\u6574\u4e2a\u65b0\u5411\u91cf\u96c6\u5408\u7684\u6bcf\u4e00\u4e2a\u7ef4\u5ea6\u53d6\u6700\u5927\u503c\u6765\u8868\u793a\u6700\u540e\u7684\u53e5\u5b50":60,"\u5bf9\u6587\u6863\u5904\u7406\u540e\u5f62\u6210\u7684\u5355\u8bcd\u5411\u91cf":64,"\u5bf9\u673a\u5668\u7ffb\u8bd1\u7684\u4eba\u5de5\u8bc4\u4f30\u5de5\u4f5c\u5f88\u5e7f\u6cdb\u4f46\u4e5f\u5f88\u6602\u8d35":65,"\u5bf9\u6bcf\u4e2a\u8f93\u5165":41,"\u5bf9\u6bcf\u4e2a\u8f93\u5165\u4e58\u4e0a\u53d8\u6362\u77e9\u9635":41,"\u5bf9\u6bd4":25,"\u5bf9\u6fc0\u6d3b\u6c42\u5bfc":41,"\u5bf9\u7528\u6237\u6765\u8bf4":3,"\u5bf9\u8bad\u7ec3\u6570\u636e\u8fdb\u884cshuffl":3,"\u5bf9\u8be5\u5411\u91cf\u8fdb\u884c\u975e\u7ebf\u6027\u53d8\u6362":60,"\u5bf9\u8c61":27,"\u5bf9\u8c61\u5b58\u50a8\u4e3a\u6587\u4ef6":62,"\u5bf9\u8f93\u51fa\u7684\u5408\u5e76":38,"\u5bf9\u9762":36,"\u5bf9check":3,"\u5bf9sparse_binary_vector\u548csparse_float_vector":[3,33],"\u5bfc\u81f4\u4e86\u6d6e\u70b9\u6570\u6ea2\u51fa":27,"\u5bfc\u81f4\u53c2\u6570\u6536\u655b\u5230\u4e86\u4e00\u4e9b\u5947\u5f02\u7684\u60c5\u51b5":27,"\u5bfc\u81f4\u53c2\u6570\u7d2f\u52a0":27,"\u5bfc\u81f4\u7f16\u8bd1paddlepaddle\u5931\u8d25":27,"\u5bfc\u81f4\u8bad\u7ec3\u65f6\u95f4\u8fc7\u957f":27,"\u5c01\u88c5\u4e86":44,"\u5c01\u88c5\u8be5\u5c42\u7684python\u63a5\u53e3":41,"\u5c06":[3,27,44,60],"\u5c06\u4e0a\u4e00\u65f6\u95f4\u6b65\u6240\u751f\u6210\u7684\u8bcd\u7684\u5411\u91cf\u6765\u4f5c\u4e3a\u5f53\u524d\u65f6\u95f4\u6b65\u7684\u8f93\u5165":39,"\u5c06\u4ed6\u4eec\u79fb\u52a8\u5230\u76ee\u5f55":62,"\u5c06\u4f1a\u81ea\u52a8\u8ba1\u7b97\u51fa\u4e00\u4e2a\u5408\u9002\u7684\u503c":47,"\u5c06\u5176\u8bbe\u7f6e\u6210":27,"\u5c06\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217\u6570\u636e\u5148\u53d8\u6362\u6210\u5355\u5c42\u65f6\u95f4\u5e8f\u5217\u6570\u636e":36,"\u5c06\u542b\u6709\u5b50\u53e5":38,"\u5c06\u542b\u6709\u8bcd\u8bed\u7684\u53e5\u5b50\u5b9a\u4e49\u4e3a\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":38,"\u5c06\u56fe\u7247\u6309\u7167\u4e0a\u8ff0\u7ed3\u6784\u5b58\u50a8\u597d\u4e4b\u540e":57,"\u5c06\u5728":57,"\u5c06\u5728\u8fd0\u884c\u65f6\u62a5\u9519":45,"\u5c06\u5916\u90e8\u7684\u5b58\u50a8\u670d\u52a1\u5728kubernetes\u4e2d\u63cf\u8ff0\u6210\u4e3a\u7edf\u4e00\u7684\u8d44\u6e90\u5f62\u5f0f":50,"\u5c06\u591a\u53e5\u8bdd\u770b\u6210\u4e00\u4e2a\u6574\u4f53\u540c\u65f6\u4f7f\u7528encoder\u538b\u7f29":36,"\u5c06\u591a\u53f0\u673a\u5668\u7684\u6d4b\u8bd5\u7ed3\u679c\u5408\u5e76":47,"\u5c06\u5927\u91cf\u7684":25,"\u5c06\u5b57\u5178\u7684\u5730\u5740\u4f5c\u4e3aargs\u4f20\u7ed9dataprovid":27,"\u5c06\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u524d\u5411\u548c\u53cd\u5411\u90e8\u5206\u6df7\u5408\u5728\u4e00\u8d77":39,"\u5c06\u6570\u636e\u5904\u7406\u6210\u89c4\u8303\u683c\u5f0f":56,"\u5c06\u6570\u636e\u7ec4\u5408\u6210batch\u8fdb\u884c\u8bad\u7ec3":3,"\u5c06\u6570\u636e\u8f6c\u6362\u4e3apaddle\u7684\u683c\u5f0f":57,"\u5c06\u65b0\u5efa\u7684\u6743\u91cd\u52a0\u5165\u6743\u91cd\u8868":41,"\u5c06\u65e5\u5fd7\u5199\u5165\u6587\u4ef6":62,"\u5c06\u672c\u5730\u7684\u4fee\u6539\u63a8\u9001\u5230":40,"\u5c06\u6837\u672c\u7684\u5730\u5740\u653e\u5165\u53e6\u4e00\u4e2a\u6587\u672c\u6587\u4ef6":3,"\u5c06\u6b64\u76ee\u5f55\u6302\u8f7d\u4e3a\u5bb9\u5668\u7684":52,"\u5c06\u6bcf\u4e2a\u6e90\u8bed\u8a00\u5230\u76ee\u6807\u8bed\u8a00\u7684\u5e73\u884c\u8bed\u6599\u5e93\u6587\u4ef6\u5408\u5e76\u4e3a\u4e00\u4e2a\u6587\u4ef6":65,"\u5c06\u73af\u5883\u53d8\u91cf\u8f6c\u6362\u6210paddle\u7684\u547d\u4ee4\u884c\u53c2\u6570":52,"\u5c06\u7ed3\u679c\u4fdd\u5b58\u5230\u6b64\u76ee\u5f55\u91cc":52,"\u5c06\u884c\u4e2d\u7684\u6570\u636e\u8f6c\u6362\u6210\u4e0einput_types\u4e00\u81f4\u7684\u683c\u5f0f":3,"\u5c06\u88ab\u5206\u4e3a":56,"\u5c06\u8bad\u7ec3\u6587\u4ef6\u4e0e\u5207\u5206\u597d\u7684\u6570\u636e\u4e0a\u4f20\u5230\u5171\u4eab\u5b58\u50a8":52,"\u5c06\u8be5\u53e5\u8bdd\u5305\u542b\u7684\u6240\u6709\u5355\u8bcd\u5411\u91cf\u6c42\u5e73\u5747":60,"\u5c06\u8df3\u8fc7\u5206\u53d1\u9636\u6bb5\u76f4\u63a5\u542f\u52a8\u6240\u6709\u8282\u70b9\u7684\u96c6\u7fa4\u4f5c\u4e1a":45,"\u5c06\u8fd9\u79cd\u8de8\u8d8a\u65f6\u95f4\u6b65\u7684\u8fde\u63a5\u7528\u4e00\u4e2a\u7279\u6b8a\u7684\u795e\u7ecf\u7f51\u7edc\u5355\u5143\u5b9e\u73b0":36,"\u5c06\u8fdc\u7a0b\u4ed3\u5e93":40,"\u5c06\u900f\u660e":45,"\u5c06ip\u6392\u5e8f\u751f\u6210\u7684\u5e8f\u53f7\u4f5c\u4e3atrain":52,"\u5c11\u4e8e5":45,"\u5c1a\u53ef":36,"\u5c31":36,"\u5c31\u4f1a\u751f\u6210\u975e\u5e38\u591a\u7684gener":3,"\u5c31\u53ef\u4ee5\u518d\u8fd0\u884c\u53e6\u4e00\u4e2anginx":30,"\u5c31\u53ef\u4ee5\u5c06\u6570\u636e\u4f20\u9001\u7ed9paddlepaddle\u4e86":3,"\u5c31\u53ef\u4ee5\u5c06\u8fd9\u4e9b\u6587\u4ef6\u6301\u4e45\u5316\u5b58\u50a8":50,"\u5c31\u5f88\u5bb9\u6613\u5bfc\u81f4\u5185\u5b58\u8d85\u9650":27,"\u5c31\u662f":36,"\u5c31\u662f\u6a21\u578b\u7684\u53c2\u6570":28,"\u5c31\u662f\u7528\u4e8e\u5c55\u793a\u4e0a\u8ff0\u5206\u6790\u5de5\u5177\u7684\u7528\u6cd5":44,"\u5c31\u80fd\u591f\u5f88\u65b9\u4fbf\u7684\u5b8c\u6210\u6570\u636e\u4e0b\u8f7d\u548c\u76f8\u5e94\u7684\u9884\u5904\u7406\u5de5\u4f5c":60,"\u5c31\u8fd9\u4e48\u7b80\u5355":30,"\u5c31\u901a\u5e38\u7684gpu\u6027\u80fd\u5206\u6790\u6765\u8bf4":44,"\u5c31\u9700\u8981\u5bf9\u8fd9\u4e2a\u7b2c\u4e09\u65b9\u8bed\u8a00\u589e\u52a0\u4e00\u4e9b\u5b9a\u4e49":25,"\u5c31\u9700\u8981\u9009\u62e9\u4f7f\u7528no":30,"\u5c3a\u5bf8":58,"\u5c40\u90e8\u5173\u8054\u6027\u8d28\u548c\u7a7a\u95f4\u4e0d\u53d8\u6027\u8d28":57,"\u5c42\u540e\u5f97\u5230\u6df1\u5ea6":63,"\u5c42\u548c\u8f93\u5165\u7684\u914d\u7f6e":41,"\u5c42\u6743\u91cd":58,"\u5c42\u6b21\u5316\u7684rnn":38,"\u5c42\u7279\u5f81":58,"\u5c42\u7684\u540d\u79f0\u4e0e":39,"\u5c42\u7684\u5927\u5c0f":41,"\u5c42\u7684\u7279\u5f81":58,"\u5c42\u7684\u7c7b\u578b":41,"\u5c42\u7684\u8f93\u5165":63,"\u5c42\u7684\u8f93\u5165\u548c\u8f93\u51fa\u4f5c\u4e3a\u4e0b\u4e00\u4e2a":63,"\u5c42\u7684\u8f93\u51fa\u88ab\u7528\u4f5c\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684":39,"\u5c42\u7ec4\u6210\u4e00\u5bf9":63,"\u5c45\u7136":36,"\u5c55\u793a\u4e86\u4e00\u79cd\u65b9\u6cd5":65,"\u5c55\u793a\u4e86\u4e0a\u8ff0\u7f51\u7edc\u6a21\u578b\u7684\u8bad\u7ec3\u6548\u679c":60,"\u5c55\u793a\u4e86\u5982\u4f55\u5c06\u6bcf\u4e2a\u7279\u5f81\u6620\u5c04\u5230\u4e00\u4e2a\u5411\u91cf":62,"\u5c5e\u6027":63,"\u5d4c\u5165\u5c42":62,"\u5d4c\u5165\u7279\u5f81\u5b57\u5178":62,"\u5d4c\u5165\u7f16\u53f7\u4f1a\u6839\u636e\u5355\u8bcd\u6392\u5e8f":62,"\u5de5\u4f5c\u6a21\u5f0f":47,"\u5de5\u4f5c\u7a7a\u95f4":45,"\u5de5\u4f5c\u7a7a\u95f4\u4e2d\u7684":45,"\u5de5\u4f5c\u7a7a\u95f4\u6839\u76ee\u5f55":45,"\u5de5\u4f5c\u7a7a\u95f4\u76ee\u5f55\u7684\u5de5\u4f5c\u7a7a\u95f4":45,"\u5de5\u4f5c\u7a7a\u95f4\u914d\u7f6e":45,"\u5de5\u5177":64,"\u5de5\u5177\u4e2d\u7684\u811a\u672c":64,"\u5de5\u5177\u6765\u7ba1\u7406":40,"\u5de5\u7a0b\u5e08":61,"\u5de6\u56fe\u6784\u9020\u7f51\u7edc\u6a21\u5757\u7684\u65b9\u5f0f\u88ab\u7528\u4e8e34\u5c42\u7684\u7f51\u7edc\u4e2d":58,"\u5de6\u8fb9\u662f":58,"\u5dee\u8bc4":60,"\u5df2\u6253\u5f00":40,"\u5df2\u7ecf\u5728\u96c6\u7fa4\u63d0\u4ea4\u73af\u5883\u4e2d\u5b8c\u6210\u8bbe\u7f6e":47,"\u5df2\u7ecf\u63d0\u4f9b\u4e86\u811a\u672c\u6765\u5e2e\u52a9\u60a8\u521b\u5efa\u8fd9\u4e24\u4e2a\u6587\u4ef6":45,"\u5e02\u573a":61,"\u5e02\u9762\u4e0a\u5df2\u7ecf\u6709nvidia\u6216\u7b2c\u4e09\u65b9\u63d0\u4f9b\u7684\u4f17\u591a\u5de5\u5177":44,"\u5e0c\u671b\u80fd\u8ba9\u6211\u4eec\u77e5\u6653":62,"\u5e2e\u52a9\u6211\u4eec\u5b8c\u6210\u5bf9\u8f93\u5165\u5e8f\u5217\u7684\u62c6\u5206":38,"\u5e2e\u52a9\u6211\u4eec\u66f4\u597d\u5730\u63cf\u8ff0\u6bb5\u843d":38,"\u5e2e\u52a9\u6211\u4eec\u6784\u9020\u4e00\u4e9b\u590d\u6742\u7684\u8f93\u5165\u4fe1\u606f":35,"\u5e38\u5e38\u51fa\u73b0":27,"\u5e38\u7528\u4f18\u5316\u7b97\u6cd5\u5305\u62ecmomentum":60,"\u5e38\u89c1\u7684\u53ef\u9009\u5b58\u50a8\u670d\u52a1\u5305\u62ec":50,"\u5e72\u51c0":36,"\u5e73\u53f0\u4e3a\u60f3\u89c2\u6d4b\u8bcd\u5411\u91cf\u7684\u7528\u6237\u63d0\u4f9b\u4e86\u5c06\u4e8c\u8fdb\u5236\u8bcd\u5411\u91cf\u6a21\u578b\u8f6c\u6362\u4e3a\u6587\u672c\u6a21\u578b\u7684\u529f\u80fd":56,"\u5e73\u5747\u7279\u5f81\u56fe\u7684\u9ad8\u5ea6\u53ca\u5bbd\u5ea6":57,"\u5e74\u9f84":61,"\u5e74\u9f84\u4ece\u4e0b\u5217\u5217\u8868\u8303\u56f4\u4e2d\u9009\u53d6":61,"\u5e74\u9f84\u548c\u804c\u4e1a":62,"\u5e76\u4e0d\u4fdd\u8bc1":41,"\u5e76\u4e0d\u662f\u4f7f\u7528\u53cc\u5c42rnn\u89e3\u51b3\u5b9e\u9645\u7684\u95ee\u9898":36,"\u5e76\u4e0d\u662fkubernetes\u4e2d\u7684node\u6982\u5ff5":52,"\u5e76\u4e0d\u771f\u6b63\u7684\u548c":36,"\u5e76\u4e14":[3,39],"\u5e76\u4e14\u5185\u5c42\u7684\u5e8f\u5217\u64cd\u4f5c\u4e4b\u95f4\u72ec\u7acb\u65e0\u4f9d\u8d56":36,"\u5e76\u4e14\u5206\u522b\u91cd\u547d\u540d\u6587\u4ef6\u540e\u7f00":65,"\u5e76\u4e14\u52a0\u4e0a\u4e0b\u9762\u7684\u547d\u4ee4\u884c\u53c2\u6570":49,"\u5e76\u4e14\u5305\u62ecunit":40,"\u5e76\u4e14\u53ea\u6709\u4e00\u4e2a\u6743\u91cd":58,"\u5e76\u4e14\u53ef\u80fd\u4f1a\u52a0\u901f\u8bad\u7ec3\u8fc7\u7a0b":27,"\u5e76\u4e14\u540e\u7eed\u4ecd\u5728\u4e0d\u65ad\u6539\u8fdb":28,"\u5e76\u4e14\u542f\u52a8\u8bad\u7ec3":52,"\u5e76\u4e14\u5728\u5185\u5b58\u8db3\u591f\u7684\u60c5\u51b5\u4e0b\u8d8a\u5927\u8d8a\u597d":3,"\u5e76\u4e14\u5728\u5e38\u89c1\u7684\u5e73\u53f0\u4e0a":25,"\u5e76\u4e14\u5728\u968f\u540e\u7684\u8bfb\u53d6\u6570\u636e\u8fc7\u7a0b\u4e2d\u586b\u5145\u8bcd\u8868":60,"\u5e76\u4e14\u5728dataprovider\u4e2d\u5b9e\u73b0\u5982\u4f55\u8bbf\u95ee\u8bad\u7ec3\u6587\u4ef6\u5217\u8868":2,"\u5e76\u4e14\u5b83\u4eec\u7684\u987a\u5e8f\u4e0e":58,"\u5e76\u4e14\u5bf9\u7528\u6237\u7684\u7279\u5f81\u505a\u540c\u6837\u7684\u64cd\u4f5c":62,"\u5e76\u4e14\u5c06\u9884\u5904\u7406\u597d\u7684\u6570\u636e\u96c6\u5b58\u653e\u5728":65,"\u5e76\u4e14\u5c55\u793a\u4e86\u5982\u4f55\u5229\u7528paddlepaddle\u6765\u89e3\u51b3\u4e00\u4e2a\u7ecf\u5178\u7684\u7ebf\u6027\u56de\u5f52\u95ee\u9898":33,"\u5e76\u4e14\u67e5\u8be2paddlepaddle\u5355\u5143\u6d4b\u8bd5\u7684\u65e5\u5fd7":27,"\u5e76\u4e14\u7b2c\u4e8c\u4e2a\u662f\u53cd\u5411lstm":64,"\u5e76\u4e14\u7f16\u8bd1\u80fd\u901a\u8fc7\u4ee3\u7801\u6837\u5f0f\u68c0\u67e5":40,"\u5e76\u4e14\u8ba9\u63a5\u53e3\u8131\u79bb\u5b9e\u73b0\u7ec6\u8282":25,"\u5e76\u4e14\u8f93\u51fa\u4e00\u4e2a":40,"\u5e76\u4e14\u901a\u8fc7\u7ed9\u51fa\u5f53\u524d\u76ee\u6807\u5355\u8bcd\u6765\u9884\u6d4b\u4e0b\u4e00\u4e2a\u76ee\u6807\u5355\u8bcd":65,"\u5e76\u4e14\u96c6\u7fa4\u4f5c\u4e1a\u4e2d\u7684\u6240\u6709\u8282\u70b9\u5c06\u5728\u6b63\u5e38\u60c5\u51b5\u4e0b\u5904\u7406\u5177\u6709\u76f8\u540c\u903b\u8f91\u4ee3\u7801\u7684\u6587\u4ef6":45,"\u5e76\u4e14\u9700\u8981\u91cd\u5199\u57fa\u7c7b\u4e2d\u7684\u4ee5\u4e0b\u51e0\u4e2a\u865a\u51fd\u6570":41,"\u5e76\u4e14softmax\u5c42\u7684\u4e24\u4e2a\u8f93\u5165\u4e5f\u4f7f\u7528\u4e86\u540c\u6837\u7684\u53c2\u6570":27,"\u5e76\u4f20\u5165\u76f8\u5e94\u7684\u547d\u4ee4\u884c\u53c2\u6570\u521d\u59cb\u5316paddlepaddl":5,"\u5e76\u4f7f\u7528":63,"\u5e76\u4f7f\u7528\u4e86dropout":60,"\u5e76\u4f7f\u7528\u8fd9\u4e2a\u795e\u7ecf\u7f51\u7edc\u6765\u5bf9\u56fe\u7247\u8fdb\u884c\u5206\u7c7b":57,"\u5e76\u521b\u5efa\u4e86\u4e00\u4e2a\u65b0\u6587\u4ef6":40,"\u5e76\u521b\u5efaoptim":33,"\u5e76\u5728\u4e58\u79ef\u7ed3\u679c\u4e0a\u518d\u52a0\u4e0a\u7ef4\u5ea6\u4e3a":41,"\u5e76\u5728\u6700\u5f00\u59cb\u521d\u59cb\u5316\u4e3a\u8d77\u59cb\u8bcd":39,"\u5e76\u5728\u7b14\u8bb0\u672c\u4e0a\u901a\u8fc7ssh\u4e0e\u5176\u8fde\u63a5":30,"\u5e76\u5728\u7c7b\u6784\u5efa\u51fd\u6570\u4e2d\u628a\u5b83\u653e\u5165\u4e00\u4e2a\u7c7b\u6210\u5458\u53d8\u91cf\u91cc":41,"\u5e76\u5bf9\u76f8\u5e94\u7684\u53c2\u6570\u8c03\u7528":41,"\u5e76\u5c06\u5176\u6295\u5c04\u5230":39,"\u5e76\u5c06\u5b83\u4eec\u6309\u7167\u542f\u53d1\u4ee3\u4ef7":65,"\u5e76\u5c06\u5b83\u4eec\u653e\u5728":65,"\u5e76\u5c06\u6bcf\u8f6e\u7684\u6a21\u578b\u7ed3\u679c\u5b58\u653e\u5728":28,"\u5e76\u5c06develop\u548ctest\u6570\u636e\u5206\u522b\u653e\u5165\u4e0d\u540c\u7684\u6587\u4ef6\u5939":65,"\u5e76\u60f3\u4f7f\u7528gpu\u6765\u8bad\u7ec3\u8bbe\u7f6e\u4e3atru":64,"\u5e76\u6307\u5b9abatch":65,"\u5e76\u63d0\u4f9b\u4e86\u7b80\u5355\u7684cache\u529f\u80fd":3,"\u5e76\u6b22\u8fce\u60a8\u6765\u53c2\u4e0e\u8d21\u732e":64,"\u5e76\u6ca1\u6709paddle\u7279\u522b\u9700\u8981\u7684\u7279\u6027":25,"\u5e76\u7531":63,"\u5e76\u7ed9\u51fa\u5206\u7c7b\u7ed3\u679c":57,"\u5e76\u7ed9\u51fa\u7684\u76f8\u5173\u6a21\u578b\u683c\u5f0f\u7684\u5b9a\u4e49":56,"\u5e76\u88ab\u53cd\u5411\u5904\u7406":63,"\u5e76\u89c2\u5bdf\u7ed3\u679c":44,"\u5e76\u8bb0\u5f55\u5b83\u7684\u7f16\u53f7":40,"\u5e76\u8bbe\u7f6e":[32,45],"\u5e76\u8fdb\u884c\u521d\u59cb\u5316\u64cd\u4f5c":33,"\u5e76\u9010\u6e10\u5c55\u793a\u66f4\u52a0\u6df1\u5165\u7684\u529f\u80fd":60,"\u5e8a\u4e0a\u7528\u54c1":36,"\u5e8a\u57ab":36,"\u5e8f\u5217\u4e2d\u542b\u6709\u5143\u7d20\u7684\u6570\u76ee\u540c":35,"\u5e8f\u5217\u5230\u5e8f\u5217":65,"\u5e8f\u5217\u6570\u636e\u662f\u81ea\u7136\u8bed\u8a00\u5904\u7406\u4efb\u52a1\u9762\u5bf9\u7684\u4e00\u79cd\u4e3b\u8981\u8f93\u5165\u6570\u636e\u7c7b\u578b":38,"\u5e8f\u5217\u662f\u4e00\u79cd\u5e38\u89c1\u7684\u6570\u636e\u7c7b\u578b":35,"\u5e8f\u5217\u751f\u6210\u4efb\u52a1\u5927\u591a\u9075\u5faaencod":38,"\u5e8f\u5217\u751f\u6210\u4efb\u52a1\u7684\u8f93\u5165":38,"\u5e8f\u5217\u7684\u5f00\u59cb":65,"\u5e8f\u5217\u7684\u6bcf\u4e2a\u5143\u7d20\u662f\u539f\u6765\u53cc\u5c42\u5e8f\u5217\u6bcf\u4e2asubseq\u5143\u7d20\u7684\u5e73\u5747\u503c":35,"\u5e8f\u5217\u7684\u7ed3\u5c3e":65,"\u5e8f\u5217\u7684\u7ed3\u675f":65,"\u5e93":45,"\u5e93\u7684\u8def\u5f84":45,"\u5e94\u7528\u524d\u5411\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":39,"\u5e94\u7528\u53cd\u5411\u9012\u5f52\u795e\u7ecf\u7f51\u7edc":39,"\u5e94\u7528\u6a21\u578b":60,"\u5e94\u8be5":36,"\u5e94\u8be5\u4e0e\u5b83\u7684memory\u540d\u5b57\u76f8\u540c":39,"\u5e94\u8be5\u964d\u4f4e\u5b66\u4e60\u7387":27,"\u5e95\u5c42\u8fdb\u7a0b":45,"\u5efa\u7acb\u4e00\u4e2a":40,"\u5efa\u7acb\u4e00\u4e2a\u6d3b\u8dc3\u7684\u5f00\u6e90\u793e\u533a":0,"\u5efa\u8bae\u5c06\u5176\u8bbe\u7f6e\u4e3a\u8f83\u5927":45,"\u5efa\u8bae\u5c06\u8be5\u53c2\u6570\u8bbe\u4e3atrue":47,"\u5f00\u53d1\u4eba\u5458\u4f7f\u7528":40,"\u5f00\u53d1\u4eba\u5458\u53ef\u4ee5\u5728docker\u5f00\u53d1\u955c\u50cf\u4e2d\u5f00\u53d1paddlepaddl":30,"\u5f00\u53d1\u955c\u50cf":[30,40],"\u5f00\u53d1\u955c\u50cf\u5305\u542b\u4e86\u4ee5\u4e0b\u5de5\u5177":30,"\u5f00\u59cb":28,"\u5f00\u59cb\u6807\u8bb0":39,"\u5f00\u59cb\u8bad\u7ec3\u6a21\u578b":60,"\u5f00\u59cb\u9636\u6bb5":44,"\u5f02\u6b65\u968f\u673a\u68af\u5ea6\u4e0b\u964d":46,"\u5f15\u5165lstm\u6a21\u578b\u4e3b\u8981\u662f\u4e3a\u4e86\u514b\u670d\u6d88\u5931\u68af\u5ea6\u7684\u95ee\u9898":64,"\u5f15\u5165paddlepaddle\u7684pydataprovider2\u5305":3,"\u5f15\u53d1":20,"\u5f15\u5bfc\u5c42":39,"\u5f15\u7528":45,"\u5f15\u7528memory\u5f97\u5230\u8fd9layer\u4e0a\u4e00\u65f6\u523b\u8f93\u51fa":38,"\u5f3a\u70c8\u63a8\u8350":36,"\u5f3a\u70c8\u63a8\u8350\u4f7f\u7528virtualenv\u6765\u521b\u9020\u4e00\u4e2a\u5e72\u51c0\u7684python\u73af\u5883":62,"\u5f52\u4e00\u5316\u6982\u7387\u5411\u91cf":39,"\u5f53":49,"\u5f53\u4f20\u9012\u76f8\u540c\u7684\u6837\u672c\u6570\u65f6":64,"\u5f53\u4f60\u6267\u884c\u547d\u4ee4":41,"\u5f53\u51fd\u6570\u8fd4\u56de\u7684\u65f6\u5019":3,"\u5f53\u524d\u5355\u8bcd\u5728\u76f8\u6bd4\u4e4b\u4e0b\u603b\u662f\u88ab\u5f53\u4f5c\u771f\u503c":65,"\u5f53\u524d\u5355\u8bcd\u662f\u89e3\u7801\u5668\u6700\u540e\u4e00\u6b65\u7684\u8f93\u51fa":65,"\u5f53\u524d\u65f6\u95f4\u6b65\u5904\u7684memory\u7684\u8f93\u51fa\u4f5c\u4e3a\u4e0b\u4e00\u65f6\u95f4\u6b65memory\u7684\u8f93\u5165":39,"\u5f53\u524d\u7684\u5b9e\u73b0\u65b9\u5f0f\u4e0b":41,"\u5f53\u524d\u7684\u8f93\u5165y\u548c\u4e0a\u4e00\u4e2a\u65f6\u95f4\u6b65\u7684\u8f93\u51farnn_state\u505a\u4e86\u4e00\u4e2a\u5168\u94fe\u63a5":36,"\u5f53\u524d\u8bc4\u4f30\u4e2d":65,"\u5f53\u524dbatch\u7684cost":65,"\u5f53\u524dlog_period\u4e2abatch\u6240\u6709\u6837\u672c\u7684\u5e73\u5747\u5206\u7c7b\u9519\u8bef\u7387":60,"\u5f53\u524dlog_period\u4e2abatch\u6240\u6709\u6837\u672c\u7684\u5e73\u5747cost":60,"\u5f53\u5728\u7f51\u7edc\u5c42\u914d\u7f6e\u4e2d\u8bbe\u7f6e":47,"\u5f53\u5728\u8bad\u7ec3\u914d\u7f6e\u4e2d\u8bbe\u7f6e":47,"\u5f53\u5bb9\u5668\u56e0\u4e3a\u5404\u79cd\u539f\u56e0\u88ab\u9500\u6bc1\u65f6":50,"\u5f53\u6240\u6709pod\u90fd\u5904\u4e8erunning\u72b6\u6001":52,"\u5f53\u6a21\u578b\u53c2\u6570\u4e0d\u5b58\u5728\u65f6":47,"\u5f53\u6a21\u578b\u8bad\u7ec3\u597d\u4e86\u4e4b\u540e":60,"\u5f53\u6a21\u5f0f\u4e3a":47,"\u5f53\u7136":44,"\u5f53\u7f51\u7edc\u5c42\u7528\u4e00\u4e2a\u6279\u6b21\u505a\u8bad\u7ec3\u65f6":41,"\u5f53\u89e3\u8bfb\u6bcf\u4e00\u4e2a":39,"\u5f53\u8bad\u7ec3\u6570\u636e\u975e\u5e38\u591a\u65f6":3,"\u5f53\u8d85\u8fc7\u8be5\u9608\u503c\u65f6":47,"\u5f53\u8f93\u5165\u662f\u7ef4\u5ea6\u5f88\u9ad8\u7684\u7a00\u758f\u6570\u636e\u65f6":49,"\u5f53\u9700\u8981\u5feb\u901f\u6216\u8005\u9891\u7e41\u7684\u8bc4\u4f30\u65f6":65,"\u5f53classif":65,"\u5f62\u6210recurr":38,"\u5f62\u6210recurrent\u8fde\u63a5":38,"\u5f62\u72b6":58,"\u5f88":[36,60],"\u5f88\u591a":36,"\u5f88\u591a\u5f00\u53d1\u8005\u4f1a\u4f7f\u7528\u8fdc\u7a0b\u7684\u5b89\u88c5\u6709gpu\u7684\u670d\u52a1\u5668\u5de5\u4f5c":30,"\u5f88\u5b89\u9759":36,"\u5f88\u5e72\u51c0":36,"\u5f88\u65b9\u4fbf":36,"\u5f88\u6709\u53ef\u80fd\u5b9e\u9645\u5e94\u7528\u5c31\u662f\u6ca1\u6709\u6309\u7167\u60a8\u7684\u9884\u671f\u60c5\u51b5\u8fd0\u884c":44,"\u5f88\u9002\u5408\u6784\u5efa\u7528\u4e8e\u7406\u89e3\u56fe\u7247\u5185\u5bb9\u7684\u6a21\u578b":57,"\u5f88\u96be\u4fdd\u8bc1\u591a\u8bed\u8a00\u4ee3\u7801\u98ce\u683c\u7684\u4e00\u81f4\u6027":25,"\u5f88\u96be\u6574\u4f53\u4fee\u6b63":3,"\u5f8b\u5e08":61,"\u5f97":36,"\u5f97\u4f7f\u7528":25,"\u5f97\u5230\u53e5\u5b50\u7684\u8868\u793a":60,"\u5f97\u5230\u6700\u597d\u8f6e\u6b21\u4e0b\u7684\u6a21\u578b":62,"\u5faa\u73af\u5c55\u5f00\u7684\u6bcf\u4e2a\u65f6\u95f4\u6b65\u603b\u662f\u80fd\u591f\u5f15\u7528\u6240\u6709\u8f93\u5165":38,"\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u4e2d":39,"\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u4f5c\u4e3a\u4f7f\u7528":39,"\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u548c":39,"\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u5728\u6bcf\u4e2a\u65f6\u95f4\u6b65\u9aa4\u987a\u5e8f\u5730\u5904\u7406\u5e8f\u5217":39,"\u5faa\u73af\u7f51\u7edc\u4ece":39,"\u5fc5\u987b":41,"\u5fc5\u987b\u4e00\u81f4":3,"\u5fc5\u987b\u4f7f\u7528python\u5173\u952e\u8bcd":3,"\u5fc5\u987b\u5c06\u524d\u4e00\u4e2a\u5b50\u53e5\u7684\u6700\u540e\u4e00\u4e2a\u5143\u7d20":36,"\u5fc5\u987b\u6307\u5411\u4e00\u4e2apaddlepaddle\u5b9a\u4e49\u7684lay":38,"\u5fc5\u987b\u662f\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":38,"\u5fc5\u987b\u662f\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":35,"\u5fc5\u987b\u7531\u53ea\u8bfbmemory\u7684":39,"\u5fd8\u8bb0\u95e8\u548c\u8f93\u51fa\u95e8":64,"\u5feb":[36,64],"\u5feb\u901f\u5165\u95e8":59,"\u5feb\u901f\u5728\u672c\u5730\u542f\u52a8\u4e00\u4e2a\u5355\u673a\u7684kubernetes\u670d\u52a1\u5668":50,"\u5feb\u901f\u90e8\u7f72\u96c6\u7fa4":50,"\u6027\u4ef7\u6bd4":36,"\u6027\u522b":[61,62],"\u6027\u80fd\u5206\u6790":44,"\u6027\u80fd\u5206\u6790\u5de5\u5177\u662f\u7528\u4e8e\u7ed9\u5e94\u7528\u7a0b\u5e8f\u7684\u6027\u80fd\u505a\u5b9a\u91cf\u5206\u6790\u7684":44,"\u6027\u80fd\u5206\u6790\u662f\u6027\u80fd\u4f18\u5316\u7684\u5173\u952e\u4e00\u6b65":44,"\u6027\u80fd\u8c03\u4f18":46,"\u603b\u4f53\u6765\u8bf4":36,"\u603b\u8ba1\u7684\u53c2\u6570\u4e2a\u6570":56,"\u603b\u8bc4\u520610\u5206":64,"\u6050\u6016\u7247":61,"\u60a8\u4f1a\u5728\u63a5\u4e0b\u6765\u7684\u90e8\u5206\u4e2d\u83b7\u5f97\u66f4\u591a\u7684\u7ec6\u8282\u4ecb\u7ecd":44,"\u60a8\u53ef\u4ee5\u4efb\u610f\u4f7f\u7528\u4e00\u4e2a\u6216\u4e24\u4e2a\u6765\u5bf9\u611f\u5174\u8da3\u7684\u4ee3\u7801\u6bb5\u505a\u6027\u80fd\u5206\u6790":44,"\u60a8\u53ef\u4ee5\u5bfc\u5165":44,"\u60a8\u53ef\u4ee5\u91c7\u7528\u4e0b\u9762\u4e94\u4e2a\u6b65\u9aa4":44,"\u60a8\u5c06\u4e86\u89e3\u5982\u4f55":39,"\u60a8\u5c31\u80fd\u83b7\u5f97\u5982\u4e0b\u7684\u5206\u6790\u7ed3\u679c":44,"\u60a8\u6309\u5982\u4e0b\u6b65\u9aa4\u64cd\u4f5c\u5373\u53ef":44,"\u60a8\u6700\u597d\u5148\u786e\u8ba4":44,"\u60a8\u9700\u8981\u66f4\u6539":30,"\u60a8\u9996\u5148\u9700\u8981\u5728\u76f8\u5173\u4ee3\u7801\u6bb5\u4e2d\u52a0\u5165":44,"\u60ac\u7591\u7247":61,"\u60c5\u6001\u52a8\u8bcd":63,"\u60c5\u611f\u5206\u6790":59,"\u60c5\u611f\u5206\u6790\u4e5f\u5e38\u7528\u4e8e\u57fa\u4e8e\u5927\u91cf\u8bc4\u8bba\u548c\u4e2a\u4eba\u535a\u5ba2\u6765\u76d1\u63a7\u793e\u4f1a\u5a92\u4f53":64,"\u60c5\u611f\u5206\u6790\u662f\u81ea\u7136\u8bed\u8a00\u7406\u89e3\u4e2d\u6700\u5178\u578b\u7684\u95ee\u9898\u4e4b\u4e00":64,"\u60c5\u611f\u5206\u6790\u6709\u8bb8\u591a\u5e94\u7528\u573a\u666f":64,"\u60ca\u9669\u7535\u5f71":61,"\u60f3\u4e86\u89e3\u66f4\u591a\u7ec6\u8282\u53ef\u4ee5\u53c2\u8003pydataprovider\u90e8\u5206\u7684\u6587\u6863":64,"\u610f\u5473\u7740\u4e0d\u540c\u65f6\u95f4\u6b65\u7684\u8f93\u5165\u90fd\u662f\u76f8\u540c\u7684\u503c":39,"\u610f\u601d\u662f\u4e0d\u4f7f\u7528\u5e73\u5747\u53c2\u6570\u6267\u884c\u6d4b\u8bd5":47,"\u610f\u601d\u662f\u4e0d\u4fdd\u5b58\u7ed3\u679c":47,"\u610f\u601d\u662f\u4f7f\u7528\u7b2ctest":47,"\u610f\u601d\u662f\u5728gpu\u6a21\u5f0f\u4e0b\u4f7f\u75284\u4e2agpu":47,"\u611f\u89c9":36,"\u620f\u5267":61,"\u6210\u529f\u8bad\u7ec3\u4e14\u9000\u51fa\u7684pod\u6570\u76ee\u4e3a3\u65f6":52,"\u6211\u4eec\u4e0d\u80fd\u901a\u8fc7\u5e38\u89c4\u7684\u68af\u5ea6\u68c0\u67e5\u7684\u65b9\u5f0f\u6765\u8ba1\u7b97\u68af\u5ea6":41,"\u6211\u4eec\u4e3b\u8981\u4f1a\u4ecb\u7ecdnvprof\u548cnvvp":44,"\u6211\u4eec\u4ec5\u4ec5\u5bf9\u795e\u7ecf\u7f51\u7edc\u7684\u8f93\u5165\u8fdb\u884c\u4e86\u63cf\u8ff0":33,"\u6211\u4eec\u4ec5\u4ec5\u662f\u5c06\u6bcf\u4e2a\u7279\u5f81\u79cd\u7c7b\u6620\u5c04\u5230\u4e00\u4e2a\u7279\u5f81\u5411\u91cf\u4e2d":62,"\u6211\u4eec\u4ec5\u6709\u4e00\u4e2a\u8f93\u5165":41,"\u6211\u4eec\u4ec5\u7528":62,"\u6211\u4eec\u4ecb\u7ecd\u5982\u4f55\u5728":51,"\u6211\u4eec\u4ecb\u7ecd\u5982\u4f55\u5728kubernetes\u96c6\u7fa4\u4e0a\u8fdb\u884c\u5206\u5e03\u5f0fpaddlepaddle\u8bad\u7ec3\u4f5c\u4e1a":52,"\u6211\u4eec\u4ece\u63d0\u524d\u7ed9\u5b9a\u7684\u7c7b\u522b\u96c6\u5408\u4e2d\u9009\u62e9\u5176\u6240\u5c5e\u7c7b\u522b":60,"\u6211\u4eec\u4ee5mnist\u624b\u5199\u8bc6\u522b\u4e3a\u4f8b":3,"\u6211\u4eec\u4f1a\u53d1\u73b0\u6570\u636e\u96c6":65,"\u6211\u4eec\u4f1a\u5728":30,"\u6211\u4eec\u4f1a\u7ee7\u7eed\u4f7f\u7528\u73b0\u6709\u7684\u5185\u5b58\u5757":41,"\u6211\u4eec\u4f1a\u91cd\u65b0\u5206\u914d\u5185\u5b58":41,"\u6211\u4eec\u4f7f\u7528":[41,45,64],"\u6211\u4eec\u4f7f\u7528\u4e0d\u540c\u7684layer\u8fdb\u884c\u7ec4\u5408":33,"\u6211\u4eec\u4f7f\u7528\u4e86":36,"\u6211\u4eec\u4f7f\u7528\u4e86\u4e00\u4e2a\u7f16\u89e3\u7801\u6a21\u578b\u7684\u6269\u5c55":65,"\u6211\u4eec\u4f7f\u7528\u4e86\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":64,"\u6211\u4eec\u4f7f\u7528\u5176\u4e2d\u7684":65,"\u6211\u4eec\u4f7f\u7528\u52a8\u6001\u5e93\u6765\u5206\u53d1paddl":25,"\u6211\u4eec\u4f7f\u7528\u6700\u5927\u6982\u7387\u7684\u6807\u7b7e\u4f5c\u4e3a\u7ed3\u679c":63,"\u6211\u4eec\u4f7f\u7528\u96c6\u675f\u641c\u7d22":65,"\u6211\u4eec\u4f7f\u7528paddlepaddle\u5728ilsvrc\u7684\u9a8c\u8bc1\u96c6\u517150":58,"\u6211\u4eec\u5047\u8bbe\u4e00\u53f0\u673a\u5668\u4e0a\u67094\u4e2agpu":49,"\u6211\u4eec\u5047\u8bbe\u623f\u4ea7\u7684\u4ef7\u683c":28,"\u6211\u4eec\u5148\u4ece\u4e00\u6761\u968f\u673a\u7684\u76f4\u7ebf":28,"\u6211\u4eec\u51c6\u5907\u4e86\u4e00\u4e2a\u811a\u672c":57,"\u6211\u4eec\u5373\u53ef\u5b8c\u6210\u795e\u7ecf\u7f51\u7edc\u7684\u642d\u5efa":33,"\u6211\u4eec\u53ea\u4f7f\u7528\u5df2\u7ecf\u6807\u6ce8\u8fc7\u7684\u8bad\u7ec3\u96c6\u548c\u6d4b\u8bd5\u96c6":64,"\u6211\u4eec\u53ea\u6240\u4ee5\u4f7f\u7528lstm\u6765\u6267\u884c\u8fd9\u4e2a\u4efb\u52a1\u662f\u56e0\u4e3a\u5176\u6539\u8fdb\u7684\u8bbe\u8ba1\u5e76\u4e14\u5177\u6709\u95e8\u673a\u5236":64,"\u6211\u4eec\u53ea\u6f14\u793a\u4e00\u4e2a\u5355\u673a\u4f5c\u4e1a":51,"\u6211\u4eec\u53ea\u9700\u8981\u4f7f\u7528lstm":36,"\u6211\u4eec\u53ea\u9700\u8981\u8fd0\u884c":60,"\u6211\u4eec\u53ea\u9700\u8981\u8fd0\u884c\u4e0b\u9762\u547d\u4ee4\u628a\u7f16\u8bd1\u597d\u7684paddlepaddle\u6253\u5305\u6210\u4e00\u4e2a":40,"\u6211\u4eec\u53ef\u4ee5\u4f7f\u7528":57,"\u6211\u4eec\u53ef\u4ee5\u4f7f\u7528\u5176\u4ed6layer\u8fdb\u884c\u7ec4\u5408":33,"\u6211\u4eec\u53ef\u4ee5\u4f7f\u7528\u5b83\u6765\u751f\u6210\u5e8f\u5217":39,"\u6211\u4eec\u53ef\u4ee5\u521b\u5efatrainer\u6765\u5bf9\u7f51\u7edc\u8fdb\u884c\u8bad\u7ec3":33,"\u6211\u4eec\u53ef\u4ee5\u5728":40,"\u6211\u4eec\u53ef\u4ee5\u5b9a\u4e49\u5982\u4e0b\u7684layer\u7ec4\u5408":33,"\u6211\u4eec\u53ef\u4ee5\u5b9a\u4e49\u5982\u4e0blayer\u6765\u63cf\u8ff0\u795e\u7ecf\u7f51\u7edc\u7684\u8f93\u5165":33,"\u6211\u4eec\u53ef\u4ee5\u5c06":45,"\u6211\u4eec\u53ef\u4ee5\u6309\u7167\u5982\u4e0b\u5c42\u6b21\u5b9a\u4e49\u975e\u5e8f\u5217":35,"\u6211\u4eec\u53ef\u4ee5\u751f\u6210":62,"\u6211\u4eec\u53ef\u4ee5\u76f4\u63a5\u89c2\u5bdf\u6a21\u578b\u7684\u53c2\u6570\u662f\u5426\u7b26\u5408\u9884\u671f\u6765\u8fdb\u884c\u68c0\u9a8c":28,"\u6211\u4eec\u53ef\u4ee5\u770b\u5230\u5728\u76ee\u5f55":64,"\u6211\u4eec\u53ef\u4ee5\u8bbe\u8ba1\u642d\u5efa\u4e00\u4e2a\u7075\u6d3b\u7684":38,"\u6211\u4eec\u53ef\u4ee5\u8fd0\u884c\u4ee5\u4e0b\u547d\u4ee4\u6765\u505ableu\u8bc4\u4f30":65,"\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7\u4ee5\u4e0b\u547d\u4ee4\u6765\u8bad\u7ec3\u6a21\u578b":65,"\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7\u4ee5\u4e0b\u547d\u4ee4\u6765\u8fdb\u884c\u4ece\u6cd5\u8bed\u5230\u82f1\u8bed\u7684\u6587\u672c\u7ffb\u8bd1":65,"\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7\u5982\u4e0b\u547d\u4ee4\u8fdb\u884c\u9884\u5904\u7406\u5de5\u4f5c":57,"\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7\u65e5\u5fd7\u67e5\u770b\u5bb9\u5668\u8bad\u7ec3\u7684\u60c5\u51b5":52,"\u6211\u4eec\u57285\u5929\u91cc\u8bad\u7ec3\u4e8616\u4e2apass":65,"\u6211\u4eec\u5728\u51fd\u6570\u7684\u7ed3\u5c3e\u8fd4\u56de":39,"\u6211\u4eec\u5728\u62e5\u670950\u4e2a\u8282\u70b9\u7684\u96c6\u7fa4\u4e2d\u8bad\u7ec3\u6a21\u578b":65,"\u6211\u4eec\u5728\u8bad\u7ec3\u4e4b\u524d\u9700\u8981\u5e38\u89c1\u4e00\u4e2a\u6a21\u578b\u914d\u7f6e\u6587\u4ef6":65,"\u6211\u4eec\u5728initialzier\u51fd\u6570\u91cc\u521d\u59cb\u5316\u8bcd\u8868":60,"\u6211\u4eec\u5bf9\u6a21\u578b\u8fdb\u884c\u4e86\u4ee5\u4e0b\u66f4\u6539":39,"\u6211\u4eec\u5c06":[52,62],"\u6211\u4eec\u5c06\u4e00\u6bb5\u8bdd\u770b\u6210\u53e5\u5b50\u7684\u6570\u7ec4":36,"\u6211\u4eec\u5c06\u4ecb\u7ecd\u5982\u4f55\u542f\u52a8\u5206\u5e03\u5f0f\u8bad\u7ec3\u4f5c\u4e1a":51,"\u6211\u4eec\u5c06\u4ee5":[45,60],"\u6211\u4eec\u5c06\u4ee5\u6700\u57fa\u672c\u7684\u903b\u8f91\u56de\u5f52\u7f51\u7edc\u4f5c\u4e3a\u8d77\u70b9":60,"\u6211\u4eec\u5c06\u4f7f\u7528":39,"\u6211\u4eec\u5c06\u4f7f\u7528\u7b80\u5355\u7684":39,"\u6211\u4eec\u5c06\u4f7f\u7528cifar":57,"\u6211\u4eec\u5c06\u539f\u59cb\u6570\u636e\u7684\u6bcf\u4e00\u7ec4":36,"\u6211\u4eec\u5c06\u5728\u540e\u9762\u4ecb\u7ecd\u8bad\u7ec3\u548c\u9884\u6d4b\u6d41\u7a0b\u7684\u811a\u672c":60,"\u6211\u4eec\u5c06\u5b83\u4eec\u5212\u5206\u4e3a\u4e0d\u540c\u7684\u7c7b\u522b":46,"\u6211\u4eec\u5c06\u5bf9\u8fd9\u4e24\u4e2a\u6b65\u9aa4\u7ed9\u51fa\u4e86\u8be6\u7ec6\u7684\u89e3\u91ca":60,"\u6211\u4eec\u5c06\u653e\u7f6e\u4f9d\u8d56\u5e93":45,"\u6211\u4eec\u5c06\u8bc4\u5206\u5206\u6210\u4e24\u90e8\u5206":62,"\u6211\u4eec\u5c06\u9610\u91ca\u5982\u4f55\u5728\u96c6\u7fa4\u4e0a\u8fd0\u884c\u5206\u5e03\u5f0f":45,"\u6211\u4eec\u5c31\u53ef\u4ee5\u7740\u624b\u5bf9\u5206\u7c7b\u6a21\u578b\u8fdb\u884c\u8bad\u7ec3\u4e86":57,"\u6211\u4eec\u5c31\u53ef\u4ee5\u8bad\u7ec3\u6a21\u578b\u4e86":60,"\u6211\u4eec\u5c31\u53ef\u4ee5\u8fdb\u884c\u9884\u6d4b\u4e86":60,"\u6211\u4eec\u5c31\u5b8c\u6210\u4e86\u4e00\u6b21\u4ee3\u7801\u8d21\u732e\u7684\u8fc7\u7a0b":40,"\u6211\u4eec\u5c55\u793a\u5982\u4f55\u7528paddlepaddle\u89e3\u51b3":28,"\u6211\u4eec\u5df2\u7ecf\u5b9e\u73b0\u4e86\u5927\u591a\u6570\u5e38\u7528\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u67b6\u6784":39,"\u6211\u4eec\u5e0c\u671b\u80fd\u5728\u8fd9\u4e2a\u57fa\u7840\u4e0a\u4e0d\u65ad\u7684\u6539\u8fdb":0,"\u6211\u4eec\u5e0c\u671b\u80fd\u591f\u68c0\u9a8c\u6a21\u578b\u7684\u597d\u574f":28,"\u6211\u4eec\u5e94\u5f53\u4f1a\u5f97\u5230\u4e00\u4e2a\u540d\u4e3acifar":57,"\u6211\u4eec\u5efa\u8bae\u4f60\u4e3a\u4f60\u7684python\u5c01\u88c5\u5b9e\u73b0\u4e00\u4e2a":41,"\u6211\u4eec\u5efa\u8bae\u4f60\u5728\u5199\u65b0\u7f51\u7edc\u5c42\u65f6\u628a\u6d4b\u8bd5\u4ee3\u7801\u653e\u5165\u65b0\u7684\u6587\u4ef6\u4e2d":41,"\u6211\u4eec\u603b\u7ed3\u4e86\u5404\u4e2a\u7f51\u7edc\u7684\u590d\u6742\u5ea6\u548c\u6548\u679c":60,"\u6211\u4eec\u611f\u8c22":65,"\u6211\u4eec\u628apaddlepaddle\u7684\u7f16\u8bd1\u73af\u5883\u6253\u5305\u6210\u4e00\u4e2a\u955c\u50cf":30,"\u6211\u4eec\u63a8\u8350\u4f7f\u7528":30,"\u6211\u4eec\u63a8\u8350\u4f7f\u7528\u6700\u65b0\u7248\u672c\u7684cudnn":29,"\u6211\u4eec\u63a8\u8350\u4f7f\u7528docker\u955c\u50cf\u6765\u90e8\u7f72\u73af\u5883":31,"\u6211\u4eec\u63d0\u4f9b\u4e24\u4e2a\u7f51\u7edc":64,"\u6211\u4eec\u63d0\u4f9b\u4e86\u4e00\u4e2a\u6570\u636e\u9884\u5904\u7406\u811a\u672c":64,"\u6211\u4eec\u63d0\u4f9b\u4e86\u4e00\u4e2a\u793a\u4f8b\u811a\u672c":58,"\u6211\u4eec\u63d0\u4f9b\u4e86\u56fd\u5185\u7684\u955c\u50cf\u670d\u52a1\u5668\u4f9b\u5927\u5bb6\u4f7f\u7528":30,"\u6211\u4eec\u63d0\u4f9b\u4e86\u811a\u672c\u6765\u6784\u5efa\u5b57\u5178\u548c\u9884\u5904\u7406\u6570\u6910":64,"\u6211\u4eec\u63d0\u4f9b\u4e86c":58,"\u6211\u4eec\u63d0\u4f9b\u53ef\u4ee5\u76f4\u63a5\u8fd0\u884cpaddlepaddl":30,"\u6211\u4eec\u662f\u5bf9\u6bcf\u4e00\u4e2a\u5b50\u5e8f\u5217\u53d6\u6700\u540e\u4e00\u4e2a\u5143\u7d20":36,"\u6211\u4eec\u6700\u7ec8\u7684\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165python\u6216\u8005\u5176\u4ed6\u4efb\u4f55\u8bed\u8a00\u7684\u89e3\u91ca\u5668":25,"\u6211\u4eec\u6709\u4e00\u4e2a\u5e8f\u5217\u4f5c\u4e3a\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u72b6\u6001":39,"\u6211\u4eec\u7528":61,"\u6211\u4eec\u7528\u4ee5\u4e0b\u7684\u4e00\u4e9b":62,"\u6211\u4eec\u7528\u7f16\u53f7\u4f5c\u4e3akei":62,"\u6211\u4eec\u7528paddlepaddle\u89e3\u51b3\u4e86\u5355\u53d8\u91cf\u7ebf\u6027\u56de\u5f52\u95ee\u9898":28,"\u6211\u4eec\u7684\u5b57\u5178\u4f7f\u7528\u5185\u90e8\u7684\u5206\u8bcd\u5de5\u5177\u5bf9\u767e\u5ea6\u77e5\u9053\u548c\u767e\u5ea6\u767e\u79d1\u7684\u8bed\u6599\u8fdb\u884c\u5206\u8bcd\u540e\u4ea7\u751f":56,"\u6211\u4eec\u7684\u6807\u51c6\u5f00\u53d1\u6d41\u7a0b\u662f\u628a\u8fd9\u4e9b\u5de5\u5177\u90fd\u88c5\u8fdb\u4e00\u4e2adocker":40,"\u6211\u4eec\u7684\u8bad\u7ec3\u66f2\u7ebf\u5982\u4e0b":63,"\u6211\u4eec\u770b\u4e00\u4e0b\u5355\u5c42rnn\u7684\u914d\u7f6e":36,"\u6211\u4eec\u770b\u4e00\u4e0b\u8bed\u4e49\u76f8\u540c\u7684\u53cc\u5c42rnn\u7684\u7f51\u7edc\u914d\u7f6e":36,"\u6211\u4eec\u771f\u8bda\u5730\u611f\u8c22\u60a8\u7684\u5173\u6ce8":64,"\u6211\u4eec\u771f\u8bda\u5730\u611f\u8c22\u60a8\u7684\u8d21\u732e":40,"\u6211\u4eec\u79f0\u4e4b\u4e3a\u4e00\u4e2a0\u5c42\u7684\u5e8f\u5217":35,"\u6211\u4eec\u8fd8\u53ef\u4ee5\u767b\u5f55\u5230\u5bbf\u4e3b\u673a\u4e0a\u67e5\u770b\u8bad\u7ec3\u7ed3\u679c":51,"\u6211\u4eec\u8fd8\u5c06\u7f16\u7801\u5411\u91cf\u6295\u5c04\u5230":39,"\u6211\u4eec\u9009\u53d6\u5355\u53cc\u5c42\u5e8f\u5217\u914d\u7f6e\u4e2d\u7684\u4e0d\u540c\u90e8\u5206":36,"\u6211\u4eec\u901a\u5e38\u5728\u6240\u6709\u8282\u70b9\u4e0a\u521b\u5efa\u4e00\u4e2a":45,"\u6211\u4eec\u901a\u5e38\u5c06\u4e00\u53e5\u8bdd\u7406\u89e3\u6210\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217":36,"\u6211\u4eec\u901a\u8fc7\u8bfb\u53d6":52,"\u6211\u4eec\u9075\u5faa":65,"\u6211\u4eec\u91c7\u7528\u4e0a\u9762\u7684\u7279\u5f81\u6a21\u677f":63,"\u6211\u4eec\u91c7\u7528\u5355\u5c42lstm\u6a21\u578b":60,"\u6211\u4eec\u91c7\u7528\u7b2c\u4e8c\u4e2a\u65b9\u6cd5":30,"\u6211\u4eec\u9700\u8981\u5148\u521b\u5efa\u4e00\u4e2a\u6a21\u578b\u914d\u7f6e\u6587\u4ef6":57,"\u6211\u4eec\u9700\u8981\u521b\u5efa\u6a21\u578b\u914d\u7f6e\u6587\u4ef6":65,"\u6211\u4eec\u9700\u8981\u5236\u4f5c\u4e00\u4e2a\u5305\u542b\u8bad\u7ec3\u6570\u636e\u7684paddle\u955c\u50cf":51,"\u6211\u4eec\u9700\u8981\u5728\u96c6\u7fa4\u7684\u6240\u6709\u8282\u70b9\u4e0a\u5b89\u88c5":45,"\u6211\u4eec\u9700\u8981\u8ba1\u7b97":41,"\u6211\u4eec\u9700\u8981\u8bbe\u7f6e":62,"\u6211\u4eec\u9700\u8981\u9884\u5904\u7406\u6570\u6910\u5e76\u6784\u5efa\u4e00\u4e2a\u5b57\u5178":64,"\u6211\u4eec\u975e\u5e38\u6b22\u8fce\u60a8\u7528paddlepaddle\u6784\u5efa\u66f4\u597d\u7684\u793a\u4f8b":62,"\u6211\u4eec\u9884\u8bad\u7ec3\u5f97\u52304\u79cd\u4e0d\u540c\u7ef4\u5ea6\u7684\u8bcd\u5411\u91cf":56,"\u6211\u4eec\u9996\u5148\u5904\u7406\u7535\u5f71\u6216\u7528\u6237\u7684\u7279\u5f81\u6587\u4ef6":62,"\u6211\u4eec\u9996\u5148\u9700\u8981\u6839\u636e\u795e\u7ecf\u7f51\u7edc\u7ed3\u6784\u6765\u521b\u5efa\u6240\u9700\u8981\u4f18\u5316\u7684paramet":33,"\u6211\u4eec\u9ed8\u8ba4\u4f7f\u7528imdb\u7684\u6d4b\u8bd5\u6570\u636e\u96c6\u4f5c\u4e3a\u9a8c\u8bc1":64,"\u6211\u5220\u9664\u4e86":40,"\u6216":[3,44,57,63],"\u6216\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":35,"\u6216\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u53d8\u6210\u4e00\u4e2a0\u5c42\u5e8f\u5217":35,"\u6216\u4e00\u4e2a\u5411\u91cf":38,"\u6216\u4e0d\u786e\u5b9a":61,"\u6216\u5355\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u53d8\u6210\u4e00\u4e2a0\u5c42\u5e8f\u5217":35,"\u6216\u662f\u624b\u52a8\u7f16\u8f91\u751f\u6210":62,"\u6216\u6700\u5927\u503c":35,"\u6216\u6d4b\u8bd5\u6587\u4ef6\u5217\u8868":2,"\u6216\u7b2c\u4e00\u4e2a":35,"\u6216\u7b2c\u4e00\u4e2a\u5143\u7d20":35,"\u6216\u8005":[25,27,30,32,35,36,44],"\u6216\u8005\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":38,"\u6216\u8005\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":[35,38],"\u6216\u8005\u4ece\u5de5\u5177\u7684\u754c\u9762\u91cc\u8fd0\u884c\u60a8\u7684\u5e94\u7528":44,"\u6216\u8005\u53cd\u5411\u5730\u4ece":39,"\u6216\u8005\u5728cpu\u6a21\u5f0f\u4e0b\u4f7f\u75284\u4e2a\u7ebf\u7a0b":47,"\u6216\u8005\u5df2\u7ecf\u5728\u96c6\u7fa4\u63d0\u4ea4\u73af\u5883\u4e2d\u81ea\u52a8\u8bbe\u7f6e":46,"\u6216\u8005\u6570\u636e\u5e93\u8fde\u63a5\u8def\u5f84\u7b49":2,"\u6216\u8005\u6570\u7ec4\u7684\u6570\u7ec4\u8fd9\u4e2a\u6982\u5ff5":36,"\u6216\u8005\u662f\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":35,"\u6216\u8005\u662f\u4e00\u4e2a\u5c0f\u7684\u6587\u672c\u7247\u6bb5":64,"\u6216\u8005\u662f\u51fd\u6570\u8c03\u7528\u7684\u9891\u7387\u548c\u8017\u65f6\u7b49":44,"\u6216\u8005\u66f4\u65e9":27,"\u6216\u8005\u6bcf\u4e00\u4e2a\u7cfb\u5217\u91cc\u7684\u7279\u5f81\u6570\u636e":36,"\u6216\u8005\u76f4\u63a5\u4f7f\u7528\u4e0b\u9762\u7684shell\u547d\u4ee4":58,"\u6216\u8005\u76f4\u63a5\u6254\u6389\u975e\u5e38\u957f\u7684\u5e8f\u5217":27,"\u6216\u8005\u8f93\u5165\u6570\u636e\u5c3a\u5ea6\u8fc7\u5927":27,"\u6216\u8005\u91c7\u7528\u5e76\u884c\u8ba1\u7b97\u6765\u52a0\u901f\u67d0\u4e9b\u5c42\u7684\u66f4\u65b0":49,"\u6216\u8005\u9700\u8981\u53d1\u5e03\u60a8\u7684\u5e94\u7528\u7684\u955c\u50cf":30,"\u6216\u8005\u9700\u8981\u66f4\u9ad8\u7684\u6548\u7387":2,"\u6216\u8bbe\u7f6e\u4e3anone":2,"\u6216gpu":47,"\u6216gpu\u4e2a\u6570":64,"\u6218\u4e89\u7247":61,"\u623f":36,"\u623f\u95f4":36,"\u6240\u4ee5":[3,27],"\u6240\u4ee5\u4e00\u822c\u9700\u8981\u5bf9\u8bad\u7ec3\u7528\u7684\u6a21\u578b\u914d\u7f6e\u6587\u4ef6\u7a0d\u4f5c\u76f8\u5e94\u4fee\u6539\u624d\u80fd\u5728\u9884\u6d4b\u65f6\u4f7f\u7528":5,"\u6240\u4ee5\u4f60\u53ea\u7528\u6309\u4e0b\u9762\u7684\u7ed3\u6784\u6765\u7ec4\u7ec7\u6570\u6910\u5c31\u884c\u4e86":64,"\u6240\u4ee5\u505a\u6cd5\u53ef\u4ee5\u6709\u4e24\u79cd":27,"\u6240\u4ee5\u53ef\u4ee5\u5229\u7528\u5982\u4e0b\u65b9\u6cd5\u8bfb\u53d6\u6a21\u578b\u7684\u53c2\u6570":28,"\u6240\u4ee5\u53ef\u4ee5\u7b80\u5316\u5bf9\u73af\u5883\u7684\u8981\u6c42":51,"\u6240\u4ee5\u5916\u5c42\u8f93\u51fa\u7684\u5e8f\u5217\u5f62\u72b6":36,"\u6240\u4ee5\u5982\u679c\u9700\u8981\u81ea\u884c\u914d\u7f6e\u5f00\u53d1\u73af\u5883\u9700\u8981\u8003\u8651\u7248\u672c\u7684\u56e0\u7d20":30,"\u6240\u4ee5\u5b83\u4eec\u4f7f\u7528\u540c\u4e00\u4e2aip\u5730\u5740":50,"\u6240\u4ee5\u5bf9":36,"\u6240\u4ee5\u5f88\u591a\u65f6\u5019\u4f60\u9700\u8981\u505a\u7684\u53ea\u662f\u5b9a\u4e49\u6b63\u786e\u7684\u7f51\u7edc\u5c42\u5e76\u628a\u5b83\u4eec\u8fde\u63a5\u8d77\u6765":28,"\u6240\u4ee5\u6027\u80fd\u4e5f\u5c31\u9010\u6b65\u53d8\u6210\u4e86\u6df1\u5ea6\u5b66\u4e60\u9886\u57df\u6700\u91cd\u8981\u7684\u6307\u6807":44,"\u6240\u4ee5\u6211\u4eec\u4f7f\u7528\u8fd9\u4e2a\u955c\u50cf\u6765\u4e0b\u8f7d\u8bad\u7ec3\u6570\u636e\u5230docker":51,"\u6240\u4ee5\u6211\u4eec\u53ef\u4ee5\u5728\u8fd9\u4e2a\u57fa\u7840\u4e0a":52,"\u6240\u4ee5\u6211\u4eec\u63a8\u8350\u4f7f\u7528\u57fa\u4e8edocker\u6765\u6784\u5efapaddlepaddle\u7684\u6587\u6863":42,"\u6240\u4ee5\u6211\u4eec\u9700\u8981\u5c06\u8f93\u5165\u6570\u636e\u6807\u8bb0\u6210":36,"\u6240\u4ee5\u63a8\u8350\u4f7f\u7528\u663e\u5f0f\u6307\u5b9a\u7684\u65b9\u5f0f\u6765\u8bbe\u7f6einput_typ":3,"\u6240\u4ee5\u653e\u4e00\u4e2a\u7a7a\u5217\u8868":28,"\u6240\u4ee5\u8bad\u7ec3":45,"\u6240\u4ee5\u8f93\u51fa\u7684value\u5305\u542b\u4e24\u4e2a\u5411\u91cf":5,"\u6240\u4ee5\u8fd9\u4e00\u6b65\u662f\u5fc5\u8981\u7684":41,"\u6240\u5bf9\u5e94\u7684\u8bcd\u8868index\u6570\u7ec4":36,"\u6240\u6709\u4ee3\u7801\u5fc5\u987b\u5177\u6709\u5355\u5143\u6d4b\u8bd5":40,"\u6240\u6709\u53c2\u6570\u7f6e\u4e3a\u96f6":47,"\u6240\u6709\u540c\u76ee\u5f55\u4e0b\u7684\u6587\u672c\u5b9e\u4f8b\u6587\u4ef6\u90fd\u662f\u540c\u7ea7\u522b\u7684":64,"\u6240\u6709\u547d\u4ee4\u884c\u9009\u9879\u53ef\u4ee5\u8bbe\u7f6e\u4e3a":45,"\u6240\u6709\u6587\u4ef6\u5217\u8868":3,"\u6240\u6709\u672c\u5730\u8bad\u7ec3":45,"\u6240\u6709\u6807\u8bb0\u7684\u6d4b\u8bd5\u96c6\u548c\u8bad\u7ec3\u96c6":64,"\u6240\u6709\u751f\u6210\u7684\u7684\u6587\u4ef6\u90fd\u5728build\u76ee\u5f55\u4e0b":30,"\u6240\u6709\u7684":[40,41],"\u6240\u6709\u7684\u4eba\u53e3\u7edf\u8ba1\u5b66\u4fe1\u606f\u7531\u7528\u6237\u81ea\u613f\u63d0\u4f9b":61,"\u6240\u6709\u7684\u5355\u6d4b\u90fd\u4f1a\u88ab\u6267\u884c\u4e00\u6b21":41,"\u6240\u6709\u7684\u64cd\u4f5c\u90fd\u662f\u9488\u5bf9\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65\u6765\u8fdb\u884c\u7684":36,"\u6240\u6709\u7684\u7528\u6237\u4fe1\u606f\u90fd\u5305\u542b\u5728":61,"\u6240\u6709\u7684\u7535\u5f71\u4fe1\u606f\u90fd\u5305\u542b\u5728":61,"\u6240\u6709\u7684\u8bc4\u5206\u6570\u636e\u90fd\u5305\u542b\u5728":61,"\u6240\u6709\u7684python\u5c01\u88c5\u90fd\u4f7f\u7528":41,"\u6240\u6709\u7684python\u5c01\u88c5\u90fd\u5728":41,"\u6240\u6709\u7f51\u7edc\u5c42\u7684\u68af\u5ea6\u68c0\u67e5\u5355\u6d4b\u90fd\u4f4d\u4e8e":41,"\u6240\u6709\u8282\u70b9\u8fd0\u884c\u96c6\u7fa4\u4f5c\u4e1a\u7684\u4e3b\u673a\u540d\u6216":45,"\u6240\u6709\u8d21\u732e\u8005":0,"\u6240\u6709\u8f93\u5165\u5e8f\u5217\u5e94\u8be5\u6709\u76f8\u540c\u7684\u957f\u5ea6":39,"\u6240\u6709\u914d\u7f6e\u90fd\u80fd\u5728":60,"\u6240\u6784\u5efa\u7f51\u7edc\u7ed3\u6784\u7684\u7684\u6df1\u5ea6\u6bd4\u4e4b\u524d\u4f7f\u7528\u7684\u7f51\u7edc\u6709\u5927\u5e45\u5ea6\u7684\u63d0\u9ad8":58,"\u6240\u793a":63,"\u6240\u8c13\u65f6\u95f4\u6b65\u4fe1\u606f":3,"\u624b\u5199\u591a\u8bed\u8a00\u7ed1\u5b9a":25,"\u624b\u5de5\u827a\u8005":61,"\u624d\u4f1a\u91ca\u653e\u8be5\u6bb5\u5185\u5b58":3,"\u624d\u4f1astop":3,"\u624d\u80fd\u4fdd\u8bc1\u548c\u5355\u5c42\u5e8f\u5217\u7684\u914d\u7f6e\u4e2d":36,"\u624d\u80fd\u5145\u5206\u5229\u7528mac":30,"\u624d\u80fd\u53d1\u6325\u5176\u5168\u90e8\u80fd\u529b":44,"\u6253\u5370\u5728\u5c4f\u5e55\u4e0a":62,"\u6253\u5370\u7684\u65e5\u5fd7\u53d8\u591a":29,"\u6253\u5f00":44,"\u6253\u5f00\u6587\u672c\u6587\u4ef6":3,"\u6253\u5f00\u6d4f\u89c8\u5668\u8bbf\u95ee\u5bf9\u5e94\u76ee\u5f55\u4e0b\u7684index":42,"\u6267\u884c":[32,63,64],"\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u5c31\u53ef\u4ee5\u9884\u5904\u7406\u6570\u6910":64,"\u6267\u884c\u4ee5\u4e0b\u64cd\u4f5c":39,"\u6267\u884c\u60a8\u7684\u4ee3\u7801":44,"\u6267\u884c\u7684\u547d\u4ee4\u5982\u4e0b":58,"\u6269\u5c55\u548c\u5ef6\u4f38":0,"\u6269\u5c55\u673a\u5236\u7b49\u529f\u80fd":50,"\u6279\u6b21\u540e\u6253\u5370\u65e5\u5fd7":62,"\u6279\u6b21\u5bf9\u5e73\u5747\u53c2\u6570\u8fdb\u884c\u6d4b\u8bd5":63,"\u6279\u6b21\u7684\u6570\u636e":62,"\u627e\u5230":39,"\u627e\u5230\u8fd0\u884c\u6162\u7684\u539f\u56e0":44,"\u627e\u5230\u8fd0\u884c\u6162\u7684\u90e8\u5206":44,"\u6280\u672f\u5458":61,"\u628a":41,"\u628a\u7528\u6237\u5728\u8d2d\u7269\u7f51\u7ad9":64,"\u628a\u7f16\u8bd1\u51fa\u6765\u7684paddlepaddle\u4e5f\u6253\u5305\u6210\u4e00\u4e2a\u955c":30,"\u628a\u8bad\u7ec3\u6570\u636e\u76f4\u63a5\u653e\u5728":51,"\u6293\u53d6\u4ea7\u54c1\u7684\u7528\u6237\u8bc4\u8bba\u5e76\u5206\u6790\u4ed6\u4eec\u7684\u60c5\u611f":64,"\u6295\u5c04\u53cd\u5411rnn\u7684\u7b2c\u4e00\u4e2a\u5b9e\u4f8b\u5230":39,"\u6295\u5c04\u7f16\u7801\u5411\u91cf\u5230":39,"\u62a5\u9519":32,"\u62bd\u53d6\u51fa\u7684\u65b0\u8bcd\u8868\u7684\u4fdd\u5b58\u8def\u5f84":56,"\u62bd\u53d6\u5bf9\u5e94\u7684\u8bcd\u5411\u91cf\u6784\u6210\u65b0\u7684\u8bcd\u8868":56,"\u62c6\u5206\u5230\u4e0d\u540c\u6587\u4ef6\u5939":65,"\u62c6\u89e3":38,"\u62c6\u89e3\u6210\u7684\u6bcf\u4e00\u53e5\u8bdd\u518d\u901a\u8fc7\u4e00\u4e2alstm\u7f51\u7edc":36,"\u62f7\u8d1d\u8bad\u7ec3\u6587\u4ef6\u5230\u5bb9\u5668\u5185":52,"\u62fc\u63a5\u6210\u4e00\u4e2a\u65b0\u7684\u5411\u91cf":60,"\u6301\u7eed\u96c6\u6210\u7cfb\u7edf\u4e5f\u53ef\u4ee5\u7b80\u5355\u5730\u590d\u73b0\u8fd9\u4e2a\u8fc7\u7a0b":30,"\u6307\u5411\u4e00\u4e2alayer":38,"\u6307\u5b9a":[27,38,39],"\u6307\u5b9a\u4e00\u53f0\u673a\u5668\u4e0a\u4f7f\u7528\u7684\u7ebf\u7a0b\u6570":47,"\u6307\u5b9a\u4f7f\u75282":27,"\u6307\u5b9a\u521d\u59cb\u5316\u6a21\u578b\u8def\u5f84":60,"\u6307\u5b9a\u52a0\u8f7d\u7684\u65b9\u5f0f":47,"\u6307\u5b9a\u5de5\u4f5c\u6a21\u578b\u8fdb\u884c\u9884\u6d4b":58,"\u6307\u5b9a\u5de5\u4f5c\u6a21\u5f0f\u6765\u63d0\u53d6\u7279\u5f81":58,"\u6307\u5b9a\u63d0\u53d6\u7279\u5f81\u7684\u5c42":58,"\u6307\u5b9a\u662f\u5426\u4f7f\u7528gpu":58,"\u6307\u5b9a\u751f\u6210\u6570\u636e\u7684\u51fd\u6570":60,"\u6307\u5b9a\u7684\u5b57\u5178\u5355\u8bcd\u6570":65,"\u6307\u5b9a\u7684\u6570\u636e\u5c06\u4f1a\u88ab\u6d4b\u8bd5":60,"\u6307\u5b9a\u7684\u8f93\u5165\u4e0d\u4f1a\u88ab":38,"\u6307\u5b9a\u8bad\u7ec3\u6570\u636e\u548c\u6d4b\u8bd5\u6570\u636e":60,"\u6307\u5b9abatch":65,"\u6307\u5b9acudnn\u7684\u6700\u5927\u5de5\u4f5c\u7a7a\u95f4\u5bb9\u9650":47,"\u6307\u5bf9\u4e8e\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217\u8f93\u5165\u6570\u636e":36,"\u6307\u793a\u4f7f\u7528\u54ea\u4e2agpu\u6838":47,"\u6307\u793a\u5728\u7b80\u5355\u7684recurrentlayer\u5c42\u7684\u8ba1\u7b97\u4e2d\u662f\u5426\u4f7f\u7528\u6279\u5904\u7406\u65b9\u6cd5":47,"\u6307\u793a\u5f53\u6307\u5b9a\u8f6e\u7684\u6d4b\u8bd5\u6a21\u578b\u4e0d\u5b58\u5728\u65f6":47,"\u6307\u793a\u662f\u5426\u4f7f\u7528\u591a\u7ebf\u7a0b\u6765\u8ba1\u7b97\u4e00\u4e2a\u795e\u7ecf\u7f51\u7edc":47,"\u6307\u793a\u662f\u5426\u5f00\u542f\u53c2\u6570\u670d\u52a1\u5668":47,"\u6307\u793a\u662f\u5426\u663e\u793a\u53c2\u6570\u670d\u52a1\u5668\u4e0a\u7684\u7a00\u758f\u53c2\u6570\u5206\u5e03\u7684\u65e5\u5fd7\u7ec6\u8282":47,"\u6307\u793a\u662f\u5426\u68c0\u67e5\u6240\u6709\u53c2\u6570\u670d\u52a1\u5668\u4e0a\u7684\u7a00\u758f\u53c2\u6570\u7684\u5206\u5e03\u662f\u5747\u5300\u7684":47,"\u6307\u793a\u6d4b\u8bd5\u4efb\u52a1":63,"\u6307\u793a\u6d4b\u8bd5\u4efb\u52a1\u7684\u6807\u8bb0":63,"\u6309\u542f\u53d1\u5f0f\u635f\u5931\u7684\u5927\u5c0f\u9012\u589e\u6392\u5e8f":47,"\u6309\u7167\u4e0b\u9762\u6b65\u9aa4\u5373\u53ef":52,"\u6309\u94ae":40,"\u633a":36,"\u633a\u597d":36,"\u6355\u83b7":60,"\u6362":36,"\u6392\u6210\u4e00\u5217\u7684\u591a\u4e2a\u5143\u7d20":35,"\u63a5\u4e0a\u4e00\u4e2a\u5168\u8fde\u63a5\u5c42":33,"\u63a5\u4e0a\u5747\u65b9\u8bef\u5dee\u5c42":33,"\u63a5\u4e0b\u6765":[60,64],"\u63a5\u4e0b\u6765\u53ef\u4ee5\u8003\u8651\u4e0b\u65f6\u95f4\u7ebf\u7684\u5206\u6790":44,"\u63a5\u4e0b\u6765\u5c31\u53ef\u4ee5\u4f7f\u7528":44,"\u63a5\u4e0b\u6765\u6211\u4eec\u521b\u5efa\u4e00\u4e2a\u539f\u59cb":40,"\u63a5\u4e0b\u6765\u6211\u4eec\u53d6\u6d88\u5bf9":40,"\u63a5\u4e0b\u6765\u6211\u4eec\u5c06\u5c55\u793a\u5982\u4f55\u7528paddlepaddle\u8bad\u7ec3\u4e00\u4e2a\u6587\u672c\u5206\u7c7b\u6a21\u578b":60,"\u63a5\u4e0b\u6765\u7b49\u5f85":40,"\u63a5\u53d7":63,"\u63a5\u53d7\u7684\u4e1c\u897f":63,"\u63a5\u53d7\u8005":63,"\u63a5\u53e3":25,"\u63a5\u53e3\u540d\u79f0":45,"\u63a5\u53e3\u63d0\u53d6\u7684\u7ed3\u679c\u662f\u4e00\u81f4\u7684":58,"\u63a5\u53e3\u6709\u4e00\u4e2a":27,"\u63a5\u53e3\u6765\u52a0\u8f7d\u6570\u636e":60,"\u63a5\u53e3\u6765\u52a0\u8f7d\u8be5\u6587\u4ef6":58,"\u63a5\u53e3\u6765\u6253\u5f00\u8be5\u6587\u4ef6":58,"\u63a5\u7740\u6211\u4eec\u5c31\u80fd\u591f\u6253\u5f00\u6d4f\u89c8\u5668\u5728":30,"\u63a7\u5236":47,"\u63a7\u5236\u5982\u4f55\u6539\u53d8\u6a21\u578b\u53c2\u6570":28,"\u63a7\u5236\u662f\u5426\u751f\u6210\u5355\u5143\u6d4b\u8bd5":30,"\u63a7\u5236\u751f\u6210\u7684\u751f\u4ea7\u955c\u50cf\u662f\u5426\u652f\u6301avx":30,"\u63a7\u5236\u751f\u6210\u7684\u751f\u4ea7\u955c\u50cf\u662f\u5426\u652f\u6301gpu":30,"\u63a8\u5bfc\u8be5\u5c42\u524d\u5411\u548c\u540e\u5411\u4f20\u9012\u7684\u65b9\u7a0b":41,"\u63a8\u8350":36,"\u63a8\u8350\u4f7f\u7528":3,"\u63a8\u8350\u6e05\u7406\u6574\u4e2a\u7f16\u8bd1\u76ee\u5f55":29,"\u63a8\u8350\u76f4\u63a5\u5b58\u653e\u5230\u8bad\u7ec3\u76ee\u5f55":2,"\u63a8\u8350\u7cfb\u7edf":45,"\u63a8\u9001\u5230\u8fdc\u7a0b\u4ed3\u5e93":40,"\u63a8\u9500\u5458":61,"\u63cf\u8ff0":29,"\u63cf\u8ff0\u7f51\u7edc\u7ed3\u6784\u548c\u4f18\u5316\u7b97\u6cd5":60,"\u63cf\u8ff0\u95ee\u9898":40,"\u63cf\u8ff0kubernetes\u4e0a\u8fd0\u884c\u7684\u4f5c\u4e1a":50,"\u63d0\u4f9b":45,"\u63d0\u4f9b\u4e86\u4e00\u4e2a\u542f\u52a8\u811a\u672c":52,"\u63d0\u4f9b\u4e86\u547d\u4ee4\u6837\u4f8b\u6765\u8fd0\u884c":45,"\u63d0\u4f9b\u4e86\u81ea\u52a8\u5316\u811a\u672c\u6765\u542f\u52a8\u4e0d\u540c\u8282\u70b9\u4e2d\u7684\u6240\u6709":45,"\u63d0\u4f9b\u51e0\u4e4e\u6240\u6709\u8bad\u7ec3\u7684\u5185\u90e8\u8f93\u51fa\u65e5\u5fd7":45,"\u63d0\u4f9b\u6269\u5c55\u7684\u957f\u5ea6\u4fe1\u606f":35,"\u63d0\u4f9b\u6700\u65b0":30,"\u63d0\u4f9b\u8bad\u7ec3\u8fc7\u7a0b\u7684":45,"\u63d0\u51fa\u7684\u4ee3\u7801\u9700\u6c42":56,"\u63d0\u793a":27,"\u642d\u5efa\u795e\u7ecf\u7f51\u7edc\u5c31\u50cf\u4f7f\u7528\u79ef\u6728\u642d\u5efa\u5b9d\u5854\u4e00\u6837":33,"\u64cd\u4f5c":36,"\u64cd\u6301\u5bb6\u52a1\u8005":61,"\u652f\u6301\u4e3b\u6d41x86\u5904\u7406\u5668\u5e73\u53f0":32,"\u652f\u6301\u53cc\u5c42\u5e8f\u5217\u4f5c\u4e3a\u8f93\u5165\u7684layer":[37,38],"\u652f\u6301nvidia":32,"\u652f\u6301rbd":50,"\u653e\u5728\u8fd9\u4e2a\u76ee\u5f55\u91cc\u7684\u6587\u4ef6\u5176\u5b9e\u662f\u4fdd\u5b58\u5230\u4e86mfs\u4e0a":52,"\u653e\u5fc3":36,"\u6545\u800c\u662f\u4e00\u4e2a\u5355\u5c42\u65f6\u95f4\u5e8f\u5217":36,"\u6548\u679c\u603b\u7ed3":60,"\u6559\u7a0b\u6587\u6863\u5230":45,"\u6559\u80b2\u5de5\u4f5c\u8005":61,"\u6570":[38,63],"\u6570\u5fc5\u987b\u4e25\u683c\u76f8\u7b49":38,"\u6570\u636e":65,"\u6570\u636e\u4e0b\u8f7d\u4e4b\u540e":57,"\u6570\u636e\u4e2d0":27,"\u6570\u636e\u5217\u8868":58,"\u6570\u636e\u5c06\u4fdd\u5b58\u5728":56,"\u6570\u636e\u5c42":[28,62],"\u6570\u636e\u5e94\u8be5\u5728\u542f\u52a8\u96c6\u7fa4\u4f5c\u4e1a\u4e4b\u524d\u51c6\u5907\u597d":45,"\u6570\u636e\u63d0\u4f9b\u5668":46,"\u6570\u636e\u63d0\u4f9b\u811a\u672c\u4ec5\u4ec5\u662f\u8bfb\u53d6meta":62,"\u6570\u636e\u63d0\u4f9b\u811a\u672c\u7684\u7ec6\u8282\u6587\u6863\u53ef\u4ee5\u53c2\u8003":62,"\u6570\u636e\u7684\u6574\u6570id":39,"\u6570\u636e\u76ee\u5f55\u4e2d\u7684\u6240\u6709\u6587\u4ef6\u88ab":45,"\u6570\u636e\u7c7b\u578b":5,"\u6570\u636e\u7f13\u5b58\u7684\u7b56\u7565":3,"\u6570\u636e\u8bbf\u95ee":1,"\u6570\u636e\u8bfb\u53d6\u5747\u4ea4\u7531\u5176\u4ed6\u8bed\u8a00\u5b8c\u6210":25,"\u6570\u636e\u8f93\u5165":38,"\u6570\u636e\u8f93\u5165\u683c\u5f0f":3,"\u6570\u636e\u96c6":61,"\u6570\u636e\u96c6\u63cf\u8ff0":62,"\u6570\u636e\u96c6\u6587\u4ef6\u5939\u540d\u79f0":65,"\u6570\u636e\u9884\u5904\u7406\u5b8c\u6210\u4e4b\u540e":60,"\u6570\u636e\u9884\u6d4b":63,"\u6570\u6910\u5b9a\u4e49":64,"\u6570\u6910\u8bf4\u660e\u6587\u6863":64,"\u6570\u6910\u96c6\u548c":64,"\u6570\u76ee":49,"\u6574\u4f53":36,"\u6574\u4f53\u6570\u636e\u548c\u539f\u59cb\u6570\u636e\u5b8c\u5168\u4e00\u6837":36,"\u6574\u4f53\u7684\u7ed3\u6784\u56fe\u5982\u4e0b":52,"\u6574\u6570":41,"\u6574\u6570\u6807\u7b7e":[3,33],"\u6574\u6d01":36,"\u6587\u4e66\u5de5\u4f5c":61,"\u6587\u4ef6":[25,40,51,63],"\u6587\u4ef6\u4e2d":[52,58,61,63],"\u6587\u4ef6\u4e2d\u6307\u5b9a\u6a21\u578b\u8def\u5f84\u548c\u8f93\u51fa\u7684\u76ee\u5f55":58,"\u6587\u4ef6\u4e2d\u6307\u5b9a\u8981\u63d0\u53d6\u7279\u5f81\u7684\u7f51\u7edc\u5c42\u7684\u540d\u5b57":58,"\u6587\u4ef6\u4e2d\u7684":58,"\u6587\u4ef6\u4e2d\u7684\u6bcf\u884c\u90fd\u5fc5\u987b\u662f\u4e00\u4e2a\u53e5\u5b50":65,"\u6587\u4ef6\u4e3a":[27,65],"\u6587\u4ef6\u4e4b\u5916":40,"\u6587\u4ef6\u4e5f\u53ef\u4ee5\u7528\u4e8e\u5bf9\u6837\u672c\u8fdb\u884c\u9884\u6d4b":58,"\u6587\u4ef6\u5185\u5bb9\u4e3a":25,"\u6587\u4ef6\u5206\u5272\u4e3a\u8bad\u7ec3\u548c\u6d4b\u8bd5\u6587\u4ef6":62,"\u6587\u4ef6\u540d\u79f0\u4e3a":62,"\u6587\u4ef6\u59390":52,"\u6587\u4ef6\u5939\u4e2d\u7684\u6bcf\u4e2a\u6587\u4ef6\u7684\u6bcf\u4e00\u884c\u5305\u542b\u4e24\u90e8\u5206":65,"\u6587\u4ef6\u7684\u5206\u9694\u7b26\u4e3a":62,"\u6587\u4ef6\u7684\u6539\u53d8":40,"\u6587\u4ef6\u7684\u683c\u5f0f\u53ef\u4ee5":62,"\u6587\u4ef6\u7a0d\u6709\u5dee\u522b":57,"\u6587\u4ef6\u7d22\u5f15":45,"\u6587\u4ef6\u7ed9\u51fa\u4e86\u5b8c\u6574\u4f8b\u5b50":60,"\u6587\u4ef6model":49,"\u6587\u5b57\u7684\u4ea4\u4e92\u5f0f\u6587\u6863":30,"\u6587\u672c\u4e2d\u7684\u5355\u8bcd\u7528\u7a7a\u683c\u5206\u9694":60,"\u6587\u672c\u4fe1\u606f\u5c31\u662f\u4e00\u4e2a\u5e8f\u5217\u6570\u636e":3,"\u6587\u672c\u5206\u7c7b\u95ee\u9898":60,"\u6587\u672c\u5377\u79ef\u5206\u53ef\u4e3a\u4e09\u4e2a\u6b65\u9aa4":60,"\u6587\u672c\u5377\u79ef\u91c7\u6837\u5c42":62,"\u6587\u672c\u6295\u5f71\u5c42":62,"\u6587\u672c\u683c\u5f0f\u7684\u5b9e\u4f8b\u6587\u4ef6":64,"\u6587\u6863":27,"\u6587\u6863\u7f16\u5199\u7b49\u5de5\u4f5c":30,"\u6587\u6863\u81ea\u52a8\u5206\u7c7b\u548c\u95ee\u7b54":63,"\u6587\u6863\u90fd\u662f\u901a\u8fc7":42,"\u6587\u7ae0":52,"\u65b0":36,"\u65b0\u5efa\u4e00\u4e2a\u6743\u91cd":41,"\u65b0\u624b\u5165\u95e8":55,"\u65b9\u4fbf":36,"\u65b9\u4fbf\u4eca\u540e\u7684\u5d4c\u5165\u5f0f\u79fb\u690d\u5de5\u4f5c":29,"\u65b9\u4fbf\u5f00\u53d1\u8005\u76f4\u63a5\u767b\u5f55\u5230\u955c\u50cf\u4e2d\u8fdb\u884c\u5f00\u53d1":30,"\u65b9\u5f0f1":27,"\u65b9\u5f0f2":27,"\u65b9\u6848\u6765\u6807\u8bb0\u6bcf\u4e2a\u53c2\u6570":63,"\u65b9\u6cd5\u4e00":49,"\u65b9\u6cd5\u4e09":49,"\u65b9\u6cd5\u4e8c":49,"\u65c1\u8fb9":36,"\u65c5\u6e38\u7f51\u7ad9":64,"\u65e0":36,"\u65e0\u4e1a\u4eba\u58eb":61,"\u65e0\u5ef6\u8fdf":47,"\u65e0\u6cd5\u505a\u5230\u5bf9\u4e8e\u5404\u79cd\u8bed\u8a00\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u7684\u9002\u914d":25,"\u65e5\u5fd7\u5c06\u4fdd\u5b58\u5728":64,"\u65e8\u5728\u5efa\u7acb\u4e00\u4e2a\u53ef\u4ee5\u88ab\u534f\u540c\u8c03\u81f3\u6700\u4f18\u7ffb\u8bd1\u6548\u679c\u7684\u5355\u795e\u7ecf\u5143\u7f51\u7edc":65,"\u65e9\u9910":36,"\u65f6":[27,35,39,41,47,52],"\u65f6\u5019":36,"\u65f6\u52a0\u4e0a":64,"\u65f6\u5e8f\u6a21\u578b\u5747\u4f7f\u7528\u8be5\u811a\u672c":60,"\u65f6\u5e8f\u6a21\u578b\u662f\u6307\u6570\u636e\u7684\u67d0\u4e00\u7ef4\u5ea6\u662f\u4e00\u4e2a\u5e8f\u5217\u5f62\u5f0f":3,"\u65f6\u76ee\u6807\u8bed\u8a00\u7684\u6587\u4ef6":65,"\u65f6\u88ab\u8bad\u7ec3\u7684":41,"\u65f6\u8bbe\u5907id\u53f7\u7684\u5206\u914d":49,"\u65f6\u95f4":36,"\u65f6\u95f4\u6233":61,"\u65f6\u95f4\u6233\u8868\u793a\u4e3a\u4ece1970":61,"\u65f6\u95f4\u6b65\u7684\u6982\u5ff5":36,"\u6620\u5c04\u5230\u4e00\u4e2a\u7ef4\u5ea6\u4e3a":41,"\u662f":[29,36],"\u662f\u4e00\u4e2a\u51681\u7684\u5411\u91cf":41,"\u662f\u4e00\u4e2a\u5185\u7f6e\u7684\u5b9a\u65f6\u5668\u5c01\u88c5":44,"\u662f\u4e00\u4e2a\u52a8\u6001\u7a0b\u5e8f\u5206\u6790\u7684\u672f\u8bed":44,"\u662f\u4e00\u4e2a\u5305\u88c5\u6570\u636e\u7684":63,"\u662f\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":35,"\u662f\u4e00\u4e2a\u53cc\u5c42\u7684\u5e8f\u5217":35,"\u662f\u4e00\u4e2a\u591a\u8bed\u8a00\u63a5\u53e3\u7684\u4ee3\u7801\u751f\u6210\u5668":25,"\u662f\u4e00\u4e2a\u5c01\u88c5\u5bf9\u8c61":44,"\u662f\u4e00\u4e2a\u5f88\u6709\u7528\u7684\u53c2\u6570":49,"\u662f\u4e00\u4e2a\u7b26\u5408\u9ad8\u65af\u5206\u5e03\u7684\u968f\u673a\u53d8\u91cf":28,"\u662f\u4e00\u4e2a\u7edf\u8ba1\u5b66\u7684\u673a\u5668\u7ffb\u8bd1\u7cfb\u7edf":65,"\u662f\u4e00\u4e2a\u914d\u7f6e\u6587\u4ef6\u7684\u4f8b\u5b50":64,"\u662f\u4e00\u4e2a\u975e\u7ebf\u6027\u7684":41,"\u662f\u4e00\u4e2apython\u7684":3,"\u662f\u4e00\u4e2aunbound":38,"\u662f\u4e00\u6761\u65f6\u95f4\u5e8f\u5217":[3,33],"\u662f\u4e00\u79cd\u4efb\u610f\u590d\u6742\u7684rnn\u5355\u5143":38,"\u662f\u4e00\u7ec4":50,"\u662f\u4e0d\u5305\u62ec\u6e90\u7801\u7684":51,"\u662f\u4e0d\u5e38\u89c1\u7684\u505a\u6cd5":25,"\u662f\u4e0d\u662f\u5f88\u7b80\u5355\u5462":3,"\u662f\u4e0d\u662f\u8981\u5bf9\u6570\u636e\u505ashuffl":3,"\u662f\u4e3b\u8981\u7684\u53ef\u6267\u884cpython\u811a\u672c":63,"\u662f\u4ec0\u4e48\u4e5f\u6ca1\u5173\u7cfb":3,"\u662f\u4f17\u591a\u8bef\u5dee\u4ee3\u4ef7\u51fd\u6570\u5c42\u7684\u4e00\u79cd":28,"\u662f\u4f7f\u5f97\u8981\u5171\u4eab\u7684\u53c2\u6570\u4f7f\u7528\u540c\u6837\u7684":27,"\u662f\u504f\u5dee":39,"\u662f\u5176\u5927\u5c0f":28,"\u662f\u51e0\u4e4e\u4e0d\u5360\u5185\u5b58\u7684":3,"\u662f\u539f\u59cb\u6cd5\u8bed\u6587\u4ef6":65,"\u662f\u5411\u91cf":41,"\u662f\u5426\u4ee5\u9006\u5e8f\u5904\u7406\u8f93\u5165\u5e8f\u5217":38,"\u662f\u5426\u4f7f\u7528\u53cc\u7cbe\u5ea6\u6d6e\u70b9\u6570":29,"\u662f\u5426\u4f7f\u7528\u65e7\u7684remoteparameterupdat":47,"\u662f\u5426\u4f7f\u7528\u6743\u91cd":41,"\u662f\u5426\u4f7f\u7528gpu":62,"\u662f\u5426\u4f7f\u7528gpu\u8bad\u7ec3":65,"\u662f\u5426\u5141\u8bb8\u6682\u5b58\u7565\u5fae\u591a\u4f59pool_size\u7684\u6570\u636e":3,"\u662f\u5426\u5185\u5d4cpython\u89e3\u91ca\u5668":29,"\u662f\u5426\u5c06\u5168\u5c40\u79cd\u5b50\u5e94\u7528\u4e8e\u672c\u5730\u7ebf\u7a0b\u7684\u968f\u673a\u6570":47,"\u662f\u5426\u5f00\u542f\u5355\u5143\u6d4b\u8bd5":29,"\u662f\u5426\u5f00\u542f\u8ba1\u65f6\u529f\u80fd":29,"\u662f\u5426\u5f00\u542frdma":29,"\u662f\u5426\u6253\u5370\u7248\u672c\u4fe1\u606f":47,"\u662f\u5426\u652f\u6301gpu":29,"\u662f\u5426\u663e\u793a":47,"\u662f\u5426\u7a00\u758f":41,"\u662f\u5426\u7f16\u8bd1\u4e2d\u82f1\u6587\u6587\u6863":29,"\u662f\u5426\u7f16\u8bd1\u542b\u6709avx\u6307\u4ee4\u96c6\u7684paddlepaddle\u4e8c\u8fdb\u5236\u6587\u4ef6":29,"\u662f\u5426\u7f16\u8bd1\u65f6\u8fdb\u884c\u4ee3\u7801\u98ce\u683c\u68c0\u67e5":29,"\u662f\u5426\u7f16\u8bd1python\u7684swig\u63a5\u53e3":29,"\u662f\u5426\u8fd0\u884c\u65f6\u52a8\u6001\u52a0\u8f7dcuda\u52a8\u6001\u5e93":29,"\u662f\u5426\u9700\u8981\u7b49\u5f85\u8be5\u8f6e\u6a21\u578b\u53c2\u6570":47,"\u662f\u56e0\u4e3ac99\u652f\u6301":25,"\u662f\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u52a0\u8f7d\u5b57\u5178\u5e76\u5b9a\u4e49\u6570\u636e\u63d0\u4f9b\u7a0b\u5e8f\u6a21\u5757\u548c\u7f51\u7edc\u67b6\u6784\u7684\u795e\u7ecf\u7f51\u7edc\u914d\u7f6e\u6587\u4ef6":63,"\u662f\u5728paddlepaddle\u4e2d\u6784\u9020\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u65f6\u6700\u91cd\u8981\u7684\u6982\u5ff5":39,"\u662f\u57fa\u7840\u7684\u8ba1\u7b97\u5355\u5143":28,"\u662f\u5b58\u6709\u4e00\u7cfb\u5217\u53d8\u6362\u77e9\u9635\u7684\u6743\u91cd":41,"\u662f\u5b58\u6709\u504f\u7f6e\u5411\u91cf\u7684\u6743\u91cd":41,"\u662f\u5e8f\u5217":62,"\u662f\u5f85\u6269\u5c55\u7684\u6570\u636e":35,"\u662f\u6211\u4eec":40,"\u662f\u6307\u4e00\u7cfb\u5217\u7684\u7279\u5f81\u6570\u636e":36,"\u662f\u6307recurrent_group\u7684\u591a\u4e2a\u8f93\u5165\u5e8f\u5217":36,"\u662f\u6570\u636e\u8f93\u5165":39,"\u662f\u6574\u4e2a\u7684\u8bcd\u5d4c\u5165":62,"\u662f\u6709\u610f\u4e49\u7684":36,"\u662f\u6784\u5efa\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u6700\u91cd\u8981\u7684\u5de5\u5177":39,"\u662f\u6a21\u578b\u53c2\u6570\u4f18\u5316\u7684\u76ee\u6807\u51fd\u6570":28,"\u662f\u6d45\u5c42\u8bed\u4e49\u89e3\u6790\u7684\u4e00\u79cd\u5f62\u5f0f":63,"\u662f\u6e90\u8bed\u8a00\u7684\u6587\u4ef6":65,"\u662f\u76ee\u6807\u82f1\u8bed\u6587\u4ef6":65,"\u662f\u77e9\u9635":41,"\u662f\u795e\u7ecf\u7f51\u7edc\u7684\u5165\u53e3":28,"\u662f\u7f51\u7edc\u548c\u6570\u636e\u914d\u7f6e\u6587\u4ef6":57,"\u662f\u7f51\u7edc\u5c42\u5b9e\u4f8b\u7684\u540d\u5b57\u6807\u8bc6\u7b26":41,"\u662f\u7f51\u7edc\u5c42\u7684\u6807\u8bc6\u7b26":41,"\u662f\u7f51\u7edc\u5c42\u7684\u7c7b\u578b":41,"\u662f\u7f51\u7edc\u5c42\u8f93\u51fa\u7684\u5927\u5c0f":41,"\u662f\u8be5\u5c42\u7684\u6807\u8bc6\u7b26":41,"\u662f\u8be5\u5c42\u7684\u7c7b\u540d":41,"\u662f\u8be5\u7f51\u7edc\u5c42\u7684":41,"\u662f\u8f93\u5165":39,"\u662f\u901a\u7528\u7269\u4f53\u5206\u7c7b\u9886\u57df\u4e00\u4e2a\u4f17\u6240\u5468\u77e5\u7684\u6570\u636e\u5e93":58,"\u662f\u9700\u8981\u4e86\u89e3\u54ea\u4e9b\u6b65\u9aa4\u62d6\u6162\u4e86\u6574\u4f53":44,"\u662fdecoder\u7684\u6570\u636e\u8f93\u5165":38,"\u662fgoogle\u5f00\u6e90\u7684\u5bb9\u5668\u96c6\u7fa4\u7ba1\u7406\u7cfb\u7edf":50,"\u662fnvidia\u6027\u80fd\u5206\u6790\u5de5\u5177":44,"\u662fpaddlepaddle\u652f\u6301\u7684\u4e00\u79cd\u4efb\u610f\u590d\u6742\u7684rnn\u5355\u5143":38,"\u662fpaddlepaddle\u8d1f\u8d23\u63d0\u4f9b\u6570\u636e\u7684\u6a21\u5757":60,"\u662fpod\u5185\u7684\u5bb9\u5668\u90fd\u53ef\u4ee5\u8bbf\u95ee\u7684\u5171\u4eab\u76ee\u5f55":50,"\u662fpython\u5c01\u88c5\u7684\u7c7b\u540d":41,"\u662frnn\u72b6\u6001":39,"\u663e":60,"\u663e\u5f0f\u6307\u5b9a\u8fd4\u56de\u7684\u662f\u4e00\u4e2a28":3,"\u665a":36,"\u666e\u901a\u7528\u6237\u8bf7\u8d70\u5b89\u88c5\u6d41\u7a0b":31,"\u6682\u4e0d\u8003\u8651\u5728\u5185":27,"\u66f4\u591a\u5173\u4e8edocker\u7684\u5b89\u88c5\u4e0e\u4f7f\u7528":27,"\u66f4\u591a\u5185\u5bb9\u53ef\u67e5\u770b\u53c2\u8003\u6587\u732e":64,"\u66f4\u591a\u7684\u7ec6\u8282\u53ef\u4ee5\u5728\u6587\u732e\u4e2d\u627e\u5230":64,"\u66f4\u597d\u5730\u5b8c\u6210\u4e00\u4e9b\u590d\u6742\u7684\u8bed\u8a00\u7406\u89e3\u4efb\u52a1":38,"\u66f4\u5feb":39,"\u66f4\u65b0":27,"\u66f4\u65b0\u6a21\u5f0f":27,"\u66f4\u65b9\u4fbf\u7684\u8bbe\u7f6e\u65b9\u5f0f":27,"\u66f4\u8be6\u7ec6\u6570\u636e\u683c\u5f0f\u548c\u7528\u4f8b\u8bf7\u53c2\u8003":60,"\u66f4\u8be6\u7ec6\u7684\u7f51\u7edc\u914d\u7f6e\u8fde\u63a5\u8bf7\u53c2\u8003":60,"\u66f4\u8be6\u7ec6\u7684\u8bf4\u660e":60,"\u66f4\u8fdb\u4e00\u6b65":38,"\u66f4\u9ad8":39,"\u66ff\u6211\u4eec\u5b8c\u6210\u4e86\u539f\u59cb\u8f93\u5165\u6570\u636e\u7684\u62c6\u5206":38,"\u6700":36,"\u6700\u4e0d\u540c\u7684\u7279\u8272\u662f\u5b83\u5e76\u6ca1\u6709\u5c06\u8f93\u5165\u8bed\u53e5\u7f16\u7801\u4e3a\u4e00\u4e2a\u5355\u72ec\u7684\u5b9a\u957f\u5411\u91cf":65,"\u6700\u4e3b\u8981\u7684\u5de5\u4f5c\u5c31\u662f\u89e3\u6790\u51fa":52,"\u6700\u4f73\u63a8\u8350":3,"\u6700\u540e":[3,40,41,45,60,64],"\u6700\u540e\u4e00\u4e2a":35,"\u6700\u540e\u4e00\u5c42cost\u4e2d\u8bb0\u5f55\u4e86\u795e\u7ecf\u7f51\u7edc\u7684\u6240\u6709\u62d3\u6251\u7ed3\u6784":33,"\u6700\u540e\u4e00\u90e8\u5206\u662f\u795e\u7ecf\u7f51\u7edc\u7684\u914d\u7f6e":28,"\u6700\u540e\u6211\u4eec\u4f7f\u7528\u94fe\u5f0f\u6cd5\u5219\u8ba1\u7b97":41,"\u6700\u597d\u7684\u6a21\u578b\u662f":64,"\u6700\u5c11\u663e\u793a\u591a\u5c11\u4e2a\u8282\u70b9":47,"\u6700\u5e38\u89c1\u7684\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u662fexcept":25,"\u6700\u65b0\u7684\u4ee3\u7801":40,"\u6700\u65b0log":64,"\u6700\u7ec8":41,"\u6700\u7ec8\u5b9e\u73b0\u4e00\u4e2a\u5c42\u6b21\u5316\u7684\u590d\u6742rnn":38,"\u6700\u7ec8\u6211\u4eec\u53ef\u4ee5\u8c03\u7528trainer\u7684train\u65b9\u6cd5\u542f\u52a8\u8bad\u7ec3":33,"\u6700\u7ec8\u751f\u6210\u540d\u4e3apaddl":30,"\u6700\u7ec8\u7684\u8f93\u51fa\u7ed3\u679c":38,"\u6700\u7ec8\u8d8b\u4e8e\u63a5\u8fd1":28,"\u6708\u6e56":36,"\u6709":36,"\u6709\u4e00\u4e2a\u57fa\u672c\u7684\u8ba4\u8bc6":50,"\u6709\u4e24\u4e2a\u7279\u6b8a\u6807\u8bc6":65,"\u6709\u4e86meta\u914d\u7f6e\u6587\u4ef6\u4e4b\u540e":62,"\u6709\u4e9b\u5c42\u53ef\u80fd\u9700\u8981\u9ad8\u7cbe\u5ea6\u6765\u4fdd\u8bc1\u68af\u5ea6\u68c0\u67e5\u5355\u6d4b\u6b63\u786e\u6267\u884c":41,"\u6709\u4e9b\u5c42\u6216\u8005\u6fc0\u6d3b\u9700\u8981\u505a\u5f52\u4e00\u5316\u4ee5\u4fdd\u8bc1\u5b83\u4eec\u7684\u8f93\u51fa\u7684\u548c\u662f\u4e00\u4e2a\u5e38\u6570":41,"\u6709\u4e9b\u7279\u5f81\u7684\u53d6\u503c\u8fbe\u5230\u6570\u767e\u4e07":27,"\u6709\u4e9b\u7535\u5f71id\u53ef\u80fd\u4e0e\u5b9e\u9645\u7535\u5f71\u4e0d\u76f8\u7b26\u5408":61,"\u6709\u5173":36,"\u6709\u5173\u5982\u4f55\u7f16\u5199\u6570\u636e\u63d0\u4f9b\u7a0b\u5e8f\u7684\u66f4\u591a\u7ec6\u8282\u63cf\u8ff0":39,"\u6709\u5173\u7ebf\u6027\u56de\u5f52\u7684\u5b9e\u9645\u5e94\u7528":33,"\u6709\u5173kubernetes\u76f8\u5173\u6982\u5ff5\u4ee5\u53ca\u5982\u4f55\u642d\u5efa\u548c\u914d\u7f6ekubernetes\u96c6\u7fa4":52,"\u6709\u52a9\u4e8e\u7406\u89e3\u7528\u6237\u5bf9\u4e0d\u540c\u516c\u53f8":64,"\u6709\u52a9\u4e8e\u8bca\u65ad\u5206\u5e03\u5f0f\u9519\u8bef":45,"\u6709\u65f6\u79f0\u4e3a":64,"\u6709\u6807\u51c6\u7684":25,"\u6709\u7684\u65f6\u5019":25,"\u6709\u7684\u65f6\u5019\u7b80\u7b80\u5355\u5355\u7684\u6539\u53d8\u5c31\u80fd\u5728\u6027\u80fd\u4e0a\u4ea7\u751f\u660e\u663e\u7684\u4f18\u5316\u6548\u679c":44,"\u670d\u52a1":36,"\u670d\u52a1\u5458":36,"\u672a\u5305\u542b\u5728\u5b57\u5178\u4e2d\u7684\u5355\u8bcd":65,"\u672a\u6807\u8bb0\u7684\u8bc4\u4ef7\u6837\u672c":64,"\u672a\u77e5\u8bcd":56,"\u672c\u4f8b\u4e2d\u4e3a0":56,"\u672c\u4f8b\u4e2d\u4e3a32":56,"\u672c\u4f8b\u4e2d\u4e3a4":56,"\u672c\u4f8b\u4e2d\u4f7f\u7528for\u5faa\u73af\u8fdb\u884c\u591a\u6b21\u8c03\u7528":3,"\u672c\u4f8b\u4e2d\u7684\u539f\u59cb\u6570\u636e\u4e00\u5171\u670910\u4e2a\u6837\u672c":36,"\u672c\u4f8b\u4e2d\u7684\u8f93\u5165\u7279\u5f81\u662f\u8bcdid\u7684\u5e8f\u5217":3,"\u672c\u4f8b\u6839\u636e\u7f51\u7edc\u914d\u7f6e\u4e2d":3,"\u672c\u4f8b\u6bcf\u884c\u4fdd\u5b58\u4e00\u6761\u6837\u672c":60,"\u672c\u4f8b\u7531\u6613\u5230\u96be\u5c55\u793a4\u79cd\u4e0d\u540c\u7684\u6587\u672c\u5206\u7c7b\u7f51\u7edc\u914d\u7f6e":60,"\u672c\u4f8b\u7684":3,"\u672c\u4f8b\u7684\u6240\u6709\u5b57\u7b26\u90fd\u5c06\u8f6c\u6362\u4e3a\u8fde\u7eed\u6574\u6570\u8868\u793a\u7684id\u4f20\u7ed9\u6a21\u578b":60,"\u672c\u4f8b\u91c7\u7528\u82f1\u6587\u60c5\u611f\u5206\u7c7b\u7684\u6570\u636e":3,"\u672c\u4f8b\u91c7\u7528adam\u4f18\u5316\u65b9\u6cd5":60,"\u672c\u5730\u6d4b\u8bd5":46,"\u672c\u5730\u8bad\u7ec3":46,"\u672c\u5730\u8bad\u7ec3\u7684\u5b9e\u9a8c":49,"\u672c\u5b9e\u4f8b\u4e2d":56,"\u672c\u5c0f\u8282\u6211\u4eec\u5c06\u4ecb\u7ecd\u6a21\u578b\u7f51\u7edc\u7ed3\u6784":60,"\u672c\u5c42\u5c3a\u5bf8":58,"\u672c\u5c42\u6709\u56db\u4e2a\u53c2\u6570":58,"\u672c\u6559\u7a0b\u4e2d\u6211\u4eec\u7ed9\u51fa\u4e86\u4e09\u4e2aresnet\u6a21\u578b":58,"\u672c\u6559\u7a0b\u5c06\u4ecb\u7ecd\u4f7f\u7528\u6df1\u5ea6\u53cc\u5411\u957f\u77ed\u671f\u8bb0\u5fc6":63,"\u672c\u6559\u7a0b\u5c06\u6307\u5bfc\u4f60\u5982\u4f55\u5728":39,"\u672c\u6559\u7a0b\u5c06\u6307\u5bfc\u60a8\u5b8c\u6210\u957f\u671f\u77ed\u671f\u8bb0\u5fc6":64,"\u672c\u6559\u7a0b\u63d0\u4f9b\u4e86\u4e00\u4e2a\u7528\u4e8eimagenet\u4e0a\u7684\u5377\u79ef\u5206\u7c7b\u7f51\u7edc\u6a21\u578b":58,"\u672c\u6587\u4e2d\u6240\u6709\u7684\u4f8b\u5b50":36,"\u672c\u6587\u4e2d\u7531\u4e8e\u8f93\u5165\u6570\u636e\u662f\u968f\u673a\u751f\u6210\u7684\u4e0d\u9700\u8981\u8bfb\u8f93\u5165\u6587\u4ef6":28,"\u672c\u6587\u4e2d\u793a\u4f8b\u6240\u4f7f\u7528\u7684\u5355\u5143\u6d4b\u8bd5\u6587\u4ef6\u662f":36,"\u672c\u6587\u4ee5paddlepaddle\u7684\u53cc\u5c42rnn\u5355\u5143\u6d4b\u8bd5\u4e3a\u793a\u4f8b":36,"\u672c\u6587\u53ea\u4f7f\u7528\u4e86\u9ed8\u8ba4\u547d\u540d\u7a7a\u95f4":50,"\u672c\u6587\u5c06\u4ecb\u7ecd\u5728kubernetes\u5bb9\u5668\u7ba1\u7406\u5e73\u53f0\u4e0a\u5feb\u901f\u6784\u5efapaddlepaddle\u5bb9\u5668\u96c6\u7fa4":52,"\u672c\u6587\u6863\u4ecb\u7ecd\u5982\u4f55\u5728paddlepaddle\u5e73\u53f0\u4e0a":56,"\u672c\u6587\u6863\u5185\u4e0d\u91cd\u590d\u4ecb\u7ecd":50,"\u672c\u6765":36,"\u672c\u6b21\u8bad\u7ec3\u6587\u4ef6\u6240\u5728\u76ee\u5f55":52,"\u672c\u6b21\u8bad\u7ec3\u7684yaml\u6587\u4ef6\u53ef\u4ee5\u5199\u6210":52,"\u672c\u6b21\u8bad\u7ec3\u8981\u6c42\u67093\u4e2apaddlepaddle\u8282\u70b9":52,"\u672c\u6b21\u8bd5\u9a8c":60,"\u672c\u793a\u4f8b\u4e2d\u4f7f\u7528\u7684\u539f\u59cb\u6570\u636e\u5982\u4e0b":36,"\u672c\u793a\u4f8b\u610f\u56fe\u4f7f\u7528\u5355\u5c42rnn\u548c\u53cc\u5c42rnn\u5b9e\u73b0\u4e24\u4e2a\u5b8c\u5168\u7b49\u4ef7\u7684\u5168\u8fde\u63a5rnn":36,"\u672c\u793a\u4f8b\u7684\u9884\u6d4b\u7ed3\u679c":64,"\u672c\u7bc7\u6559\u7a0b\u5728paddlepaddle\u4e2d\u91cd\u73b0\u4e86\u8fd9\u4e00\u826f\u597d\u7684\u8bad\u7ec3\u7ed3\u679c":65,"\u672c\u7bc7\u6559\u7a0b\u5c06\u4f1a\u6307\u5bfc\u4f60\u901a\u8fc7\u8bad\u7ec3\u4e00\u4e2a":65,"\u672c\u8d28\u4e0a\u4e0e\u8bad\u7ec3\u6a21\u578b\u4e00\u6837":65,"\u673a\u5668\u7684\u8bbe\u5907":49,"\u673a\u5668\u7ffb\u8bd1":59,"\u6743\u91cd\u66f4\u65b0\u7684\u68af\u5ea6":47,"\u6761\u4ef6\u4e0b":50,"\u6765":36,"\u6765\u4ee3\u66ff":40,"\u6765\u505a\u68af\u5ea6\u68c0\u67e5":41,"\u6765\u505ableu\u8bc4\u4f30":65,"\u6765\u505c\u6b62\u8bad\u7ec3":62,"\u6765\u5206\u6790\u6267\u884c\u6587\u4ef6":44,"\u6765\u5206\u79bb\u8bad\u7ec3\u548c\u6d4b\u8bd5\u6587\u4ef6":62,"\u6765\u5206\u9694\u6bcf\u4e00\u884c":62,"\u6765\u521d\u59cb\u5316\u53c2\u6570":27,"\u6765\u5b89\u88c5":45,"\u6765\u5b8c\u6210\u7f51\u7edc\u7684\u8bad\u7ec3":33,"\u6765\u5b9a\u4e49\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":39,"\u6765\u5bf9\u6bd4\u5206\u6790\u4e24\u8005\u8bed\u4e49\u76f8\u540c\u7684\u539f\u56e0":36,"\u6765\u5e2e\u52a9\u4f60\u7406\u89e3paddlepaddle\u7684\u5185\u90e8\u8fd0\u884c\u673a\u5236":60,"\u6765\u5f00\u542f\u672c\u5730\u7684\u8bad\u7ec3":64,"\u6765\u5f97\u5230\u67d0\u4e2a\u7279\u5b9a\u53c2\u6570\u7684\u68af\u5ea6\u77e9\u9635":41,"\u6765\u6307\u5b9a\u7f51\u7edc\u5c42\u7684\u6570\u76ee":58,"\u6765\u63a5\u53d7\u4e0d\u4f7f\u7528\u7684\u51fd\u6570\u4ee5\u4fdd\u8bc1\u517c\u5bb9\u6027":3,"\u6765\u642d\u5efa\u795e\u7ecf\u7f51\u7edc":33,"\u6765\u6ce8\u518c\u8be5\u5c42":41,"\u6765\u6df7\u5408\u4f7f\u7528gpu\u548ccpu\u8ba1\u7b97\u7f51\u7edc\u5c42\u7684\u53c2\u6570":49,"\u6765\u751f\u6210\u5e8f\u5217":65,"\u6765\u7684\u79d2\u6570":61,"\u6765\u786e\u4fdd\u628a":25,"\u6765\u786e\u5b9a\u5bf9\u5e94\u5173\u7cfb":3,"\u6765\u81ea\u5b9a\u4e49\u4f20\u6570\u636e\u7684\u8fc7\u7a0b":2,"\u6765\u83b7\u5f97\u8f93\u51fa\u7684\u68af\u5ea6":41,"\u6765\u8868\u793a":39,"\u6765\u8868\u793a\u53c2\u6570\u4f4d\u7f6e":63,"\u6765\u8868\u793a\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":39,"\u6765\u8868\u793apaddle\u5185\u90e8\u7c7b":25,"\u6765\u8ba1\u7b97\u68af\u5ea6":41,"\u6765\u8bb2\u89e3\u5982\u4f55\u4f7f\u7528\u53cc\u5c42rnn":36,"\u6765\u8bbe\u7f6e":27,"\u6765\u8bf4\u660epydataprovider2\u7684\u7b80\u5355\u4f7f\u7528\u573a\u666f":3,"\u6765\u8c03\u6574":40,"\u6765\u8fd0\u884c":45,"\u6765\u8fd0\u884c\u6027\u80fd\u5206\u6790\u548c\u8c03\u4f18":44,"\u6765\u8fd0\u884c\u955c\u50cf":30,"\u6765\u9884\u6d4b\u8fd9\u4e2a\u4e2d\u95f4\u7684\u8bcd":27,"\u676f\u5b50":36,"\u6784\u5efapaddlepaddle\u6587\u6863\u9700\u8981\u51c6\u5907\u7684\u73af\u5883\u76f8\u5bf9\u8f83\u590d\u6742":42,"\u6784\u6210\u4e86\u8f93\u51fa\u53cc\u5c42\u5e8f\u5217\u7684\u7b2ci\u4e2a":35,"\u6784\u9020":52,"\u6784\u9020paddl":5,"\u67b6\u6784\u6765\u8bad\u7ec3\u60c5\u611f\u5206\u6790\u6a21\u578b":64,"\u67d0\u4e00\u4e2a\u795e\u7ecf\u5143\u7684\u4e00\u4e2a\u8f93\u5165\u4e3a\u4e0a\u4e00\u4e2a\u65f6\u95f4\u6b65\u7f51\u7edc\u4e2d\u67d0\u4e00\u4e2a\u795e\u7ecf\u5143\u7684\u8f93\u51fa":36,"\u67d0\u4e9b\u53c2\u6570\u53ea\u53ef\u7528\u4e8e\u7279\u5b9a\u7684\u5c42\u4e2d":46,"\u67e5\u770b":[40,60],"\u67e5\u770b\u5b89\u88c5\u540e\u7684paddl":32,"\u67e5\u770b\u5f53\u524d\u72b6\u6001":40,"\u67e5\u770b\u5f53\u524d\u8fdc\u7a0b\u4ed3\u5e93\u7684\u540d\u5b57":40,"\u67e5\u770b\u6587\u4ef6\u5177\u4f53\u88ab\u4fee\u6539\u7684\u5185\u5bb9":40,"\u67e5\u770bjob\u7684\u8be6\u7ec6\u60c5\u51b5":51,"\u6807\u51c6\u5dee\u4e3a":27,"\u6807\u51c6lstm\u4ee5\u6b63\u5411\u5904\u7406\u8be5\u5e8f\u5217":63,"\u6807\u793a\u56fe\u7247\u662f\u5f69\u8272\u56fe\u6216\u7070\u5ea6\u56fe":57,"\u6807\u793a\u662f\u5426\u4e3a\u5f69\u8272\u56fe\u7247":57,"\u6807\u7b7e0\u8868\u793a\u8d1f\u9762\u7684\u8bc4\u8bba":64,"\u6807\u7b7e1\u8868\u793a\u6b63\u9762\u7684\u8bc4\u8bba":64,"\u6807\u7b7e\u4e0b\u627e\u5230\u6700\u65b0\u7684paddle\u955c\u50cf\u7248\u672c":30,"\u6807\u7b7e\u6587\u4ef6":63,"\u6807\u7b7e\u65b9\u6848\u6765\u81ea":63,"\u6807\u8bc6\u662f\u5426\u4e3a\u8fde\u7eed\u7684batch\u8ba1\u7b97":47,"\u6839\u636e\u4f60\u7684\u4efb\u52a1":49,"\u6839\u636e\u524d\u6587\u7684\u63cf\u8ff0":52,"\u6839\u636e\u5728\u6a21\u578b\u914d\u7f6e\u6587\u4ef6\u4e2d\u4f7f\u7528\u7684\u540d\u4e3a":45,"\u6839\u636e\u6570\u636e\u91cf\u89c4\u6a21":61,"\u6839\u636e\u7528\u6237\u6307\u5b9a\u7684\u5b57\u5178":56,"\u6839\u636e\u7d22\u5f15\u77e9\u9635\u548c\u5b57\u5178\u6253\u5370\u6587\u672c":39,"\u6839\u636e\u7f51\u7edc\u914d\u7f6e\u4e2d\u7684":47,"\u6839\u636e\u8fd9\u4e9b\u53c2\u6570\u7684\u4f7f\u7528\u573a\u5408":46,"\u6839\u636e\u9ed8\u8ba4\u503c\u9012\u589e":47,"\u6839\u636e\u9ed8\u8ba4\u7aef\u53e3\u53f7\u9012\u589e":47,"\u6839\u636ecpu":30,"\u6839\u636ejob\u5bf9\u5e94\u7684pod\u4fe1\u606f":51,"\u683c\u5f0f":47,"\u683c\u5f0f\u5982\u4e0b":60,"\u683c\u5f0f\u8bf4\u660e":56,"\u68af\u5ea6\u4f1a\u5c31\u5730":41,"\u68af\u5ea6\u53c2\u6570\u7684\u5206\u5757\u6570\u76ee":47,"\u68af\u5ea6\u5c31\u53ef\u4ee5\u901a\u8fc7\u8fd9\u4e2a\u65b9\u7a0b\u8ba1\u7b97\u5f97\u5230":41,"\u68af\u5ea6\u670d\u52a1\u5668\u7684\u6570\u91cf":47,"\u68af\u5ea6\u68c0\u67e5\u5355\u5143\u6d4b\u8bd5\u901a\u8fc7\u6709\u9650\u5dee\u5206\u6cd5\u6765\u9a8c\u8bc1\u4e00\u4e2a\u5c42\u7684\u68af\u5ea6":41,"\u68af\u5ea6\u68c0\u67e5\u7684\u8f93\u5165\u6570\u636e\u7684\u6279\u6b21\u5927\u5c0f":41,"\u68d2":60,"\u697c\u5c42":36,"\u6a21\u5757":57,"\u6a21\u5757\u4e2d\u7684":3,"\u6a21\u578b":63,"\u6a21\u578b\u4e00\u76f4\u4e0d\u6536\u655b":27,"\u6a21\u578b\u4fdd\u5b58\u5728\u76ee\u5f55":64,"\u6a21\u578b\u5171\u5305\u542b1":56,"\u6a21\u578b\u5217\u8868\u6587\u4ef6":63,"\u6a21\u578b\u53ca\u53c2\u6570\u4f1a\u88ab\u4fdd\u5b58\u5728\u8def\u5f84":57,"\u6a21\u578b\u5b58\u50a8\u8def\u5f84":60,"\u6a21\u578b\u5c06\u4fdd\u5b58\u5728\u76ee\u5f55":63,"\u6a21\u578b\u5c31\u8bad\u7ec3\u6210\u529f\u4e86":65,"\u6a21\u578b\u6587\u4ef6\u5c06\u88ab\u5199\u5165\u8282\u70b9":45,"\u6a21\u578b\u6765\u5c06\u6cd5\u8bed\u7ffb\u8bd1\u6210\u82f1\u8bed":65,"\u6a21\u578b\u6765\u6307\u5bfc\u4f60\u5b8c\u6210\u8fd9\u4e9b\u6b65\u9aa4":39,"\u6a21\u578b\u6f14\u793a\u5982\u4f55\u914d\u7f6e\u590d\u6742\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u6a21\u578b":39,"\u6a21\u578b\u7684\u4ee3\u7801\u53ef\u4ee5\u5728":39,"\u6a21\u578b\u7684\u7ed3\u6784\u548c\u8bad\u7ec3\u8fc7\u7a0b":56,"\u6a21\u578b\u7684\u7f16\u7801\u5668\u90e8\u5206\u5982\u4e0b\u6240\u793a":39,"\u6a21\u578b\u88ab\u4fdd\u5b58\u5728":62,"\u6a21\u578b\u8bad\u7ec3\u4f1a\u770b\u5230\u7c7b\u4f3c\u4e0a\u9762\u8fd9\u6837\u7684\u65e5\u5fd7\u4fe1\u606f":60,"\u6a21\u578b\u8bad\u7ec3\u548c\u6700\u540e\u7684\u7ed3\u679c\u9a8c\u8bc1":28,"\u6a21\u578b\u8bad\u7ec3\u7b49\u4efb\u52a1":33,"\u6a21\u578b\u8def\u5f84":[58,63],"\u6a21\u578b\u8f93\u51fa\u8def\u5f84":63,"\u6a21\u578b\u914d\u7f6e":1,"\u6a21\u578b\u914d\u7f6e\u89e3\u6790":25,"\u6a21\u578b\u91c7\u7528":56,"\u6a21\u578b\u9884\u6d4b":5,"\u6b21":36,"\u6b22\u8fce\u901a\u8fc7":40,"\u6b63\u6837\u672c":60,"\u6b63\u786e\u7684\u89e3\u51b3\u65b9\u6cd5\u662f":27,"\u6b63\u8d1f\u5bf9\u9a8c\u8bc1":46,"\u6b63\u9762\u7684\u8bc4\u8bba\u7684\u5f97\u5927\u4e8e\u7b49\u4e8e7":64,"\u6b63\u9762\u8bc4\u4ef7\u6837\u672c":64,"\u6b64\u5904":56,"\u6b64\u5904\u90fd\u4e3a2":36,"\u6b64\u6559\u7a0b\u5c06\u5411\u60a8\u5206\u6b65\u4ecb\u7ecd\u5982\u4f55\u4f7f\u7528\u5185\u7f6e\u7684\u5b9a\u65f6\u5de5\u5177":44,"\u6b64\u6570\u636e\u96c6\u5305\u542b\u7535\u5f71\u8bc4\u8bba\u53ca\u5176\u76f8\u5173\u8054\u7684\u7c7b\u522b\u6807\u7b7e":64,"\u6b64\u65f6\u60a8\u53ef\u4ee5\u8fd0\u884c\u8fd9\u4e2a\u547d\u4ee4\u5728\u5f00\u53d1\u673a\u4e0a\u8fdb\u884c\u6d4b\u8bd5\u8fd0\u884c":30,"\u6bb5\u843d\u53ef\u4ee5\u770b\u4f5c\u662f\u4e00\u4e2a\u5d4c\u5957\u7684\u53cc\u5c42\u7684\u5e8f\u5217":38,"\u6bcf100\u4e2abatch\u6253\u5370\u4e00\u6b21\u7edf\u8ba1\u4fe1\u606f":64,"\u6bcf100\u4e2abatch\u663e\u793a\u53c2\u6570\u7edf\u8ba1":63,"\u6bcf20\u4e2abatch\u6253\u5370\u4e00\u6b21\u65e5\u5fd7":64,"\u6bcf20\u4e2abatch\u8f93\u51fa\u65e5\u5fd7":63,"\u6bcf\u4e00\u4e2a\u4efb\u52a1\u6d41\u7a0b\u90fd\u53ef\u4ee5\u88ab\u5212\u5206\u4e3a\u5982\u4e0b\u4e94\u4e2a\u6b65\u9aa4":60,"\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65":36,"\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65\u4e4b\u95f4\u7684\u795e\u7ecf\u7f51\u7edc\u5177\u6709\u4e00\u5b9a\u7684\u76f8\u5173\u6027":36,"\u6bcf\u4e00\u4e2a\u6d4b\u8bd5\u5468\u671f\u6d4b\u8bd5\u4e00\u6b21\u6240\u6709\u6570\u636e":62,"\u6bcf\u4e00\u4e2a\u8282\u70b9\u90fd\u6709\u76f8\u540c\u7684\u65e5\u5fd7\u7ed3\u6784":45,"\u6bcf\u4e00\u4e2akey\u7531":62,"\u6bcf\u4e00\u7ec4\u5185\u7684\u6240\u6709\u53e5\u5b50\u548clabel":36,"\u6bcf\u4e00\u884c\u8868\u793a\u4e00\u4e2a\u5b9e\u4f8b":64,"\u6bcf\u4e2a":[39,45,63],"\u6bcf\u4e2a\u5143\u7d20\u662f\u4e00\u4e2a0\u5c42\u5e8f\u5217":35,"\u6bcf\u4e2a\u5143\u7d20\u662f\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":35,"\u6bcf\u4e2a\u5355\u5c42rnn":38,"\u6bcf\u4e2a\u5355\u8bcd\u7684\u9884\u6d4b\u9519\u8bef\u7387":65,"\u6bcf\u4e2a\u53e5\u5b50\u53c8\u662f\u5355\u8bcd\u7684\u6570\u7ec4":36,"\u6bcf\u4e2a\u53e5\u5b50\u90fd\u4ee5\u5f00\u59cb\u6807\u8bb0\u5f00\u5934":39,"\u6bcf\u4e2a\u53e5\u5b50\u90fd\u4ee5\u7ed3\u675f\u6807\u8bb0\u7ed3\u5c3e":39,"\u6bcf\u4e2a\u5b50\u5e8f\u5217\u957f\u5ea6\u53ef\u4ee5\u4e0d\u4e00\u81f4":36,"\u6bcf\u4e2a\u5b50\u6587\u4ef6\u5939\u4e0b\u5b58\u50a8\u76f8\u5e94\u5206\u7c7b\u7684\u56fe\u7247":57,"\u6bcf\u4e2a\u5b57\u5178\u5305\u542b\u603b\u517130000\u4e2a\u5355\u8bcd":65,"\u6bcf\u4e2a\u5b57\u5178\u90fd\u6709dictsize\u4e2a\u5355\u8bcd":65,"\u6bcf\u4e2a\u5c42\u5728\u5176":41,"\u6bcf\u4e2a\u5c42\u90fd\u6709\u4e00\u4e2a\u6216\u591a\u4e2ainput":60,"\u6bcf\u4e2a\u6279\u6b21\u6570\u636e":47,"\u6bcf\u4e2a\u6574\u6570\u5217\u8868\u88ab\u89c6\u4e3a\u4e00\u4e2a\u6574\u6570\u5e8f\u5217":39,"\u6bcf\u4e2a\u6587\u4ef6\u5939\u90fd\u5305\u542b\u6cd5\u8bed\u5230\u82f1\u8bed\u7684\u5e73\u884c\u8bed\u6599\u5e93":65,"\u6bcf\u4e2a\u6587\u4ef6\u662f\u4e00\u4e2a\u7535\u5f71\u8bc4\u8bba":64,"\u6bcf\u4e2a\u6587\u672c\u6587\u4ef6\u5305\u542b\u4e00\u4e2a\u6216\u8005\u591a\u4e2a\u5b9e\u4f8b":64,"\u6bcf\u4e2a\u65f6\u95f4\u6b65\u4e4b\u5185\u7684\u8fd0\u7b97\u662f\u72ec\u7acb\u7684":38,"\u6bcf\u4e2a\u65f6\u95f4\u6b65\u90fd\u7528\u4e86\u4e0a\u4e00\u4e2a\u65f6\u95f4\u6b65\u7684\u8f93\u51fa\u7ed3\u679c":36,"\u6bcf\u4e2a\u6743\u91cd\u5bf9\u5e94\u4e00\u4e2a\u8f93\u5165":41,"\u6bcf\u4e2a\u6837\u672c\u7531\u4e24\u90e8\u5206\u7ec4\u6210":36,"\u6bcf\u4e2a\u6837\u672c\u95f4\u7528\u7a7a\u884c\u5206\u5f00":36,"\u6bcf\u4e2a\u6d4b\u8bd5\u5468\u671f\u6d4b\u8bd5":62,"\u6bcf\u4e2a\u7279\u5f81\u7684meta\u914d\u7f6e":62,"\u6bcf\u4e2a\u72b6\u6001":38,"\u6bcf\u4e2a\u7c7b\u522b\u4e2d\u968f\u673a\u62bd\u53d6\u4e8610\u5f20\u56fe\u7247":57,"\u6bcf\u4e2a\u7c7b\u5305\u542b6000\u5f20":57,"\u6bcf\u4e2a\u7ebf\u7a0b":47,"\u6bcf\u4e2a\u7ebf\u7a0b\u5206\u914d\u5230128\u4e2a\u6837\u672c\u7528\u4e8e\u8bad\u7ec3":47,"\u6bcf\u4e2a\u8282\u70b9\u6709\u4e24\u4e2a6\u6838cpu":65,"\u6bcf\u4e2a\u8bad\u7ec3\u8282\u70b9\u5fc5\u987b\u6307\u5b9a\u4e00\u4e2a\u552f\u4e00\u7684id\u53f7":47,"\u6bcf\u4e2a\u8bb0\u5fc6\u5355\u5143\u5305\u542b\u56db\u4e2a\u4e3b\u8981\u7684\u5143\u7d20":64,"\u6bcf\u4e2a\u8bc4\u8bba\u7684\u7f51\u5740":64,"\u6bcf\u4e2a\u8f93\u5165\u90fd\u662f\u4e00\u4e2a":41,"\u6bcf\u4e2a\u8f93\u51fa\u8282\u70b9\u90fd\u8fde\u63a5\u5230\u6240\u6709\u7684\u8f93\u5165\u8282\u70b9\u4e0a":41,"\u6bcf\u4e2a\u91cc\u9762\u90fd\u5305\u542b202mb\u7684\u5168\u90e8\u7684\u6a21\u578b\u53c2\u6570":65,"\u6bcf\u4e2apass\u7684\u7b2c0\u4e2abatch\u5230\u5f53\u524dbatch\u6240\u6709\u6837\u672c\u7684\u5e73\u5747\u5206\u7c7b\u9519\u8bef\u7387":60,"\u6bcf\u4e2apass\u7684\u7b2c0\u4e2abatch\u5230\u5f53\u524dbatch\u6240\u6709\u6837\u672c\u7684\u5e73\u5747cost":60,"\u6bcf\u4e2apod\u5305\u542b\u4e00\u4e2apaddlepaddle\u5bb9\u5668":52,"\u6bcf\u4f4d\u7528\u6237\u81f3\u5c11\u670920\u6761\u8bc4\u5206":61,"\u6bcf\u5c42\u4e0a\u53ea\u80fd\u4fdd\u5b58\u56fa\u5b9a\u6570\u76ee\u4e2a\u6700\u597d\u7684\u72b6\u6001":47,"\u6bcf\u5c42\u4f7f\u7528\u7684gpu\u53f7\u4f9d\u8d56\u4e8e\u53c2\u6570train":49,"\u6bcf\u5f53\u6a21\u578b\u5728\u7ffb\u8bd1\u8fc7\u7a0b\u4e2d\u751f\u6210\u4e86\u4e00\u4e2a\u5355\u8bcd":65,"\u6bcf\u6279\u6b21":47,"\u6bcf\u6b21":30,"\u6bcf\u6b21\u63d0\u4ea4\u4ee3\u7801":40,"\u6bcf\u6b21\u6d4b\u8bd5\u90fd\u6d4b\u8bd5\u6240\u6709\u6570\u636e":64,"\u6bcf\u6b21\u751f\u62101\u4e2a\u5e8f\u5217":65,"\u6bcf\u6b21\u8bfb\u53d6\u4e00\u6761\u6570\u636e\u540e":60,"\u6bcf\u6b21\u90fd\u4f1a\u4ecepython\u7aef\u8bfb\u53d6\u6570\u636e":3,"\u6bcf\u884c\u5b58\u50a8\u4e00\u4e2a\u8bcd":56,"\u6bcf\u884c\u5b58\u50a8\u7684\u662f\u4e00\u4e2a\u6837\u672c\u7684\u7279\u5f81":58,"\u6bcf\u884c\u6253\u537032\u4e2a\u53c2\u6570\u4ee5":56,"\u6bcf\u884c\u8868\u793a\u4e00\u4e2a\u6279\u6b21\u4e2d\u7684\u5355\u4e2a\u8f93\u5165":41,"\u6bcf\u884c\u90fd\u662f\u4e00\u4e2a\u6cd5\u8bed\u6216\u8005\u82f1\u8bed\u7684\u53e5\u5b50":65,"\u6bcf\u8f6e\u4f1a\u5c06\u6570\u636e\u96c6\u4e2d\u7684\u6240\u6709\u8bad\u7ec3\u6837\u672c\u4f7f\u7528\u4e00\u6b21":47,"\u6bcf\u8f6e\u7ed3\u675f\u65f6\u5bf9\u6240\u6709\u6d4b\u8bd5\u6570\u636e\u8fdb\u884c\u6d4b\u8bd5":47,"\u6bcf\u8f6e\u90fd\u4f1a\u4fdd\u5b58\u9884\u6d4b\u7ed3\u679c":47,"\u6bcf\u8fd0\u884c\u591a\u5c11\u4e2a\u6279\u6b21\u6267\u884c\u4e00\u6b21\u7a00\u758f\u53c2\u6570\u5206\u5e03\u7684\u68c0\u67e5":47,"\u6bcf\u9694\u591a\u5c11batch\u6253\u5370\u4e00\u6b21\u65e5\u5fd7":60,"\u6bcfdot":47,"\u6bcflog":47,"\u6bcfsave":47,"\u6bcftest":47,"\u6bd4\u5982":[27,30,40,60],"\u6bd4\u5982\u4e00\u53e5\u8bdd\u4e2d\u7684\u6bcf\u4e00\u4e2a\u5355\u8bcd":36,"\u6bd4\u5982\u5982\u679c\u8981build\u4e00\u4e2a\u4e0d\u4f9d\u8d56gpu":40,"\u6bd4\u5982\u8bbe\u7f6e\u4e00\u4e2a\u5168\u8fde\u63a5\u5c42\u7684\u53c2\u6570\u521d\u59cb\u5316\u65b9\u5f0f\u548cbias\u521d\u59cb\u5316\u65b9\u5f0f":27,"\u6bd4\u5982\u901a\u8fc78080\u7aef\u53e3":50,"\u6bd4\u5982ide\u914d\u7f6e\u91cc":40,"\u6c34\u6e29":36,"\u6c49\u5ead":36,"\u6c60\u5316\u5c42":57,"\u6ca1":36,"\u6ca1\u6709\u4f5c\u7528":3,"\u6ca1\u6709\u4f7f\u7528avx\u6307\u4ee4\u96c6":32,"\u6ca1\u6709\u5b9e\u9645\u610f\u4e49":56,"\u6ca1\u6709\u6d4b\u8bd5\u6570\u636e":3,"\u6ca1\u6709\u8fdb\u884c\u6b63\u786e\u6027\u7684\u68c0\u67e5":61,"\u6ca1\u6709\u8fdb\u884c\u7ed3\u6784\u7684\u5fae\u8c03":62,"\u6cd5\u8bed":65,"\u6ce8\u610f":[3,29,30,33,39,41,52,57],"\u6ce8\u610f\u4e0a\u8ff0\u547d\u4ee4\u4e2d":52,"\u6ce8\u610f\u5230\u6211\u4eec\u5df2\u7ecf\u5047\u8bbe\u673a\u5668\u4e0a\u67094\u4e2agpu":49,"\u6ce8\u610f\u5e94\u8be5\u786e\u4fdd\u9ed8\u8ba4\u6a21\u578b\u8def\u5f84":64,"\u6ce8\u610f\u9884\u6d4b\u6570\u636e\u901a\u5e38\u4e0d\u5305\u542blabel":5,"\u6ce8\u610fnode":52,"\u6ce8\u91ca\u6389":64,"\u6cf3\u6c60":36,"\u6d41":36,"\u6d41\u7a0b\u6765\u63d0\u4ea4\u4ee3\u7801":40,"\u6d44":36,"\u6d4b\u8bd5":40,"\u6d4b\u8bd5\u6570\u636e":45,"\u6d4b\u8bd5\u6570\u636e\u4e5f\u5305\u542b":45,"\u6d4b\u8bd5\u6570\u636e\u548c\u6a21\u578b\u914d\u7f6e\u6587\u4ef6":45,"\u6d4b\u8bd5\u6570\u636e\u548c\u751f\u6210\u6570\u636e":65,"\u6d4b\u8bd5\u6570\u636e\u653e\u7f6e\u5728\u5de5\u4f5c\u7a7a\u95f4\u4e2d\u4e0d\u540c\u76ee\u5f55\u7684\u8981\u6c42":45,"\u6d4b\u8bd5\u6570\u636e\u7684\u6240\u6709\u76f8\u5bf9\u6216\u7edd\u5bf9\u6587\u4ef6\u8def\u5f84":45,"\u6d4b\u8bd5\u6570\u6910\u96c6":64,"\u6d4b\u8bd5\u65f6\u6307\u5b9a\u7684\u5b58\u50a8\u6a21\u578b\u5217\u8868\u7684\u6587\u4ef6":47,"\u6d4b\u8bd5\u65f6\u9ed8\u8ba4\u4e0dshuffl":3,"\u6d4b\u8bd5\u662f":40,"\u6d4b\u8bd5\u6837\u672c":45,"\u6d4b\u8bd5\u6a21\u578b\u662f\u6307\u4f7f\u7528\u8bad\u7ec3\u51fa\u7684\u6a21\u578b\u8bc4\u4f30\u5df2\u6807\u8bb0\u7684\u9a8c\u8bc1\u96c6":64,"\u6d4b\u8bd5\u7684\u6a21\u578b\u5305\u62ec\u4ece\u7b2cm\u8f6e\u5230\u7b2cn":49,"\u6d4b\u8bd5\u811a\u672c\u662f":63,"\u6d4b\u8bd5\u96c6\u548c\u8bad\u7ec3\u96c6\u76ee\u5f55\u5305\u542b\u4e0b\u9762\u7684\u6587\u4ef6":64,"\u6d4b\u8bd5model_list":46,"\u6d4b\u8bd5save_dir":46,"\u6d4f\u89c8\u4ee3\u7801":30,"\u6d6a\u6f2b\u7247":61,"\u6d6e\u70b9\u5f02\u5e38\u901a\u5e38\u7684\u539f\u56e0\u662f\u6d6e\u70b9\u6570\u6ea2\u51fa":27,"\u6d6e\u70b9\u6570\u5360\u7528\u7684\u5b57\u8282\u6570":56,"\u6d6e\u70b9\u7a00\u758f\u6570\u636e":41,"\u6dd8\u5b9d\u7b49":64,"\u6df1\u5ea6\u53cc\u5411lstm\u5c42\u63d0\u53d6softmax\u5c42\u7684\u7279\u5f81":63,"\u6df1\u5ea6\u5b66\u4e60\u5165\u95e8\u8bfe\u7a0b":34,"\u6df7\u5408":63,"\u6df7\u5408\u5f53\u524d\u8bcd\u5411\u91cf\u548cattention\u52a0\u6743\u7f16\u7801\u5411\u91cf":39,"\u6dfb\u52a0\u542f\u52a8\u811a\u672c":52,"\u6e05\u7406\u6389\u8001\u65e7\u7684paddlepaddle\u5b89\u88c5\u5305":27,"\u6e29\u99a8":36,"\u6e90":65,"\u6e90\u4ee3\u7801":[30,60],"\u6e90\u4ee3\u7801\u683c\u5f0f":40,"\u6e90\u5b57\u5178":65,"\u6e90\u5e8f\u5217":39,"\u6e90\u7801":30,"\u6e90\u7801\u4e0edemo":51,"\u6e90\u8bed\u8a00\u5230\u76ee\u6807\u8bed\u8a00\u7684\u5e73\u884c\u8bed\u6599\u5e93\u6587\u4ef6":65,"\u6e90\u8bed\u8a00\u548c\u76ee\u6807\u8bed\u8a00\u5171\u4eab\u76f8\u540c\u7684\u7f16\u7801\u5b57\u5178":56,"\u6e90\u8bed\u8a00\u548c\u76ee\u6807\u8bed\u8a00\u90fd\u662f\u76f8\u540c\u7684\u8bed\u8a00":56,"\u6e90\u8bed\u8a00\u77ed\u8bed\u548c\u76ee\u6807\u8bed\u8a00\u77ed\u8bed\u7684\u5b57\u5178\u5c06\u88ab\u5408\u5e76":56,"\u6ee4\u6ce2\u5668\u6838\u5728\u5782\u76f4\u65b9\u5411\u4e0a\u7684\u5c3a\u5bf8":58,"\u6ee4\u6ce2\u5668\u6838\u5728\u6c34\u5e73\u65b9\u5411\u4e0a\u7684\u5c3a\u5bf8":58,"\u6f14\u793a\u4e2d\u4f7f\u7528\u7684":63,"\u6f14\u793a\u91c7\u7528":63,"\u6fc0\u6d3b":41,"\u6fc0\u6d3b\u51fd\u6570\u7c7b\u578b":60,"\u6fc0\u6d3b\u65b9\u7a0b":41,"\u6fc0\u6d3b\u7684\u7c7b\u578b":41,"\u7075\u6d3b\u6027\u548c\u53ef\u6269\u5c55\u6027":0,"\u70ed\u60c5":36,"\u7136\u540e":[44,45,56,62],"\u7136\u540e\u4ea4\u7ed9\u7528\u6237\u81ea\u5b9a\u4e49\u7684\u51fd\u6570":28,"\u7136\u540e\u4ea4\u7ed9step\u51fd\u6570":38,"\u7136\u540e\u4f60\u53ea\u9700\u8981\u8fd0\u884c\u4ee5\u4e0b\u547d\u4ee4":65,"\u7136\u540e\u4f7f\u7528\u4e0b\u9762\u7684\u811a\u672c":64,"\u7136\u540e\u518d\u505a\u4e00\u6b21\u6587\u672c\u5377\u79ef\u7f51\u7edc\u64cd\u4f5c":62,"\u7136\u540e\u5229\u7528\u89c2\u6d4b\u6570\u636e\u8c03\u6574":28,"\u7136\u540e\u5355\u51fb":40,"\u7136\u540e\u53ef\u4ee5\u4f7f\u7528\u547d\u4ee4\u884c\u5de5\u5177\u521b\u5efajob":52,"\u7136\u540e\u53ef\u4ee5\u8f6c\u6362\u4e3a\u56fe\u7247":58,"\u7136\u540e\u5728":65,"\u7136\u540e\u5728\u4e0b\u4e00\u4e2a\u65f6\u95f4\u6b65\u8f93\u5165\u7ed9\u53e6\u4e00\u4e2a\u795e\u7ecf\u5143":36,"\u7136\u540e\u5728\u6d4f\u89c8\u5668\u4e2d\u8f93\u5165\u4ee5\u4e0b\u7f51\u5740":30,"\u7136\u540e\u5728\u89e3\u7801\u88ab\u7ffb\u8bd1\u7684\u8bed\u53e5\u65f6":65,"\u7136\u540e\u5728dataprovider\u91cc\u9762\u6839\u636e\u8be5\u5730\u5740\u52a0\u8f7d\u5b57\u5178":27,"\u7136\u540e\u5b9a\u4e49":39,"\u7136\u540e\u5c06\u6784\u5efa\u6210\u529f\u7684\u955c\u50cf\u4e0a\u4f20\u5230\u955c\u50cf\u4ed3\u5e93":52,"\u7136\u540e\u5f97\u5230\u5e73\u5747\u91c7\u6837\u7684\u7ed3\u679c":62,"\u7136\u540e\u6211\u4eec\u5229\u7528\u591a\u8f93\u5165\u7684":62,"\u7136\u540e\u6211\u4eec\u53d1\u73b0pass":65,"\u7136\u540e\u6211\u4eec\u5806\u53e0\u4e00\u5bf9\u5bf9\u7684":63,"\u7136\u540e\u6211\u4eec\u6c42\u8fd9\u4e24\u4e2a\u7279\u5f81\u7684\u4f59\u5f26\u76f8\u4f3c\u5ea6":62,"\u7136\u540e\u6240\u6709\u7528":40,"\u7136\u540e\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4":58,"\u7136\u540e\u628a\u8fd9\u4e2a\u5305\u542b\u4e86\u8bad\u7ec3\u6570\u636e\u7684container\u4fdd\u5b58\u4e3a\u4e00\u4e2a\u65b0\u7684\u955c\u50cf":51,"\u7136\u540e\u63d0\u4ea4\u65b0\u6dfb\u52a0\u7684":40,"\u7136\u540e\u63d0\u53d6\u9690\u85cflstm\u5c42\u7684\u6240\u6709\u65f6\u95f4\u6b65\u957f\u7684\u6700\u5927\u8bcd\u5411\u91cf\u4f5c\u4e3a\u6574\u4e2a\u5e8f\u5217\u7684\u8868\u793a":64,"\u7136\u540e\u662f\u5bf9\u5e94\u7684\u82f1\u8bed\u5e8f\u5217":65,"\u7136\u540e\u6dfb\u52a0\u4e00\u4e2a\u5168\u8fde\u63a5\u5c42":62,"\u7136\u540e\u70b9\u51fb":40,"\u7136\u540e\u7528\u5bc6\u7801":30,"\u7136\u540e\u7528pickle\u547d\u4ee4\u5c06\u7279\u5f81":62,"\u7136\u540e\u7533\u660e\u4e00\u4e2a\u5b58\u50a8\u5377":52,"\u7136\u540e\u89c2\u5bdf\u5230\u8f93\u51fa\u7684\u53d8\u5316\u4e3a":41,"\u7136\u540e\u89e3\u538b":65,"\u7136\u540e\u89e3\u7801\u5668\u901a\u8fc7\u8fd9\u4e2a\u5411\u91cf\u751f\u6210\u4e00\u4e2a\u76ee\u6807\u8bed\u53e5":65,"\u7136\u540e\u8f93\u51fa\u9884\u6d4b\u5206\u6570":62,"\u7136\u540e\u8fd4\u56de\u7ed9paddlepaddle\u8fdb\u7a0b":3,"\u7136\u540e\u8fdb\u884c\u968f\u673a\u6253\u4e71":62,"\u7136\u540e\u901a\u8fc7\u51fd\u6570":52,"\u7136\u540e\u901a\u8fc7\u81ea\u8eab\u7684ip\u5730\u5740\u5728":52,"\u7136\u800c":[39,47],"\u7136\u800c\u6709\u4e9b\u8bc4\u8bba\u4e0a\u4e0b\u6587\u975e\u5e38\u957f":64,"\u7248\u672c":32,"\u7248\u672c\u5728":40,"\u7279\u522b\u611f\u8c22paddlepaddle\u7684":0,"\u7279\u522b\u662f\u5728lstm\u7b49rnn\u4e2d":27,"\u7279\u522b\u662f\u5f53\u76f8\u540c\u7684\u8bcd\u5728\u53e5\u5b50\u4e2d\u51fa\u73b0\u591a\u4e8e\u4e00\u6b21\u65f6":63,"\u7279\u5f81":62,"\u7279\u5f81\u56fe\u5747\u503c":58,"\u7279\u5f81\u56fe\u65b9\u5dee":58,"\u7279\u5f81\u5c06\u4f1a\u5b58\u5230":58,"\u7279\u5f81\u6587\u4ef6":63,"\u7279\u5f81\u7684\u7c7b\u578b\u548c\u7ef4\u5ea6":62,"\u72af\u7f6a\u7247":61,"\u73af\u5883\u53d8\u91cf":52,"\u73af\u5883\u53d8\u91cf\u6765\u6307\u5b9a\u7279\u5b9a\u7684gpu":27,"\u73b0\u5728\u6211\u4eec\u53ef\u4ee5\u5f00\u59cbpaddle\u8bad\u7ec3\u4e86":62,"\u73b0\u9636\u6bb5paddle\u6709\u4e00\u4e2a\u95ee\u9898\u662f":25,"\u751a\u81f3\u4e0d\u540c\u7ade\u4e89\u5bf9\u624b\u4ea7\u54c1\u7684\u504f\u597d":64,"\u751a\u81f3\u80fd\u89e3\u91ca\u4e3a\u4ec0\u4e48\u67d0\u4e2a\u64cd\u4f5c\u82b1\u4e86\u5f88\u957f\u65f6\u95f4":44,"\u751f\u4ea7\u955c\u50cf":[30,40],"\u751f\u4ea7\u955c\u50cf\u7684\u751f\u6210\u5206\u4e3a\u4e24\u6b65":30,"\u751f\u6210":52,"\u751f\u6210\u5404\u79cd\u8bed\u8a00\u7684\u7ed1\u5b9a\u4ee3\u7801":25,"\u751f\u6210\u540e\u7684\u6587\u6863\u5206\u522b\u5b58\u50a8\u5728\u7f16\u8bd1\u76ee\u5f55\u7684":42,"\u751f\u6210\u5e8f\u5217\u7684\u6700\u5927\u957f\u5ea6":39,"\u751f\u6210\u5f53\u524d\u5c42\u7684\u6240\u6709\u540e\u7ee7\u72b6\u6001":65,"\u751f\u6210\u6570\u636e\u7684\u76ee\u5f55":65,"\u751f\u6210\u6587\u6863":25,"\u751f\u6210\u751f\u4ea7\u955c\u50cf\u53ea\u9700\u8981\u8fd0\u884c":30,"\u751f\u6210\u7684\u6570\u636e\u5c06\u4f1a\u5b58\u50a8\u5728\u8fd9\u4e2avolume\u4e0b":52,"\u751f\u6210\u7684\u6570\u636e\u7f13\u5b58\u5728\u5185\u5b58\u91cc":27,"\u751f\u6210\u7684\u7ed3\u679c\u6587\u4ef6":65,"\u751f\u6210\u7684html\u7248\u672c\u7684c":30,"\u751f\u6210\u7684meta\u914d\u7f6e\u6587\u4ef6\u5982\u4e0b\u6240\u793a":62,"\u751f\u6210\u7ed3\u679c\u6587\u4ef6\u7684\u8def\u5f84":39,"\u751f\u6210\u7f51\u7edc\u5c42\u914d\u7f6e":41,"\u751f\u6210\u81ea\u5df1\u76ee\u5f55\u4e0b\u7684\u4ed3\u5e93":40,"\u751f\u6210\u8bad\u7ec3\u9700\u8981\u7684\u6837\u672c":62,"\u751f\u6210\u8fd0\u884c\u7a0b\u5e8f":30,"\u751f\u6210api\u6587\u6863":25,"\u751f\u6210docker\u955c\u50cf\u7684\u65b9\u5f0f\u6709\u4e24\u4e2a":30,"\u7528":[61,62,63],"\u75280\u548c1\u8868\u793a":3,"\u7528\u4e86\u4e24\u4e2a\u6708\u4e4b\u540e\u8fd9\u4e2a\u663e\u793a\u5668\u5c4f\u5e55\u788e\u4e86":60,"\u7528\u4e8e":45,"\u7528\u4e8e\u5207\u5206\u5355\u5355\u8bcd\u548c\u6807\u70b9\u7b26\u53f7":64,"\u7528\u4e8e\u521d\u59cb\u5316\u53c2\u6570\u548c\u8bbe\u7f6e":41,"\u7528\u4e8e\u5c06\u4e0b\u4e00\u884c\u7684\u6570\u636e\u8f93\u5165\u51fd\u6570\u6807\u8bb0\u6210\u4e00\u4e2apydataprovider2":3,"\u7528\u4e8e\u5c06\u53c2\u6570\u4f20\u9012\u7ed9\u7f51\u7edc\u914d\u7f6e":49,"\u7528\u4e8e\u5c06\u8bcdid\u8f6c\u6362\u4e3a\u8bcd\u7684\u5b57\u5178\u6587\u4ef6":39,"\u7528\u4e8e\u6307\u5b9a\u7f51\u7edc\u914d\u7f6e\u6587\u4ef6":47,"\u7528\u4e8e\u653e\u7f6e":45,"\u7528\u4e8e\u6784\u6210\u65b0\u7684\u8bcd\u8868":56,"\u7528\u4e8e\u6807\u8bc6\u751f\u6210\u7684\u6587\u4ef6\u4e2d\u7684\u76f8\u5e94\u8f93\u51fa":39,"\u7528\u4e8e\u7a00\u758f\u8bad\u7ec3\u4e2d":47,"\u7528\u4e8e\u7edf\u8ba1\u8bcd\u9891\u7684bow\u6a21\u578b\u7279\u5f81":64,"\u7528\u4e8e\u81ea\u5b9a\u4e49\u6bcf\u6761\u6570\u636e\u7684batch":3,"\u7528\u4e8e\u8ba1\u7b97\u7f16\u7801\u5411\u91cf\u7684\u52a0\u6743\u548c":39,"\u7528\u4e8e\u8bbe\u7f6e\u8bad\u7ec3\u7b97\u6cd5":57,"\u7528\u4e8e\u8bfb\u53d6\u8bad\u7ec3":45,"\u7528\u4e8e\u96c6\u7fa4\u901a\u4fe1\u901a\u9053\u7684\u7aef\u53e3\u6570":45,"\u7528\u53cc\u5411\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7f16\u7801":39,"\u7528\u591a\u5bf9\u6548\u679c\u5b8c\u5168\u76f8\u540c\u7684":36,"\u7528\u6237":45,"\u7528\u62371\u7684\u7279\u5f81":62,"\u7528\u6237\u4e5f\u53ef\u4ee5\u5728c":2,"\u7528\u6237\u53ea\u9700\u5b9a\u4e49rnn\u5728\u4e00\u4e2a\u65f6\u95f4\u6b65\u5185\u5b8c\u6210\u7684\u8ba1\u7b97":38,"\u7528\u6237\u53ea\u9700\u6267\u884c":63,"\u7528\u6237\u53ea\u9700\u6267\u884c\u4ee5\u4e0b\u547d\u4ee4\u5c31\u53ef\u4ee5\u4e0b\u8f7d\u5e76\u5904\u7406\u539f\u59cb\u6570\u636e":63,"\u7528\u6237\u53ef\u4ee5\u4f7f\u7528\u5f00\u53d1\u955c\u50cf\u4ee3\u66ff\u914d\u7f6e\u672c\u5730\u73af\u5883":30,"\u7528\u6237\u53ef\u4ee5\u4f7f\u7528ssh\u767b\u5f55\u5230\u8fd9\u53f0\u670d\u52a1\u5668\u4e0a\u5e76\u6267\u884c":30,"\u7528\u6237\u53ef\u4ee5\u5728\u8f93\u51fa\u7684\u6587\u672c\u6a21\u578b\u4e2d\u770b\u5230":56,"\u7528\u6237\u53ef\u4ee5\u6839\u636e\u8bad\u7ec3\u65e5\u5fd7":60,"\u7528\u6237\u53ef\u4ee5\u81ea\u5b9a\u4e49beam":47,"\u7528\u6237\u53ef\u4ee5\u8bbe\u7f6e":49,"\u7528\u6237\u53ef\u4ee5\u901a\u8fc7\u7b80\u5355\u4f7f\u7528python\u63a5\u53e3":2,"\u7528\u6237\u53ef\u4ee5\u901a\u8fc7\u7f51\u9875\u6d4f\u89c8\u6587\u6863":30,"\u7528\u6237\u53ef\u5728\u8c03\u7528cmake\u7684\u65f6\u5019\u8bbe\u7f6e\u5b83\u4eec":29,"\u7528\u6237\u53ef\u5728cmake\u7684\u547d\u4ee4\u884c\u4e2d":29,"\u7528\u6237\u5728\u4f7f\u7528paddlepaddl":27,"\u7528\u6237\u5b9a\u4e49\u7684\u53c2\u6570":3,"\u7528\u6237\u5c06\u914d\u7f6e\u4e0e\u8bad\u7ec3\u6570\u636e\u5207\u5206\u597d\u653e\u5728\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf\u9884\u5148\u5206\u914d\u597d\u7684\u76ee\u5f55\u4e2d":52,"\u7528\u6237\u5e94\u8be5\u63d0\u4f9b\u7f51\u7edc\u914d\u7f6e\u6587\u4ef6":63,"\u7528\u6237\u5f3a\u5236\u6307\u5b9a\u7279\u5b9a\u7684python\u7248\u672c":27,"\u7528\u6237\u6307\u5b9a\u65b0\u7684\u5b57\u5178\u7684\u8def\u5f84":56,"\u7528\u6237\u6587\u4ef6\u4e2d\u6709\u56db\u79cd\u7c7b\u578b\u7684\u5b57\u6bb5":62,"\u7528\u6237\u7279\u5f81":62,"\u7528\u6237\u8fd8\u53ef\u4ee5\u6839\u636e\u6982\u7387\u5206\u5e03\u77e9\u9635\u5b9e\u73b0\u67f1\u641c\u7d22\u6216\u7ef4\u7279\u6bd4\u89e3\u7801":63,"\u7528\u6237\u9700\u8981\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u6307\u5b9a":49,"\u7528\u6237\u9700\u8981\u6307\u5b9a\u672c\u673a\u4e0apython\u7684\u8def\u5f84":27,"\u7528\u6237\u9884\u6d4b\u7684\u547d\u4ee4\u884c\u754c\u9762\u5982\u4e0b":62,"\u7528\u6237id":61,"\u7528\u6237id\u8303\u56f4\u4ece1\u52306040":61,"\u7528\u6765\u4ece\u53c2\u6570\u670d\u52a1\u5668\u9884\u53d6\u53c2\u6570\u77e9\u9635\u76f8\u5e94\u7684\u884c":41,"\u7528\u6765\u4f30\u8ba1\u7ebf\u6027\u51fd\u6570\u7684\u53c2\u6570w":28,"\u7528\u6765\u5177\u4f53\u63cf\u8ff0":62,"\u7528\u6765\u5177\u4f53\u8bf4\u660e\u6570\u636e\u96c6\u7684\u5b57\u6bb5\u548c\u6587\u4ef6\u683c\u5f0f":62,"\u7528\u6765\u8ba1\u7b97\u6a21\u578b\u7684\u8bef\u5dee":28,"\u7528\u8fd9\u4e2a\u955c\u50cf\u521b\u5efa\u7684\u5bb9\u5668\u9700\u8981\u6709\u4ee5\u4e0b\u4e24\u4e2a\u529f\u80fd":52,"\u7531":38,"\u7531\u4e8e\u4e0d\u540c\u7684paddle\u7684\u7248\u672c\u53ef\u80fd\u9700\u8981\u4e0d\u540c\u7684\u4f9d\u8d56\u548c\u5de5\u5177":30,"\u7531\u4e8e\u5b83\u5185\u90e8\u5305\u542b\u4e86\u6bcf\u7ec4\u6570\u636e\u4e2d\u7684\u6240\u6709\u53e5\u5b50":36,"\u7531\u4e8e\u5bb9\u5668\u4e4b\u95f4\u5171\u4eabnet":50,"\u7531\u4e8e\u5df2\u7ecf\u77e5\u9053\u4e86\u771f\u5b9e\u7b54\u6848":28,"\u7531\u4e8e\u610f\u5916\u7684\u526f\u672c\u8bb0\u5f55\u548c\u6d4b\u8bd5\u8bb0\u5f55":61,"\u7531\u4e8e\u6211\u4eec\u60f3\u8981\u7684\u53d8\u6362\u662f\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217":36,"\u7531\u4e8e\u6211\u4eec\u652f\u6301\u8bad\u7ec3\u6570\u636e\u6709\u4e0d\u540c\u7684\u6279\u6b21\u5927\u5c0f":41,"\u7531\u4e8e\u6570\u636e\u8bb8\u53ef\u7684\u539f\u56e0":63,"\u7531\u4e8e\u6807\u51c6\u7684\u7ffb\u8bd1\u7ed3\u679c\u5df2\u7ecf\u4e0b\u8f7d\u5230\u8fd9\u91cc":65,"\u7531\u4e8e\u6bcf\u4e2a\u5377\u79ef\u5c42\u540e\u9762\u8fde\u63a5\u7684\u662fbatch":58,"\u7531\u4e8e\u8fd9\u4e2a\u5730\u5740\u4f1a\u88abdataprovider\u4f7f\u7528":2,"\u7531\u4e8e\u8fd9\u6837\u505a\u53ef\u4ee5\u907f\u514d\u5f88\u591a\u6b7b\u9501\u95ee\u9898":3,"\u7531\u4e8e\u987a\u5e8f\u8c03\u7528\u8fd9\u4e9bgenerator\u4e0d\u4f1a\u51fa\u73b0\u4e0a\u8ff0\u95ee\u9898":3,"\u7531\u4e8ec":25,"\u7531\u4e8epaddlepaddle\u5df2\u7ecf\u5b9e\u73b0\u4e86\u4e30\u5bcc\u7684\u7f51\u7edc\u5c42":28,"\u7531\u4e8estep":38,"\u7531\u4e8etest_data\u5305\u542b\u4e24\u6761\u9884\u6d4b\u6570\u636e":5,"\u7531\u8bcd\u8bed\u6784\u6210\u7684\u53e5\u5b50":35,"\u7531grouplen":61,"\u7535\u5f711\u7684\u7279\u5f81":62,"\u7535\u5f71\u4fe1\u606f\u4ee5\u53ca\u7535\u5f71\u8bc4\u5206":61,"\u7535\u5f71\u540d\u5b57\u6bb5":62,"\u7535\u5f71\u540d\u79f0":61,"\u7535\u5f71\u548c\u7528\u6237":62,"\u7535\u5f71\u548c\u7528\u6237\u6709\u8bb8\u591a\u7684\u7279\u5f81":62,"\u7535\u5f71\u5927\u90e8\u5206\u662f\u624b\u5de5\u8f93\u5165\u6570\u636e":61,"\u7535\u5f71\u7279\u5f81":62,"\u7535\u5f71\u7c7b\u578b":61,"\u7535\u5f71\u7c7b\u578b\u5982\u7b26\u5408\u591a\u79cd\u7528\u7ba1\u9053\u7b26\u53f7":61,"\u7535\u5f71id":61,"\u7535\u5f71id\u8303\u56f4\u4ece1\u52303952":61,"\u7535\u8111":36,"\u767e\u4e07\u6570\u636e\u96c6":61,"\u7684":[36,40,45,51,52,60,64],"\u768410\u7ef4\u6574\u6570\u6807\u7b7e":3,"\u7684\u4e00\u4e2a\u7b80\u5355\u8c03\u7528\u5982\u4e0b":38,"\u7684\u4e00\u4e2a\u7ebf\u6027\u51fd\u6570":28,"\u7684\u4e00\u79cd":65,"\u7684\u4e3a0":47,"\u7684\u4e3b\u8981\u90e8\u5206":63,"\u7684\u4efb\u4e00\u4e00\u79cd":27,"\u7684\u4efb\u52a1":65,"\u7684\u4f7f\u7528\u793a\u4f8b\u5982\u4e0b":35,"\u7684\u504f\u7f6e\u5411\u91cf":41,"\u7684\u5185\u5b58":27,"\u7684\u5185\u5bb9\u5982\u4e0b\u6240\u793a":65,"\u7684\u5185\u5bb9\u6765\u5b9a\u5236imag":52,"\u7684\u5185\u6838block\u4f7f\u7528\u60c5\u51b5":44,"\u7684\u5206\u7c7b\u4efb\u52a1\u4e2d\u8d62\u5f97\u4e86\u7b2c\u4e00\u540d":58,"\u7684\u522b\u540d":[6,7,13,14,15],"\u7684\u5355\u8bcd\u7ea7\u522b\u7684\u5377\u79ef\u795e\u7ecf\u7f51\u7edc":62,"\u7684\u53cd\u5411\u4f20\u64ad\u5c06\u4f1a\u6253\u5370\u65e5\u5fd7\u4fe1\u606f":47,"\u7684\u53d8\u6362\u77e9\u9635":41,"\u7684\u53e5\u5b50\u7684\u60c5\u611f":64,"\u7684\u540d\u5b57":3,"\u7684\u540d\u79f0\u76f8\u540c":39,"\u7684\u540e\u7f00":61,"\u7684\u5411\u91cf":41,"\u7684\u542f\u52a8\u53c2\u6570":52,"\u7684\u542f\u52a8\u53c2\u6570\u5e76\u6267\u884c\u8fdb\u7a0b":52,"\u7684\u547d\u540d\u98ce\u683c\u5e76\u4e0d\u80fd\u9002\u5e94\u5176\u4ed6\u7b2c\u4e09\u65b9\u8bed\u8a00":25,"\u7684\u5730\u5740":50,"\u7684\u5730\u65b9":40,"\u7684\u5747\u5300\u5206\u5e03":27,"\u7684\u5934\u6587\u4ef6":25,"\u7684\u5dee\u8ddd\u4e0d\u65ad\u51cf\u5c0f":28,"\u7684\u5e73\u5747\u503c":35,"\u7684\u5e8f\u5217\u5f62\u72b6\u4e00\u81f4":36,"\u7684\u5f00\u53d1\u5de5\u4f5c\u90fd\u5e94\u8be5\u5728\u4e00\u4e2a\u65b0\u7684\u5206\u652f\u4e0a\u5b8c\u6210":40,"\u7684\u603b":45,"\u7684\u63a5\u53e3\u6837\u5f0f":25,"\u7684\u63cf\u8ff0\u8bf4\u660e\u4e2d":40,"\u7684\u6570\u636e\u8bfb\u53d6\u811a\u672c\u548c\u7c7b\u4f3c\u4e8e":60,"\u7684\u6570\u76ee\u4e00\u81f4":35,"\u7684\u6587\u4ef6\u4e5f\u5e26\u5230\u65b0\u5206\u652f\u4e0a":40,"\u7684\u65b9\u6cd5\u5df2\u88ab\u8bc1\u660e\u662f\u4e00\u79cd\u5f3a\u5927\u7684\u6a21\u578b":65,"\u7684\u65b9\u7a0b":41,"\u7684\u65f6\u5019\u5982\u679c\u62a5\u4e00\u4e9b\u4f9d\u8d56\u672a\u627e\u5230\u7684\u9519\u8bef\u662f\u6b63\u5e38\u7684":32,"\u7684\u65f6\u95f4\u6b65\u4fe1\u606f\u6210\u6b63\u6bd4":27,"\u7684\u66f4\u8be6\u7ec6\u51c6\u786e\u7684\u5b9a\u4e49":36,"\u7684\u6700\u5c0f\u503c":47,"\u7684\u6700\u65b0\u4ee3\u7801\u5e76\u66f4\u65b0\u5f53\u524d\u5206\u652f":40,"\u7684\u67b6\u6784\u7684\u793a\u4f8b":39,"\u7684\u6837\u5f0f":40,"\u7684\u6838\u5fc3\u662f\u8bbe\u8ba1step\u51fd\u6570\u7684\u8ba1\u7b97\u903b\u8f91":38,"\u7684\u6bb5\u843d\u5b9a\u4e49\u4e3a\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":38,"\u7684\u6d4b\u8bd5\u6570\u636e\u96c6":63,"\u7684\u6e90\u7801\u4ee5\u53ca\u751f\u6210\u6587\u6863\u9700\u8981\u591a\u79cd\u5f00\u53d1\u5de5\u5177":40,"\u7684\u6e90\u7801\u91cc\u4f7f\u7528\u4e86":25,"\u7684\u7248\u672c\u53f7":56,"\u7684\u7279\u5f81":58,"\u7684\u72b6\u6001":38,"\u7684\u7528\u6237\u53c2\u8003":45,"\u7684\u76ee\u5f55\u7ed3\u6784\u4e3a":62,"\u7684\u771f\u5b9e\u5173\u7cfb\u4e3a":28,"\u7684\u77e9\u9635":41,"\u7684\u795e\u7ecf\u7f51\u7edc\u673a\u5668\u7ffb\u8bd1":65,"\u7684\u7a20\u5bc6\u5411\u91cf\u4f5c\u4e3a\u8f93\u5165":41,"\u7684\u7aef\u5230\u7aef\u7cfb\u7edf\u6765\u89e3\u51b3srl\u4efb\u52a1":63,"\u7684\u7b2ci\u4e2a\u503c":41,"\u7684\u7b2cj\u4e2a\u503c":41,"\u7684\u7d22\u5f15\u6587\u4ef6\u5f15\u7528\u8bad\u7ec3":45,"\u7684\u7ed3\u6784\u5982\u4e0b":62,"\u7684\u7ef4\u5ea6":56,"\u7684\u884c\u6570\u5e94\u8be5\u4e00\u81f4":65,"\u7684\u89c4\u8303":25,"\u7684\u8bad\u7ec3\u6a21\u578b\u811a\u672c":60,"\u7684\u8bdd":27,"\u7684\u8def\u5f84\u4e2d":64,"\u7684\u8f93\u5165":38,"\u7684\u8f93\u51fa":44,"\u7684\u8f93\u51fa\u4fe1\u606f\u5165\u624b\u662f\u4e2a\u4e0d\u9519\u7684\u9009\u62e9":44,"\u7684\u8f93\u51fa\u51fd\u6570\u8fd4\u56de\u7684\u662f\u4e0b\u4e00\u4e2a\u65f6\u523b\u8f93\u51fa\u8bcd\u7684":39,"\u7684\u8f93\u51fa\u683c\u5f0f":36,"\u7684\u8f93\u51fa\u88ab\u7528\u4f5c":39,"\u7684\u8fd4\u56de\u503c\u4e00\u81f4":61,"\u7684\u8fdc\u7a0b\u4ed3\u5e93\u7684\u540d\u5b57":40,"\u7684\u90e8\u5206":45,"\u7684\u914d\u7f6e":56,"\u7684\u9875\u9762\u5220\u9664\u8fdc\u7a0b\u4ed3\u5e93\u7684\u5206\u652f":40,"\u7684docker\u955c\u50cf":30,"\u76ee\u524d":[38,63],"\u76ee\u524d\u4f7f\u7528":40,"\u76ee\u524d\u5d4c\u5165python\u89e3\u91ca\u5668":25,"\u76ee\u524d\u5df2\u88ab\u767e\u5ea6\u5185\u90e8\u591a\u4e2a\u4ea7\u54c1\u7ebf\u5e7f\u6cdb\u4f7f\u7528":0,"\u76ee\u524d\u652f\u6301\u4e24\u79cd":35,"\u76ee\u524d\u652f\u6301fail":47,"\u76ee\u524d\u8be5\u53c2\u6570\u4ec5\u7528\u4e8eaucvalidationlayer\u548cpnpairvalidationlayer\u5c42":47,"\u76ee\u524d\u8fd8\u672a\u652f\u6301":38,"\u76ee\u524dpaddle\u7684\u8fdb\u7a0b\u6a21\u578b\u662fc":25,"\u76ee\u5f55":[45,51,52,64],"\u76ee\u5f55\u4e0b":[41,60,65],"\u76ee\u5f55\u4e0b\u627e\u5230":60,"\u76ee\u5f55\u4e0b\u7684demo\u8bad\u7ec3\u51fa\u6765":5,"\u76ee\u5f55\u4e0b\u7684python\u5305":27,"\u76ee\u5f55\u4e2d":[45,62],"\u76ee\u5f55\u4e2d\u7684":[44,45],"\u76ee\u5f55\u4e2dpaddl":52,"\u76ee\u5f55\u4f1a\u51fa\u73b0\u5982\u4e0b\u51e0\u4e2a\u65b0\u7684\u6587\u4ef6":63,"\u76ee\u5f55\u5c31\u6210\u4e3a\u4e86\u5171\u4eab\u5b58\u50a8":52,"\u76ee\u5f55\u7ed3\u6784\u5982\u4e0b":65,"\u76ee\u5f55\u91cc\u63d0\u4f9b\u4e86\u8be5\u6570\u636e\u7684\u4e0b\u8f7d\u811a\u672c\u548c\u9884\u5904\u7406\u811a\u672c":60,"\u76ee\u6807":65,"\u76ee\u6807\u51fd\u6570\u662f\u6807\u7b7e\u7684\u4ea4\u53c9\u71b5":63,"\u76ee\u6807\u5411\u91cf":39,"\u76ee\u6807\u5b57\u5178":65,"\u76f4\u5230\u8bad\u7ec3\u6536\u655b\u4e3a\u6b62":27,"\u76f4\u5230\u903c\u8fd1\u771f\u5b9e\u89e3":28,"\u76f4\u63a5\u4f7f\u7528c\u8bed\u8a00\u7684":25,"\u76f4\u63a5\u5bfc\u51fa\u5230c\u7684\u63a5\u53e3\u6bd4\u8f83\u56f0\u96be":25,"\u76f4\u63a5\u8fd0\u884c":30,"\u76f4\u63a5\u8fd4\u56de\u8ba1\u7b97\u7ed3\u679c":5,"\u76f4\u63a5\u8fdb\u5165\u8bad\u7ec3\u6a21\u578b\u7ae0\u8282":60,"\u76f8\u5173\u6982\u5ff5\u662f":3,"\u76f8\u5173\u7684\u9e1f\u7c7b\u6570\u636e\u96c6\u53ef\u4ee5\u4ece\u5982\u4e0b\u5730\u5740\u4e0b\u8f7d":57,"\u76f8\u5173\u8bba\u6587":63,"\u76f8\u53cd":65,"\u76f8\u540c\u540d\u5b57\u7684\u53c2\u6570":27,"\u76f8\u5bf9":36,"\u76f8\u5bf9\u4e8epaddlepaddle\u7a0b\u5e8f\u8fd0\u884c\u65f6\u7684\u8def\u5f84":2,"\u76f8\u5bf9mnist\u800c\u8a00":3,"\u76f8\u5e94\u7684\u6570\u636e\u8bfb\u53d6\u811a\u672c\u548c\u8bad\u7ec3\u6a21\u578b\u811a\u672c":60,"\u76f8\u5e94\u7684\u6570\u636e\u8fed\u4ee3\u5668\u5982\u4e0b":63,"\u76f8\u5e94\u7684\u6807\u8bb0\u53e5\u5b50\u662f":63,"\u76f8\u5f53":36,"\u77e9\u9635":46,"\u7814\u7a76\u4eba\u5458\u5206\u6790\u4e86\u51e0\u4e2a\u5173\u4e8e\u6d88\u8d39\u8005\u4fe1\u5fc3\u548c\u653f\u6cbb\u89c2\u70b9\u7684\u8c03\u67e5":64,"\u7814\u7a76\u751f":61,"\u786e\u4fdd\u7f16\u8bd1\u5668\u9009\u9879":40,"\u793a":60,"\u793a\u4f8b":[27,58],"\u793a\u4f8b3\u5bf9\u4e8e\u5355\u5c42rnn\u548c\u53cc\u5c42rnn\u6570\u636e\u5b8c\u5168\u76f8\u540c":36,"\u793a\u4f8b3\u7684\u914d\u7f6e\u4f7f\u7528\u4e86\u5355\u5c42rnn\u548c\u53cc\u5c42rnn":36,"\u793a\u4f8b3\u7684\u914d\u7f6e\u5206\u522b\u4e3a":36,"\u793e\u533a\u53c2\u4e0e\u56f0\u96be":25,"\u793e\u533a\u8d21\u732e\u4ee3\u7801\u5b66\u4e60\u6210\u672c\u9ad8":25,"\u795e\u7ecf\u7f51\u7edc\u4e5f\u9700\u8981\u4e00\u4e9b\u7279\u5b9a\u7684layer\u4f5c\u4e3a\u8f93\u5165\u63a5\u53e3":33,"\u795e\u7ecf\u7f51\u7edc\u53c2\u6570\u4ee5\u53ca\u8fed\u4ee3\u65b9\u7a0b":33,"\u795e\u7ecf\u7f51\u7edc\u5728\u8bad\u7ec3\u7684\u65f6\u5019":27,"\u795e\u7ecf\u7f51\u7edc\u673a\u5668\u7ffb\u8bd1":65,"\u795e\u7ecf\u7f51\u7edc\u7684\u7f51\u7edc\u7ed3\u6784\u4e2d\u5177\u6709\u6709\u5411\u73af\u7ed3\u6784":36,"\u795e\u7ecf\u7f51\u7edc\u7684\u8bad\u7ec3\u672c\u8eab\u662f\u4e00\u4e2a\u975e\u5e38\u6d88\u8017\u5185\u5b58\u548c\u663e\u5b58\u7684\u5de5\u4f5c":27,"\u795e\u7ecf\u7f51\u7edc\u914d\u7f6e":28,"\u79bb":36,"\u79d1\u5b66\u5bb6":61,"\u79d1\u5e7b\u7247":61,"\u79f0\u4e3a":[39,40],"\u79f0\u4e3a\u5f00\u53d1\u955c\u50cf":30,"\u79f0\u4e3a\u751f\u4ea7\u955c\u50cf":30,"\u79f0\u4e4b\u4e3a\u53cc\u5c42\u5e8f\u5217\u7684\u4e00\u4e2a\u5b50\u5e8f\u5217":35,"\u79f0\u4e4b\u4e3a\u96c6\u675f\u5927\u5c0f":47,"\u7a00\u758f\u6570\u636e\u7684\u683c\u5f0f":41,"\u7a00\u758f\u66f4\u65b0\u7684\u7aef\u53e3\u6570\u91cf":52,"\u7a00\u758f\u768401\u5411\u91cf":[3,33],"\u7a00\u758f\u7684\u5411\u91cf":[3,33],"\u7a00\u758f\u77e9\u9635\u7684\u4e58\u79ef\u5e94\u7528\u4e8e\u524d\u5411\u4f20\u64ad\u8fc7\u7a0b":49,"\u7a0b\u5e8f\u4ece\u6b64\u76ee\u5f55\u62f7\u8d1d\u6587\u4ef6\u5230\u5bb9\u5668\u5185\u8fdb\u884c\u8bad\u7ec3":52,"\u7a0b\u5e8f\u505c\u6b62":47,"\u7a0b\u5e8f\u5458":61,"\u7a0b\u5e8f\u76f4\u63a5\u9000\u51fa":47,"\u7a20\u5bc6\u5411\u91cf":41,"\u7a20\u5bc6\u66f4\u65b0\u7684\u7aef\u53e3\u6570\u91cf":52,"\u7a20\u5bc6\u7684\u6d6e\u70b9\u6570\u5411\u91cf":[3,33],"\u7a97\u6237":36,"\u7acb\u523b\u9000\u51fa":27,"\u7aef\u53e3":45,"\u7aef\u53e3\u6570":45,"\u7aef\u53e3\u9644\u52a0\u5230\u4e3b\u673a\u540d\u4e0a":45,"\u7aef\u81ea\u5b9a\u4e49\u4e00\u4e2a":2,"\u7aef\u8bfb\u53d6\u6570\u636e":27,"\u7b2c":36,"\u7b2c\u4e00\u4e2a\u53c2\u6570\u662fsettings\u5bf9\u8c61":3,"\u7b2c\u4e00\u4e2a\u65b9\u6cd5\u7684\u597d\u5904\u662f\u7b80\u5355\u5feb\u6377":30,"\u7b2c\u4e00\u4e2a\u6837\u672c\u540c\u65f6encode\u4e24\u6761\u6570\u636e\u6210\u4e24\u4e2a\u5411\u91cf":36,"\u7b2c\u4e00\u4e2apass\u4f1a\u4ecepython\u7aef\u8bfb\u53d6\u6570\u636e":3,"\u7b2c\u4e00\u5929":36,"\u7b2c\u4e00\u6b65\u662f\u8fd0\u884c":30,"\u7b2c\u4e00\u7ae0\u8282":33,"\u7b2c\u4e00\u884c\u4ece":64,"\u7b2c\u4e00\u884c\u5b58\u7684\u662f\u56fe\u50cf":58,"\u7b2c\u4e00\u884c\u662f":56,"\u7b2c\u4e00\u884c\u7684":65,"\u7b2c\u4e00\u90e8\u5206\u5b9a\u4e49\u4e86\u6570\u636e\u8f93\u5165":28,"\u7b2c\u4e00\u90e8\u5206\u662f\u56fe\u7247\u7684\u6807\u7b7e":3,"\u7b2c\u4e09":64,"\u7b2c\u4e09\u5217\u662f\u751f\u6210\u7684\u82f1\u8bed\u5e8f\u5217":65,"\u7b2c\u4e09\u6b65":58,"\u7b2c\u4e8c":64,"\u7b2c\u4e8c\u4e2a\u65b9\u6cd5\u7684\u597d\u5904\u662fdockerfile\u53ef\u4ee5\u628a\u6574\u4e2a\u751f\u6210\u6d41\u7a0b\u63cf\u8ff0\u5f88\u6e05\u695a":30,"\u7b2c\u4e8c\u5217\u662f\u96c6\u675f\u641c\u7d22\u7684\u5f97\u5206":65,"\u7b2c\u4e8c\u6b65":[56,58],"\u7b2c\u4e8c\u6b65\u662f\u8fd0\u884c":30,"\u7b2c\u4e8c\u884c\u5b58\u7684\u662f\u56fe\u50cf":58,"\u7b2c\u4e8c\u90e8\u5206\u4e3b\u8981\u662f\u9009\u62e9\u5b66\u4e60\u7b97\u6cd5":28,"\u7b2c\u4e8c\u90e8\u5206\u662f28":3,"\u7b2ci\u884c\u7b2cj\u5217\u7684\u6570\u503c":41,"\u7b49\u5176\u4ed6":65,"\u7b49\u53c2\u6570":52,"\u7b49\u591a\u79cd\u516c\u6709\u4e91\u73af\u5883":50,"\u7b49\u5f85\u8fd9\u4e2a\u7a0b\u5e8f\u6267\u884c\u6210\u529f\u5e76\u8fd4\u56de0\u5219\u6210\u529f\u9000\u51fa":50,"\u7b49\u7b49":[60,65],"\u7b49\u90fd\u5c5e\u4e8e\u4e00\u4e2a\u547d\u540d\u7a7a\u95f4":50,"\u7b80\u4ecb":43,"\u7b80\u5355\u6765\u8bf4":44,"\u7b80\u5355\u7684\u5168\u8fde\u63a5\u7f51\u7edc":27,"\u7b80\u5355\u7684\u57fa\u4e8e\u5b57\u6bcd\u7684\u8bcd\u5d4c\u5165":62,"\u7b80\u5355\u7684\u6027\u80fd\u5206\u6790":44,"\u7b80\u5355\u7684\u6574\u4e2a\u8bcd\u5d4c\u5165":62,"\u7b80\u5355\u7684pydataprovider2\u6837\u4f8b\u5c31\u8bf4\u660e\u5b8c\u6bd5\u4e86":3,"\u7b80\u5355\u7684yaml\u6587\u4ef6\u5982\u4e0b":51,"\u7b80\u76f4":36,"\u7b97\u6cd5":[27,28,39,64],"\u7b97\u6cd5\u4e2d\u7684beam\u5927\u5c0f":39,"\u7b97\u6cd5\u914d\u7f6e":64,"\u7ba1\u7406\u4eba\u5458":61,"\u7ba1\u7406\u5458":61,"\u7c7b\u4f3c":35,"\u7c7b\u4f3c\u5730":63,"\u7c7b\u4f5c\u4e3a\u53c2\u6570\u7684\u62bd\u8c61":41,"\u7c7b\u522b\u4e2a\u6570":57,"\u7c7b\u522b\u4e2d\u7684\u53c2\u6570\u53ef\u7528\u4e8e\u6240\u6709\u573a\u5408":46,"\u7c7b\u522bid":60,"\u7c7b\u522bid\u548c\u6587\u672c\u4fe1\u606f\u7528":60,"\u7c7b\u540d\u548cc":25,"\u7c7b\u578b":[25,47,62],"\u7c7b\u578b\u53ef\u4ee5\u662fpaddlepaddle\u652f\u6301\u7684\u4efb\u610f\u8f93\u5165\u6570\u636e\u7c7b\u578b":35,"\u7c7b\u578b\u662fsparse_binary_vector":[3,33],"\u7c7b\u578b\u662fsparse_float_vector":[3,33],"\u7c7b\u578b\u6765\u8bbe\u7f6e":3,"\u7c7b\u578b\u7684":36,"\u7c7b\u7684\u6784\u9020\u51fd\u6570\u548c\u6790\u6784\u51fd\u6570":41,"\u7c7b\u9700\u8981\u5b9e\u73b0\u521d\u59cb\u5316":41,"\u7cfb\u7edf\u7f16\u8bd1wheel\u5305\u7684\u65f6\u5019":27,"\u7ea2\u697c\u68a6":56,"\u7eaa\u5f55\u7247":61,"\u7eafcpu\u955c\u50cf\u4ee5\u53cagpu\u955c\u50cf\u90fd\u4f1a\u7528\u5230avx\u6307\u4ee4\u96c6":30,"\u7ebf\u6027\u56de\u5f52\u7684\u8f93\u5165\u662f\u4e00\u6279\u70b9":28,"\u7ebf\u6027\u56de\u5f52\u7684\u8f93\u51fa\u662f\u4ece\u8fd9\u6279\u70b9\u4f30\u8ba1\u51fa\u6765\u7684\u53c2\u6570":28,"\u7ebf\u6027\u8ba1\u7b97\u7f51\u7edc\u5c42":28,"\u7ebf\u7a0bid\u53f7":49,"\u7ec6\u8282\u63cf\u8ff0":48,"\u7ecf\u5e38\u4f1a\u6d88\u8017\u657010gb\u7684\u5185\u5b58\u548c\u6570gb\u7684\u663e\u5b58":27,"\u7ed3\u5408":50,"\u7ed3\u675f\u6807\u8bb0":39,"\u7ed3\u6784\u5982\u4e0b":64,"\u7ed3\u6784\u5982\u4e0b\u56fe":56,"\u7ed3\u679c\u4fdd\u5b58\u5728":63,"\u7ed3\u679c\u53d1\u73b0\u5b83\u4eec\u4e0e\u540c\u65f6\u671f\u7684twitter\u6d88\u606f\u4e2d\u7684\u60c5\u7eea\u8bcd\u9891\u7387\u76f8\u5173":64,"\u7ed3\u8bba":25,"\u7ed9":36,"\u7ed9\u51fa\u56fe\u7247\u5c3a\u5bf8":57,"\u7ed9\u51fa\u8f93\u5165\u6570\u636e\u6240\u5728\u8def\u5f84":57,"\u7ed9\u5b9a\u52a8\u8bcd":63,"\u7ed9\u5b9a\u7684\u6587\u672c\u53ef\u4ee5\u662f\u4e00\u4e2a\u6587\u6863":64,"\u7ed9\u5b9aencoder\u8f93\u51fa\u548c\u5f53\u524d\u8bcd":38,"\u7ee7\u7eed\u8bad\u7ec3\u6216\u9884\u6d4b":3,"\u7ef4\u57fa\u767e\u79d1\u4e2d\u6587\u9875\u9762":36,"\u7ef4\u57fa\u767e\u79d1\u9875\u9762":36,"\u7ef4\u5ea6\u4e3aword_dim":60,"\u7ef4\u5ea6\u662f\u7c7b\u522b\u4e2a\u6570":60,"\u7ef4\u5ea6\u662f\u8bcd\u5178\u5927\u5c0f":60,"\u7ef4\u62a4":50,"\u7ef4\u7a7a\u95f4":39,"\u7ef4\u7a7a\u95f4\u5b8c\u6210":39,"\u7f13\u5b58\u6c60\u7684\u51cf\u5c0f":27,"\u7f13\u5b58\u8bad\u7ec3\u6570\u636e\u5230\u5185\u5b58":3,"\u7f16\u5199\u597d\u6570\u636e\u63d0\u4f9b\u811a\u672c\u540e":62,"\u7f16\u5199\u5b8cyaml\u6587\u4ef6\u540e":52,"\u7f16\u5199\u672c\u6b21\u8bad\u7ec3\u7684yaml\u6587\u4ef6":52,"\u7f16\u53f7":62,"\u7f16\u53f7\u4ece0\u5f00\u59cb":27,"\u7f16\u53f7\u5b57\u6bb5":62,"\u7f16\u7801\u5411\u91cf":39,"\u7f16\u7801\u5668\u8f93\u51fa":39,"\u7f16\u7801\u6e90\u5e8f\u5217":39,"\u7f16\u89e3\u7801\u6a21\u578b\u5c06\u4e00\u4e2a\u6e90\u8bed\u53e5\u7f16\u7801\u4e3a\u4e00\u4e2a\u5b9a\u957f\u7684\u5411\u91cf":65,"\u7f16\u8bd1":[30,40],"\u7f16\u8bd1\u5668\u6ca1\u6709":25,"\u7f16\u8bd1\u578b\u8bed\u8a00":25,"\u7f16\u8bd1\u5b8c\u6210\u540e":42,"\u7f16\u8bd1\u6210\u52a8\u6001\u5e93":47,"\u7f16\u8bd1\u6d41\u7a0b\u4e3b\u8981\u63a8\u8350\u9ad8\u7ea7\u7528\u6237\u67e5\u770b":31,"\u7f16\u8bd1\u751f\u6210":42,"\u7f16\u8bd1\u9009\u9879":29,"\u7f16\u8f91":50,"\u7f29\u653e\u53c2\u6570":58,"\u7f51\u7edc":[63,64],"\u7f51\u7edc\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf":50,"\u7f51\u7edc\u540d\u79f0":60,"\u7f51\u7edc\u5c42\u53ef\u4ee5\u6709\u591a\u4e2a\u8f93\u5165":41,"\u7f51\u7edc\u5c42\u7684\u6807\u8bc6\u7b26\u4e3a":41,"\u7f51\u7edc\u5c42\u7684\u7c7b\u578b":41,"\u7f51\u7edc\u5c42\u7684\u7ec6\u8282\u53ef\u4ee5\u901a\u8fc7\u4e0b\u9762\u8fd9\u4e9b\u4ee3\u7801\u7247\u6bb5\u6765\u6307\u5b9a":41,"\u7f51\u7edc\u5c42\u7684\u8f93\u51fa\u662f\u7ecf\u8fc7\u6fc0\u6d3b\u51fd\u6570\u4e4b\u540e\u7684\u503c":47,"\u7f51\u7edc\u5c42\u914d\u7f6e\u5305\u542b\u4ee5\u4e0b\u51e0\u9879":41,"\u7f51\u7edc\u63a5\u53e3\u5361":45,"\u7f51\u7edc\u6a21\u5757":58,"\u7f51\u7edc\u6a21\u578b\u5c06\u8f93\u51fa\u6807\u7b7e\u7684\u6982\u7387\u5206\u5e03":63,"\u7f51\u7edc\u7684\u8bad\u7ec3\u8fc7\u7a0b":64,"\u7f51\u7edc\u7ed3\u6784\u5982\u4e0b\u56fe\u6240\u793a":62,"\u7f51\u7edc\u901a\u4fe1":41,"\u7f51\u7edc\u914d\u7f6e":[45,60,64],"\u7f51\u7edc\u914d\u7f6e\u6587\u4ef6":[58,63],"\u800c":[28,39,51,62],"\u800c\u4e0d\u4f7f\u7528imdb\u6570\u6910\u96c6\u4e2d\u7684imdb":64,"\u800c\u4e0d\u652f\u6301pypy\u89e3\u91ca\u5668":25,"\u800c\u4e0d\u662f\u4f7f\u7528\u540c\u6b65":45,"\u800c\u4e0d\u662f\u6e90\u7801\u76ee\u5f55\u91cc":27,"\u800c\u4e0d\u662f\u7279\u5f81\u7684\u96c6\u5408":36,"\u800c\u4e0d\u662f\u7ec4\u5408\u4e0a\u4e0b\u6587\u7ea7\u522b\u4fe1\u606f":64,"\u800c\u4e0d\u7528\u5173\u5fc3\u6570\u636e\u5982\u4f55\u4f20\u8f93":3,"\u800c\u4e14":65,"\u800c\u4e4b\u524d\u7684\u53c2\u6570\u5c06\u4f1a\u88ab\u5220\u9664":47,"\u800c\u4ece\u5e94\u7528\u7684\u89d2\u5ea6":44,"\u800c\u4f18\u5316\u6027\u80fd\u7684\u9996\u8981\u4efb\u52a1":44,"\u800c\u5176\u4ed6\u5c42\u4f7f\u7528cpu\u8ba1\u7b97":49,"\u800c\u53cc\u5c42rnn\u662f\u53ef\u4ee5\u5904\u7406\u8fd9\u79cd\u8f93\u5165\u6570\u636e\u7684\u7f51\u7edc\u7ed3\u6784":36,"\u800c\u53f3\u56fe\u7684\u74f6\u9888\u8fde\u63a5\u6a21\u5757\u7528\u4e8e50\u5c42":58,"\u800c\u5728cpp\u91cc\u9762\u5b9e\u73b0\u8fd9\u4e2ac\u7684\u63a5\u53e3":25,"\u800c\u591a\u8bed\u8a00\u63a5\u53e3\u9700\u8981\u76f4\u63a5\u8bfb\u53d6\u751f\u6210\u7684\u4e8c\u8fdb\u5236":25,"\u800c\u5927\u591a\u6570\u65b9\u6cd5\u53ea\u662f\u5229\u7528n":64,"\u800c\u5bf9\u4e8e\u53cc\u5c42\u5e8f\u5217":36,"\u800c\u5bf9\u4e8e\u6bcf\u4e00\u4e2a\u5185\u5c42\u7279\u5f81\u6570\u636e\u800c\u8a00":36,"\u800c\u5bf9\u4e8egolang":25,"\u800c\u5bf9\u4e8egolang\u9519\u8bef\u5904\u7406\u5e94\u8be5\u4f7f\u7528\u8fd4\u56de\u503c":25,"\u800c\u5c06\u8fd9\u4e2a\u6bb5\u843d\u7684\u6bcf\u4e00\u53e5\u8bdd\u7528lstm\u7f16\u7801\u6210\u4e00\u4e2a\u5411\u91cf":36,"\u800c\u5f53\u524d\u5df2\u7ecf\u67095":44,"\u800c\u662f\u76f4\u63a5\u4ece\u5185\u5b58\u7684\u7f13\u5b58\u91cc\u8bfb\u53d6\u6570\u636e":27,"\u800c\u66f4\u6df1\u5165\u7684\u5206\u6790":44,"\u800c\u6709\u4e9b\u53c2\u6570\u9700\u8981\u5728\u96c6\u7fa4\u591a\u673a\u8bad\u7ec3\u4e2d\u4f7f\u7528\u7b49":46,"\u800c\u6ca1\u6709\u77ed\u65f6\u8bb0\u5fc6\u7684\u635f\u5931":64,"\u800c\u6e90\u5e8f\u5217\u7684\u7f16\u7801\u5411\u91cf\u53ef\u4ee5\u88ab\u65e0\u8fb9\u754c\u7684memory\u8bbf\u95ee":39,"\u800c\u795e\u7ecf\u7f51\u7edc\u662f\u6211\u4eec\u8981\u642d\u5efa\u7684\u5b9d\u5854":33,"\u800c\u7a00\u758f\u66f4\u65b0\u5728\u53cd\u5411\u4f20\u64ad\u4e4b\u540e\u7684\u6743\u91cd\u66f4\u65b0\u65f6\u8fdb\u884c":49,"\u800c\u7cfb\u7edf\u4e2d\u7684":27,"\u800c\u8fd9\u4e00\u53e5\u8bdd\u5c31\u53ef\u4ee5\u8868\u793a\u6210\u8fd9\u4e9b\u4f4d\u7f6e\u7684\u6570\u7ec4":36,"\u800c\u8fd9\u4e2acontext\u53ef\u80fd\u4f1a\u975e\u5e38\u5927":3,"\u800c\u8fd9\u6bcf\u4e00\u4e2a\u6570\u7ec4\u5143\u7d20":36,"\u800c\u975e\u9759\u6001\u52a0\u8f7dcuda\u52a8\u6001\u5e93":29,"\u800cpaddlepaddle\u5219\u4f1a\u5e2e\u7528\u6237\u505a\u4ee5\u4e0b\u5de5\u4f5c":3,"\u800crnn\u662f\u6700\u6d41\u884c\u7684\u9009\u62e9":38,"\u800cswig\u53ea\u80fd\u7b80\u5355\u7684\u66b4\u9732c":25,"\u800ctrainer\u9700\u8981\u8bfb\u53d6\u8bad\u7ec3\u6570\u636e\u8fdb\u884c\u8bad\u7ec3":33,"\u800cweight":57,"\u800cy_predict\u662f\u63a5\u6536x\u4f5c\u4e3a\u8f93\u5165":33,"\u804c\u4e1a":61,"\u804c\u4e1a\u4ece\u4e0b\u9762\u6240\u5217\u4e2d\u9009\u62e9":61,"\u80fd\u591f\u5904\u7406\u53cc\u5c42\u5e8f\u5217":38,"\u80fd\u591f\u5bf9\u53cc\u5411\u5e8f\u5217\u8fdb\u884c\u5904\u7406\u7684\u6709":38,"\u80fd\u591f\u627e\u5230\u8fd9\u91cc\u4f7f\u7528\u7684\u6240\u6709\u6570\u636e":60,"\u80fd\u591f\u8bb0\u5f55\u4e0a\u4e00\u4e2asubseq":38,"\u80fd\u83b7\u53d6":45,"\u811a\u672c":[45,57,62],"\u811a\u672c\u4fdd\u5b58\u5728":57,"\u811a\u672c\u5f00\u59cb\u65f6":52,"\u811a\u672c\u63d0\u4f9b\u4e86\u4e00\u4e2a\u9884\u6d4b\u63a5\u53e3":64,"\u811a\u672c\u65f6\u9700\u8981\u52a0\u4e0a":64,"\u811a\u672c\u8fd0\u884c\u5b8c\u6210\u540e":57,"\u81ea\u52a8\u5173\u95ed\u5bf9\u5e94\u7684":40,"\u81ea\u52a8\u5730\u5c06\u8fd9\u4e9b\u9009\u9879\u5e94\u7528\u5230":45,"\u81ea\u52a8\u5b8c\u6210\u8fd9\u4e00\u8fc7\u7a0b":38,"\u81ea\u52a8\u83b7\u53d6\u4e0a\u4e00\u4e2a\u751f\u6210\u7684\u8bcd":39,"\u81ea\u5e95\u5411\u4e0a\u6cd5":60,"\u81ea\u7136\u8bed\u8a00\u5904\u7406\u7b49":49,"\u81ea\u7531\u804c\u4e1a\u8005":61,"\u81f3\u6b64":[3,36,40],"\u8212\u9002":36,"\u826f\u597d\u7684\u6587\u6863":25,"\u827a\u672f\u5bb6":61,"\u8282\u70b9\u4e2d\u7684":45,"\u82e5":41,"\u82e5\u5e72\u4e2a\u53e5\u5b50\u6784\u6210\u4e00\u4e2a\u6bb5\u843d":35,"\u82e5\u6709\u4e0d\u4e00\u81f4\u4e4b\u5904":44,"\u82e5\u6709\u5fc5\u8981":41,"\u82e5\u8f93\u51fa\u662f\u5355\u5c42\u5e8f\u5217":35,"\u82e5\u8f93\u51fa\u662f\u53cc\u5c42\u5e8f\u5217":35,"\u82f1\u6587\u6587\u6863\u76ee\u5f55":42,"\u82f1\u8bed":65,"\u8303\u56f4":49,"\u83b7\u53d6":40,"\u83b7\u53d6\u5229\u7528":60,"\u83b7\u53d6\u5b57\u5178\u7ef4\u5ea6":64,"\u83b7\u53d6\u8be5\u6761\u6837\u672c\u7c7b\u522bid":60,"\u83b7\u53d6\u901a\u8fc7":64,"\u83b7\u53d6trainer":52,"\u83b7\u5f97\u53c2\u6570\u5c3a\u5bf8":41,"\u867d\u7136":28,"\u867d\u7136\u6bcf\u4e2agenerator\u5728\u6ca1\u6709\u8c03\u7528\u7684\u65f6\u5019":3,"\u867d\u7136\u8fd9\u4e9b\u6587\u4ef6\u5e76\u975e\u90fd\u9700\u8981\u96c6\u7fa4\u8bad\u7ec3":45,"\u867d\u7136paddle\u770b\u8d77\u6765\u5305\u542b\u4e86\u4f17\u591a\u53c2\u6570":46,"\u884c":56,"\u884c\u4f18\u5148\u6b21\u5e8f\u5b58\u50a8":58,"\u884c\u5185\u4f7f\u7528":3,"\u884c\u653f\u5de5\u4f5c":61,"\u884c\u955c\u50cf\u5305\u62ec\u7eafcpu\u7248\u672c\u548cgpu\u7248\u672c\u4ee5\u53ca\u5176\u5bf9\u5e94\u7684\u975eavx\u7248\u672c":30,"\u8868\u660e\u4e86\u8fd9\u4e9b\u884c\u7684\u6807\u53f7":41,"\u8868\u660e\u8fd9\u4e2a\u5c42\u7684\u4e00\u4e2a\u5b9e\u4f8b\u662f\u5426\u9700\u8981\u504f\u7f6e":41,"\u8868\u793a\u4e00\u4e2akubernetes\u96c6\u7fa4\u4e2d\u7684\u4e00\u4e2a\u5de5\u4f5c\u8282\u70b9":50,"\u8868\u793a\u4e3adeviceid":49,"\u8868\u793a\u5728\u96c6\u7fa4\u4f5c\u4e1a":45,"\u8868\u793a\u5973\u6027":61,"\u8868\u793a\u5c06\u5916\u5c42\u7684outer_mem\u4f5c\u4e3a\u5185\u5c42memory\u7684\u521d\u59cb\u72b6\u6001":36,"\u8868\u793a\u5f53\u524d\u96c6\u7fa4\u4f5c\u4e1a\u7684\u8282\u70b9":45,"\u8868\u793a\u6a21\u578b\u914d\u7f6e\u6587\u4ef6":45,"\u8868\u793a\u751f\u6210\u6570\u636e\u7684\u5e8f\u5217id":65,"\u8868\u793a\u7528\u4e8e\u8bad\u7ec3\u6216\u9884\u6d4b":3,"\u8868\u793a\u7537\u6027":61,"\u8868\u793a\u7684\u6bcf\u4e2a\u5355\u8bcd":60,"\u8868\u793a\u7b2c0\u4e2abatch\u5230\u5f53\u524dbatch\u7684\u5206\u7c7b\u9519\u8bef":64,"\u8868\u793a\u8bad\u7ec3\u4e86xx\u4e2a\u6837\u672c":64,"\u8868\u793a\u8bad\u7ec3\u4e86xx\u4e2abatch":64,"\u8868\u793a\u8bfb\u8005\u6240\u4f7f\u7528\u7684docker\u955c\u50cf\u4ed3\u5e93\u5730\u5740":52,"\u8868\u793a\u8fc7\u4e8620\u4e2abatch":60,"\u8868\u793a\u8fc7\u4e862560\u4e2a\u6837\u672c":60,"\u8868\u793a\u8fd9\u4e2ajob\u7684\u540d\u5b57":52,"\u88ab":40,"\u88ab\u6269\u5c55\u4e3a\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":35,"\u88ab\u653e\u5728":41,"\u88ab\u79f0\u4e3a":39,"\u897f\u90e8\u7247":61,"\u8981\u4e0b\u8f7d\u548c\u89e3\u538b\u6570\u636e\u96c6":62,"\u8981\u4e0b\u8f7d\u89e3\u538b\u8fd9\u4e2a\u6a21\u578b":65,"\u8981\u4f7f\u7528\u547d\u4ee4\u884c\u5206\u6790\u5de5\u5177":44,"\u8981\u5728\u5df2\u6709\u7684kubernetes\u96c6\u7fa4\u4e0a\u8fdb\u884cpaddlepaddle\u7684\u5206\u5e03\u5f0f\u8bad\u7ec3":52,"\u8981\u5728\u6240\u6709\u8282\u70b9\u4e0a\u5b58\u5728":45,"\u8981\u5bf9\u4e00\u4e2a\u56fe\u7247\u7684\u8fdb\u884c\u5206\u7c7b\u9884\u6d4b":57,"\u8981\u5c06\u5b57\u6bb5\u914d\u7f6e\u6587\u4ef6\u8f6c\u5316\u4e3ameta\u914d\u7f6e\u6587\u4ef6":62,"\u8981\u6c42\u5355\u5c42\u5e8f\u5217\u542b\u6709\u5143\u7d20\u7684\u6570\u76ee":35,"\u8981\u751f\u6210\u7684\u76ee\u6807\u5e8f\u5217":38,"\u8981\u8c03\u7528":41,"\u89e3\u51b3\u529e\u6cd5\u662f":27,"\u89e3\u51b3\u65b9\u6848\u662f":27,"\u89e3\u538b":65,"\u89e3\u6790\u5668\u80fd\u901a\u8fc7\u6587\u4ef6\u7684\u6269\u5c55\u540d\u81ea\u52a8\u8bc6\u522b\u6587\u4ef6\u7684\u683c\u5f0f":62,"\u89e3\u6790\u6570\u636e\u96c6\u4e2d\u7684\u6bcf\u4e00\u4e2a\u5b57\u6bb5":62,"\u89e3\u6790\u6a21\u578b\u914d\u7f6e\u6587\u4ef6":5,"\u89e3\u6790\u73af\u5883\u53d8\u91cf\u5f97\u5230":52,"\u89e3\u6790\u8bad\u7ec3\u6a21\u578b\u65f6\u7528\u7684\u914d\u7f6e\u6587\u4ef6":5,"\u89e3\u7801\u5668\u4f7f\u7528":39,"\u89e3\u7801\u5668\u57fa\u4e8e\u7f16\u7801\u6e90\u5e8f\u5217\u548c\u6700\u540e\u751f\u6210\u7684\u76ee\u6807\u8bcd\u9884\u6d4b\u4e0b\u4e00\u76ee\u6807\u8bcd":39,"\u89e3\u7801\u5668\u662f\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":39,"\u89e3\u7801\u5668\u6839\u636e\u4e0a\u4e0b\u6587\u5411\u91cf\u9884\u6d4b\u51fa\u4e00\u4e2a\u76ee\u6807\u5355\u8bcd":65,"\u89e3\u91ca":60,"\u89e3\u91ca\u578b\u8bed\u8a00\u53ea\u80fd\u8c03\u7528\u52a8\u6001\u5e93":25,"\u89e3\u91ca\u6027\u8bed\u8a00\u5b9e\u9645\u8fd0\u884c\u7684\u4e8c\u8fdb\u5236\u662f\u89e3\u91ca\u5668\u672c\u8eab":25,"\u8ba1\u7b97":39,"\u8ba1\u7b97\u504f\u7f6e\u7684\u68af\u5ea6":41,"\u8ba1\u7b97\u5355\u5143\u662f\u795e\u7ecf\u7f51\u7edc\u7684\u6838\u5fc3":28,"\u8ba1\u7b97\u53cd\u5411rnn\u7684\u7b2c\u4e00\u4e2a\u5b9e\u4f8b":39,"\u8ba1\u7b97\u53d8\u6362\u77e9\u9635\u7684\u5927\u5c0f\u548c\u683c\u5f0f":41,"\u8ba1\u7b97\u5f53\u524d\u5c42\u6743\u91cd\u7684\u68af\u5ea6":41,"\u8ba1\u7b97\u6bcf\u4e2a\u8bcd\u7684\u8bcd\u5411\u91cf":39,"\u8ba1\u7b97\u6fc0\u6d3b\u51fd\u6570\u7684\u68af\u5ea6":41,"\u8ba1\u7b97\u7684\u7ec6\u8282\u5c06\u5728\u4e0b\u9762\u7684\u5c0f\u8282\u7ed9\u51fa":41,"\u8ba1\u7b97\u8bef\u5dee\u51fd\u6570":28,"\u8ba1\u7b97\u8f6c\u6362\u77e9\u9635\u548c\u8f93\u5165\u7684\u68af\u5ea6":41,"\u8ba1\u7b97\u8f93\u5165\u548c\u53c2\u6570\u7684\u68af\u5ea6":41,"\u8ba1\u7b97\u8f93\u5165\u5c42\u7684\u504f\u5dee":41,"\u8ba1\u7b97\u8f93\u51fa":41,"\u8ba9\u6a21\u578b\u80fd\u591f\u5f97\u5230\u8bad\u7ec3\u66f4\u65b0":60,"\u8ba9\u795e\u7ecf\u7f51\u7edc\u53ef\u4ee5\u8fdb\u884c\u8bad\u7ec3\u6216\u9884\u6d4b":2,"\u8ba9\u8fd9\u4e2a\u793a\u4f8b\u53d8\u5f97\u66f4\u597d":62,"\u8bad\u7ec3":[46,64],"\u8bad\u7ec3\u4e0e\u5e94\u7528":1,"\u8bad\u7ec3\u4f5c\u4e1a":45,"\u8bad\u7ec3\u53ca\u6d4b\u8bd5\u8bef\u5dee\u66f2\u7ebf\u56fe\u4f1a\u88ab":57,"\u8bad\u7ec3\u53ef\u4ee5\u8bbe\u7f6e\u4e3atrue":63,"\u8bad\u7ec3\u540e":63,"\u8bad\u7ec3\u5931\u8d25\u65f6\u53ef\u4ee5\u68c0\u67e5\u9519\u8bef\u65e5\u5fd7":45,"\u8bad\u7ec3\u597d\u4e00\u4e2a\u6df1\u5c42\u795e\u7ecf\u7f51\u7edc\u901a\u5e38\u8981\u8017\u8d39\u975e\u5e38\u957f\u7684\u65f6\u95f4":44,"\u8bad\u7ec3\u5b8c\u6210\u540e":28,"\u8bad\u7ec3\u6570\u636e\u548c\u6d4b\u8bd5\u6570\u636e\u7684\u76ee\u5f55":65,"\u8bad\u7ec3\u6570\u636e\u662f":3,"\u8bad\u7ec3\u6570\u636e\u6709\u95ee\u9898":27,"\u8bad\u7ec3\u6570\u6910\u96c6":64,"\u8bad\u7ec3\u65f6":52,"\u8bad\u7ec3\u65f6\u6240\u9700\u8bbe\u7f6e\u7684\u4e3b\u8981\u53c2\u6570\u5982\u4e0b":60,"\u8bad\u7ec3\u65f6\u9ed8\u8ba4shuffl":3,"\u8bad\u7ec3\u6a21\u578b\u4e4b\u524d":64,"\u8bad\u7ec3\u6a21\u578b\u540e":39,"\u8bad\u7ec3\u7684\u635f\u5931\u51fd\u6570\u9ed8\u8ba4\u6bcf\u969410\u4e2abatch\u6253\u5370\u4e00\u6b21":65,"\u8bad\u7ec3\u7684\u811a\u672c\u662f":63,"\u8bad\u7ec3\u7ed3\u675f\u540e\u67e5\u770b\u8f93\u51fa\u7ed3\u679c":52,"\u8bad\u7ec3\u811a\u672c":60,"\u8bad\u7ec3\u811a\u672c\u7b49\u7b49":60,"\u8bad\u7ec3\u81f3\u591a":62,"\u8bad\u7ec3\u8282\u70b9\u6570\u91cf":52,"\u8bad\u7ec3\u8bed\u8a00\u6a21\u578b\u8ddd\u79bb":27,"\u8bad\u7ec3\u8f6e\u6b21":60,"\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u53c2\u6570\u6216\u8005\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u7684\u68af\u5ea6\u5c3a\u5ea6\u8fc7\u5927":27,"\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u6d4b\u8bd5test_period":46,"\u8bad\u7ec3\u8fc7\u7a0b\u662f\u5426\u4e3a\u672c\u5730\u6a21\u5f0f":47,"\u8bad\u7ec3\u8fc7\u7a0b\u662f\u5426\u4f7f\u7528gpu":47,"\u8bad\u7ec3\u914d\u7f6e\u4e2d\u7684\u8bbe\u5907\u5c5e\u6027\u5c06\u4f1a\u65e0\u6548":47,"\u8bad\u7ec3\u96c6\u548c\u6d4b\u8bd5\u96c6\u6587\u4ef6\u5217\u8868":64,"\u8bad\u7ec3\u96c6\u5df2\u7ecf\u505a\u4e86\u968f\u673a\u6253\u4e71\u6392\u5e8f\u800c\u6d4b\u8bd5\u96c6\u6ca1\u6709":64,"\u8bad\u7ec3\u96c6\u5df2\u7ecf\u968f\u673a\u6253\u4e71":64,"\u8bad\u7ec3\u96c6\u5e73\u5747\u503c":57,"\u8bad\u7ec3dot_period":46,"\u8bb0\u5fc6\u6a21\u5757":39,"\u8bba\u6587":58,"\u8bbe\u4e3a\u5df2\u90e8\u7f72\u7684\u5de5\u4f5c\u7a7a\u95f4\u76ee\u5f55":45,"\u8bbe\u4e3a\u672c\u5730":45,"\u8bbe\u7f6e\u4e3a":41,"\u8bbe\u7f6e\u4e3atrue\u4f7f\u7528\u672c\u5730\u8bad\u7ec3\u6216\u8005\u4f7f\u7528\u96c6\u7fa4\u4e0a\u7684\u4e00\u4e2a\u8282\u70b9":47,"\u8bbe\u7f6e\u4e3atrue\u4f7f\u7528gpu\u6a21\u5f0f":47,"\u8bbe\u7f6e\u4efb\u52a1\u7684\u6a21\u5f0f\u4e3a\u6d4b\u8bd5":65,"\u8bbe\u7f6e\u4fdd\u5b58\u6a21\u578b\u7684\u8f93\u51fa\u8def\u5f84":65,"\u8bbe\u7f6e\u5168\u5c40\u5b66\u4e60\u7387":64,"\u8bbe\u7f6e\u5185\u5b58\u4e2d\u6682\u5b58\u7684\u6570\u636e\u6761\u6570":3,"\u8bbe\u7f6e\u5185\u5b58\u4e2d\u6700\u5c0f\u6682\u5b58\u7684\u6570\u636e\u6761\u6570":3,"\u8bbe\u7f6e\u53c2\u6570\u7684\u540d\u5b57":27,"\u8bbe\u7f6e\u547d\u4ee4\u884c\u53c2\u6570":[27,43,62],"\u8bbe\u7f6e\u5b57\u5178\u6587\u4ef6":64,"\u8bbe\u7f6e\u5de5\u4f5c\u6a21\u5f0f\u4e3a\u8bad\u7ec3":64,"\u8bbe\u7f6e\u5e73\u5747sgd\u7a97\u53e3":64,"\u8bbe\u7f6e\u6210":27,"\u8bbe\u7f6e\u6210\u4e00\u4e2a\u5c0f\u4e00\u4e9b\u7684\u503c":27,"\u8bbe\u7f6e\u6570\u636e\u904d\u5386\u6b21\u6570":63,"\u8bbe\u7f6e\u6807\u7b7e\u7c7b\u522b\u5b57\u5178":64,"\u8bbe\u7f6e\u6a21\u578b\u8def\u5f84":64,"\u8bbe\u7f6e\u7684\u547d\u4ee4\u884c\u53c2\u6570":64,"\u8bbe\u7f6e\u795e\u7ecf\u7f51\u7edc\u7684\u914d\u7f6e\u6587\u4ef6":65,"\u8bbe\u7f6e\u7c7b\u522b\u6570":64,"\u8bbe\u7f6e\u7ebf\u7a0b\u6570":[63,64],"\u8bbe\u7f6e\u7f51\u7edc\u914d\u7f6e":64,"\u8bbe\u7f6e\u8f93\u51fa\u7684\u5c3a\u5bf8":41,"\u8bbe\u7f6e\u8f93\u51fa\u8def\u5f84\u4ee5\u4fdd\u5b58\u8bad\u7ec3\u5b8c\u6210\u7684\u6a21\u578b":64,"\u8bbe\u7f6e\u8fd9\u4e2apydataprovider2\u8fd4\u56de\u4ec0\u4e48\u6837\u7684\u6570\u636e":3,"\u8bbe\u7f6e\u9ed8\u8ba4\u8bbe\u5907\u53f7\u4e3a0":49,"\u8bbe\u7f6ebatch":64,"\u8bbe\u7f6ecpu\u7ebf\u7a0b\u6570\u6216\u8005gpu\u8bbe\u5907\u6570":65,"\u8bbe\u7f6egpu":47,"\u8bbe\u7f6epass":64,"\u8bbe\u7f6epasses\u7684\u6570\u91cf":65,"\u8bbf\u95eekubernetes\u7684\u63a5\u53e3\u6765\u67e5\u8be2\u6b64job\u5bf9\u5e94\u7684\u6240\u6709pod\u4fe1\u606f":52,"\u8bc4\u4ef7\u9884\u6d4b\u7684\u6548\u679c":28,"\u8bc4\u4f30\u8be5\u4ea7\u54c1\u7684\u8d28\u91cf":60,"\u8bc4\u5206":[61,62],"\u8bc4\u5206\u6587\u4ef6\u7684\u6bcf\u4e00\u884c\u4ec5\u4ec5\u63d0\u4f9b\u7535\u5f71\u6216\u7528\u6237\u7684\u7f16\u53f7\u6765\u4ee3\u8868\u76f8\u5e94\u7684\u7535\u5f71\u6216\u7528\u6237":62,"\u8bc4\u5206\u88ab\u8c03\u6574\u4e3a5\u661f\u7684\u89c4\u6a21":61,"\u8bcd\u5411\u91cf":56,"\u8bcd\u5411\u91cf\u6a21\u578b":59,"\u8bcd\u5411\u91cf\u6a21\u578b\u540d\u79f0":56,"\u8bcd\u672c\u8eab\u548c\u8bcd\u9891":56,"\u8bcd\u9891\u6700\u9ad8\u7684":65,"\u8bd5\u7740\u8ba9\u8f93\u51fa\u7684\u5206\u6790\u6570\u636e\u548c\u7406\u8bba\u503c\u5bf9\u5e94":44,"\u8be5":[45,63],"\u8be5\u51fd\u6570\u5177\u6709\u4e24\u4e2a\u53c2\u6570":3,"\u8be5\u51fd\u6570\u5728\u521d\u59cb\u5316\u7684\u65f6\u5019\u4f1a\u88ab\u8c03\u7528":3,"\u8be5\u51fd\u6570\u7684\u529f\u80fd\u662f":3,"\u8be5\u53c2\u6570\u5728\u7f51\u7edc\u914d\u7f6e\u7684output":47,"\u8be5\u53c2\u6570\u5728\u96c6\u7fa4\u63d0\u4ea4\u73af\u5883\u4e2d\u81ea\u52a8\u8bbe\u7f6e":47,"\u8be5\u53c2\u6570\u5df2\u7ecf\u5728\u96c6\u7fa4\u63d0\u4ea4\u73af\u5883\u4e2d\u5b8c\u6210\u8bbe\u7f6e":47,"\u8be5\u53c2\u6570\u5fc5\u987b\u80fd\u88abflag":47,"\u8be5\u53c2\u6570\u6307\u793a\u662f\u5426\u6253\u5370\u65e5\u5fd7\u622a\u65ad\u4fe1\u606f":47,"\u8be5\u53c2\u6570\u6307\u793a\u662f\u5426\u6253\u5370\u9519\u8bef\u622a\u65ad\u65e5\u5fd7":47,"\u8be5\u53c2\u6570\u7528\u4e8e\u6307\u5b9a\u52a8\u6001\u5e93\u8def\u5f84":47,"\u8be5\u53c2\u6570\u7684\u610f\u601d\u662f\u8bad\u7ec3num":47,"\u8be5\u53c2\u6570\u9ed8\u8ba4\u4e3anull":47,"\u8be5\u5bf9\u8c61\u5177\u6709\u4ee5\u4e0b\u4e24\u4e2a\u5c5e\u6027":3,"\u8be5\u5c42\u4ec5\u9700\u8981\u8fd9\u4e9b\u975e\u96f6\u6837\u672c\u4f4d\u7f6e\u6240\u5bf9\u5e94\u7684\u53d8\u6362\u77e9\u9635\u7684\u90a3\u4e9b\u884c":41,"\u8be5\u5c42\u795e\u7ecf\u5143\u4e2a\u6570":60,"\u8be5\u622a\u65ad\u4f1a\u5f71\u54cd":47,"\u8be5\u6279\u6b21\u7684\u8f93\u5165\u4e2d\u4ec5\u6709\u4e00\u4e2a\u5b50\u96c6\u662f\u975e\u96f6\u7684":41,"\u8be5\u63a5\u53e3\u4f7f\u7528\u591a\u7ebf\u7a0b\u8bfb\u53d6\u6570\u636e":3,"\u8be5\u63a5\u53e3\u53ef\u7528\u4e8e\u9884\u6d4b\u548c\u5b9a\u5236\u5316\u8bad\u7ec3":29,"\u8be5\u6570\u636e\u53ca\u6709\u5f88\u591a\u4e0d\u540c\u7684\u7248\u672c":61,"\u8be5\u6570\u636e\u96c6":56,"\u8be5\u6570\u636e\u96c6\u4e8e2003\u5e742\u6708\u53d1\u5e03":61,"\u8be5\u6570\u636e\u96c6\u5305\u542b\u4e00\u4e9b\u7528\u6237\u4fe1\u606f":61,"\u8be5\u6570\u76ee\u662f\u63d0\u524d\u5b9a\u4e49\u597d\u7684":47,"\u8be5\u6587\u4ef6\u53ef\u4ee5\u4ece\u5b57\u6bb5\u914d\u7f6e\u6587\u4ef6\u751f\u6210":62,"\u8be5\u6587\u4ef6\u662f\u7531cpickle\u4ea7\u751f\u7684":58,"\u8be5\u6587\u4ef6\u662fpython\u7684pickle\u5bf9\u8c61":62,"\u8be5\u6587\u4ef6\u8d1f\u8d23\u4ea7\u751f\u56fe\u7247\u6570\u636e\u5e76\u4f20\u9012\u7ed9paddle\u7cfb\u7edf":57,"\u8be5\u6a21\u578b\u4f9d\u7136\u4f7f\u7528\u903b\u8f91\u56de\u5f52\u5206\u7c7b\u7f51\u7edc\u7684\u6846\u67b6":60,"\u8be5\u6a21\u578b\u5728\u957f\u8bed\u53e5\u7ffb\u8bd1\u7684\u573a\u666f\u4e0b\u6548\u679c\u63d0\u5347\u66f4\u52a0\u660e\u663e":65,"\u8be5\u6a21\u578b\u7684\u7f51\u7edc\u914d\u7f6e\u5982\u4e0b":28,"\u8be5\u6a21\u578b\u7684\u8bf4\u660e\u5982\u4e0b\u56fe\u6240\u793a":39,"\u8be5\u6a21\u578b\u7f51\u7edc\u53ea\u662f\u7528\u4e8e\u8fdb\u884cdemo\u5c55\u793apaddle\u5982\u4f55\u5de5\u4f5c":62,"\u8be5\u76ee\u5f55\u4e0b\u4f1a\u751f\u6210\u5982\u4e0b\u4e24\u4e2a\u5b50\u76ee\u5f55":42,"\u8be5\u793a\u4f8b\u5c06\u5c55\u793apaddle\u5982\u4f55\u8fdb\u884c\u8bcd\u5411\u91cf\u5d4c\u5165":62,"\u8be5\u793a\u4f8b\u7684\u795e\u7ecf\u7f51\u7edc\u914d\u7f6e\u6587\u4ef6":62,"\u8be5\u7b97\u6cd5\u6bcf\u6279\u91cf":28,"\u8be5\u7c7b\u7684\u5b9e\u73b0\u7ec6\u8282\u5728":41,"\u8be5\u811a\u672c\u4ec5\u4ec5\u662f\u5f00\u59cb\u4e00\u4e2apaddle\u8bad\u7ec3\u8fc7\u7a0b":62,"\u8be5\u811a\u672c\u4f1a\u751f\u6210\u4e00\u4e2adot\u6587\u4ef6":58,"\u8be5\u811a\u672c\u5c06\u8f93\u51fa\u9884\u6d4b\u5206\u7c7b\u7684\u6807\u7b7e":57,"\u8be5\u8bed\u53e5\u4f1a\u4e3a\u6bcf\u4e2a\u5c42\u521d\u59cb\u5316\u5176\u6240\u9700\u8981\u7684\u53d8\u91cf\u548c\u8fde\u63a5":41,"\u8be6\u7ec6\u4ecb\u7ecd\u53ef\u4ee5\u53c2\u8003":36,"\u8be6\u7ec6\u4fe1\u606f\u8bf7\u68c0\u67e5":45,"\u8be6\u7ec6\u5185\u5bb9\u8bf7\u53c2\u89c1":60,"\u8be6\u7ec6\u5730\u5c55\u793a\u4e86\u6574\u4e2a\u7279\u5f81\u63d0\u53d6\u7684\u8fc7\u7a0b":58,"\u8be6\u7ec6\u6587\u6863\u53c2\u8003":27,"\u8be6\u7ec6\u7684\u53c2\u6570\u89e3\u91ca":60,"\u8be6\u7ec6\u7684cmake\u4f7f\u7528\u65b9\u6cd5\u53ef\u4ee5\u53c2\u8003":29,"\u8be6\u7ec6\u89c1":35,"\u8bed\u4e49\u89d2\u8272\u6807\u6ce8":[59,63],"\u8bed\u8a00\u6a21\u578b":56,"\u8bf4\u660e":[29,52],"\u8bf4\u660e\u6bcf\u4e2a\u7279\u5f81\u6587\u4ef6\u5177\u4f53\u5b57\u6bb5\u662f":62,"\u8bf4\u660e\u8fd9\u4e2a\u5c42\u7684\u8f93\u5165":41,"\u8bf7\u53c2\u7167\u7f51\u7edc\u914d\u7f6e\u7684\u6587\u6863\u4e86\u89e3\u66f4\u8be6\u7ec6\u7684\u4fe1\u606f":49,"\u8bf7\u53c2\u8003":[3,27,33,36,39,41,60],"\u8bf7\u53c2\u8003\u5982\u4e0b\u8868\u683c":60,"\u8bf7\u53c2\u8003\u9875\u9762":62,"\u8bf7\u53c2\u8003layer\u6587\u6863":57,"\u8bf7\u53c2\u89c1":40,"\u8bf7\u53c2\u9605":39,"\u8bf7\u53c2\u9605\u60c5\u611f\u5206\u6790\u7684\u6f14\u793a\u4ee5\u4e86\u89e3\u6709\u5173\u957f\u671f\u77ed\u671f\u8bb0\u5fc6\u5355\u5143\u7684\u66f4\u591a\u4fe1\u606f":63,"\u8bf7\u5b89\u88c5cuda":32,"\u8bf7\u628a\u6587\u6863\u91cc\u547d\u4ee4\u4e2d\u7684paddlepaddl":30,"\u8bf7\u6307\u5b9a\u8be5\u76ee\u5f55":47,"\u8bf7\u67e5\u770b":56,"\u8bf7\u6ce8\u610f":[30,39,51,56],"\u8bf7\u770b\u4e0b\u9762\u7684\u4f8b\u5b50":49,"\u8bf7\u786e\u4fdd":40,"\u8bf7\u8bb0\u4f4f":45,"\u8bf7\u8fd0\u884c":30,"\u8bf7\u9009\u62e9\u6b63\u786e\u7684\u7248\u672c":27,"\u8bf8\u5982\u56fe\u50cf\u5206\u7c7b":49,"\u8bfb\u53d612\u4e2a\u91c7\u6837\u6570\u636e\u8fdb\u884c\u968f\u673a\u68af\u5ea6\u8ba1\u7b97\u6765\u66f4\u65b0\u66f4\u65b0":28,"\u8bfb\u53d6\u6570\u636e":3,"\u8bfb\u53d6\u6bcf\u4e00\u884c":3,"\u8bfb\u53d6volume\u4e2d\u7684\u6570\u636e\u8fdb\u884c\u8fd9\u6b21\u5206\u5e03\u5f0f\u8bad\u7ec3":52,"\u8bfb\u8005\u53ef\u4ee5\u67e5\u770b":52,"\u8bfb\u8005\u9700\u8981\u66ff\u6362\u6210\u81ea\u5df1\u4f7f\u7528\u7684\u4ed3\u5e93\u5730\u5740":52,"\u8c03\u7528":[41,57],"\u8c03\u7528\u4e00\u6b21":3,"\u8c03\u7528\u4e0a\u9762\u7684process\u51fd\u6570\u83b7\u5f97\u89c2\u6d4b\u6570\u636e":28,"\u8c03\u7528\u7684pydataprovider2\u662f":3,"\u8c03\u7528\u7b2c\u4e8c\u6b21\u7684\u65f6\u5019":3,"\u8c03\u7528\u8be5\u51fd\u6570\u540e":41,"\u8c03\u7528\u8fd9\u4e2apydataprovider2\u7684\u65b9\u6cd5":3,"\u8c13\u8bcd\u4e0a\u4e0b\u6587":63,"\u8d1f\u6837\u672c":60,"\u8d1f\u9762\u7684\u8bc4\u8bba\u7684\u5f97\u5206\u5c0f\u4e8e\u7b49\u4e8e4":64,"\u8d1f\u9762\u8bc4\u4ef7\u6837\u672c":64,"\u8d44\u6e90\u5bf9\u8c61\u7684\u540d\u5b57\u662f\u552f\u4e00\u7684":50,"\u8d77":36,"\u8def\u5f84\u4e0b":[28,58],"\u8df3\u8f6c\u5230":40,"\u8f6c\u4e3ajpeg\u6587\u4ef6\u5e76\u5b58\u5165\u7279\u5b9a\u7684\u76ee\u5f55":57,"\u8f6c\u6362\u8fc7\u6765\u7684":58,"\u8f6e":62,"\u8f83":36,"\u8f93\u5165":[35,39],"\u8f93\u5165\u548c\u8f93\u51fa\u90fd\u662f\u5355\u5c42\u5e8f\u5217":38,"\u8f93\u5165\u548c\u8f93\u51fa\u90fd\u662f\u53cc\u5c42\u5e8f\u5217":38,"\u8f93\u5165\u56fe\u7247\u7684\u9ad8\u5ea6\u53ca\u5bbd\u5ea6":57,"\u8f93\u5165\u5c42\u5c3a\u5bf8":58,"\u8f93\u5165\u6570\u636e\u4e3a\u4e00\u4e2a\u5b8c\u6574\u7684\u65f6\u95f4\u5e8f\u5217":36,"\u8f93\u5165\u6570\u636e\u4e3a\u5728\u5355\u5c42rnn\u6570\u636e\u91cc\u9762":36,"\u8f93\u5165\u6570\u636e\u6574\u4f53\u4e0a\u662f\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217":36,"\u8f93\u5165\u6570\u636e\u7684\u5b57\u5178\u7ef4\u6570\u662f1\u767e\u4e07":49,"\u8f93\u5165\u6570\u6910\u6240\u5728\u76ee\u5f55":64,"\u8f93\u5165\u6587\u672c":56,"\u8f93\u5165\u6587\u672c\u4e2d\u6ca1\u6709\u5934\u90e8":56,"\u8f93\u5165\u662f\u5426\u662f\u8f6c\u7f6e\u7684":41,"\u8f93\u5165\u662f\u7531\u4e00\u4e2alist\u4e2d\u7684\u7f51\u7edc\u5c42\u5b9e\u4f8b\u7684\u540d\u5b57\u7ec4\u6210\u7684":41,"\u8f93\u5165\u7279\u5f81\u56fe\u7684\u901a\u9053\u6570\u76ee":58,"\u8f93\u5165\u7684":56,"\u8f93\u5165\u7684\u539f\u59cb\u6570\u636e\u96c6\u8def\u5f84":65,"\u8f93\u5165\u7684\u540d\u5b57":41,"\u8f93\u5165\u7684\u5927\u5c0f":41,"\u8f93\u5165\u7684\u6587\u672c\u683c\u5f0f\u5982\u4e0b":56,"\u8f93\u5165\u7684\u6587\u672c\u8bcd\u5411\u91cf\u6a21\u578b\u540d\u79f0":56,"\u8f93\u5165\u7684\u7c7b\u578b":41,"\u8f93\u5165\u95e8":64,"\u8f93\u5165\u9884\u6d4b\u6837\u672c":64,"\u8f93\u5165n\u4e2a\u5355\u8bcd":60,"\u8f93\u51fa":[35,39],"\u8f93\u51fa\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":38,"\u8f93\u51fa\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":38,"\u8f93\u51fa\u4e3an\u4e2aword_dim\u7ef4\u5ea6\u5411\u91cf":60,"\u8f93\u51fa\u51fd\u6570":39,"\u8f93\u51fa\u5e8f\u5217\u7684\u7c7b\u578b":35,"\u8f93\u51fa\u5e8f\u5217\u7684\u8bcd\u8bed\u6570\u548c\u8f93\u5165\u5e8f\u5217\u4e00\u81f4":38,"\u8f93\u51fa\u5e94\u8be5\u7c7b\u4f3c\u5982\u4e0b":62,"\u8f93\u51fa\u6587\u4ef6\u7684\u683c\u5f0f\u8bf4\u660e":56,"\u8f93\u51fa\u65e5\u5fd7\u4fdd\u5b58\u5728\u8def\u5f84":64,"\u8f93\u51fa\u65e5\u5fd7\u8bf4\u660e\u5982\u4e0b":64,"\u8f93\u51fa\u67092\u5217":56,"\u8f93\u51fa\u7279\u5f81\u56fe\u7684\u901a\u9053\u6570\u76ee":58,"\u8f93\u51fa\u7684\u4e8c\u8fdb\u5236\u8bcd\u5411\u91cf\u6a21\u578b\u540d\u79f0":56,"\u8f93\u51fa\u7684\u6587\u672c\u6a21\u578b\u540d\u79f0":56,"\u8f93\u51fa\u7684\u68af\u5ea6":47,"\u8f93\u51fa\u76ee\u5f55":58,"\u8f93\u51fa\u7ed3\u679c\u53ef\u80fd\u4f1a\u968f\u7740\u5bb9\u5668\u7684\u6d88\u8017\u800c\u88ab\u5220\u9664":51,"\u8fc7\u4e86\u4e00\u4e2a\u5f88\u7b80\u5355\u7684recurrent_group":36,"\u8fc7\u5b8c\u6240\u6709\u8bad\u7ec3\u6570\u636e\u5373\u4e3a\u4e00\u4e2apass":27,"\u8fd0":30,"\u8fd0\u884c":32,"\u8fd0\u884c\u4e0b\u9762\u547d\u4ee4\u5373\u53ef":62,"\u8fd0\u884c\u4ee5\u4e0b\u547d\u4ee4\u6765\u4e0b\u8f7d\u8fd9\u4e2a\u811a\u672c":65,"\u8fd0\u884c\u4ee5\u4e0b\u6307\u4ee4":30,"\u8fd0\u884c\u4ee5\u4e0b\u7684\u547d\u4ee4\u4e0b\u8f7d\u548c\u83b7\u53d6\u6211\u4eec\u7684\u5b57\u5178\u548c\u9884\u8bad\u7ec3\u6a21\u578b":56,"\u8fd0\u884c\u4ee5\u4e0b\u7684\u547d\u4ee4\u4e0b\u8f7d\u6570\u636e\u96c6":56,"\u8fd0\u884c\u4ee5\u4e0b\u8bad\u7ec3\u547d\u4ee4":28,"\u8fd0\u884c\u4ee5\u53ca\u53d1\u5e03\u60a8\u7684ai\u7a0b\u5e8f":30,"\u8fd0\u884c\u5206\u5e03\u5f0f\u4f5c\u4e1a":45,"\u8fd0\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3":[27,43,60],"\u8fd0\u884c\u5355\u5143\u6d4b\u8bd5":30,"\u8fd0\u884c\u5931\u8d25":49,"\u8fd0\u884c\u5b8c\u4ee5\u4e0a\u547d\u4ee4":56,"\u8fd0\u884c\u5b8c\u6210\u540e":45,"\u8fd0\u884c\u5b8c\u811a\u672c":64,"\u8fd0\u884c\u6210\u529f\u4ee5\u540e":56,"\u8fd0\u884c\u6210\u529f\u540e\u76ee\u5f55":64,"\u8fd0\u884c\u65e5\u5fd7":45,"\u8fd0\u884c\u7684\u4e00\u4e9b\u53c2\u6570\u901a\u8fc7\u8fd9\u79cd\u65b9\u5f0f\u4f20\u9012\u5230\u5bb9\u5668\u5185":52,"\u8fd1":36,"\u8fd1\u671f\u63d0\u51fa\u7684nmt\u6a21\u578b\u901a\u5e38\u90fd\u5c5e\u4e8e\u7f16\u89e3\u7801\u6a21\u578b":65,"\u8fd4\u56de":[8,9,10,11,16,17,20,22],"\u8fd4\u56de0":3,"\u8fd4\u56de8\u4e2a\u7279\u5f81list\u548c1\u4e2a\u6807\u7b7elist":63,"\u8fd4\u56de\u4e00\u6761\u5b8c\u6574\u7684\u6837\u672c":3,"\u8fd4\u56de\u6570\u636e\u7684\u6bcf\u4e00\u6761\u6837\u672c\u7ed9":62,"\u8fd4\u56de\u65f6":3,"\u8fd4\u56de\u7684\u662f":[3,33],"\u8fd4\u56de\u7684\u987a\u5e8f\u9700\u8981\u548cinput_types\u4e2d\u5b9a\u4e49\u7684\u987a\u5e8f\u4e00\u81f4":3,"\u8fd4\u56de\u7b2ci\u4e2a\u8f93\u5165\u77e9\u9635":41,"\u8fd4\u56de\u7c7b\u578b":[8,9,10,11,16,17,20,22],"\u8fd8\u4f1a":36,"\u8fd8\u4f1a\u8f93\u51fa\u4e00\u4e2a":40,"\u8fd8\u662f":36,"\u8fd8\u6709":36,"\u8fd8\u80fd\u5904\u7406\u5176\u4ed6\u7528\u6237\u81ea\u5b9a\u4e49\u7684\u6570\u636e":64,"\u8fd8\u91c7\u7528\u4e86\u4e24\u4e2a\u5176\u4ed6\u7279\u5f81":63,"\u8fd8\u9700\u8981\u8fdb\u884c\u9884\u5904\u7406":57,"\u8fd9":[27,36,60],"\u8fd9\u4e00\u5757\u7684\u8017\u65f6\u6bd4\u4f8b\u771f\u7684\u592a\u9ad8":44,"\u8fd9\u4e00\u8fc7\u7a0b\u5bf9\u7528\u6237\u662f\u5b8c\u5168\u900f\u660e\u7684":38,"\u8fd9\u4e09\u4e2a\u6b65\u9aa4\u53ef\u914d\u7f6e\u4e3a":60,"\u8fd9\u4e0e\u672c\u5730\u8bad\u7ec3\u76f8\u540c":45,"\u8fd9\u4e24\u4e2a\u6587\u4ef6\u5939\u4e0b\u5404\u81ea\u670910\u4e2a\u5b50\u6587\u4ef6\u5939":57,"\u8fd9\u4e24\u4e2a\u6807\u51c6":63,"\u8fd9\u4e2a":[36,50],"\u8fd9\u4e2a\u4efb\u52a1\u7684\u914d\u7f6e\u4e3a":27,"\u8fd9\u4e2a\u4efb\u52a1\u7684dataprovider\u4e3a":27,"\u8fd9\u4e2a\u51fd\u6570\u7684":39,"\u8fd9\u4e2a\u51fd\u6570\u8fdb\u884c\u53d8\u6362":36,"\u8fd9\u4e2a\u51fd\u6570\u9700\u8981\u8bbe\u7f6e":39,"\u8fd9\u4e2a\u5411\u91cf\u4e0e\u6e90\u4e2d\u641c\u7d22\u51fa\u7684\u4f4d\u7f6e\u548c\u6240\u6709\u4e4b\u524d\u751f\u6210\u7684\u76ee\u6807\u5355\u8bcd\u6709\u5173":65,"\u8fd9\u4e2a\u5730\u5740\u5219\u4e3a\u5b83\u7684\u7edd\u5bf9\u8def\u5f84\u6216\u76f8\u5bf9\u8def\u5f84":2,"\u8fd9\u4e2a\u5730\u5740\u6765\u8868\u793a\u6b64\u6b65\u9aa4\u6240\u6784\u5efa\u51fa\u7684\u955c\u50cf":52,"\u8fd9\u4e2a\u57fa\u7c7b":41,"\u8fd9\u4e2a\u5b57\u5178\u662f\u6574\u6570\u6807\u7b7e\u548c\u5b57\u7b26\u4e32\u6807\u7b7e\u7684\u4e00\u4e2a\u5bf9\u5e94":64,"\u8fd9\u4e2a\u5e8f\u5217\u7684\u6bcf\u4e2a\u5143\u7d20\u53c8\u662f\u4e00\u4e2a\u5e8f\u5217":38,"\u8fd9\u4e2a\u63a5\u53e3\u9700\u8981\u505a\u5230":25,"\u8fd9\u4e2a\u6570\u636e\u4e5f\u88ab\u5355\u5c42rnn\u7f51\u7edc\u76f4\u63a5\u4f7f\u7528":36,"\u8fd9\u4e2a\u6570\u91cf\u79f0\u4e3abeam":65,"\u8fd9\u4e2a\u6587\u4ef6\u5177\u6709\u72ec\u7279\u7684\u8bed\u6cd5":25,"\u8fd9\u4e2a\u663e\u793a\u5668\u5f88\u68d2":60,"\u8fd9\u4e2a\u6a21\u578b\u5bf9\u4e8e\u7f16\u89e3\u7801\u6a21\u578b\u6765\u8bf4":65,"\u8fd9\u4e2a\u795e\u7ecf\u7f51\u7edc\u5355\u5143\u5c31\u53ebmemori":36,"\u8fd9\u4e2a\u7a0b\u5e8f\u662f\u60a8\u5728\u5f00\u53d1\u673a\u4e0a\u4f7f\u7528\u5f00\u53d1\u955c\u50cf\u5b8c\u6210\u5f00\u53d1":30,"\u8fd9\u4e2a\u7c7b\u7684\u53c2\u6570\u5305\u62ec":41,"\u8fd9\u4e2a\u7c7b\u9700\u8981\u7ee7\u627f":41,"\u8fd9\u4e2a\u7cfb\u7edf\u5c06srl\u4efb\u52a1\u89c6\u4e3a\u5e8f\u5217\u6807\u6ce8\u95ee\u9898":63,"\u8fd9\u4e2a\u8282\u70b9\u53ef\u4ee5\u662f\u7269\u7406\u673a\u6216\u8005\u865a\u62df\u673a":50,"\u8fd9\u4e2a\u8868\u683c":50,"\u8fd9\u4e2a\u8fc7\u7a0b\u5bf9\u7528\u6237\u4e5f\u662f\u900f\u660e\u7684":38,"\u8fd9\u4e2a\u8fc7\u7a0b\u5c31\u662f\u6a21\u578b\u7684\u8bad\u7ec3\u8fc7\u7a0b":28,"\u8fd9\u4e2a\u8fc7\u7a0b\u9664\u4e86\u7f16\u8bd1paddlepaddle\u4e3a":40,"\u8fd9\u4e2a\u914d\u7f6e\u4e0e":56,"\u8fd9\u4e2a\u914d\u7f6e\u6587\u4ef6":50,"\u8fd9\u4e2a\u914d\u7f6e\u662f\u5426\u7528\u6765\u751f\u6210":65,"\u8fd9\u4e2a\u955c\u50cf\u5305\u542b\u4e86paddle\u76f8\u5173\u7684\u5f00\u53d1\u5de5\u5177\u4ee5\u53ca\u7f16\u8bd1\u548c\u8fd0\u884c\u73af\u5883":30,"\u8fd9\u4e2a\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u751f\u6210\u4e00\u7cfb\u5217\u6743\u91cd":39,"\u8fd9\u4e2a\u95ee\u9898\u662fpydataprovider\u8bfb\u6570\u636e\u65f6\u5019\u7684\u903b\u8f91\u95ee\u9898":3,"\u8fd9\u4e2adataprovider\u8f83\u590d\u6742":3,"\u8fd9\u4e2ajob\u624d\u7b97\u6210\u529f\u7ed3\u675f":52,"\u8fd9\u4e2alayer\u7684\u8f93\u51fa\u4f1a\u4f5c\u4e3a\u6574\u4e2a":38,"\u8fd9\u4e5f\u4f1a\u6781\u5927\u51cf\u5c11\u6570\u636e\u8bfb\u5165\u7684\u8017\u65f6":27,"\u8fd9\u4e9b":45,"\u8fd9\u4e9b\u53c2\u6570\u7684\u5177\u4f53\u63cf\u8ff0":52,"\u8fd9\u4e9b\u53c2\u6570\u7684\u7b80\u77ed\u4ecb\u7ecd\u5982\u4e0b":62,"\u8fd9\u4e9b\u540d\u5b57\u5fc5\u987b\u8981\u5199\u5bf9":41,"\u8fd9\u4e9b\u6570\u636e\u4f1a\u88ab\u7528\u6765\u66f4\u65b0\u53c2\u6570":27,"\u8fd9\u4e9b\u6570\u636e\u4f7f\u7528\u7684\u5185\u5b58\u4e3b\u8981\u548c\u4e24\u4e2a\u53c2\u6570\u6709\u5173\u7cfb":27,"\u8fd9\u4e9b\u6587\u4ef6\u5c06\u4f1a\u88ab\u4fdd\u5b58\u5728":58,"\u8fd9\u4e9b\u6a21\u578b\u90fd\u662f\u7531\u539f\u4f5c\u8005\u63d0\u4f9b\u7684\u6a21\u578b":58,"\u8fd9\u4e9b\u7279\u5f81\u503c\u4e0e\u4e0a\u8ff0\u4f7f\u7528c":58,"\u8fd9\u4e9b\u7279\u5f81\u548c\u6807\u7b7e\u5b58\u50a8\u5728":63,"\u8fd9\u4e9b\u7279\u5f81\u6570\u636e\u4e4b\u95f4\u7684\u987a\u5e8f\u662f\u6709\u610f\u4e49\u7684":36,"\u8fd9\u4efd\u6559\u7a0b\u5c55\u793a\u4e86\u5982\u4f55\u5728paddlepaddle\u4e2d\u5b9e\u73b0\u4e00\u4e2a\u81ea\u5b9a\u4e49\u7684\u7f51\u7edc\u5c42":41,"\u8fd9\u4efd\u7b80\u77ed\u7684\u4ecb\u7ecd\u5c06\u5411\u4f60\u5c55\u793a\u5982\u4f55\u5229\u7528paddlepaddle\u6765\u89e3\u51b3\u4e00\u4e2a\u7ecf\u5178\u7684\u7ebf\u6027\u56de\u5f52\u95ee\u9898":28,"\u8fd9\u4f1a\u63d0\u793a\u5f53\u524d\u76ee\u5f55\u7684\u4e00\u4e9b\u53d8\u5316":40,"\u8fd9\u4f1a\u81ea\u52a8\u8fdb\u884c\u7f51\u7edc\u914d\u7f6e\u4e2d\u58f0\u660e\u7684\u6fc0\u6d3b\u64cd\u4f5c":41,"\u8fd9\u4f7f\u5f97nmt\u6a21\u578b\u5f97\u4ee5\u89e3\u653e\u51fa\u6765":65,"\u8fd9\u4fbf\u662f\u4e00\u79cd\u53cc\u5c42rnn\u7684\u8f93\u5165\u6570\u636e":36,"\u8fd9\u51e0\u4e2a\u7f16\u8bd1\u9009\u9879\u7684\u8bbe\u7f6e":29,"\u8fd9\u53ef\u4ee5\u8ba9\u5176\u4ed6\u4eba\u77e5\u9053\u8fd9\u6b21\u63d0\u4ea4\u505a\u4e86\u54ea\u4e9b\u6539\u53d8":40,"\u8fd9\u53ef\u4ee5\u901a\u8fc7":40,"\u8fd9\u548c\u5355\u5c42rnn\u7684\u914d\u7f6e\u662f\u7b49\u4ef7\u7684":36,"\u8fd9\u56db\u4e2a\u7b80\u5355\u7684\u7279\u5f81\u662f\u6211\u4eec\u7684srl\u7cfb\u7edf\u6240\u9700\u8981\u7684":63,"\u8fd9\u56db\u6761\u6570\u636e\u540c\u65f6\u5904\u7406\u7684\u53e5\u5b50\u6570\u91cf\u4e3a":36,"\u8fd9\u5728\u5f88\u5927\u7a0b\u5ea6\u4e0a\u4f18\u4e8e\u5148\u524d\u7684\u6700\u5148\u8fdb\u7684\u7cfb\u7edf":63,"\u8fd9\u5728\u6784\u9020\u975e\u5e38\u590d\u6742\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u65f6\u662f\u6709\u7528\u7684":39,"\u8fd9\u5bf9\u4e8e\u901a\u5e38\u7684java\u7684\u5f00\u53d1\u8005\u6765\u8bf4":25,"\u8fd9\u5c06\u82b1\u8d39\u6570\u5206\u949f\u7684\u65f6\u95f4":65,"\u8fd9\u5df2\u7ecf\u5728":64,"\u8fd9\u610f\u5473\u7740":39,"\u8fd9\u610f\u5473\u7740\u6a21\u578b\u5728\u8bad\u7ec3\u6570\u636e\u4e0a\u4e0d\u65ad\u7684\u6539\u8fdb":28,"\u8fd9\u610f\u5473\u7740\u9664\u4e86\u6307\u5b9adevic":49,"\u8fd9\u65f6\u8fdb\u884c\u77e9\u9635\u4e58\u6cd5\u8fd0\u7b97\u5c31\u53ef\u80fd\u5bfc\u81f4\u6d6e\u70b9\u6570\u6ea2\u51fa":27,"\u8fd9\u662f\u4e00\u4e2a\u57fa\u4e8e\u7edf\u8ba1\u7684\u673a\u5668\u7ffb\u8bd1\u7cfb\u7edf":64,"\u8fd9\u662f\u4e00\u79cd\u975e\u5e38\u7075\u6d3b\u7684\u6570\u636e\u7ec4\u7ec7\u65b9\u5f0f":35,"\u8fd9\u662f\u56e0\u4e3a":25,"\u8fd9\u662f\u56e0\u4e3a\u5b83\u53d1\u6398\u51fa\u4e86\u56fe\u7247\u7684\u4e24\u7c7b\u91cd\u8981\u4fe1\u606f":57,"\u8fd9\u662f\u666e\u901a\u7684\u5355\u5c42\u65f6\u95f4\u5e8f\u5217\u7684dataprovider\u4ee3\u7801":36,"\u8fd9\u662f\u76ee\u524dcmake\u5bfb\u627epython\u7684\u903b\u8f91\u5b58\u5728\u7f3a\u9677":27,"\u8fd9\u662f\u96c6\u675f\u641c\u7d22\u7684\u7ed3\u679c":65,"\u8fd9\u6765\u81ea\u4e8epaddlepaddle\u7684\u5185\u5b58\u4e2d":65,"\u8fd9\u6837":[28,45],"\u8fd9\u6837\u505a\u53ef\u4ee5\u6781\u5927\u7684\u51cf\u5c11\u5185\u5b58\u5360\u7528":27,"\u8fd9\u6837\u53ef\u4ee5\u51cf\u5c0fgpu\u5185\u5b58":49,"\u8fd9\u6837\u5bb9\u5668\u7684":52,"\u8fd9\u6837\u5c31\u4f1a\u751f\u6210\u4e24\u4e2a\u6587\u4ef6":62,"\u8fd9\u6837\u5f00\u53d1\u4eba\u5458\u53ef\u4ee5\u4ee5\u4e00\u81f4\u7684\u65b9\u5f0f\u5728\u4e0d\u540c\u7684\u5e73\u53f0\u4e0a\u5de5\u4f5c":30,"\u8fd9\u6837\u7684\u88c5\u9970\u5668":41,"\u8fd9\u6837\u7684\u8bdd":51,"\u8fd9\u6837\u7684\u8bdd\u6bcf\u4f4d\u7528\u6237\u5728\u6d4b\u8bd5\u6587\u4ef6\u4e2d\u5c06\u4e0e\u8bad\u7ec3\u6587\u4ef6\u542b\u6709\u540c\u6837\u7684\u4fe1\u606f":62,"\u8fd9\u6b63\u662f\u5b83\u4eec\u901f\u5ea6\u5feb\u7684\u539f\u56e0":44,"\u8fd9\u6bb5\u7b80\u77ed\u7684\u914d\u7f6e\u5c55\u793a\u4e86paddlepaddle\u7684\u57fa\u672c\u7528\u6cd5":28,"\u8fd9\u7528\u4e8e\u5728\u591a\u7ebf\u7a0b\u548c\u591a\u673a\u4e0a\u66f4\u65b0\u53c2\u6570":41,"\u8fd9\u79cd\u521d\u59cb\u5316\u65b9\u5f0f\u5728\u4e00\u822c\u60c5\u51b5\u4e0b\u4e0d\u4f1a\u4ea7\u751f\u5f88\u5dee\u7684\u7ed3\u679c":27,"\u8fd9\u79cd\u60c5\u51b5\u4e0b\u4e0d\u9700\u8981\u91cd\u5199\u8be5\u51fd\u6570":41,"\u8fd9\u79cd\u65b9\u5f0f\u5fc5\u987b\u4f7f\u7528paddle\u5b58\u50a8\u7684\u6a21\u578b\u8def\u5f84\u683c\u5f0f":49,"\u8fd9\u79cd\u751f\u6210\u6280\u672f\u53ea\u7528\u4e8e\u7c7b\u4f3c\u89e3\u7801\u5668\u7684\u751f\u6210\u8fc7\u7a0b":39,"\u8fd9\u79cd\u7c7b\u578b\u7684\u8f93\u5165\u5fc5\u987b\u901a\u8fc7":38,"\u8fd9\u79cd\u96c6\u7fa4\u8282\u70b9\u7ba1\u7406\u65b9\u5f0f\u4f1a\u5728\u5c06\u6765\u4f7f\u7528":52,"\u8fd9\u7bc7\u6587\u6863":40,"\u8fd9\u7bc7\u6587\u7ae0":65,"\u8fd9\u7ec4\u8bed\u4e49\u76f8\u540c\u7684\u793a\u4f8b\u914d\u7f6e\u5982\u4e0b":36,"\u8fd9\u901a\u8fc7\u83b7\u5f97\u53cd\u5411\u5faa\u73af\u7f51\u7edc\u7684\u7b2c\u4e00\u4e2a\u5b9e\u4f8b":39,"\u8fd9\u90fd\u9700\u8981\u8fd9\u4e2a\u63a5\u53e3\u6309\u7167\u7ea6\u5b9a\u4fd7\u6210\u7684\u89c4\u5219\u6765\u6ce8\u91ca\u5b8c\u5907":25,"\u8fd9\u91cc":[27,30,39,40,50,52,58,63],"\u8fd9\u91cc\u4ecb\u7ecd\u5982\u751f\u6210\u9020\u8fd9\u4e2a\u5f00\u53d1\u955c\u50cf":30,"\u8fd9\u91cc\u4ee5":60,"\u8fd9\u91cc\u4f7f\u7528\u4e00\u4e2a\u57fa\u4e8emomentum\u7684\u968f\u673a\u68af\u5ea6\u4e0b\u964d":28,"\u8fd9\u91cc\u4f7f\u7528\u4e86\u4e09\u79cd\u7f51\u7edc\u5355\u5143":28,"\u8fd9\u91cc\u4f7f\u7528\u4e86paddlepaddle\u7684python\u63a5\u53e3\u6765\u52a0\u8f7d\u6570\u6910":64,"\u8fd9\u91cc\u4f7f\u7528\u4e86paddlepaddle\u9884\u5b9a\u4e49\u597d\u7684rnn\u5904\u7406\u51fd\u6570":36,"\u8fd9\u91cc\u4f7f\u7528\u7b80\u5355\u7684":27,"\u8fd9\u91cc\u5229\u7528\u5b83\u5efa\u6a21\u53d8\u91cf\u4e4b\u95f4\u7684\u7ebf\u6027\u5173\u7cfb":28,"\u8fd9\u91cc\u53ea\u52a0\u8f7d":65,"\u8fd9\u91cc\u53ea\u7b80\u5355\u4ecb\u7ecd\u4e86\u5355\u673a\u8bad\u7ec3":60,"\u8fd9\u91cc\u5c06\u4ecb\u7ecdpaddlepaddle\u7684\u57fa\u672c\u4f7f\u7528\u6982\u5ff5":33,"\u8fd9\u91cc\u5c55\u793a\u5982\u4f55\u4f7f\u7528\u89c2\u6d4b\u6570\u636e\u6765\u62df\u5408\u8fd9\u4e00\u7ebf\u6027\u5173\u7cfb":28,"\u8fd9\u91cc\u6211\u4eec\u4f7f\u7528":62,"\u8fd9\u91cc\u6211\u4eec\u4f7f\u7528\u7684\u662f\u4e00\u4e2a\u5c0f\u7684vgg\u7f51\u7edc":57,"\u8fd9\u91cc\u6211\u4eec\u4f7f\u7528\u7684\u662fgpu\u6a21\u5f0f\u8fdb\u884c\u8bad\u7ec3":57,"\u8fd9\u91cc\u6211\u4eec\u5728movielens\u6570\u636e\u96c6\u63cf\u8ff0\u4e00\u79cd":62,"\u8fd9\u91cc\u6211\u4eec\u5c55\u793a\u4e00\u4efd\u7b80\u5316\u8fc7\u7684\u4ee3\u7801":41,"\u8fd9\u91cc\u6211\u4eec\u7528paddl":30,"\u8fd9\u91cc\u6211\u4eec\u901a\u8fc7\u5728kubernetes\u96c6\u7fa4\u4e0a\u542f\u52a8\u4e00\u4e2ajob\u6765\u4e0b\u8f7d\u5e76\u5207\u5272\u6570\u636e":52,"\u8fd9\u91cc\u6307\u5b9a\u8bcd\u5178":60,"\u8fd9\u91cc\u6570\u636e\u5c42\u6709\u4e24\u4e2a":28,"\u8fd9\u91cc\u662f\u4e00\u4e2a\u4f8b\u5b50":65,"\u8fd9\u91cc\u6709\u4e00\u4e9b\u4e0d\u540c\u7684\u53c2\u6570\u9700\u8981\u6307\u5b9a":65,"\u8fd9\u91cc\u68c0\u9a8c\u8fd0\u884c\u65f6\u95f4\u6a21\u578b\u7684\u6536\u655b":45,"\u8fd9\u91cc\u6bcf\u4e2a5\u4e2abatch\u6253\u5370\u4e00\u4e2a\u70b9":65,"\u8fd9\u91cc\u6bcf\u9694100\u4e2abatch\u663e\u793a\u4e00\u6b21\u53c2\u6570\u7edf\u8ba1\u4fe1\u606f":65,"\u8fd9\u91cc\u6bcf\u969410\u4e2abatch\u6253\u5370\u4e00\u6b21\u65e5\u5fd7":65,"\u8fd9\u91cc\u7684\u5217\u51fa\u7684\u548c":57,"\u8fd9\u91cc\u76f4\u63a5\u901a\u8fc7\u9884\u6d4b\u811a\u672c":60,"\u8fd9\u91cc\u7ed9\u51fa\u96c6\u4e2d\u5e38\u89c1\u7684\u90e8\u7f72\u65b9\u6cd5":50,"\u8fd9\u91cc\u8bbe\u7f6e\u4e3a\u4f7f\u7528cpu":65,"\u8fd9\u91cc\u8bbe\u7f6e\u4e3afals":65,"\u8fd9\u91cc\u8bbe\u7f6e\u4e3atrue":65,"\u8fd9\u91cc\u91c7\u7528adam\u4f18\u5316\u65b9\u6cd5":60,"\u8fdb\u4e3b\u4ed3\u5e93\u540e":40,"\u8fdb\u5165":64,"\u8fdb\u5165\u5bb9\u5668":51,"\u8fdb\u5165\u5f00\u53d1\u955c\u50cf\u5e76\u5f00\u59cb\u5de5\u4f5c":30,"\u8fdb\u7a0b\u542f\u52a8\u7684\u5fc5\u8981\u53c2\u6570":52,"\u8fdb\u7a0b\u7684":45,"\u8fdb\u7a0b\u7684\u542f\u52a8\u53c2\u6570":52,"\u8fdb\u7a0b\u7684\u8fd0\u884c\u73af\u5883":52,"\u8fdb\u7a0b\u9700\u8981\u7684":52,"\u8fdb\u884c\u4e86":36,"\u8fdb\u884c\u4f7f\u7528":57,"\u8fdb\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3\u7684\u65b9\u6848":52,"\u8fdb\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3\u7684\u65b9\u6cd5":52,"\u8fdb\u884c\u5f00\u53d1":40,"\u8fdb\u884c\u62c6\u89e3":36,"\u8fdb\u884c\u6fc0\u6d3b\u64cd\u4f5c":41,"\u8fdb\u884c\u8bfb\u5165\u548c\u9884\u5904\u7406\u4ece\u800c\u5f97\u5230\u771f\u5b9e\u8f93\u5165":28,"\u8fdb\u884c\u9884\u6d4b":60,"\u8fdb\u9636\u6307\u5357":[33,55],"\u8fde\u63a5":38,"\u8fde\u63a5\u4e09\u4e2alstm\u9690\u85cf\u5c42":64,"\u9000\u4f11\u4eba\u5458":61,"\u9000\u51fa\u5bb9\u5668":51,"\u9002\u4e2d":36,"\u9002\u5408\u81ea\u5df1\u5b9e\u9a8c":30,"\u9009":36,"\u9009\u62e9":36,"\u9009\u62e9\u6d4b\u8bd5\u7ed3\u679c\u6700\u597d\u7684\u6a21\u578b\u6765\u9884\u6d4b":60,"\u9009\u62e9\u76ee\u6807\u5206\u652f":40,"\u9009\u62e9\u8def\u5f84\u6765\u52a8\u6001\u52a0\u8f7dnvidia":47,"\u9009\u62e9\u8fc7\u540e\u7684":65,"\u9009\u62e9\u9002\u5408\u60a8\u7684\u573a\u666f\u7684\u5408\u9002\u65b9\u6848":50,"\u9009\u81ea\u4e0b\u5217\u7c7b\u578b":61,"\u9009\u9879":[29,56],"\u9012\u5f52\u795e\u7ecf\u7f51\u7edc":46,"\u901a\u5e38":[45,64],"\u901a\u5e38\u4f1a\u4f7f\u7528\u73af\u5883\u53d8\u91cf\u914d\u7f6ejob\u7684\u914d\u7f6e\u4fe1\u606f":52,"\u901a\u5e38\u4f7f\u7528\u7a00\u758f\u8bad\u7ec3\u6765\u52a0\u901f\u8ba1\u7b97\u8fc7\u7a0b":49,"\u901a\u5e38\u505a\u6cd5\u662f\u4ece\u4e00\u4e2a\u6bd4\u8f83\u5927\u7684learning_rate\u5f00\u59cb\u8bd5":27,"\u901a\u5e38\u540d\u5b57\u662f":40,"\u901a\u5e38\u60c5\u51b5\u4e0b":44,"\u901a\u5e38\u6211\u4eec\u4f1a\u5b89\u88c5ceph\u7b49\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf\u6765\u5b58\u50a8\u8bad\u7ec3\u6570\u636e":51,"\u901a\u5e38\u6bcf\u4e2ajob\u5305\u62ec\u4e00\u4e2a\u6216\u8005\u591a\u4e2apod":50,"\u901a\u5e38\u7684\u505a\u6cd5\u662f\u4f7f\u7528":39,"\u901a\u5e38\u7684\u505a\u6cd5\u662f\u5c06\u914d\u7f6e\u5b58\u4e8e":41,"\u901a\u5e38\u8981\u6c42\u65f6\u95f4\u6b65\u4e4b\u95f4\u5177\u6709\u4e00\u4e9b\u4f9d\u8d56\u6027":36,"\u901a\u5e38\u90fd\u4f1a\u4f7f\u7528\u4e0b\u9762\u8fd9\u4e9b\u547d\u4ee4\u884c\u53c2\u6570":49,"\u901a\u7528":46,"\u901a\u77e5":36,"\u901a\u8fc7":[27,36,40,41,45,60],"\u901a\u8fc7\u4e24\u4e2a\u5d4c\u5957\u7684":38,"\u901a\u8fc7\u4ea4\u66ff\u4f7f\u7528\u5377\u79ef\u548c\u6c60\u5316\u5904\u7406":57,"\u901a\u8fc7\u4f7f\u7528":29,"\u901a\u8fc7\u51fd\u6570":52,"\u901a\u8fc7\u5377\u79ef\u64cd\u4f5c\u4ece\u56fe\u7247\u6216\u7279\u5f81\u56fe\u4e2d\u63d0\u53d6\u7279\u5f81":57,"\u901a\u8fc7\u547d\u4ee4\u884c\u53c2\u6570":27,"\u901a\u8fc7\u5f15\u7528memory\u5f97\u5230\u8fd9\u4e2alayer\u4e0a\u4e00\u4e2a\u65f6\u523b\u7684\u8f93\u51fa":38,"\u901a\u8fc7\u5f15\u7528memory\u5f97\u5230\u8fd9\u4e2alayer\u4e0a\u4e00\u4e2a\u65f6\u523b\u8f93\u51fa":38,"\u901a\u8fc7\u6240\u6709\u5355\u5143\u6d4b\u8bd5":40,"\u901a\u8fc7\u6240\u6709\u8bad\u7ec3\u96c6\u4e00\u6b21\u79f0\u4e3a\u4e00\u904d":64,"\u901a\u8fc7\u67e5\u770b\u4e70\u5bb6\u5bf9\u67d0\u4e2a\u4ea7\u54c1\u7684\u8bc4\u4ef7\u53cd\u9988":60,"\u901a\u8fc7\u7ec4\u5408\u4e0d\u540c\u7684layer":33,"\u901a\u8fc7\u7f16\u8bd1\u4f1a\u751f\u6210py_paddle\u8f6f\u4ef6\u5305":5,"\u901a\u8fc7\u7f51\u7edc\u5c42\u7684\u6807\u8bc6\u7b26\u6765\u6307\u5b9a":41,"\u901a\u8fc7\u8c03\u7528":5,"\u901a\u8fc7\u914d\u7f6e\u7c7b\u4f3c\u4e8e":60,"\u901a\u8fc7data":38,"\u901a\u8fc7volum":50,"\u903b\u8f91\u56de\u5f52":60,"\u9053\u6b49":36,"\u9069":36,"\u9075\u5faa\u5982\u4e0b\u7684\u683c\u5f0f":61,"\u9075\u5faa\u6587\u7ae0":56,"\u90a3\u4e48":[38,41],"\u90a3\u4e480\u5c42\u5e8f\u5217\u5373\u4e3a\u4e00\u4e2a\u8bcd\u8bed":38,"\u90a3\u4e48\u53ef\u4ee5\u8ba4\u4e3a\u8bad\u7ec3\u4e0d\u6536\u655b":27,"\u90a3\u4e48\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u4e0d\u4f1a\u6267\u884c\u6d4b\u8bd5\u64cd\u4f5c":2,"\u90a3\u4e48\u5982\u4f55\u5224\u65ad\u8bad\u7ec3\u4e0d\u6536\u655b\u5462":27,"\u90a3\u4e48\u5e38\u6570\u8f93\u51fa\u6240\u80fd\u8fbe\u5230\u7684\u6700\u5c0fcost\u662f":27,"\u90a3\u4e48\u5f53check\u51fa\u6570\u636e\u4e0d\u5408\u6cd5\u65f6":3,"\u90a3\u4e48\u6211\u4eec\u53ef\u4ee5\u5224\u65ad\u4e3a\u8bad\u7ec3\u4e0d\u6536\u655b":27,"\u90a3\u4e48\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7\u6536\u96c6\u5e02\u573a\u4e0a\u623f\u5b50\u7684\u5927\u5c0f\u548c\u4ef7\u683c":28,"\u90a3\u4e48\u63a8\u8350\u4f7f\u7528":39,"\u90a3\u4e48\u63a8\u8350\u4f7f\u7528\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u65b9\u6cd5":39,"\u90a3\u4e48\u6536\u655b\u53ef\u80fd\u5f88\u6162":27,"\u90a3\u4e48\u6700\u597d\u5c06\u6570\u636e\u6587\u4ef6\u5728\u6bcf\u6b21\u8bfb\u53d6\u4e4b\u524d\u505a\u4e00\u6b21shuffl":27,"\u90a3\u4e48\u8bad\u7ec3\u6709\u53ef\u80fd\u4e0d\u6536\u655b":27,"\u90a3\u4e48\u8be5\u4f18\u5316\u7b97\u6cd5\u81f3\u5c11\u9700\u8981":27,"\u90a3\u4e48fc1\u548cfc2\u5c42\u5c06\u4f1a\u4f7f\u7528\u7b2c1\u4e2agpu\u6765\u8ba1\u7b97":49,"\u90a3\u4e48paddlepaddle\u4f1a\u6839\u636elayer\u7684\u58f0\u660e\u987a\u5e8f":3,"\u90a3\u4e5f\u5c31\u4e0d\u9700\u8981\u6025\u7740\u4f18\u5316\u6027\u80fd\u5566":44,"\u90a3\u4f30\u8ba1\u8fd9\u91cc\u7684\u6f5c\u529b\u5c31\u6ca1\u5565\u597d\u6316\u7684\u4e86":44,"\u90a3\u51cf\u5c11\u5b66\u4e60\u738710\u500d\u7ee7\u7eed\u8bd5\u9a8c":27,"\u90a3\u6211\u4f1a\u671f\u671b\u5206\u6790\u5de5\u5177\u7edf\u8ba1\u5230\u901f\u5ea6\u662f100gb":44,"\u90a3\u7a0b\u5e8f\u5206\u6790\u5de5\u5177\u662f\u5fc5\u4e0d\u53ef\u5c11\u7684\u5229\u5668":44,"\u90ae\u7f16":61,"\u90e8\u7f72\u548c\u914d\u7f6e\u6bd4\u8f83\u7b80\u5355":50,"\u90fd":36,"\u90fd\u4f1a\u4ea7\u751f\u5f53\u524d\u5c42\u72b6\u6001\u7684\u6240\u6709\u7ee7\u627f\u7ed3\u679c":47,"\u90fd\u4f7f\u7528\u968f\u673a\u503c\u521d\u59cb\u5316":28,"\u90fd\u53ea\u662f\u4ecb\u7ecd\u53cc\u5c42rnn\u7684api\u63a5\u53e3":36,"\u90fd\u662f\u5bf9layer1\u5143\u7d20\u7684\u62f7\u8d1d":35,"\u90fd\u662f\u5c06\u6bcf\u4e00\u53e5\u5206\u597d\u8bcd\u540e\u7684\u53e5\u5b50":36,"\u90fd\u662fabi\u8c03\u7528\u6807\u51c6\u7684":25,"\u90fd\u7528":40,"\u90fd\u9700\u8981\u5199\u63d0\u4ea4\u8bf4\u660e":40,"\u90fd\u9700\u8981\u8c03\u7528\u4e00\u6b21":41,"\u914d\u7f6e":64,"\u914d\u7f6e\u4e86\u7f51\u7edc":62,"\u914d\u7f6e\u521b\u5efa\u5b8c\u6bd5\u540e":57,"\u914d\u7f6e\u5982\u4e0b":56,"\u914d\u7f6e\u6253\u5f00":44,"\u914d\u7f6e\u6587\u4ef6":60,"\u914d\u7f6e\u6587\u4ef6\u63a5\u53e3\u662ffc_layer":41,"\u914d\u7f6e\u6a21\u578b\u6587\u4ef6":56,"\u914d\u7f6e\u7b49\u6587\u4ef6\u7684\u76ee\u5f55\u89c6\u4e3a":45,"\u914d\u7f6e\u7b80\u5355\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u4f8b\u5b50":39,"\u914d\u7f6e\u7f51\u7edc\u5c42\u7684\u8f93\u5165":41,"\u914d\u7f6eapi":35,"\u9152\u5e97":36,"\u91c7\u6837\u5c42":62,"\u91c7\u7528":63,"\u91c7\u7528\u53e6\u4e00\u79cd\u65b9\u6cd5\u6765\u5806\u53e0lstm\u5c42":63,"\u91c7\u7528\u5747\u5300\u5206\u5e03\u6216\u8005\u9ad8\u65af\u5206\u5e03\u521d\u59cb\u5316":47,"\u91c7\u7528multi":27,"\u91cc\u4ecb\u7ecd\u4e86\u7528paddle\u6e90\u7801\u4e2d\u7684\u811a\u672c\u4e0b\u8f7d\u8bad\u7ec3\u6570\u636e\u7684\u8fc7\u7a0b":51,"\u91cc\u4f1a\u7ee7\u7eed\u5b89\u88c5":32,"\u91cc\u6240\u6709\u7684\u7b26\u53f7\u90fd\u5199\u5165\u81ea\u5df1\u7684\u7a0b\u5e8f\u7684\u4e8c\u8fdb\u5236\u6587\u4ef6\u91cc":25,"\u91cc\u6307\u5b9a\u56fe\u50cf\u6570\u636e\u5217\u8868":58,"\u91cc\u7684\u65e5\u5fd7":45,"\u91cc\u9762\u6db5\u76d6\u4e86":30,"\u91cc\u9762\u6db5\u76d6\u4e86paddlepaddle\u8fd0\u884c\u6240\u9700\u7684\u6240\u6709\u73af\u5883":30,"\u91cd\u547d\u540d\u6210":25,"\u91cd\u65b0\u7f16\u8bd1paddlepaddl":44,"\u9488\u5bf9\u4efb\u52a1\u8fd0\u884c\u5b8c\u6210\u540e\u5bb9\u5668\u81ea\u52a8\u9000\u51fa\u7684\u573a\u666f":51,"\u9488\u5bf9\u5185\u5b58\u548c\u663e\u5b58":27,"\u9488\u5bf9\u6587\u672c":62,"\u94fe\u63a5\u4f55\u79cdblas\u5e93\u7b49":29,"\u94fe\u63a5\u5230\u81ea\u5df1\u7684\u7a0b\u5e8f\u91cc":25,"\u94fe\u63a5\u5f85\u8865\u5145":60,"\u9500\u552e":61,"\u9519\u8bef\u5904\u7406":25,"\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u662f\u8fd4\u56de\u503c":25,"\u9519\u8bef\u5904\u7406\u7684\u65b9\u5f0f\u4e5f\u4e0d\u5c3d\u76f8\u540c":25,"\u9519\u8bef\u7387":60,"\u9519\u8bef\u7684define_py_data_sources2\u7c7b\u4f3c":27,"\u955c\u50cf\u91cc\u6709":51,"\u957f\u5ea6":27,"\u95e8\u63a7\u5faa\u73af\u5355\u5143\u5355\u6b65\u51fd\u6570\u548c\u8f93\u51fa\u51fd\u6570":39,"\u95e8\u63a7\u5faa\u73af\u5355\u5143\u7684\u8f93\u51fa\u88ab\u7528\u4f5c\u8f93\u51famemori":39,"\u95ee\u9898":28,"\u95f4\u9694":60,"\u9650\u5236\u5957\u63a5\u5b57\u53d1\u9001\u7f13\u51b2\u533a\u7684\u5927\u5c0f":47,"\u9650\u5236\u5957\u63a5\u5b57\u63a5\u6536\u7f13\u51b2\u533a\u7684\u5927\u5c0f":47,"\u9664\u4e86":3,"\u9664\u4e86boot_lay":36,"\u9664\u53bbdata\u5c42":60,"\u9664\u8bcd\u5411\u91cf\u6a21\u578b\u5916\u7684\u53c2\u6570\u5c06\u4f7f\u7528\u6b63\u6001\u5206\u5e03\u968f\u673a\u521d\u59cb\u5316":56,"\u9664\u96f6\u7b49\u95ee\u9898":27,"\u968f\u540e\u53ef\u4ee5\u7528\u8fd9\u4e2a\u5f00\u53d1\u955c\u50cf\u5f00build":40,"\u968f\u673a\u521d\u59cb\u4e0d\u5b58\u5728\u7684\u53c2\u6570":63,"\u968f\u673a\u6570\u7684\u79cd\u5b50":47,"\u968f\u673a\u6570seed":46,"\u968f\u7740\u8f6e\u6570\u589e\u52a0\u8bef\u5dee\u4ee3\u4ef7\u51fd\u6570\u7684\u8f93\u51fa\u5728\u4e0d\u65ad\u7684\u51cf\u5c0f":28,"\u9694\u5f00":58,"\u96c6":61,"\u96c6\u675f\u641c\u7d22\u4e2d\u7684\u6269\u5c55\u5e7f\u5ea6":65,"\u96c6\u675f\u641c\u7d22\u4f7f\u7528\u5e7f\u5ea6\u4f18\u5148\u641c\u7d22\u6765\u6784\u5efa\u641c\u7d22\u6811":65,"\u96c6\u675f\u641c\u7d22\u4f7f\u7528\u5e7f\u5ea6\u4f18\u5148\u641c\u7d22\u7684\u65b9\u5f0f\u6784\u5efa\u67e5\u627e\u6811":47,"\u96c6\u7fa4\u4e0a\u542f\u52a8\u4e00\u4e2a\u5355\u673a\u4f7f\u7528cpu\u7684paddle\u8bad\u7ec3\u4f5c\u4e1a":51,"\u96c6\u7fa4\u4f5c\u4e1a\u4e2d\u6240\u6709\u8fdb\u7a0b\u7684\u73af\u5883\u8bbe\u7f6e":45,"\u96c6\u7fa4\u4f5c\u4e1a\u5c06\u4f1a\u5728\u51e0\u79d2\u540e\u542f\u52a8":45,"\u96c6\u7fa4\u5de5\u4f5c":45,"\u96c6\u7fa4\u6d4b\u8bd5":46,"\u96c6\u7fa4\u8bad\u7ec3":46,"\u96c6\u7fa4\u8fdb\u7a0b":45,"\u96c6\u7fa4\u901a\u4fe1\u4fe1\u9053\u7684\u7aef\u53e3\u6570":45,"\u96c6\u7fa4\u901a\u4fe1\u901a\u9053\u7684":45,"\u96c6\u7fa4\u901a\u4fe1\u901a\u9053\u7684\u7aef\u53e3\u53f7":45,"\u9700\u5728nvvp\u754c\u9762\u4e2d\u9009\u4e0a\u624d\u80fd\u5f00\u542f":44,"\u9700\u8981\u4f7f\u7528\u5176\u5236\u5b9a\u7684\u65b9\u5f0f\u6302\u8f7d\u540e\u5e76\u5bfc\u5165\u6570\u636e":52,"\u9700\u8981\u4fdd\u6301\u5f53\u524d\u5206\u652f\u76ee\u5f55":40,"\u9700\u8981\u5148\u6302\u8f7d\u5230\u670d\u52a1\u5668node\u4e0a\u518d\u901a\u8fc7kubernet":50,"\u9700\u8981\u540c\u6b65\u539f\u4ed3\u5e93":40,"\u9700\u8981\u542f\u52a8\u7684\u8282\u70b9\u4e2a\u6570\u4ee5\u53ca":52,"\u9700\u8981\u5728":45,"\u9700\u8981\u5728\u521b\u5efa\u5bb9\u5668\u524d\u6302\u8f7d\u5377\u4ee5\u4fbf\u6211\u4eec\u4fdd\u5b58\u8bad\u7ec3\u7ed3\u679c":51,"\u9700\u8981\u5728\u7cfb\u7edf\u91cc\u5148\u5b89\u88c5\u597ddocker\u5de5\u5177\u5305":42,"\u9700\u8981\u5b89\u88c5graphviz\u6765\u8f6c\u6362dot\u6587\u4ef6\u4e3a\u56fe\u7247":58,"\u9700\u8981\u5bf9":50,"\u9700\u8981\u5c06\u5176parameter\u8bbe\u7f6e\u6210":27,"\u9700\u8981\u5c06\u6807\u8bb0\u7684\u8bad\u7ec3\u548c\u6d4b\u8bd5\u6837\u672c\u79fb\u52a8\u5230\u53e6\u4e00\u4e2a\u8def\u5f84":64,"\u9700\u8981\u6307\u5b9a\u4e0e\u67d0\u4e00\u4e2a\u8f93\u5165\u7684\u5e8f\u5217\u4fe1\u606f\u662f\u4e00\u81f4\u7684":36,"\u9700\u8981\u6307\u5b9alayer\u7684\u8f93\u5165\u6765\u6e90":33,"\u9700\u8981\u660e\u786e\u6307\u5b9a":47,"\u9700\u8981\u6709\u4e00\u4e2a\u5916\u90e8\u7684\u5b58\u50a8\u670d\u52a1\u6765\u4fdd\u5b58\u8bad\u7ec3\u6240\u9700\u6570\u636e\u548c\u8bad\u7ec3\u8f93\u51fa":50,"\u9700\u8981\u6709\u7a33\u5b9a\u7684\u5bfc\u51fa\u7b26\u53f7":25,"\u9700\u8981\u6839\u636e\u4e0d\u540c\u7684\u5206\u5e03\u5f0f\u5b58\u50a8\u6765\u7ed1\u5b9a\u4e00\u4e2a":52,"\u9700\u8981\u6ce8\u610f\u7684\u662f":[47,62],"\u9700\u8981\u6ce8\u610f\u7684\u662f\u68af\u5ea6\u68c0\u67e5\u4ec5\u4ec5\u9a8c\u8bc1\u4e86\u68af\u5ea6\u7684\u8ba1\u7b97":41,"\u9700\u8981\u6ce8\u610f\u7684\u662fpaddlepaddle\u76ee\u524d\u53ea\u652f\u6301\u5b50\u5e8f\u5217\u6570\u76ee\u4e00\u6837\u7684\u591a\u8f93\u5165\u53cc\u5c42rnn":36,"\u9700\u8981\u9075\u5faa\u4ee5\u4e0b\u7ea6\u5b9a":38,"\u975e\u5e38\u6570":41,"\u975e\u96f6\u6570\u5b57\u7684\u4e2a\u6570":41,"\u97f3\u4e50\u5267":61,"\u987a\u5e8f":36,"\u9884\u5904\u7406\u6570\u636e\u4e00\u822c\u7684\u547d\u4ee4\u4e3a":62,"\u9884\u5904\u7406\u811a\u672c":64,"\u9884\u5b9a\u4e49\u7f51\u7edc":64,"\u9884\u5b9a\u4e49\u7f51\u7edc\u5982\u56fe3\u6240\u793a":64,"\u9884\u63d0\u4ea4\u94a9\u5b50":40,"\u9884\u6d4b\u540e":63,"\u9884\u6d4b\u63a5\u53e3\u811a\u672c":64,"\u9884\u6d4b\u6982\u7387\u53d6\u5e73\u5747":58,"\u9884\u6d4b\u7a0b\u5e8f\u5c06\u8bfb\u53d6\u7528\u6237\u7684\u8f93\u5165":62,"\u9884\u6d4b\u7ed3\u679c\u4ee5\u6587\u672c\u7684\u5f62\u5f0f\u4fdd\u5b58\u5728":60,"\u9884\u6d4b\u811a\u672c\u662f":63,"\u9884\u6d4bid":60,"\u9884\u6d4bimdb\u7684\u672a\u6807\u8bb0\u8bc4\u8bba\u7684\u4e00\u4e2a\u5b9e\u4f8b\u5982\u4e0b":64,"\u9884\u8bad\u7ec3\u6a21\u578b\u4f7f\u7528\u7684\u5b57\u5178\u7684\u8def\u5f84":56,"\u9884\u8bad\u7ec3\u8bcd\u5411\u91cf\u5b57\u5178\u6a21\u578b\u7684\u8def\u5f84":56,"\u989c\u8272\u901a\u9053\u987a\u5e8f\u4e3a":58,"\u989d\u5916\u7684\u53c2\u6570":60,"\u9996\u5148":[3,28,36,39,41,56,58,60,63,64],"\u9996\u5148\u4e0b\u8f7dcifar":57,"\u9996\u5148\u5728\u7cfb\u7edf\u8def\u5f84":29,"\u9996\u5148\u5b89\u88c5\u5e76\u5728\u5f53\u524d\u76ee\u5f55\u8fd0\u884c\u5b83":40,"\u9996\u5148\u5b89\u88c5paddlepaddl":64,"\u9996\u5148\u5bf9\u8f93\u5165\u505a\u4e00\u4e2a\u5c0f\u7684\u6270\u52a8":41,"\u9996\u5148\u6211\u4eec\u9700\u8981\u63a8\u5bfc\u8be5\u7f51\u7edc\u5c42\u7684":41,"\u9996\u5148\u662f\u6cd5\u8bed\u5e8f\u5217":65,"\u9996\u5148\u901a\u8fc7":40,"\u9996\u5148\u9700\u8981\u52a0\u8f7d\u76f8\u5e94\u7684python\u5e93":33,"\u9a71\u52a8":42,"\u9ad8\u4e2d\u6bd5\u4e1a\u751f":61,"\u9ad8\u4eae\u90e8\u5206":36,"\u9ad8\u53ef\u7528":50,"\u9ad8\u5ea6\u652f\u6301\u7075\u6d3b\u548c\u9ad8\u6548\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u914d\u7f6e":39,"\u9ad8\u6548\u6027":0,"\u9ad8\u65af\u5206\u5e03":27,"\u9ed1\u8272\u7535\u5f71":61,"\u9ed8\u8ba4":[3,47,65],"\u9ed8\u8ba4\u4e00\u4e2apass\u4fdd\u5b58\u4e00\u6b21\u6a21\u578b":60,"\u9ed8\u8ba4\u4e0d\u663e\u793a":47,"\u9ed8\u8ba4\u4e0d\u8bbe\u7f6e":38,"\u9ed8\u8ba4\u4e3a0":[47,49],"\u9ed8\u8ba4\u4e3a1":[3,49],"\u9ed8\u8ba4\u4e3a100":49,"\u9ed8\u8ba4\u4e3a4096mb":47,"\u9ed8\u8ba4\u4e3a\u4e0d\u4f7f\u7528":62,"\u9ed8\u8ba4\u4e3a\u7b2c\u4e00\u4e2a\u8f93\u5165":38,"\u9ed8\u8ba4\u4e3anull":47,"\u9ed8\u8ba4\u4f7f\u7528concurrentremoteparameterupdat":47,"\u9ed8\u8ba4\u503c":[29,35,49],"\u9ed8\u8ba4\u521d\u59cb\u72b6\u4e3a0":38,"\u9ed8\u8ba4\u60c5\u51b5\u4e0b":[27,45,64],"\u9ed8\u8ba4\u60c5\u51b5\u4e0b\u4f7f\u7528\u6b64\u7f51\u7edc":64,"\u9ed8\u8ba4\u6307\u5b9a\u7b2c\u4e00\u4e2a\u8f93\u5165":36,"\u9ed8\u8ba4\u7528\u6765\u5207\u5206\u5355\u8bb0\u548c\u6807\u70b9\u7b26\u53f7":64,"\u9ed8\u8ba4\u7684":51,"\u9ed8\u8ba4\u8bbe\u7f6e\u4e3a\u771f":49,"\u9ed8\u8ba4\u914d\u7f6e\u5982\u4e0b":45,"adamax\u7b49":60,"amazon\u7535\u5b50\u4ea7\u54c1\u8bc4\u8bba\u6570\u636e":60,"api\u4e2d\u4f7f\u7528":25,"api\u5bf9\u6bd4\u4ecb\u7ecd":37,"api\u63a5\u53e3":50,"async_sgd\u8fdb\u884c\u8bad\u7ec3\u65f6":27,"atlas\u7684\u8def\u5f84":29,"avx\u7684\u955c\u50cf":30,"awselasticblockstore\u7b49":50,"batch\u4e2d\u5305\u542b":27,"batches\u4e2a\u6279\u6b21\u4fdd\u5b58\u4e00\u6b21\u53c2\u6570":47,"batches\u6b21":47,"bin\u548c\u8bc4\u5206\u6587\u4ef6":62,"blas\u7684\u8def\u5f84":29,"book\u4e00\u5b9a\u662f\u60a8\u6700\u597d\u7684\u9009\u62e9":30,"book\u662f\u4e3a\u7528\u6237\u548c\u5f00\u53d1\u8005\u5236\u4f5c\u7684\u4e00\u4e2a\u4ea4\u4e92\u5f0f\u7684jupyt":30,"book\u7684":33,"book\u7684docker\u955c\u50cf":30,"bool\u578b\u53c2\u6570":3,"boolean":[10,16,25],"build\u6307\u4ee4\u6309\u7167dockerfile\u751f\u6210\u955c\u50cf":30,"build\u8fd9\u4e2a\u547d\u4ee4\u7684":30,"byte":27,"c99\u662f\u76ee\u524dc\u6700\u5e7f\u6cdb\u7684\u4f7f\u7528\u6807\u51c6":25,"c\u6709\u6807\u51c6\u7684abi":25,"c\u8bed\u8a00\u662f\u6709\u5bfc\u51fa\u7b26\u53f7\u7684\u6807\u51c6\u7684":25,"caoying\u7684pul":65,"case":[10,16,26,44],"class":[7,10,12,14,15,16,17,18,19,20,22,23,25,27,41,64],"cmake\u4e2d\u5c06":44,"cmake\u627e\u5230\u7684python\u5e93\u548cpython\u89e3\u91ca\u5668\u7248\u672c\u53ef\u80fd\u6709\u4e0d\u4e00\u81f4\u73b0\u8c61":27,"cmake\u7f16\u8bd1\u65f6":29,"cmake\u914d\u7f6e\u4e2d\u5c06":44,"conf\u4f5c\u4e3a\u914d\u7f6e":65,"const":41,"container\u4e2d":51,"cost\u63a5\u6536y_predict\u4e0ey\u4f5c\u4e3a\u8f93\u5165":33,"cost\u8fd8\u5927\u4e8e\u8fd9\u4e2a\u6570":27,"count\u4e2agpu\u4e0a\u4f7f\u7528\u6570\u636e\u5e76\u884c\u6765\u8ba1\u7b97\u67d0\u4e00\u5c42":49,"count\u548cgpu":49,"cpu\u7248\u672c":32,"cuda\u5e73\u53f0":32,"cuda\u5e93":47,"cudnn\u5e93":[29,47],"dat\u4e2d":62,"data\u76ee\u5f55\u4e2d\u5b58\u653e\u5207\u5206\u597d\u7684\u6570\u636e":52,"dataprovider\u5171\u8fd4\u56de\u4e24\u4e2a\u6570\u636e":36,"dataprovider\u5171\u8fd4\u56de\u4e24\u7ec4\u6570\u636e":36,"dataprovider\u662fpaddlepaddle\u8d1f\u8d23\u63d0\u4f9b\u6570\u636e\u7684\u6a21\u5757":2,"dataprovider\u7684\u4ecb\u7ecd":[4,60],"dataprovider\u7f13\u51b2\u6c60\u5185\u5b58":27,"deb\u5b89\u88c5\u5305":32,"decay\u5219\u4e3a0":57,"decoder\u5faa\u73af\u5c55\u5f00\u7684\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65\u4f1a\u5f15\u7528\u5168\u90e8\u7ed3\u679c":38,"decoder\u63a5\u53d7\u4e24\u4e2a\u8f93\u5165":38,"decoder\u6bcf\u6b21\u9884\u6d4b\u4ea7\u751f\u4e0b\u4e00\u4e2a\u6700\u53ef\u80fd\u7684\u8bcd\u8bed":38,"decoer\u67b6\u6784":38,"default":[7,9,10,11,12,15,16,17,19,20,22,23,49,51,52,64],"demo\u9884\u6d4b\u8f93\u51fa\u5982\u4e0b":5,"dictionary\u662f\u4ece\u7f51\u7edc\u914d\u7f6e\u4e2d\u4f20\u5165\u7684dict\u5bf9\u8c61":3,"dictionary\u7531\u89e3\u6790\u81ea\u52a8\u751f\u6210":62,"dir\u4e2d\u670916\u4e2a\u5b50\u76ee\u5f55":65,"docker\u5b58\u5728\u95ee\u9898":30,"docker\u5b89\u88c5\u8bf7\u53c2\u8003":42,"docker\u7684\u5b98\u7f51":42,"docker\u955c\u50cf\u6765\u670d\u52a1html\u4ee3\u7801":30,"dockerfile\u4f4d\u4e8epaddlepaddl":30,"dockers\u8bbe\u7f6e":30,"dropout\u7684\u6bd4\u4f8b":41,"elec\u6d4b\u8bd5\u96c6":60,"embedding\u6a21\u578b\u9700\u8981\u7a0d\u5fae\u6539\u53d8\u63d0\u4f9b\u6570\u636e\u7684python\u811a\u672c":60,"encode\u6210\u7684\u6700\u540e\u4e00\u4e2a\u5411\u91cf":36,"encoder\u548cdecoder\u53ef\u4ee5\u662f\u80fd\u591f\u5904\u7406\u5e8f\u5217\u7684\u4efb\u610f\u795e\u7ecf\u7f51\u7edc\u5355\u5143":38,"encoder\u8f93\u51fa":38,"entropy\u4f5c\u4e3acost":27,"evaluator\u7684\u503c\u4f4e\u4e8e0":65,"export":[27,30,32,57],"f\u4ee3\u8868\u4e00\u4e2a\u6d6e\u70b9\u6570":[3,33],"false\u7684\u60c5\u51b5":3,"fc1\u548cfc2\u5c42\u5728gpu\u4e0a\u8ba1\u7b97":49,"fc3\u5c42\u4f7f\u7528cpu\u8ba1\u7b97":49,"final":[11,17,62],"float":[3,7,9,10,12,15,16,18,20,28,44,58,62],"float\u7b49":49,"function":[8,10,11,12,16,17,18,20,23,26,39,64],"gen\u6587\u4ef6\u5939\u4e2d\u7684\u6587\u4ef6\u5217\u8868":65,"generator\u4fbf\u4f1a\u5b58\u4e0b\u5f53\u524d\u7684\u4e0a\u4e0b\u6587":3,"generator\u81f3\u5c11\u9700\u8981\u8c03\u7528\u4e24\u6b21\u624d\u4f1a\u77e5\u9053\u662f\u5426\u505c\u6b62":3,"git\u6d41\u5206\u652f\u6a21\u578b":40,"github\u9996\u9875":40,"golang\u53ef\u4ee5\u4f7f\u7528":25,"golang\u7684":25,"gpu\u4e8c\u8fdb\u5236\u6587\u4ef6":29,"gpu\u5219\u8fd8\u9700\u8981\u9ad8\u5e76\u884c\u6027":44,"gpu\u53cc\u7f13\u5b58":3,"gpu\u548c\u975eavx\u533a\u5206\u4e86\u5982\u4e0b4\u4e2a\u955c\u50cf":30,"gpu\u6027\u80fd\u5206\u6790\u4e0e\u8c03\u4f18":43,"gpu\u6838\u5728\u8bad\u7ec3\u914d\u7f6e\u4e2d\u6307\u5b9a":47,"gpu\u7248\u672c":32,"gpu\u7248\u672c\u5e76\u60f3\u4f7f\u7528":63,"gpu\u7684docker\u955c\u50cf\u7684\u65f6\u5019":27,"gram\u7ea7\u522b\u7684\u77e5\u8bc6":64,"group\u6559\u7a0b":37,"gru\u6216lstm":39,"gru\u6a21\u578b":60,"gru\u6a21\u578b\u914d\u7f6e":60,"h\u5e76\u4e0d\u56f0\u96be":25,"hot\u7a20\u5bc6\u5411\u91cf":62,"html\u5373\u53ef\u8bbf\u95ee\u672c\u5730\u6587\u6863":42,"i\u4ee3\u8868\u4e00\u4e2a\u6574\u6570":[3,33],"id\u4e3a0\u7684\u6982\u7387":60,"id\u4e3a1\u7684\u6982\u7387":60,"id\u6307\u5b9a\u4f7f\u7528\u54ea\u4e2agpu\u6838":47,"id\u6307\u5b9a\u7684gpu":49,"id\u65e0\u6548":47,"image\u91cc":51,"imdb\u6570\u636e\u96c6\u5305\u542b25":64,"imdb\u6709\u4e24\u4e2a\u6807\u7b7e":64,"imdb\u7684\u6570\u6910\u96c6":64,"import":[3,5,9,10,16,23,27,28,33,56,57,58,62,64,65],"include\u4e0b\u9700\u8981\u5305\u542bcbla":29,"include\u4e0b\u9700\u8981\u5305\u542bmkl":29,"init_hook\u53ef\u4ee5\u4f20\u5165\u4e00\u4e2a\u51fd\u6570":3,"int":[3,7,9,10,11,12,15,16,17,20,25,26,36,41,49,60,62,63],"interface\u6587\u4ef6\u7684\u5199\u6cd5\u975e\u5e38":25,"issue\u7f16\u53f7":40,"job\u542f\u52a8\u540e\u4f1a\u521b\u5efa\u8fd9\u4e9bpod\u5e76\u5f00\u59cb\u6267\u884c\u4e00\u4e2a\u7a0b\u5e8f":50,"job\u6216\u8005\u5e94\u7528\u7a0b\u5e8f\u5728\u5bb9\u5668\u4e2d\u8fd0\u884c\u65f6\u751f\u6210\u7684\u6570\u636e\u4f1a\u5728\u5bb9\u5668\u9500\u6bc1\u65f6\u6d88\u5931":50,"job\u662f\u672c\u6b21\u8bad\u7ec3\u5bf9\u5e94\u7684job":52,"job\u7684\u540d\u5b57":52,"kubernetes\u4e3a\u8fd9\u6b21\u8bad\u7ec3\u521b\u5efa\u4e863\u4e2apod\u5e76\u4e14\u8c03\u5ea6\u5230\u4e863\u4e2anode\u4e0a\u8fd0\u884c":52,"kubernetes\u5206\u5e03\u5f0f\u8bad\u7ec3":43,"kubernetes\u5355\u673a\u8bad\u7ec3":43,"kubernetes\u53ef\u4ee5\u5728\u7269\u7406\u673a\u6216\u865a\u62df\u673a\u4e0a\u8fd0\u884c":50,"kubernetes\u53ef\u4ee5\u901a\u8fc7yaml\u6587\u4ef6\u6765\u521b\u5efa\u76f8\u5173\u5bf9\u8c61":52,"kubernetes\u5c31\u4f1a\u521b\u5efa3\u4e2apod\u4f5c\u4e3apaddlepaddle\u8282\u70b9\u7136\u540e\u62c9\u53d6\u955c\u50cf":52,"kubernetes\u63d0\u4f9b\u4e86\u591a\u79cd\u96c6\u7fa4\u90e8\u7f72\u7684\u65b9\u6848":50,"kubernetes\u652f\u6301\u591a\u79cdvolum":50,"kubernetes\u6709job\u7c7b\u578b\u7684\u8d44\u6e90\u6765\u652f\u6301":51,"kubernetes\u96c6\u7fa4\u5c31\u662f\u7531node\u8282\u70b9\u4e0emaster\u8282\u70b9\u7ec4\u6210\u7684":50,"label\u662f\u539f\u59cb\u6570\u636e\u4e2d\u5bf9\u4e8e\u6bcf\u4e00\u53e5\u8bdd\u7684\u5206\u7c7b\u6807\u7b7e":36,"labels\u662f\u6bcf\u7ec4\u5185\u6bcf\u4e2a\u53e5\u5b50\u7684\u6807\u7b7e":36,"layer1\u5fc5\u987b\u662f\u4e00\u4e2a0\u5c42\u5e8f\u5217":35,"layer1\u5fc5\u987b\u662f\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":35,"layer\u62ff\u5230\u7684\u7528\u6237\u8f93\u5165":38,"layer\u662f\u6211\u4eec\u7684\u79ef\u6728":33,"layer\u7c7b\u53ef\u4ee5\u81ea\u52a8\u8ba1\u7b97\u4e0a\u9762\u7684\u5bfc\u6570":41,"lib\u4e0b\u9700\u8981\u5305\u542bcblas\u548catlas\u4e24\u4e2a\u5e93":29,"lib\u4e0b\u9700\u8981\u5305\u542bcblas\u5e93":29,"lib\u4e0b\u9700\u8981\u5305\u542bopenblas\u5e93":29,"lib\u76ee\u5f55\u4e0b\u9700\u8981\u5305\u542bmkl_cor":29,"list\u4e2d\u7684\u6bcf\u4e00\u884c\u90fd\u4f20\u9012\u7ed9process\u51fd\u6570":3,"list\u5199\u5165\u90a3\u4e2a\u6587\u672c\u6587\u4ef6\u7684\u5730\u5740":3,"list\u548ctest":2,"list\u5982\u4e0b\u6240\u793a":49,"list\u5b58\u653e\u5728\u672c\u5730":2,"list\u6307\u5b9a\u6d4b\u8bd5\u7684\u6a21\u578b\u5217\u8868":49,"long":[10,11,16,17,20],"lstm\u67b6\u6784\u7684\u6700\u5927\u4f18\u70b9\u662f\u5b83\u53ef\u4ee5\u5728\u957f\u65f6\u95f4\u95f4\u9694\u5185\u8bb0\u5fc6\u4fe1\u606f":64,"lstm\u6a21\u578b":60,"lstm\u6a21\u578b\u914d\u7f6e":60,"lstm\u7f51\u7edc\u7c7b\u4f3c\u4e8e\u5177\u6709\u9690\u85cf\u5c42\u7684\u6807\u51c6\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":64,"memory\u4e0d\u80fd\u72ec\u7acb\u5b58\u5728":38,"memory\u4e5f\u53ef\u4ee5\u5177\u6709":39,"memory\u4e5f\u53ef\u4ee5\u662f\u5e8f\u5217":39,"memory\u53ea\u80fd\u5728":38,"memory\u53ef\u4ee5\u7f13\u5b58\u4e0a\u4e00\u4e2a\u65f6\u523b\u67d0\u4e00\u4e2a\u795e\u7ecf\u5143\u7684\u8f93\u51fa":36,"memory\u6307\u5411\u4e00\u4e2alay":38,"memory\u662f\u5728\u5355\u6b65\u51fd\u6570\u4e2d\u5faa\u73af\u4f7f\u7528\u7684\u72b6\u6001":39,"memory\u662fpaddlepaddle\u5b9e\u73b0rnn\u65f6\u5019\u4f7f\u7528\u7684\u4e00\u4e2a\u6982\u5ff5":36,"memory\u7684":39,"memory\u7684\u521d\u59cb\u72b6\u6001":38,"memory\u7684\u65f6\u95f4\u5e8f\u5217\u957f\u5ea6\u4e00\u81f4\u7684\u60c5\u51b5":36,"memory\u7684\u66f4\u591a\u8ba8\u8bba\u8bf7\u53c2\u8003\u8bba\u6587":38,"memory\u7684\u8f93\u51fa\u5b9a\u4e49\u5728":39,"memory\u7684i":38,"memory\u9ed8\u8ba4\u521d\u59cb\u5316\u4e3a0":38,"mkl\u7684\u8def\u5f84":29,"mkl_sequential\u548cmkl_intel_lp64\u4e09\u4e2a\u5e93":29,"mnist\u662f\u4e00\u4e2a\u5305\u542b\u670970":3,"mode\u548cattent":65,"mode\u7684python\u51fd\u6570":65,"model\u53ef\u4ee5\u901a\u8fc7":5,"model\u6765\u5b9e\u73b0\u624b\u5199\u8bc6\u522b\u7684\u9884\u6d4b\u4ee3\u7801":5,"movielens\u6570\u636e\u96c6":62,"name\u662f\u4f53\u88c1":62,"name\u662f\u5e74\u9f84":62,"name\u662f\u6027\u522b":62,"name\u662f\u7535\u5f71\u540d":62,"name\u662f\u804c\u4e1a":62,"name\u7ec4\u5408\u53ef\u4ee5\u627e\u5230\u672c\u6b21\u8bad\u7ec3\u9700\u8981\u7684\u6587\u4ef6\u8def\u5f84":52,"new":[10,16,20,24,26,40,41],"nfs\u7684\u90e8\u7f72\u65b9\u6cd5\u53ef\u4ee5\u53c2\u8003":50,"nmt\u6a21\u578b\u53d7\u5236\u4e8e\u6e90\u8bed\u53e5\u7684\u7f16\u7801":65,"noavx\u7248\u672c":32,"normalization\u5c42":58,"normalization\u5c42\u7684\u53c2\u6570":58,"notebook\u662f\u4e00\u4e2a\u5f00\u6e90\u7684web\u7a0b\u5e8f":30,"null":[10,16,41,47,62],"num_gradient_servers\u53c2\u6570":52,"openblas\u7684\u8def\u5f84":29,"out\u4e0b\u5305\u542b":57,"out\u7684\u6587\u4ef6\u5939":57,"outer_mem\u662f\u4e00\u4e2a\u5b50\u53e5\u7684\u6700\u540e\u4e00\u4e2a\u5411\u91cf":36,"output\u6587\u4ef6\u5939\u5b58\u653e\u8bad\u7ec3\u7ed3\u679c\u4e0e\u65e5\u5fd7":52,"packages\u91cc\u9762":27,"packages\u91cc\u9762\u7684python\u5305":27,"paddepaddle\u901a\u8fc7\u7f16\u8bd1\u65f6\u6307\u5b9a\u8def\u5f84\u6765\u5b9e\u73b0\u5f15\u7528\u5404\u79cdbla":29,"paddle\u4e00\u4e2a\u52a8\u6001\u5e93\u53ef\u4ee5\u5728\u4efb\u4f55linux\u7cfb\u7edf\u4e0a\u8fd0\u884c":25,"paddle\u4e2d\u7684\u4e00\u6761pass\u8868\u793a\u8bad\u7ec3\u6570\u636e\u96c6\u4e2d\u6240\u6709\u7684\u6837\u672c\u4e00\u6b21":65,"paddle\u4e2d\u7ecf\u5e38\u4f1a\u5c06\u65f6\u95f4\u5e8f\u5217\u6210\u4e3a":36,"paddle\u4e8c\u8fdb\u5236\u5728\u8fd0\u884c\u65f6\u6355\u83b7\u4e86\u6d6e\u70b9\u6570\u5f02\u5e38":27,"paddle\u5185\u5d4c\u7684python\u89e3\u91ca\u5668\u548c\u5916\u90e8\u4f7f\u7528\u7684python\u5982\u679c\u7248\u672c\u4e0d\u540c":25,"paddle\u5185\u90e8\u7684\u7c7b\u4e3ac":25,"paddle\u66ff\u6362\u6210":30,"paddle\u7684\u5404\u7248\u672c\u955c\u50cf\u53ef\u4ee5\u53c2\u8003":51,"paddle\u7684\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0\u5305\u62ec\u4e00\u4e0b\u51e0\u4e2a\u65b9\u9762":25,"paddle\u7684\u94fe\u63a5\u65b9\u5f0f\u6bd4\u8f83\u590d\u6742":25,"paddle\u7684dock":51,"paddle\u7684docker\u5f00\u53d1\u955c\u50cf\u5e26\u6709\u4e00\u4e2a\u901a\u8fc7":30,"paddle\u955c\u50cf":51,"paddle\u9700\u8981\u4e00\u4e2a\u591a\u8bed\u8a00\u63a5\u53e3":25,"paddle\u9759\u6001\u5e93\u94fe\u63a5\u590d\u6742":25,"paddlepaddle\u4e2d":[35,38],"paddlepaddle\u4e2d\u7684\u4e00\u4e2apass\u610f\u5473\u7740\u5bf9\u6570\u636e\u96c6\u4e2d\u7684\u6240\u6709\u6837\u672c\u8fdb\u884c\u4e00\u6b21\u8bad\u7ec3":64,"paddlepaddle\u4e2d\u7684\u8bb8\u591alayer\u5e76\u4e0d\u5728\u610f\u8f93\u5165\u662f\u5426\u662f\u65f6\u95f4\u5e8f\u5217":36,"paddlepaddle\u4e2d\u901a\u8fc7reader\u6765\u52a0\u8f7d\u6570\u636e":33,"paddlepaddle\u4e3a\u6df1\u5ea6\u5b66\u4e60\u7814\u7a76\u4eba\u5458\u63d0\u4f9b\u4e86\u4e30\u5bcc\u7684api":33,"paddlepaddle\u4f1a\u5728\u8c03\u7528\u8bfb\u53d6\u6570\u636e\u7684python\u811a\u672c\u4e4b\u524d":60,"paddlepaddle\u4f7f\u7528\u5747\u503c0":27,"paddlepaddle\u4f7f\u7528avx":27,"paddlepaddle\u4f7f\u7528swig\u5bf9\u5e38\u7528\u7684\u9884\u6d4b\u63a5\u53e3\u8fdb\u884c\u4e86\u5c01\u88c5":5,"paddlepaddle\u4fdd\u7559\u6dfb\u52a0\u53c2\u6570\u7684\u6743\u529b":3,"paddlepaddle\u5148\u4ece\u4e00\u4e2a\u6587\u4ef6\u5217\u8868\u91cc\u83b7\u5f97\u6570\u636e\u6587\u4ef6\u5730\u5740":28,"paddlepaddle\u53d1\u5e03\u65b0\u7248\u672c\u7684\u65f6\u5019\u90fd\u4f1a\u53d1\u5e03\u5bf9\u5e94\u7248\u672c\u7684\u751f\u4ea7\u955c\u50cf\u4ee5\u53ca\u5f00\u53d1\u955c\u50cf":30,"paddlepaddle\u53ef\u4ee5\u4f7f\u7528cudnn":29,"paddlepaddle\u53ef\u4ee5\u6267\u884c\u7528\u6237\u7684python\u811a\u672c\u7a0b\u5e8f\u6765\u8bfb\u53d6\u5404\u79cd\u683c\u5f0f\u7684\u6570\u636e\u6587\u4ef6":60,"paddlepaddle\u53ef\u4ee5\u6bd4\u8f83\u7b80\u5355\u7684\u5224\u65ad\u54ea\u4e9b\u8f93\u51fa\u662f\u5e94\u8be5\u8de8\u8d8a\u65f6\u95f4\u6b65\u7684":36,"paddlepaddle\u53ef\u4ee5\u901a\u8fc7\u8be5\u673a\u5236\u5224\u65ad\u662f\u5426\u5df2\u7ecf\u6536\u96c6\u9f50\u6240\u6709\u7684\u68af\u5ea6":41,"paddlepaddle\u5728\u5b9e\u73b0rnn\u7684\u65f6\u5019":36,"paddlepaddle\u5b58\u7684\u662f\u6709\u503c\u4f4d\u7f6e\u7684\u7d22\u5f15":[3,33],"paddlepaddle\u5b9a\u4e49\u7684\u53c2\u6570":3,"paddlepaddle\u5c06\u4ee5\u8bbe\u7f6e\u53c2\u6570\u7684\u65b9\u5f0f\u6765\u8bbe\u7f6e":60,"paddlepaddle\u5c06\u5728\u89c2\u6d4b\u6570\u636e\u96c6\u4e0a\u8fed\u4ee3\u8bad\u7ec330\u8f6e":28,"paddlepaddle\u5c06\u6bcf\u4e2a\u6a21\u578b\u53c2\u6570\u4f5c\u4e3a\u4e00\u4e2anumpy\u6570\u7ec4\u5355\u72ec\u5b58\u4e3a\u4e00\u4e2a\u6587\u4ef6":28,"paddlepaddle\u5c06train":3,"paddlepaddle\u5f00\u53d1\u955c\u50cf\u5c31\u88ab\u6784\u5efa\u5b8c\u6bd5\u4e86":30,"paddlepaddle\u63d0\u4f9b\u4e86\u5f88\u591a\u4f18\u79c0\u7684\u5b66\u4e60\u7b97\u6cd5":28,"paddlepaddle\u63d0\u4f9b\u4e86ubuntu":32,"paddlepaddle\u63d0\u4f9b\u6570\u4e2a\u9884\u7f16\u8bd1\u7684\u4e8c\u8fdb\u5236\u6765\u8fdb\u884c\u5b89\u88c5":31,"paddlepaddle\u652f\u6301\u4e0d\u540c\u7c7b\u578b\u7684\u8f93\u5165\u6570\u636e":33,"paddlepaddle\u652f\u6301\u4ee5\u4e0b\u4efb\u610f\u4e00\u79cdblas\u5e93":29,"paddlepaddle\u652f\u6301\u5927\u91cf\u7684\u8ba1\u7b97\u5355\u5143\u548c\u4efb\u610f\u6df1\u5ea6\u7684\u7f51\u7edc\u8fde\u63a5":28,"paddlepaddle\u652f\u6301\u975e\u5e38\u591a\u7684\u4f18\u5316\u7b97\u6cd5":27,"paddlepaddle\u652f\u6301sparse\u7684\u8bad\u7ec3":27,"paddlepaddle\u662f\u4e00\u4e2a\u6700\u65e9\u7531\u767e\u5ea6\u79d1\u5b66\u5bb6\u548c\u5de5\u7a0b\u5e08\u5171\u540c\u7814\u53d1\u7684\u5e76\u884c\u5206\u5e03\u5f0f\u6df1\u5ea6\u5b66\u4e60\u5e73\u53f0":0,"paddlepaddle\u662f\u6e90\u4e8e\u767e\u5ea6\u7684\u4e00\u4e2a\u6df1\u5ea6\u5b66\u4e60\u5e73\u53f0":[28,33],"paddlepaddle\u6bcf\u6b21\u53d1\u5e03\u65b0\u7248\u672c\u90fd\u4f1a\u53d1\u5e03\u5bf9\u5e94\u7684\u5f00\u53d1\u955c\u50cf\u4f9b\u5f00\u53d1\u8005\u76f4\u63a5\u4f7f\u7528":30,"paddlepaddle\u7684\u5185\u5b58\u5360\u7528\u4e3b\u8981\u5206\u4e3a\u5982\u4e0b\u51e0\u4e2a\u65b9\u9762":27,"paddlepaddle\u7684\u53c2\u6570\u4f7f\u7528\u540d\u5b57":27,"paddlepaddle\u7684\u6570\u636e\u5305\u62ec\u56db\u79cd\u4e3b\u8981\u7c7b\u578b":3,"paddlepaddle\u7684\u6587\u6863\u5305\u62ec\u82f1\u6587\u6587\u6863":42,"paddlepaddle\u7684\u6587\u6863\u6784\u5efa\u6709\u76f4\u63a5\u6784\u5efa\u548c\u57fa\u4e8edocker\u6784\u5efa\u4e24\u79cd\u65b9\u5f0f":42,"paddlepaddle\u7684\u6e90\u7801":40,"paddlepaddle\u7684\u7f16\u8bd1\u9009\u9879":31,"paddlepaddle\u7684bas":41,"paddlepaddle\u7684docker\u5bb9\u5668\u4f7f\u7528\u65b9\u5f0f":31,"paddlepaddle\u76ee\u524d\u53ea\u652f\u6301\u5728\u6bcf\u4e2a\u65f6\u95f4\u6b65\u4e2d":36,"paddlepaddle\u76ee\u524d\u552f\u4e00\u5b98\u65b9\u652f\u6301\u7684\u8fd0\u884c\u7684\u65b9\u5f0f\u662fdocker\u5bb9\u5668":30,"paddlepaddle\u76ee\u524d\u5df2\u7ecf\u5f00\u653e\u6e90\u7801":0,"paddlepaddle\u76ee\u524d\u63d0\u4f9b\u4e24\u79cd\u53c2\u6570\u521d\u59cb\u5316\u7684\u65b9\u5f0f":27,"paddlepaddle\u8c03\u7528process\u51fd\u6570\u6765\u8bfb\u53d6\u6570\u636e":60,"paddlepaddle\u8d1f\u8d23\u5b8c\u6210\u4fe1\u606f\u548c\u68af\u5ea6\u5728\u65f6\u95f4\u5e8f\u5217\u4e0a\u7684\u4f20\u64ad":38,"paddlepaddle\u8d1f\u8d23\u5b8c\u6210\u4fe1\u606f\u548c\u8bef\u5dee\u5728\u65f6\u95f4\u5e8f\u5217\u4e0a\u7684\u4f20\u64ad":38,"paddlepaddle\u955c\u50cf\u9700\u8981\u63d0\u4f9b":52,"paddlepaddle\u9700\u8981\u7528\u6237\u5728\u7f51\u7edc\u914d\u7f6e":2,"paddlepaddle\u9700\u8981\u7684\u6240\u6709\u7f16\u8bd1\u5de5\u5177":30,"pass\u4e2a\u6a21\u578b\u5230\u7b2c":47,"pass\u5230":65,"pass\u5c06\u4e0d\u8d77\u4f5c\u7528":47,"pass\u8f6e\u5f00\u59cb\u8bad\u7ec3":47,"pass\u8f6e\u7684\u6a21\u578b\u7528\u4e8e\u6d4b\u8bd5":47,"passes\u8f6e":47,"path\u6307\u5b9a\u6d4b\u8bd5\u7684\u6a21\u578b":49,"period\u4e2a\u6279\u6b21\u5bf9\u6240\u6709\u6d4b\u8bd5\u6570\u636e\u8fdb\u884c\u6d4b\u8bd5":47,"period\u4e2a\u6279\u6b21\u6253\u5370\u65e5\u5fd7\u8fdb\u5ea6":47,"period\u4e2a\u6279\u6b21\u8f93\u51fa\u53c2\u6570\u7edf\u8ba1":47,"period\u4e2a\u6279\u6b21\u8f93\u51fa\u7b26\u53f7":47,"period\u4e2abatch\u5904\u7406\u7684\u5f53\u524d\u635f\u5931":64,"period\u4e2abatch\u7684\u5206\u7c7b\u9519\u8bef":64,"period\u6574\u9664":47,"period\u8f6e\u4fdd\u5b58\u8bad\u7ec3\u53c2\u6570":47,"pod\u4e2d\u7684\u5bb9\u5668\u5171\u4eabnet":50,"pod\u662fkubernetes\u7684\u6700\u5c0f\u8c03\u5ea6\u5355\u5143":50,"pooling\u5bf9\u7279\u5f81\u56fe\u4e0b\u91c7\u6837":57,"process\u51fd\u6570\u4f1a\u7528yield\u8bed\u53e5\u8f93\u51fa\u8fd9\u6761\u6570\u636e":60,"prod\u7684\u751f\u4ea7\u955c\u50cf":30,"public":[20,41,51,64],"py_paddle\u91cc\u9762\u63d0\u4f9b\u4e86\u4e00\u4e2a\u5de5\u5177\u7c7b":5,"pydataprovider2\u4f1a\u5c3d\u53ef\u80fd\u591a\u7684\u4f7f\u7528\u5185\u5b58":3,"pydataprovider2\u63d0\u4f9b\u4e86\u4e24\u79cd\u7b80\u5355\u7684cache\u7b56\u7565":3,"pydataprovider2\u662fpaddlepaddle\u4f7f\u7528python\u63d0\u4f9b\u6570\u636e\u7684\u63a8\u8350\u63a5\u53e3":3,"pydataprovider2\u7684\u4f7f\u7528":[2,4,27,39,60,62],"pydataprovider\u4f7f\u7528\u7684\u662f\u5f02\u6b65\u52a0\u8f7d":27,"python\u4ee3\u7801\u5c06\u968f\u673a\u4ea7\u751f2000\u4e2a\u89c2\u6d4b\u70b9":28,"python\u53ef\u4ee5\u89e3\u9664\u6389\u5185\u90e8\u53d8\u91cf\u7684\u5f15\u7528":3,"python\u5c01\u88c5\u7684\u5b9e\u73b0\u4f7f\u5f97\u6211\u4eec\u53ef\u4ee5\u5728\u914d\u7f6e\u6587\u4ef6\u4e2d\u4f7f\u7528\u65b0\u5b9e\u73b0\u7684\u7f51\u7edc\u5c42":41,"query\u6539\u5199":65,"rate\u4e3a0":65,"rate\u4e3a5":65,"rate\u88ab\u8bbe\u7f6e\u4e3a0":57,"recommendation\u6587\u4ef6\u5939\u5185\u5b58\u653e\u8bad\u7ec3\u6587\u4ef6":52,"repo\u7684\u6839\u76ee\u5f55":30,"research\u5b9e\u9a8c\u5ba4\u641c\u96c6\u6574\u7406":61,"resnet\u6a21\u578b":59,"return":[3,10,11,16,17,19,20,22,23,28,33,36,39,41,52,58,60,62],"rnn\u5373\u65f6\u95f4\u9012\u5f52\u795e\u7ecf\u7f51\u7edc":36,"rnn\u5bf9\u4e8e\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65\u901a\u8fc7\u4e86\u4e00\u4e2alstm\u7f51\u7edc":36,"rnn\u603b\u662f\u5f15\u7528\u4e0a\u4e00\u65f6\u523b\u9884\u6d4b\u51fa\u7684\u8bcd\u7684\u8bcd\u5411\u91cf":38,"rnn\u6a21\u578b":60,"rnn\u76f8\u5173\u6a21\u578b":43,"search\u7684\u65b9\u6cd5":47,"sentences\u662f\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217\u7684\u6570\u636e":36,"seq\u53c2\u6570\u5fc5\u987b\u4e3afals":38,"server\u4e2a\u6279\u6b21\u6253\u5370\u65e5\u5fd7\u8fdb\u5ea6":47,"sh\u6765\u8bad\u7ec3\u6a21\u578b":57,"sh\u8c03\u7528\u4e86":58,"short":[10,11,16,17],"simd\u6307\u4ee4\u63d0\u9ad8cpu\u6267\u884c\u6548\u7387":27,"size\u4e3a1":65,"size\u4e3a50":65,"size\u4e3a512":47,"size\u53ef\u80fd\u4f1a\u5bf9\u8bad\u7ec3\u7ed3\u679c\u4ea7\u751f\u5f71\u54cd":27,"size\u5927\u5c0f\u4e3a128":64,"size\u662f3":65,"size\u672c\u8eab\u662f\u795e\u7ecf\u7f51\u7edc\u7684\u8d85\u53c2\u6570":27,"size\u7684\u503c":3,"softmax\u5c42":56,"softmax\u6fc0\u6d3b\u7684\u8f93\u51fa\u7684\u548c\u603b\u662f1":41,"sparse\u8bad\u7ec3\u9700\u8981\u8bad\u7ec3\u7279\u5f81\u662f":27,"srl\u4f5c\u4e3a\u5f88\u591a\u81ea\u7136\u8bed\u8a00\u5904\u7406\u4efb\u52a1\u4e2d\u7684\u4e2d\u95f4\u6b65\u9aa4\u662f\u5f88\u6709\u7528\u7684":63,"ssh\u65b9\u5f0f\u7684\u4e00\u4e2a\u4f18\u70b9\u662f\u6211\u4eec\u53ef\u4ee5\u4ece\u591a\u4e2a\u7ec8\u7aef\u8fdb\u5165\u5bb9\u5668":30,"ssh\u8fdb\u5165\u5bb9\u5668":30,"static":10,"step\u51fd\u6570\u4e2d\u7684memori":38,"step\u51fd\u6570\u5185\u90e8\u53ef\u4ee5\u81ea\u7531\u7ec4\u5408paddlepaddle\u652f\u6301\u7684\u5404\u79cdlay":38,"subseq\u7684\u6bcf\u4e2a\u5143\u7d20\u662f\u4e00\u4e2a0\u5c42\u5e8f\u5217":35,"super":41,"swig\u652f\u6301\u7684\u8bed\u8a00\u6216\u8005\u89e3\u91ca\u5668\u6709\u5c40\u9650":25,"swig\u66b4\u9732\u7684\u63a5\u53e3\u4fdd\u7559\u4e86c":25,"swig\u751f\u6210\u7684\u4ee3\u7801\u4e0d\u80fd\u4fdd\u8bc1\u591a\u8bed\u8a00\u4ee3\u7801\u98ce\u683c\u7684\u4e00\u81f4\u6027":25,"swig\u76f4\u63a5\u8bfb\u53d6c":25,"swig\u9700\u8981\u5199\u4e00\u4e2ainterface\u6587\u4ef6":25,"swig_paddle\u4e2d\u7684\u9884\u6d4b\u63a5\u53e3\u7684\u53c2\u6570\u662f\u81ea\u5b9a\u4e49\u7684c":5,"t\u6307\u5b9a\u4e86\u751f\u6210\u7684\u955c\u50cf\u7684\u540d\u5b57":30,"test\u548cgen\u8fd9\u4e09\u4e2a\u6587\u4ef6\u5939\u662f\u56fa\u5b9a\u7684":65,"tests\u7684paddlepaddl":40,"tflops\u4e86":44,"trainer\u63a5\u6536\u4e09\u4e2a\u53c2\u6570":33,"true":[7,9,10,11,12,15,16,17,19,20,22,23,26,27,36,39,41,49,52,58,62,63,64,65],"true\u8868\u793a\u53cd\u5411\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":39,"try":[12,18,24,26,27,62],"type\u662fon":62,"ubuntu\u7684deb\u5b89\u88c5\u5305\u7b49":31,"ubuntu\u90e8\u7f72paddlepaddl":31,"update\u53c2\u6570\u65f6\u624d\u6709\u6548":47,"utf8\u7f16\u7801":56,"uts\u7b49linux":50,"v2\u4e4b\u540e\u7684\u4efb\u4f55\u4e00\u4e2a\u7248\u672c\u6765\u7f16\u8bd1\u8fd0\u884c":29,"vocab\u4e2d\u6bcf\u4e2a\u5207\u5206\u5355\u8bcd\u7684\u9884\u671f\u8bc4\u7ea7":64,"vocab\u505a\u4e3a\u5b57\u5178":64,"void":[25,41],"volume\u6302\u8f7d\u5230\u5bb9\u5668\u4e2d":50,"w0\u548c":58,"wbias\u662f\u9700\u8981\u5b66\u4e60\u7684\u53c2\u6570":58,"while":[7,9,15,20,26,52,65],"words\u5373\u4e3a\u8fd9\u4e2a\u6570\u636e\u4e2d\u7684\u5355\u5c42\u65f6\u95f4\u5e8f\u5217":36,"words\u662f\u539f\u59cb\u6570\u636e\u4e2d\u7684\u6bcf\u4e00\u53e5\u8bdd":36,"x\u4e0ey\u4e3a\u4e4b\u524d\u63cf\u8ff0\u7684\u8f93\u5165\u5c42":33,"x\u548cwindow":30,"x\u548cwindows\u4e0a\u7684\u786c\u4ef6\u8d44\u6e90":30,"y\u8868\u793a\u8f93\u5165\u6570\u636e\u662f\u4e00\u4e2a\u7ef4\u5ea6\u4e3a1\u7684\u7a20\u5bc6\u5411\u91cf":33,"yaml\u6587\u4ef6\u4e2d\u5404\u4e2a\u5b57\u6bb5\u7684\u5177\u4f53\u542b\u4e49":52,"yaml\u6587\u4ef6\u63cf\u8ff0\u4e86\u8fd9\u6b21\u8bad\u7ec3\u4f7f\u7528\u7684docker\u955c\u50cf":52,"zero\u4e09\u79cd\u64cd\u4f5c":47,AGE:51,AWS:[50,53,54],Abs:6,And:[9,10,12,16,18,20,26],But:[10,11,16,17,27],EOS:[10,16],For:[5,8,9,10,12,16,18,20,23,26,32,44],IDs:20,NFS:50,Not:[23,24],One:[9,10,11,17,22],QoS:51,TLS:23,That:[10,16,20,26],The:[3,7,8,9,10,11,12,14,15,16,17,18,20,22,23,24,26,41,52,60,62,63,65],Their:[10,16,24],Then:[10,16,62],There:[9,10,16,20,22,23,24],Use:[23,26],Used:[11,17],Using:64,Will:[20,22],With:[10,11,16,17],Yes:30,___embedding_0__:52,___embedding_1__:52,__init__:41,__list_to_map__:62,__main__:[5,58],__meta__:62,__mse_cost_0__:52,__name__:[5,58],__rnn_step__:39,_link:[11,17],_proj:[10,16],_recurrent_group:39,_res2_1_branch1_bn:58,_source_language_embed:[39,56],_target_language_embed:[39,56],abc:[10,16],abl:[10,16,23],about:[10,11,16,17,63,65],abov:[3,10,16,23,24,44],abs:[11,17],accept:[20,23,26,63],access:[10,11,16,17,23],accord:[9,10,16],accrod:[11,17],accumul:24,accuraci:9,acl:64,aclimdb:64,across:[10,16],act:[10,11,16,17,27,28,33,36,39],act_typ:60,activ:[4,10,11,16,17,21,33,60],activi:[11,17],actual:[10,16],adadelta:[12,27,60],adagrad:[12,60],adam:[12,23,60,64],adamax:12,adamoptim:[56,60,64,65],adapt:[9,12,18],add:[10,11,16,17,20,40,62],add_input:41,add_test:41,add_to:[10,16],add_unittest_without_exec:41,addbia:41,added:[9,40],addit:[10,11,16,17],address:24,addrow:41,addto:10,addtolay:[10,16],adversari:26,affect:[10,16],afi:3,after:[10,16,20],again:[23,24],age:[20,52,62],agg_level:[10,16,35,36],aggregatelevel:[10,16,35,36],aircraft:65,airplan:57,aistat:[10,16],alex:[10,16,64],alexnet_pass1:49,alexnet_pass2:49,algo_hrnn_demo:36,algorithm:[10,12,16,18,56,64,65],align:[10,11,16,17,20,65],all:[3,7,9,10,12,15,16,18,23,24,38,52,62,63,64],alloc:[7,15],allow:[23,60],allow_only_one_model_on_one_gpu:[46,47,49],almost:[11,17],alreadi:[24,27,32],also:[9,10,11,16,17,20,23,26,44,60],alwai:[10,11,16,17,22,26,52],amazon:51,ambigu:26,amd64:50,analysi:[63,64],ani:[10,11,16,17,20,23,24,26],annot:63,annual:63,anoth:[10,16,23],anyth:[20,26,63],api:[20,22,23,44,52,55,64],api_pydataprovider2_sequential_model:8,api_trainer_config:62,apiserv:50,apivers:[50,51,52],apo:65,append:[3,22,26,36,39,52,62],append_gradient_machin:22,appleyard:44,appli:[10,11,16,17],applic:[44,51],approach:[10,16,60],apt:[32,57],arbitrari:[10,16],architectur:65,archiv:[20,25],arg:[3,8,9,10,11,12,16,17,20,27,28,52,57,58,60,62,63,64],arg_nam:[10,16],argpars:52,args_ext:52,argument:[3,8,10,16,20,52,60,62,63],argumentpars:52,argv:58,around:[3,10,16],arrai:[5,10,16,20,22,26,33,58],articl:40,arxiv:[10,11,16,17,64],ask:24,assert:5,assign:[10,16],associ:63,assum:[10,16],astyp:26,async:[12,24,46],async_count:[46,47],async_lagged_grad_discard_ratio:47,async_lagged_ratio_default:[46,47],async_lagged_ratio_min:[46,47],asynchron:24,atla:29,atlas_root:29,attenion:[11,17],attent:[10,11,16,17,65],attr:[7,11,15,16,17],attribut:[4,10,11,16,17,21],auc:[9,46],author:50,authorized_kei:45,auto:[25,41,44],automat:[10,16,23,65],automaticli:[10,16],automobil:57,avail:24,averag:[9,10,12,16,19,63],average_test_period:[46,47,63],averagepool:[10,16],avg:[13,35,44,60],avgcost:[9,60,62,64,65],avgpool:[10,16,60],avoid:[24,44],avx:30,await:51,awar:[23,24],azur:50,b2t:56,b363:51,b8561f5c79193550d64fa47418a9e67ebdd71546186e840f88de5026b8097465:51,back:24,backward:[10,11,14,16,17,41],backward_first:39,backwardactiv:41,bag:60,baidu:[10,16,51],balasubramanyan:64,bank:63,bardward:[11,17],bare:[50,51],barrierstatset:44,base:[6,12,16,17,19,20,23],baseactiv:[10,11,16],baseev:22,basematrix:41,basenam:9,basepool:13,basepoolingtyp:[10,11,16,17],baseregular:12,basestr:[7,8,9,10,11,15,16,17,19,22,62],bash:[30,40,42,51,52],basic:[10,16,22],batch:[9,10,11,12,16,17,18,20,22,23,24,28,33,45,51,52,57,58,60,62,64,65],batch_0:58,batch_id:[22,33],batch_norm:[10,17],batch_norm_lay:11,batch_norm_typ:[10,16],batch_read:26,batch_siz:[12,20,22,27,28,33,45,56,57,60,62,64,65],batchsiz:[10,16,41],bcd:[10,16],bcebo:20,beam:[10,16,39,65],beam_gen:[10,16,39],beam_search:[22,38,39],beam_siz:[10,16,39,46,47,49],beamsiz:65,becaus:[10,16,20,23,24,26,36],been:63,befor:[10,11,16,17,24,26,27,62],begin:[9,10,16],beginiter:[22,23],beginn:39,beginpass:[22,23],begintrain:23,being:[24,26],belong:[10,16],below:[10,16,20,24,26],benefit:[11,17],bengio:[10,16],bertolami:64,besid:[10,16,20],best:[8,10,16,62],best_model_path:63,besteffort:51,beta1:[12,18],beta2:[12,18],beta:58,better:[10,11,16,17],between:[10,12,16,18,24,65],bgr:58,bi_lstm:[11,17],bia:[10,11,12,16,17,18,41,58],bias:[10,16],bias_attr:[10,11,16,17,27,28,36,39],bias_param:27,bias_param_attr:[11,17],biases_:41,biasparameter_:41,biassiz:41,bidi:51,bidirect:[11,17,63,64],bidirectional_lstm_net:64,bigger:24,bilinear:[10,16],bilinear_interpol:[10,16],bilinearfwdbwd:44,bin:[30,32,45,50,51,52,62],binari:[9,10,16,20,60],bird:57,bitext:65,blank:[10,16],bleu:65,block:[10,16],block_expand:10,block_i:[10,16],block_x:[10,16],bn_attr:17,bn_bias_attr:[11,17],bn_layer_attr:11,bn_param_attr:[11,17],bollen:64,book:20,bool:[7,9,10,11,12,15,16,17,19,20,41,47,49,60,62,64],boot:[10,16,38,39],boot_bia:[10,16],boot_bias_active_typ:[10,16],boot_lay:[10,16,36,39],boot_with_const_id:[10,16],bos_id:[10,16,39],both:[7,10,11,14,15,16,17,23,24],bottom:60,bow:60,branch:[10,16,23,40],brelu:6,brendan:64,broadcast:24,broken:40,browser:30,bryan:64,buf_siz:20,buffer:[20,26],buffered_read:26,bug:40,build:[20,30,40,52,53,54,65],build_dict:20,build_doc:42,built:44,bunk:64,c11:25,c703c041:40,cach:[27,60,62,63],cache_pass_in_mem:[3,27,60,62,63],cachetyp:[3,27,60,62,63],calc_batch_s:[3,63],calcul:[9,10,11,12,16,17,18,24],call:[10,11,16,17,23,44,52,60],callabl:[10,16,20],callback:41,calrnn:36,caltech:57,can:[7,8,9,10,11,15,16,17,20,23,24,26,44,60],can_over_batch_s:[3,63],candid:[10,16],caption:65,card:45,care:[11,17,26],cat:[30,52,57,58,64],categori:[10,16,20,24,60],categorig:20,categoryfil:51,caus:24,ccb2_pc30:65,cde:[10,16],cdn:20,ceil:[10,16],ceil_mod:[10,16],cell:[10,11,16,17],ceph:50,certif:[23,50],cffi:25,cfg:51,cgo:25,chain:20,challeng:24,chanc:23,chang:[10,16,20,26,40,64],channel:[10,16,44],char_bas:62,check:[3,20,27,32,40,41,47],check_align:20,check_eq:41,check_fail_continu:3,check_l:41,check_sparse_distribution_batch:[46,47],check_sparse_distribution_in_pserv:[46,47],check_sparse_distribution_ratio:[46,47],check_sparse_distribution_unbalance_degre:[46,47],checkgrad:47,checkgrad_ep:47,checkout:40,chunk:9,chunk_schem:9,chunktyp:9,cifar:57,cifar_vgg_model:57,claimnam:52,clang:[25,30,40],class1:64,class2:64,class_dim:64,classic:[10,16],classif:[10,16,60,64,65],classifi:[9,58],classification_cost:[27,36,57,60],classification_error_evalu:[60,64,65],classification_threshold:9,clean:[27,40],client:50,clip:[7,12,15,47,60],clock:[10,16],clone:30,close:[3,26,40],cloud:24,cluster:[23,24,45,50,52],cluster_train:45,cmake:[27,29,40,42,44],cmakelist:41,cna:[10,16],cnn:[51,60],code:[3,5,20,23,26,30,41,51,62],coded_stream:27,codedinputstream:27,coeff:[10,16],coeffici:[10,16],collabor:24,collect:[10,16,20,22],collectbia:41,color:[57,58],colour:20,column:[9,10,16,26],com:[10,11,16,17,20,30,32,40,50,51,58],combin:[10,11,16,17,20,22,62],command:[41,49,51,52,53,54],commandlin:[44,52],comment:[11,17,36,52,60],commit:51,common_util:[45,62],commun:24,compil:32,complet:[10,11,16,17,20,22,24,51,52],complex:[11,17,26],complic:[10,16],compos:[20,23],composenotalign:20,comput:[10,11,16,17,23,24,63,64],conat:16,conat_lay:10,concat:[10,65],concat_lay:39,concaten:[11,17],concept:23,concern:23,concurr:24,condit:[10,16,51],conf:[5,10,16,27,36,45,52,56,58,65],conf_paddle_gradient_num:52,conf_paddle_n:52,conf_paddle_port:52,conf_paddle_ports_num:52,conf_paddle_ports_num_spars:52,config:[7,10,11,15,16,17,28,33,41,45,46,47,49,50,51,52,57,60,62,63,64,65],config_:47,config_arg:[46,47,49,58,60,63,64],config_bas:[16,17,22],config_fil:63,config_gener:[45,62],config_lay:41,config_pars:[5,41],configprotostr:27,configur:[8,10,16,41,56,58,65],conflict:40,confront:65,conll05st:63,conll:[20,63],connect:[11,17,51,60],connectionist:[10,16,64],connor:64,consequ:[10,11,16,17],consid:[9,10,12,16,18],consider:[11,17],consist:[10,16,20,26],construct:[23,62],construct_featur:62,consum:24,contact:24,contain:[3,8,9,10,11,16,17,19,20,22,23,51,52,60],context:[3,10,11,16,17,20,39,50],context_attr:[11,17],context_len:[10,11,16,17,60,62],context_proj_layer_nam:11,context_proj_nam:17,context_proj_param_attr:[11,17],context_project:[11,17,62],context_start:[10,11,16,17,60],continu:24,contrast:[10,16],control:[7,15,51,65],conv:[11,17],conv_act:[11,17],conv_attr:17,conv_batchnorm_drop_r:[11,17],conv_bias_attr:[11,17],conv_filter_s:[11,17],conv_layer_attr:11,conv_num_filt:[11,17],conv_op:[10,16],conv_pad:[11,17],conv_param_attr:[11,17],conv_shift:10,conv_strid:[11,17],conv_with_batchnorm:[11,17],conveni:23,convert:[3,5,20,26,60,62],convlay:[10,16],convolut:[10,11,16,17],convoper:[10,16],convtran:[10,16],convtranslay:[10,16],cool:40,copi:[22,23,62],core:[7,15],corpora:65,corpu:[20,63],correct:[9,10,16],correctli:[9,20],correspoind:23,correspond:23,corss_entropi:23,cos:[10,16],cos_sim:62,cosin:[10,16],cost:[12,18,22,23,28,33,62,64,65],cost_id:[10,16],could:[9,10,16,20,22,23,26],couldn:32,count:[24,26,44,47,49,51,62,63,64,65],counter:24,cpickl:62,cpp:[25,27,36,41,44,52,60,62,65],cpu:[3,7,10,15,16,30,32,44,49,51,52,63],cpuinfo:30,crash:[24,44],creat:[7,10,15,16,20,22,23,24,33,40,41,51,52],create_bias_paramet:41,create_input_paramet:41,createfromconfigproto:5,creator:20,cretor:20,crf:[10,63],crf_decod:10,critic:64,crlf:40,crop:58,crop_siz:58,cross:[10,16,27,60],cross_entropi:[16,23],cross_entropy_with_selfnorm:16,crt:50,csc:41,cslm:65,csr:41,ctc:10,ctc_layer:9,ctest:[30,40],ctrl:[45,62],ctx:63,ctx_0:63,ctx_0_slot:63,ctx_n1:63,ctx_n1_slot:63,ctx_n2:63,ctx_n2_slot:63,ctx_p1:63,ctx_p1_slot:63,ctx_p2:63,ctx_p2_slot:63,cub:57,cuda:[32,44,45,47],cuda_dir:[46,47],cuda_so:[27,30],cuda_visible_devic:27,cudaconfigurecal:44,cudadevicegetattribut:44,cudaeventcr:44,cudaeventcreatewithflag:44,cudafre:44,cudagetdevic:44,cudagetdevicecount:44,cudagetdeviceproperti:44,cudagetlasterror:44,cudahostalloc:44,cudalaunch:44,cudamalloc:44,cudamemcpi:44,cudaprofilerstart:44,cudaprofilerstop:44,cudaprofilestop:44,cudaruntimegetvers:44,cudasetdevic:44,cudasetupargu:44,cudastreamcr:44,cudastreamcreatewithflag:44,cudastreamsynchron:44,cudeviceget:44,cudevicegetattribut:44,cudevicegetcount:44,cudevicegetnam:44,cudevicetotalmem:44,cudnn:[10,16,19],cudnn_batch_norm:[10,16],cudnn_conv:[10,16],cudnn_conv_workspace_limit_in_mb:[46,47],cudnn_convt:[10,16],cudnn_dir:[46,47],cudnnv5:29,cudrivergetvers:44,cuinit:44,cumul:[10,16],curl:50,current:[3,10,12,16,24,50,60],current_word:39,currentcost:[9,60,62,64,65],currentev:[9,60,62,64,65],curv:23,custom:23,custom_batch_read:26,cutoff:20,cycl:24,cyclic:[10,16],cython:25,dalla:3,dan:63,darwin:50,dat:[20,45,62],data:[3,8,11,12,17,18,22,23,24,27,32,33,36,45,46,51,52,53,56,57,58,60,62,63,64,65],data_config:5,data_dir:[56,57,64,65],data_feed:20,data_fil:28,data_initialz:60,data_lay:[3,9,27,28,36,39,57,60,62,63],data_nam:20,data_provid:8,data_read:[20,26],data_reader_creator_random_imag:26,data_sourc:8,data_typ:[16,20,33],databas:20,datadim:[10,16],datalay:[10,16],dataprovid:[2,8,27,28,39,45,52,60,62,63],dataprovider_:60,dataprovider_bow:60,dataprovider_emb:60,dataproviderconvert:5,datasci:[10,16],dataset:[26,33,58,60,61,64,65],datasourc:[4,62],date:63,db_lstm:63,dcudnn_root:29,dead:24,deb:[32,40],decai:[12,18],decid:[23,26],declar:[10,11,16],decod:[10,11,16,17,38,39,65],decoder_boot:39,decoder_group_nam:39,decoder_input:39,decoder_mem:39,decoder_prev:[11,17],decoder_s:39,decoder_st:[11,17,39],deconv:[10,16],deconvolut:[10,16],decor:[3,20],deep:[10,16,44,57,58],deer:57,def:[3,5,10,16,20,23,26,27,28,33,36,39,41,52,58,60,62,63],defalut:[10,16],default_decor:52,default_devic:49,default_valu:49,defin:[3,8,9,10,11,16,17,20,23,26,27,33,60,62],define_py_data_sources2:[3,8,27,28,57,58,60,62],defini:65,definit:[20,24,56],degre:[10,16],del:62,delar:60,delet:[24,40],delimit:[9,62],demand:24,demo:[5,10,16,20,39,45,51,53,56,57,58,60,62,64,65],dens:[10,16,20,62],dense_vector:[3,5,16,20,28,33,62],dense_vector_sequ:20,depend:24,deriv:[14,23],descent:[10,12,16,24],describ:[23,51,60],descript:52,deseri:22,design:[10,16,20,25],desir:[24,51],detail:[7,10,11,12,15,16,17,18],detect:[9,40],determin:[10,16,20],dev:[27,30,40,57,62,65],develop:[40,65],deviat:[7,15],devic:[7,15,27,30,49],deviceid:49,devid:[10,16],dez:64,dfs:11,dict:[3,8,20,22,27,36,52,60,62,64,65],dict_dim:[27,36,64],dict_fil:[9,36,39,60,63],dict_nam:8,dict_siz:20,dictionai:60,dictionari:[3,8,9,10,16,20,22,23,27,60,65],dictrionari:60,dictsiz:65,diff:40,differ:[8,9,10,16,24],digit:[10,16],dim:[20,41,56,64],dimens:[10,14,16,19,20,27,60],dimes:[10,16],din:62,dir:[45,52,58,62,63,64,65],direct:[10,11,16,17],directli:[11,17],directori:[40,44,51],disabl:27,discard:[20,24,40,47],discount:[10,16],discov:24,discuss:23,dispatch:24,disput:65,dist_train:23,distanc:9,distribut:[10,16,47,53,54],distribute_test:[46,47],disucss:23,divid:[12,18],diy_beam_search_prob_so:[46,47],dmkl_root:29,do_forward_backward:26,doc:[5,11,17,20,30,42,52],doc_cn:42,docker:[27,30,40,51,52,53,54],docker_build:23,docker_push:23,dockerfil:[30,40,52],dockerhub:30,document:[11,17],documentari:3,doe:[11,17,24,26],doesn:[7,10,15,16,20,23,26],dog:[57,58],don:[11,17,23,26],done:[10,11,16,17,24,44,52],dot:65,dot_period:[47,49,52,57,62,64,65],dotmuloper:[10,16],dotmulproject:[10,16],doubl:47,down:44,download:[20,24,51],download_cifar:57,doxygen:40,dpkg:32,dpython_execut:27,dpython_include_dir:27,dpython_librari:27,drop_rat:[7,15],dropout:[7,10,15,16],dropout_lay:10,dropout_r:[11,17],drwxr:51,dso_handl:32,dtoh:44,dtype:[5,28,58],dubai:65,due:62,dure:[3,10,16,24,60,65],dwith_gpu:29,dwith_profil:44,dwith_tim:44,dynam:[3,26],dynamic_cast:41,each:[3,9,10,16,19,20,22,24,26,60,62],each_feature_vector:14,each_meta:62,each_pixel_str:3,each_sequ:[10,16,35,36],each_time_step_output:14,each_timestep:[10,16,35],each_word:3,eaqual:[10,16],eas:[20,26],easi:26,easier:[23,26],easili:[23,26],ec2:50,echo:[27,30,62,64],edit:9,edu:[20,51,57],efg:[10,16],either:[10,16,20,22,23,60],electron:51,elem_dim:[10,16],element:[9,10,11,16,17,20,22,26],elif:[23,62],els:[10,16,23,30,36,41,58,60,62],emb1:36,emb2:36,emb:[27,36,51,60],emb_group:36,emb_sum:27,embed:[10,23,56,62,64],embedding_lay:[27,36,39,60,62],embedding_nam:39,embedding_s:39,empir:[10,16],emplace_back:41,empti:[9,20,24,28],enabl:[7,15,44],enable_grad_shar:[46,47],enable_parallel_vector:47,enc_proj:[11,17,39],enc_seq:[11,17],enc_vec:39,encod:[11,17,36,65],encoded_proj:[11,17,39],encoded_sequ:[11,17,39],encoded_vector:39,encoder1:36,encoder1_expand:36,encoder1_rep:36,encoder2:36,encoder2_rep:36,encoder_last:[10,16],encoder_proj:39,encoder_s:39,end:[9,10,16,26,39,40,63,64],end_pass:23,enditer:[22,23,33],endpass:[22,23],endtrain:23,english:[10,16,65],ensembl:[11,17],ensur:24,entir:[10,11,16,17],entri:20,entropi:[10,16,60],enumer:[10,14,27,60,62],enumerate_data_types_of_data_lay:20,env:[27,52],environ:[23,27,44,51],eol:40,eos:10,eos_id:[10,16,39],epsilon:[12,18],equal:[10,11,12,16,17,24,36],equat:[10,11,12,16,17,18],equival:[10,16,23],error:[7,9,10,12,15,16,18,23,27,47,60,62,64,65],error_clipping_threshold:[7,15],errorr:9,especi:[11,17],essenc:23,essenti:[10,16,23],estim:[10,16,23],eta:51,etc:[12,20,26,65],etcd:24,eth0:[45,52],eval:[9,60,62,64,65],eval_bleu:65,evalu:[4,10,16,22,44,45,62,64,65],evaluate_pass:64,evaluator_bas:9,even:[23,26],event:[33,51],event_handl:[22,23,33],everi:[9,10,11,16,17,20,23,24],exactli:[9,10,11,16,17],exampl:[8,9,10,11,12,16,17,18,20,22,26,58,60],exc_path:27,exceed:[10,16],except:[20,62],excluded_chunk_typ:9,exconv:[10,16],exconvt:[10,16],exdb:20,exe:50,exec:30,execut:24,exist:[23,24,26,64],exit:51,exp:6,expand:[10,36],expand_a:[10,16,35,36],expand_lay:36,expand_level:[10,16,35],expandconvlay:[10,16],expandlevel:[10,16,35],expect:[10,16],expir:24,explain:[9,24],explan:[10,16],explicit:41,explicitli:23,explor:[10,16],exponenti:14,express:23,extend:62,extens:12,extern:25,extra:[10,11,15,16,17],extra_input:16,extra_lay:22,extraattr:[7,15,49],extraattribut:[16,17],extraattributenon:16,extract:[10,16,56,58,63],extract_fea_c:58,extract_fea_pi:58,extralayerattribut:[7,10,11,15,16],extralayeroutput:11,extrapaddl:17,extrem:[10,16],f1205:27,f120da72:51,fa0wx:51,fabric:45,facotr:[10,16],factor:[7,10,12,15,16,18],factori:25,fail:[27,32,47,51],failur:24,fake_imag:26,fals:[7,9,10,11,12,15,16,17,18,20,26,27,28,33,36,39,41,49,51,56,60,62,63,64,65],false_label:26,false_read:26,faq:55,fast:[10,16,44],faster:[10,11,16,17],fbd1f2bb71f4:51,fc1:[41,49],fc2:49,fc3:49,fc4:49,fc_act:[11,17],fc_attr:[11,17],fc_bias_attr:[11,17],fc_layer:[16,27,28,36,49,60,62],fc_layer_nam:11,fc_mat:22,fc_name:17,fc_param:27,fc_param_attr:[11,17],fclayer:41,fdata:[36,63],fea:58,fea_output:58,feat:64,featur:[3,10,14,16,20,40,58,60,62,63],feature_a:27,feature_b:27,feature_map:62,feed:[11,17,20,22,23,33],feeder:20,fernan:64,festiv:3,fetch:[20,40],few:[24,26],fewer:[10,16],fg0:[10,16],field:[10,16,22,62],figur:[23,56,65],file1:65,file2:65,file:[3,9,10,16,20,22,23,24,26,40,58,60,61,62,64,65],file_list:3,file_nam:[28,36,58,60,63],filenam:[3,27,62],filer:[10,16],fill:[10,16,24,60],filter:[10,16],filter_s:[10,11,16,17],filter_size_i:[10,16],find:[10,12,16,18,24,32],fine:[7,15],finish:[24,51],first:[10,16,20,23,24,60,62],first_seq:39,firstn:20,firstseen:51,fit:20,fix:[7,15,25,40],flag:20,flexiabl:26,flexibl:[10,11,16,17,23],flight:65,float32:[5,20,26,28,58],floor:[10,16],fly:60,folder:65,follow:[9,10,11,12,16,17,18,20,23,24,26,53,54,62],forbid:23,force_load:25,forget:[12,18,23],form:[11,12,17,18],format:[9,40,41],former:23,formula:[10,11,16,17],formular:[10,16],forward:[11,14,17,41],forwardactiv:41,forwardtest:5,found:[10,16],frame:9,framework:[23,60],free:20,french:65,frequenc:20,frequent:26,frog:57,from:[3,5,10,11,16,17,20,22,24,26,27,28,30,38,44,51,56,57,60,62,63,64,65],from_sequ:35,from_timestep:[10,16,35],fromfil:[26,28,58],fulfil:44,full:[10,16,24],full_matrix_project:[11,17,36,39],fulli:60,fullmatrixproject:[10,16],fully_matrix_project:[11,17],fullyconnectedlay:41,func:[3,20],further:[10,16],fusion:62,gain:[10,16],gamma:58,gan:23,gate:[10,11,16,17],gate_act:[10,11,16,17,36],gate_recurr:[10,16],gather:[10,16,62],gauss:[7,15],gcc:[25,30],gce:50,gcepersistentdisk:50,gdebi:32,gen:[10,16,65],gen_conf:65,gen_data:65,gen_result:65,gen_trans_fil:39,gender:[20,52,62],gener:[3,9,10,11,16,17,20,22,23,24,26,44,49,52,56,60,65],generatedinput:[38,39],genr:[52,62],gereat:9,get:[3,10,11,16,17,20,22,32,41,51,57,60,62,63,64],get_batch_s:63,get_best_pass:64,get_config_arg:[49,60,62,64],get_data:[51,60,63],get_dict:20,get_embed:20,get_imdb:64,get_input_lay:41,get_model:58,get_movie_title_dict:20,get_output_attr:17,get_output_layer_attr:11,get_sample_from_lin:27,get_shap:22,get_word_dict:20,getbatchs:41,getenv:[23,52],gethostbynam:52,gethostnam:52,getidmap:52,getinput:41,getinputgrad:41,getinputvalu:41,getoutputgrad:41,getoutputvalu:41,getparameterptr:41,getpodlist:52,getsiz:41,gettranspos:41,getw:41,getweight:41,getwgrad:41,gildea:63,gist:[11,17],git:[30,40],github:[10,11,16,17,30,32,40,58],give:[3,24],given:[20,22,26,60],global:[7,12,15,23,24,44,62],global_learning_r:[7,15],globalstat:44,globalstatinfo:44,globe:3,glusterf:50,godoc:25,goe:[10,11,16,17,24],good:[10,16,26],goodfellow13:[10,16],googl:[23,27],googleapi:50,gpu:[7,10,12,15,16,19,27,30,32,44,49,58,63,64,65],gpu_id:[27,47,49],gpugpu_id:46,grab:24,grad:47,grad_share_block_num:[46,47],gradient:[7,9,10,12,15,16,18,22,24,47,52,60],gradient_clipping_threshold:[7,12,15,60,64],gradient_machin:22,gradientmachin:[5,22,52,62,65],gram:56,graph:[10,16,22,24],grave:64,greater:[10,16],grep:[30,64],groudtruth:39,ground:[9,10,16,65],group:[11,17],group_id:62,group_input:[36,39],grouplen:[20,61],gru:[10,16,60],gru_attr:17,gru_bias_attr:[11,17],gru_decod:39,gru_decoder_with_attent:39,gru_encoder_decod:[56,65],gru_layer_attr:11,gru_memori:[11,17],gru_siz:60,gru_step:[17,39],gru_step_lay:[11,39],grumemori:[11,17,39],gserver:[10,16,41],gsizex:44,guid:51,gur_group:[11,17],gzip:51,hadoop:23,handl:[23,26],handler:22,handwrit:64,harvest:60,has:[10,11,12,16,17,18,20,23,24,44,60,63],has_kei:22,hassubseq:36,have:[9,10,11,16,17,20,23,24,26],head:64,header:[28,58,62],height:[10,16,20,25,26,41],held:24,hello:23,help:[5,40],helper:[8,10,11,16,17],here:[7,10,11,15,16,17,20,23,26],heurist:[10,16,65],hidden:[10,11,16,17,27,62],hidden_a:27,hidden_b:27,hidden_dim:36,hidden_s:[11,17,62],hierach:38,hierarch:[10,16,36],high:[7,15],highest:20,highli:20,him:23,hint:5,hl_dso_load:32,hl_get_sync_flag:41,hold:[23,24],home:[45,51,52],hook2:36,hook:[36,62,63],horizont:[10,16],hors:57,horst:64,host:[45,51],hostnetwork:52,hostpath:[50,51,52],hot:60,hous:[3,20],how:[7,10,15,16,23,24],howardjohnson:36,howev:[11,17,26],howto:52,hpp:25,html:[20,42,57],htod:44,http:[10,11,16,17,20,30,32,40,50,51,57,58,61,65],huber:[10,16],huge:[10,16],huina:64,hyper:[10,16],hyperplan:20,i0601:62,i0706:65,i0719:65,i1116:52,i1117:44,ib0:45,ics:20,icwsm:64,id_input:[9,39],idea:[10,16,26],identityoffsetproject:[10,16],identityproject:[10,16],idmap:52,ids:[9,10,16,27,60,62],idx:41,ieee:64,ignor:[3,9,10,16],ijcnlp:64,illustr:24,ilsvrc:58,imag:[19,20,23,26,40,51,52,53,54,57,58,65],image_a:26,image_b:26,image_classif:57,image_fil:26,image_lay:26,image_list_provid:58,image_nam:23,image_path:26,image_provid:57,image_reader_cr:26,image_s:58,imagepullpolici:52,imageri:[10,16],images_reader_cr:26,imdber:64,img:[3,10,16,57],img_conv:17,img_conv_lay:11,img_norm_typ:10,img_pool:17,img_pool_lay:11,img_siz:57,imgsiz:44,imgsizei:44,imgsizex:44,immutable_paramet:23,implement:[10,11,12,16,17,18,20],importerror:62,inarg:5,inc_path:27,includ:[10,11,16,17,20,23,25,40,44],incorrect:[10,16],increas:[24,27],incupd:41,inde:[20,26],independ:[10,16],index:[9,10,16,19,20,22,24,36,42,62],indexslot:[10,16],indic:[9,10,16],industri:24,infer:[23,24],infiniband:45,info:[9,10,16,20,33,36,41,45,52],inform:[9,20],ininst:23,init:[7,15,33,41,52,62,63],init_hook:[36,60,62,63],init_hook_wrapp:8,init_model_path:[46,47,49,56,60,63],initi:[3,7,10,15,16,20,47,60],initial_max:[7,15,27],initial_mean:[7,10,15,16,27],initial_min:[7,15,27],initial_std:[7,10,15,16,27],initpaddl:5,inlcud:[11,17],inner:[27,36],inner_:36,inner_mem:36,inner_param_attr:[11,17],inner_rnn_output:36,inner_rnn_st:36,inner_rnn_state_:36,inner_step:36,inner_step_impl:36,input1:[10,11,16,17],input2:[10,16],input:[3,9,10,11,14,16,17,19,20,22,26,27,28,33,35,36,38,39,41,49,52,56,57,60,62,63,65],input_data:41,input_data_target:41,input_featur:14,input_fil:[28,63],input_hassub_sequence_data:41,input_id:[10,16],input_imag:[11,17,57],input_index:41,input_label:41,input_lay:[10,41],input_nam:23,input_sequence_data:41,input_sequence_label:41,input_sparse_float_value_data:41,input_sparse_non_value_data:41,input_t:41,input_typ:[27,28,36,39,60,62],inputdef:41,inputlayers_:41,inputtyp:20,insert:40,insid:[9,10,16,24,26],instal:[27,32,40,45,51,57,62],instanc:[10,12,16,24],instanti:24,instead:[10,16,19,26],int32:47,integ:[3,9,10,16,20,25,33,60],integer_sequ:27,integer_valu:[3,20,27,33,36,60],integer_value_sequ:[3,20,36,39,60,63],integer_value_sub_sequ:36,integr:63,inter:[10,16],intercept:[10,16],interfac:[7,10,11,15,16,17,45],intergr:[10,16],intern:[10,11,16,17,20,22],internet:24,interpol:10,interpret:9,introduc:24,invalid:26,invok:[3,10,16,22,44,62],iob:9,ioe:9,ip_str:52,ipc:50,ips:52,ipt:[10,16,27,36,39],ipython:23,is_async:12,is_gener:[10,16,56,65],is_kei:62,is_layer_typ:10,is_predict:[60,62,64],is_seq:[10,16,39,62],is_sequ:62,is_stat:[7,15],is_test:[58,63,64],is_train:3,isinst:[5,33],ispodallrun:52,isspars:41,item:[10,16,20,22,26,52],iter:[3,10,11,12,16,17,18,20,22,23,24,26],its:[3,9,10,11,16,17,23,24,32,44],itself:[11,17,24],java:25,jeremi:44,jie:[63,64],jmlr:[10,16],job:[9,20,45,46,47,49,50,52,58,60,63,64,65],job_dispatch_packag:45,job_id:20,job_mod:56,job_nam:52,job_namespac:52,job_path:52,job_path_output:52,job_workspac:45,jobnam:52,jobpath:52,jobselector:52,johan:64,join:[24,36],joint:65,jointli:[11,17,65],journal:[63,64],jpg:58,json:[45,51,62],jth:[11,17],jupyt:30,just:[9,10,11,14,16,17,20],jypyt:23,k8s:52,k8s_data:52,k8s_job:23,k8s_token:23,k8s_train:52,k8s_user:23,kaim:[10,16],kaimingh:58,kebilinearinterpbw:44,kebilinearinterpfw:44,keep:[10,16,24],kei:[3,20,22,24,40,44,50,52,62],kernel:[10,16],key1:47,key2:47,keyword:52,kill:24,kind:[23,24,50,51,52],kingsburi:63,know:[11,17,23],kriz:[20,57],ksimonyan:[11,17],kube:50,kube_cluster_tl:23,kube_ctrl_start_job:23,kube_list_containers_in_job_and_return_current_containers_rank:23,kubeadm:50,kubectl:[50,51,52],kubernet:[23,24,43,45,52,53,54],kubernetes_service_host:23,kwarg:[3,9,10,11,12,16,17,18,20,36,60,62,63],l1_rate:[7,15],l2_rate:[7,15],l2regular:[57,60,64],label:[3,9,10,12,16,18,20,22,26,27,28,33,36,51,57,58,60,62,63,64],label_dict:63,label_dim:[10,16,36,60],label_fil:[26,63],label_lay:[10,26],label_list:63,label_path:26,label_slot:63,labeledbow:64,labelselector:52,lag:47,lake:3,lambdacost:[10,16],lambdarank:[10,16],languag:[10,16,20,56],larg:[19,20,40,65],larger:[7,9,10,12,15,16],last:[9,10,11,16,17,35,36],last_seq:36,last_time_step_output:[10,16],lastseen:51,later:60,latest:[10,16,24,27,51,52],launcher:23,layer1:[10,11,16,17,35],layer2:[10,16,35],layer3:[10,16],layer:[4,5,7,9,11,15,17,19,20,21,22,26,33,35,38,39,41,58,60,62,63],layer_0:41,layer_attr:[10,16,39,49],layer_num:[49,58],layer_s:[10,16],layer_typ:[10,16],layerbas:41,layerconfig:41,layergradutil:41,layermap:41,layeroutout:[10,16],layeroutput:[9,11,16,62],layers_test:27,lbl:[9,57],ld_library_path:[32,45],learn:[7,9,10,11,12,15,16,17,18,20,23,26,30,44,57,58,63,64,65],learnabl:[10,16],learning_method:[12,28,56,57,60,62,64,65],learning_r:[7,12,15,27,28,56,57,60,62,64,65],leas:24,least:[9,10,16,24],lecun:20,left:[10,16],leman:65,len:[3,10,16,36,39,41,52,60,62,63],length:[10,11,16,17,20,51],less:[10,16,23],less_than:23,let02:51,let:[10,16,23],level:[7,10,15,16,38],lib64:[27,30,32,45,47],lib:[29,32],lib_path:27,libcuda:[27,30],libjpeg:57,libnvidia:[27,30],libpaddl:[25,40],libprotobuf:27,librari:[10,16,32,45,47],life:24,like:[9,10,16,20,24,26,58],limit:[10,16,20,27,44],line:[3,9,20,27,36,40,49,60,62,63],line_count:27,linear:[6,10,16,33],linear_comb:10,linearactiv:[10,16,28],linguist:63,link:[10,11,16,17,38,64],linux:[30,50],lipeng:56,lipton:64,list:[2,3,8,9,10,11,16,20,22,23,28,45,49,57,58,60,62,63,64,65],lium:65,live:24,liwicki:64,load:[10,16,23,24,28,52,58,62,63,64,65],load_data_arg:5,load_featur:58,load_feature_c:58,load_feature_pi:58,load_missing_parameter_strategi:[46,47,49,56,63],loadparamet:5,loadsave_parameters_in_pserv:[46,47],local:[7,15,24,29,32,45,46,47,52],localhost:[30,50],localip:52,lock:24,log:[6,27,41,45,47,51,52,57,62,63,64,65],log_barrier_abstract:[46,47],log_barrier_lowest_nod:[46,47],log_barrier_show_log:[46,47],log_clip:[46,47],log_error_clip:[46,47],log_period:[47,49,51,52,57,60,62,63,64,65],log_period_serv:[46,47],logarithm:14,logger:[3,36],logist:60,look:[3,9,60],loop:26,loss:[10,16,60],low:[10,16],lst:62,lstm:[10,16,36,39,51,60],lstm_attr:17,lstm_bias_attr:[11,17],lstm_cell_attr:[11,17],lstm_group:[11,17,36],lstm_group_input:36,lstm_input:36,lstm_last:36,lstm_layer_attr:11,lstm_nest_group:36,lstm_output:36,lstm_size:60,lstm_step:[11,17],lstmemori:[11,17,36,39],lstmemory_group:[10,16,36],ltr:[10,16],mac:30,machan:[11,17],machin:[10,11,12,16,17,20,22,38,64,65],made:24,mai:[8,9,10,16,26],main:5,maintain:[10,16],major:65,make:[3,10,23,24,26,32,40,41,44,64],manag:24,mandarin:[10,16],mani:[10,11,16,17],manufactur:65,mao:64,map:[10,16,20,22,23,33,62],map_read:20,mapreduc:23,marcu:64,mark:63,mark_slot:63,market:64,martha:63,mask:[7,10,15,16],master:[23,50,64],mat_param_attr:[11,17],math:[11,17,25,41,44],matirx:[10,16],matplotlib:57,matrix:[9,10,11,16,17,20,22,25,41],matrixptr:41,max:[7,10,13,15,16,20,35,44,49,62],max_id:[16,22],max_job_id:20,max_length:[10,16,39],max_movie_id:20,max_sort_s:[10,16],max_user_id:20,maxid:[9,10],maxid_lay:9,maxim:[10,16],maximum:[9,20],maxinum:19,maxout:10,maxpool:[10,16],mayb:[10,11,16,17],md5:20,mean:[7,9,10,11,12,15,16,17,18,19,20,22,26,27,47,58,60,62],mean_img_s:57,mean_meta:58,mean_meta_224:58,mean_valu:58,mechan:[10,11,16,17],meet:63,mem:[10,16,36],member:23,memcpi:44,memori:[11,17,39,44,51,60],memory_nam:[10,16],memory_threshold_on_load_data:[46,47],mere:[11,17],merg:40,mergedict:[56,65],messag:[40,51],meta:[45,57,58,62],meta_config:[45,62],meta_fil:62,meta_gener:[45,62],meta_path:57,meta_to_head:62,metadata:[51,52],metal:50,metaplotlib:23,method:[3,8,10,11,12,16,18,22,62,65],mfs:52,might:[10,16],million:20,min:[7,15,44,49,62],min_pool_s:[3,27],mini:[10,16,20,22,24],mini_batch:26,minibatch:[10,16],minibatch_data:20,minikub:50,minim:[12,18],minimum:[10,16],minut:24,miss:63,mix:[11,17],mixed_attr:17,mixed_bias_attr:[11,17],mixed_lay:[11,16,36,39,63],mixed_layer_attr:11,mixedlayertyp:[10,16],mkl:29,mkl_root:29,ml_data:[45,62],mnist:[3,5,26],mnist_model:5,mnist_provid:3,mnist_random_image_batch_read:26,mnist_train:[3,26],mnist_train_batch_read:26,mnt:52,mod:63,mode:[10,16,40,52,64],model:[10,11,12,16,17,20,24,49,56,57,60,62,63,64,65],model_averag:12,model_config:5,model_list:[47,49,63,64],model_output:64,model_path:49,model_zoo:[56,58],modelaverag:12,modifi:40,modul:[3,8,11,17,20,22,27,28,57,58,60,62],modulo:[10,16],momentum:[7,12,15,27,33,60],momentumoptim:[28,57],mon:51,mono:[10,16],month:65,mood:64,moosef:50,more:[9,10,11,16,17,20,23,24,26,27,44],morin:[10,16],mose:[64,65],moses_bleu:65,mosesdecod:64,most:[10,16,20,23,26],mountpath:[51,52],move:[10,16,24],movi:[3,20,62],movie_categori:20,movie_featur:62,movie_head:62,movie_id:[52,62],movie_info:20,movie_meta:62,movie_nam:62,movie_review:20,movieinfo:20,movielen:61,moving_average_fract:[10,16],mpi:45,mse:10,mse_cost:[28,33,62],much:[10,16,24,26],mul:41,multi:[10,16,58,65],multi_binary_label_cross_entropi:16,multi_crop:58,multinomi:[10,16],multipl:[9,10,11,16,17,20,23],multipli:[9,10,16],must:[9,10,11,14,16,17,26,32,41],mypaddl:[51,52],name:[3,7,8,9,10,11,15,16,17,19,20,22,23,24,27,28,30,33,36,39,41,44,49,50,51,52,53,54,57,60,62,65],namespac:[25,41,50,51,52],nativ:[10,16],nchw:[10,16],ndarrai:22,ndarri:22,ndcg:[10,16],ndcg_num:[10,16],necessari:[10,16,60],need:[10,11,16,17,20,23,27,44,52,60],neg:[3,9,10,16,60,63,64],neg_distribut:[10,16],nest:20,net:[10,11,16,17,64],net_conf:64,net_diagram:58,network:[4,5,7,9,10,12,15,16,18,20,21,22,23,26,33,36,45,49,52,56,57,58,62,63,64,65],network_config:49,neural:[10,11,12,16,17,18,20,22,23,36,38,56,62,63,64,65],neuralnetwork:[10,16],never:[20,26,51,52],next:[3,10,16,20,24],nfs:52,nfsdir:52,nginx:30,nic:[45,46,47,52],nine:20,nlp:[10,16],nltk:20,nmt:65,nnz:41,no_cach:3,no_sequ:[3,33,62],noah:64,noavx:30,node0:52,node:[10,16,50,51,52],node_0:52,node_1:52,node_2:52,nodebook:30,nodefil:45,nois:[10,16],non:[10,16,24],none:[3,5,7,8,9,10,11,12,15,16,17,18,19,20,22,23,28,39,58,60],norm_by_tim:[10,16],normal:[10,11,16,17,20,51,52,58],notat:[10,16],note:[7,10,11,12,15,16,17,19,22,23,26,32,64],noth:[14,22,40],novel:64,now:[10,16,24,38],np_arrai:20,ntst1213:65,ntst14:65,nullptr:[32,41],num:[10,16,45,47,52,63,64,65],num_channel:[10,11,16,17,57],num_chunk_typ:9,num_class:[10,11,16,17,57],num_filt:[10,11,16,17],num_gradient_serv:[46,47],num_group:[10,16],num_neg_sampl:[10,16],num_parameter_serv:23,num_pass:[22,28,33,46,47,49,51,52,60,62,63,64,65],num_repeat:[10,16],num_result:9,num_results_per_sampl:[10,16],number:[9,10,16,20,24,26,65],numchunktyp:9,numdevices_:49,numlogicaldevices_:49,numofallsampl:9,numofwrongpredict:9,numpi:[20,22,26,28,33,58],numsampl:44,numtagtyp:9,nvcc:30,nvidia:[27,30],obj:[3,8,27,28,57,58,60,62],object:[3,7,8,9,10,11,12,15,16,17,18,20,22,23,25,44,60,62],observ:[12,18],occup:[52,62],occur:[20,22],oct:51,odd:[10,16],off:[29,30,32,40],offlin:24,offset:[10,16,62],often:9,ograd:41,omit:[27,60],on_init:3,onc:[10,16,24],one:[3,8,9,10,11,12,14,16,17,18,19,20,23,24,26,60,63,64],one_host_dens:62,one_hot_dens:62,onli:[9,10,11,16,17,19,20,22,23,36,38],onlin:[12,18,24,26],open:[3,10,16,23,26,27,28,36,42,58,60,62,63],openbla:29,openblas_root:29,oper:[10,11,12,16,17,18],opinion:64,opt:[23,29,52],optim:[4,7,15,21,22,27,33],option:[9,10,16,23],order:[10,11,16,17,20,26,52],org:[10,11,16,17,20,30,61],organ:[10,16],origin:[10,16,20,40],other:[9,10,11,12,16,17,20,60,62],otherchunktyp:9,otherwis:[8,10,16,20,23,24,26],our:23,out:[10,16,22,23,36,38,39,57],out_dir:52,out_left:[10,16],out_mem:39,out_right:[10,16],out_size_i:[10,16],out_size_x:[10,16],outer:36,outer_mem:36,outer_rnn_st:36,outer_rnn_state_:36,outer_step:36,output:[7,9,10,14,15,16,17,19,20,22,23,26,27,28,36,39,45,49,51,52,56,57,58,60,62,63,64],output_:[10,16],output_dir:58,output_fil:63,output_id:[10,16],output_lay:[22,58],output_max_index:19,output_mem:[10,16,39],outputh:[10,16],outputw:[10,16],outsid:[3,10,11,16,17],outv:41,over:[10,11,16,17,23],overrid:24,packag:[20,27],pad:10,pad_c:[10,16],pad_h:[10,16],pad_w:[10,16],padding_attr:[10,16],padding_i:[10,16],padding_x:[10,16],paddl:[3,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,22,23,24,25,27,28,30,32,33,40,41,42,44,45,49,51,52,56,57,60,62,63,64,65],paddle_error:25,paddle_matrix:25,paddle_matrix_shap:25,paddle_n:[45,52],paddle_output:51,paddle_port:[45,52],paddle_ports_num:[45,52],paddle_ports_num_for_spars:45,paddle_ports_num_spars:52,paddle_process_by_paddl:52,paddle_pserver2:45,paddle_root:56,paddle_server_num:52,paddle_source_root:56,paddle_train:[45,52],paddledev:[27,30,51,52],paddlepaddl:[10,11,12,16,17,20,24,26,27,30,32,39,40,44,45,53,54,56,63],pair:9,palceholder_just_ignore_the_embed:56,palmer:63,paper:[10,16,65],para:56,paraconvert:56,parallel:[44,49,51,52,65],parallel_nn:[7,15,46,47],param:[7,10,15,16,62],param_attr:[10,11,16,17,27,28,39],paramattr:[7,10,15,16,27,28,39],paramet:[4,9,10,11,12,16,17,18,19,20,21,26,33,47,52,62,63,64,65],parameter_attribut:[10,16],parameter_block_s:[46,47],parameter_block_size_for_spars:[46,47],parameter_learning_r:[7,15],parameter_nam:[22,23],parameter_serv:23,parameterattribut:[7,10,11,15,16,17],parameterclient2:52,parametermap:41,parameters_:41,parameterset:23,parametris:[12,18],paramutil:62,paraphras:[56,65],paraphrase_data:56,paraphrase_model:56,paraphrase_modeldata:56,paraspars:41,parent:10,pars:[20,62],parse_config:5,parse_known_arg:52,parsefromstr:27,parser:52,part:[62,64],partial:[10,16],partit:24,paserv:52,pass:[3,8,10,16,20,22,24,26,27,28,33,40,44,47,49,51,52,57,60,62,63,64,65],pass_id:[22,33],pass_idx:26,passtyp:41,past:23,path:[9,20,22,24,26,32,45,47,50,51,52,63,64],pattern:[20,24,25,62,64],paul:63,paus:24,pave:65,pdf:[10,11,16,17],pem:23,pend:24,penn:63,per:[10,16,20,26],perform:[10,11,16,17,44,46],period:[24,47,63,64,65],perl:[64,65],peroid:[10,16],persistentvolum:50,persistentvolumeclaim:[50,52],person:23,pickl:62,picklabl:8,pid:50,piec:[10,11,16,17],pillow:57,pip:[27,40,45,57,62],pixel:[3,10,16,20],pixels_float:3,pixels_str:3,place:[3,24],plain:[9,10,16,22],plan:24,pleas:[7,10,11,12,15,16,17,18,23,24,26,27,32,52],plot:[23,57],plotcurv:57,png:57,pnpairvalid:46,pod:[50,51,52],podip:52,podlist:52,point:44,polar:20,poll:64,pool3:41,pool:[4,11,17,21,62],pool_attr:[11,17],pool_bias_attr:[11,17],pool_layer_attr:11,pool_pad:[11,17],pool_siz:[3,10,11,16,17],pool_size_i:[10,16],pool_strid:[11,17],pool_typ:[10,11,16,17],pooling_lay:[11,27,60,62],pooling_typ:[10,16,27,35,60],poolingtyp:19,port:[45,46,47,51,52],port_num:46,ports_num:[47,52],ports_num_for_spars:[46,47,49,52],pos:[62,64],pose:24,posit:[3,9,10,16,20,60],positive_label:9,posix:50,possibl:23,potenti:44,power:10,practic:[8,10,16],pre:[10,11,16,17,20,23,56,64,65],pre_dictandmodel:56,precis:9,pred:[60,63],predetermin:[10,16],predic:[20,63],predicate_dict:63,predicate_dict_fil:63,predicate_slot:63,predict:[3,5,9,10,12,16,18,22,27,45,56,57,58,60,62,63,64],predict_fil:[46,47],predict_output_dir:[46,47,60],predict_sampl:5,predin:57,prefer:50,prefetch:41,prefix:24,pregrad:41,premodel:56,prepar:53,preprocess:[20,45,56,57,60,62,64,65],present:[23,40],prev_batch_st:[46,47],prevent:[12,18,23,24],previou:[10,11,16,17,24],price:20,principl:23,print:[5,7,15,22,23,28,33],printallstatu:44,printer:9,printstatu:44,prite:9,privat:40,prob:[9,22],probabilist:[10,16,56],probabl:[9,10,16,22],problem:[10,12,16,18,23],proc:30,proce:[20,26],proceed:[10,16,63],process2:36,process:[3,7,8,10,11,12,15,16,17,23,27,28,36,39,52,60,62,63],process_predict:60,process_test:8,process_train:8,processdata:[57,58],processor:44,prod:[30,40],produc:[11,17,20,24,26],productgraph:51,profil:44,proflier:44,prog:52,program:[20,23,26,44,52],progress:24,proj:[10,16],project:[10,11,16,17],promis:[10,11,16,17],prone:23,prop:63,propag:[12,18],properli:60,proposit:63,protect:41,proto:19,protobuf:27,provid:[8,10,16,20,23,27,28,36,46,60,62,63],prune:[10,16],ps_desir:24,pserver:[45,46,47,52],pserver_num_thread:[46,47],pseudo:23,psize:41,purpos:[24,44],push:52,push_back:41,put:[24,60],pwd:[30,40],py_paddl:[5,20],pydataprovid:27,pydataprovider2:[3,5,28,52,62],pydataproviderwrapp:8,pyramid:[10,16],pyramid_height:[10,16],python:[8,22,23,25,27,30,40,41,45,56,57,58,62,63,64,65],pythonpath:[27,57],pzo:64,queri:[10,16,65],question:[10,16,23],quick:51,quick_start:[51,52,53,60],quick_start_data:51,quickstart:51,rac:[10,16],rais:20,ramnath:64,ran:44,rand:[44,47,49,63],random:[7,10,15,16,20,26,28],rang:[10,16,20,26,52,60],rank:[10,16,23,58,60],rare:3,rate:[7,9,12,15,18,20,45,52,62],ratio:47,raw:[10,16],raw_meta:62,rdma_tcp:[46,47],reach:24,read:[3,20,22,23,24,26,28,58,60,62],read_from_realistic_imag:23,read_from_rng:23,read_mnist_imag:23,read_next_from_fil:27,read_ranking_model_data:23,reader:[22,33],reader_creator_bool:26,reader_creator_random_imag:[20,26],reader_creator_random_image_and_label:[20,26],readi:[24,51],readm:[40,62,64],real:26,real_process:3,realist:23,reason:[10,11,17,23,24,51],recal:9,receiv:[8,24],recognit:[10,16,58,64],recommend:[11,17,23,45,52,62],record:[62,63],recordio:23,recov:24,rectangular:[10,16],recurr:[36,37,63,64],recurrent_group:[11,17,36,38,39],recurrent_lay:11,recurrentgroup:9,reduc:[12,18],ref:62,refer:[7,8,10,11,12,15,16,17,18,24,29],referenc:[10,16],reference_cblas_root:29,refine_unknown_arg:52,regex:62,register_gpu_profil:44,register_lay:41,register_timer_info:44,registri:51,regress:9,regular:[7,12,15,57,60,64],rel:[11,17],relat:[8,24,62],relationship:20,releas:[32,50,63],reliabl:24,relu:[6,10,16],reluactiv:10,rememb:[10,16],remot:[7,15,40,45,47,49],remov:[20,40],reorgan:[10,16],repeat:10,repres:[10,12,16,20,60],represent:60,request:[24,51,65],requir:[9,10,16,23,24,45,62],res5_3_branch2c_bn:58,res5_3_branch2c_conv:58,res:63,research:[10,16,20,57],reserveoutput:41,reset:[10,16,24],reshap:26,reshape_s:[10,16],residu:58,resnet_101:58,resnet_152:58,resnet_50:58,resolv:[40,51],respons:[10,16,51],rest:[10,16],restart:[24,51],restartpolici:[51,52],resu:26,result:[3,9,10,14,16,22,44,60,65],result_fil:[9,39],ret_val:62,return_seq:[11,17],reus:26,reveal:23,revers:[10,11,16,17,38,39],review:[20,40,51,64],reviews_electronics_5:51,rewrit:65,rgb:[10,16],rgen:64,rho:[12,18],right:[10,16],rmsprop:[12,60],rmspropoptim:62,rnn:[10,11,16,17,38,39,46,64],rnn_bias_attr:39,rnn_layer_attr:39,rnn_out:39,rnn_state:36,rnn_state_:36,rnn_step:[10,16],rnn_use_batch:[46,47],rnnlm:20,robot:57,role:[20,23,63,64],roman:64,root:[12,18,19,30,45,51,52,56],root_dir:45,rot:[10,16],rotat:10,routin:62,routledg:64,row:[9,10,16,20],row_id:[10,16],rstrip:52,rtype:[10,16,62],run:[23,24,27,30,40,44,45,51,52,53,54,62],runinitfunct:[44,52],runtim:[3,27],s_fusion:62,s_id:62,same:[3,8,9,10,11,16,17,23,36,60],samping_id:[10,16],sampl:[3,9,20,60,62,64,65],sample_id:9,sample_num:9,santiago:64,save:[3,10,16,20,24,51,62,63,64,65],save_dir:[28,47,49,51,52,57,60,62,63,64,65],save_only_on:[46,47],saving_period:[46,47,52],saving_period_by_batch:[46,47,49],saw:3,scalar:[10,16],scale:[10,14,62],scalingproject:[10,16],scatter:[10,16],scheduler_factor:[7,15],scheme:[9,12],schmidhub:64,schwenk:65,scienc:64,score:[9,10,16,62],script:[20,42],seaplane_s_000978:57,search:[10,16,24,39,65],seat:65,second:[10,16,20,23,26,60,62],sed:64,see:[10,11,16,17,23,27,60],seed:[44,47],segment:9,sel_fc:[10,16],select:[10,16],selectiv:[10,16],selector:51,self:41,selfnorm:[10,16],semant:[20,23,63,64],semantic_role_label:39,semat:23,sen_len:63,send:24,sens:10,sent:[23,51],sent_id:39,sentanc:27,sentenc:[3,10,16,20,36,39,63],sentence_last_state1:36,sentence_last_state2:36,sentiment:[3,63,64],sentiment_data:64,sentiment_net:64,sentimental_provid:3,sentimental_train:3,separ:[9,60,62],seq:[10,16,36,62],seq_pool:[10,16,35],seq_text_print:9,seq_to_seq_data:[56,65],seq_typ:[20,62],seqlastin:36,seqtext_printer_evalu:39,seqtoseq:[10,16,27,39,56,65],seqtoseq_net:[10,16,39,56,65],sequel:3,sequenc:[3,9,10,11,14,16,17,19,20,27,33,36,38,60,62,64,65],sequence_conv_pool:60,sequence_layer_group:[10,16,36],sequence_nest_layer_group:[10,16,36],sequencegen:36,sequencesoftmax:6,sequencestartposit:[10,16],sequencetextprint:9,sequencetyp:[3,33],sequenti:[10,16,63],seri:[11,17,36,64],serial:22,server:[23,45,47,50,52],serverless:24,set:[3,7,9,10,11,15,16,17,20,22,23,24,27,28,36,39,44,45,51,56,57,58,60,62,63,64,65],set_active_typ:41,set_default_parameter_nam:[7,15],set_drop_r:41,set_input:[10,16],set_siz:41,set_typ:41,settotalbyteslimit:27,setup:[41,60],sever:[10,16],sgd:[12,18,22,23,24,28,33,45,46,64],shape:[10,16,22,33],shard:24,share:[10,16,63],shared_bia:[11,17],shared_bias:[10,16],ship:57,shold:64,shorten:[10,16],should:[9,10,12,16,20,22,23,26,38],should_be_fals:23,should_be_tru:23,should_shuffl:[3,36,63],show:[12,18,20,24,63,64,65],show_check_sparse_distribution_log:[46,47],show_layer_stat:[46,47],show_parameter_stats_period:[46,47,49,51,63,64,65],shown:[9,10,16,23],shuf:[27,62],shuffl:[20,27],side:[10,16,22],sigint:45,sigmoid:[6,10,16,17],sigmoidactiv:[10,11,36],similar:[10,16,26,62],similarli:[10,16],simpl:[9,10,11,14,16,17,20,22,52],simple_attent:39,simple_gru:60,simple_lstm:[10,16,60],simple_rnn:[10,16,39],simpli:[10,16,23],simplifi:23,sinc:[10,16,24,26],singl:[9,11,12,17,20,24],size:[3,9,10,11,12,16,17,18,20,24,26,27,28,33,36,39,41,57,58,60,62,64,65],size_a:[10,16],size_b:[10,16],size_t:41,skip:[26,28,40,58],sleep:52,slide:[10,12,16,18,20,24],slope:[10,16],slot:[62,63],slot_dim:62,slot_nam:62,slottyp:62,small:20,small_messag:[46,47],small_vgg:57,smaller:[10,16,24],smith:64,snap:51,sock_recv_buf_s:[46,47],sock_send_buf_s:[46,47],socket:52,softmax:[6,10,11,16,17,23,27,39,41,60],softmax_param:27,softmax_param_attr:[11,17],softmax_selfnorm_alpha:[10,16],softmaxactiv:[27,36,39,60],softrelu:6,solv:23,some:[7,10,12,15,16,20,22,23],some_python_class:25,somecppclass:25,somedata:22,somegotyp:25,someth:[10,16],sometim:[12,18,26],soon:24,sort:[10,16,20,52,62,64],sourc:[8,10,16,20,26,65],source_dict_dim:39,source_language_word:39,space:9,space_seperated_tokens_from_dictionary_according_to_seq:9,space_seperated_tokens_from_dictionary_according_to_sub_seq:9,spars:[7,10,12,15,16,18,20,27,41,45,47,49,52,60],sparse_binary_vector:[3,20,27,33,60],sparse_binary_vector_sequ:20,sparse_float_vector:[3,33],sparse_non_value_slot:20,sparse_upd:[7,15,27],sparse_value_slot:20,sparse_vector:[20,27],sparse_vector_sequ:20,sparseparam:41,sparseprefetchrowcpumatrix:41,spatial:[10,16],spec:[51,52],special:[10,16],specifi:[9,10,16,20,23,32,60],speech:[10,16],speed:[11,17],sphinx:[25,30,42],split:[3,10,16,36,45,60,62,63],split_count:52,spp:10,squar:[6,10,12,16,18,19],squarerootn:13,squarerootnpool:[10,16],srand:47,src:[52,65],src_backward:39,src_dict:[27,39],src_dict_path:27,src_embed:39,src_forward:39,src_id:39,src_root:5,src_word_id:39,srl:[20,63],ssd:16,ssh:[30,45],sshd:30,sstabl:23,stabl:50,stacked_lstm_net:64,stacked_num:64,stackexchang:[10,16],stage:40,stake:65,standard:[7,15],stanford:[20,51],stanh:6,start:[10,16,22,24,27,47,51,52],start_paddl:52,start_pass:[46,47],start_pserv:[46,47],startpaddl:52,startup:24,stat:[44,47,63,64,65],state:[10,11,16,17,24,38,51],state_act:[10,11,16,17,36],statfulset:52,staticinput:[10,16,38,39],statist:[10,16],statset:44,statu:[9,40,44,51,52],status:51,std:[41,47],stderr:45,stdout:45,step:[10,11,12,16,17,19,24,36,38,39],stepout:36,stochast:[12,18,24],stock:64,stop:[10,16],storag:50,store:[9,10,16,20,24,60,62],str:[22,49,52],strategi:[19,24,47,63],street:[10,16,63],strict:26,stride:[10,16],stride_i:[10,16],stride_x:[10,16],string:[3,8,9,10,16,41,47],strip:[27,36,60,62,63],structur:[20,60],stub:[10,16],stuff:40,stun:3,style:[10,16],sub:[9,10,16,20,23],sub_sequ:[3,33],subgradi:[12,18],subnet:23,subobjectpath:51,subseq:[35,38],subsequenceinput:[10,16,36],succeed:51,success:51,successfulcr:51,sudo:[32,57],suffic:26,suggest:[10,16],sum:[9,10,12,13,16,18],sum_:[10,16],sum_to_one_norm:10,sumpool:[10,16,27],support:[7,9,10,12,15,16,19,20,24,26,36],sure:[32,64],swap_channel:58,swig:25,swig_paddl:[5,20],symbol:[10,16],symlink:40,sync:24,syncflag:41,synchron:[12,18,24],syntax:26,sys:58,system:[24,27,64],t2b:56,t_i:[10,16],tab:60,tabl:[10,16],tableproject:[10,16],tag:[9,20,30],tagtyp:9,tainer_id:52,take:[3,9,10,11,16,17,23],tanh:[6,10,11,16,17,41],tanhactiv:[10,11,36,39],target:[10,16,20,22,65],target_dict_dim:39,target_language_word:39,targetinlink:[10,16,36],task:[9,10,16,63],tbd:[25,36,42],tconf:64,tcp:47,tear:44,technic:24,tee:[51,57,62,63,64,65],tell:24,tellig:64,templat:[51,52],tempor:[10,16],tensor:10,term:[10,11,16,17],termin:51,tesh:63,test100:20,test10:20,test:[2,3,8,9,10,16,20,22,23,26,40,41,44,45,47,49,57,58,60,62,63,64,65],test_all_data_in_one_period:[51,57,62,63,64],test_compar:27,test_comparespars:27,test_comparetwonet:27,test_comparetwoopt:27,test_config_pars:27,test_data:[5,65],test_fcgrad:41,test_gpuprofil:44,test_layergrad:41,test_list:[3,8,27,28,57,60],test_networkcompar:27,test_part_000:64,test_pass:[46,47,49,65],test_period:[46,47,49],test_predict:27,test_pydataprovid:27,test_pydataprovider2:27,test_pydataproviderwrapp:27,test_ratio:62,test_recurrent_machine_gener:27,test_recurrentgradientmachin:[27,36],test_swig_api:27,test_train:27,test_traineronepass:27,test_wait:[46,47],testa:23,testb:23,testbilinearfwdbwd:44,testconfig:41,tester:[62,65],testfcgrad:41,testfclay:41,testlayergrad:41,testq:23,testresult:22,testutil:41,text:[3,9,11,17,20,23,56,60,64,65],text_conv:60,text_conv_pool:62,text_fil:[20,64],tflop:44,tgz:20,than:[7,9,10,11,12,15,16,17,24,27],thei:[23,24,44],them:[11,17,23,24,26,44,60,62],therein:[10,16],thi:[3,7,8,9,10,11,12,15,16,17,18,20,22,23,24,26,44,60,62,64],thing:3,think:23,third:[10,16,24],those:[24,63],thread:44,thread_local_rand_use_global_se:[46,47],threadid:49,threadloc:44,three:[9,10,12,16,24,26,58],threshold:[7,9,12,15,24,47],through:[10,16,24],throughput:44,thu:[10,16],tier:51,time:[10,11,16,17,19,20,23,24,26,36,44,47,51,52,64],timelin:[10,16],timeout:24,timer:44,timestamp:[10,16],timestep:[10,16],titl:[20,52,62],tmp:3,to_your_paddle_clone_path:42,todo:[9,11,17,20,24],toend:[10,16],togeth:[10,11,16,17,20,22],token:[9,10,16,23,39,64],too:20,tool:[42,52],top:[9,58],top_k:9,topolog:[20,23],topolopi:22,toronto:[20,57],total:[9,22,24,26,44,51,65],total_pass:26,touch:64,tourist:65,track:[10,24,40],tractabl:[10,16],tradit:[10,16],trail:20,train100:20,train10:20,train:[2,3,7,8,9,10,12,15,16,18,20,27,28,32,33,45,47,49,51,52,53,54,56,57,58,60,62,63,64,65],train_arg:52,train_args_dict:52,train_args_list:52,train_conf:[56,65],train_config_dir:52,train_data:65,train_i:33,train_list:[3,8,27,28,57,58,60],train_part_000:64,train_read:33,train_x:33,trainabl:[10,16],trainer:[3,5,23,28,33,41,47,49,52,62,63,64,65],trainer_config:[2,3,5,28,45,51,52,60,62,64],trainer_config_help:[3,6,7,8,9,10,11,12,13,27,28,41,57,62],trainer_count:[27,46,47,49,51,52,62,63,64,65],trainer_id:[47,52],trainerconfighelp:27,trainerid:52,trainerintern:[60,62,65],tran:[10,41],transact:[24,64],transform:[10,16],transform_param_attr:[11,17],translat:[10,11,16,17,56,65],transpos:[10,16],transposedfullmatrixproject:[10,16],travel:[3,11],travi:40,treat:[10,16],tree:[10,16,52],trg:65,trg_dict:39,trg_dict_path:39,trg_embed:39,trg_id:39,trg_ids_next:39,tricki:25,trn:60,truck:57,true_imag:26,true_label:26,true_read:26,truth:[9,10,16,65],tst:60,tune:[7,15,46],tupl:[8,10,11,16,20,22,26],ture:[10,16],turn:[10,16,26,38],tutori:[51,52,53,54,56,65],tweet:64,twitter:64,two:[10,11,16,17,23,26,44,60],txt:[3,41,45,50,60,62,64],type:[3,9,10,11,12,16,17,19,20,23,24,25,26,28,33,36,41,49,51,58,60,62,63],type_nam:[10,62],typedef:25,typic:9,ubyt:26,uci:20,ufldl:[10,16],uid:51,uint64:25,uint64_t:25,unconstrain:64,undeterminist:44,uniform:[7,10,15,16,20,26],uninstal:27,uniqu:[23,24],unique_ptr:41,unit:[10,11,16,17],unittest:27,univ:65,unix:45,unk:[56,65],unk_idx:[60,63],unknown:[10,16],unseg:[10,16],unsup:64,unsupbow:64,until:[24,52],untrack:40,unus:62,updat:[7,10,12,15,16,24,40,45,49],update_equ:[22,33],updatecallback:41,upgrad:27,upload:24,upstream:40,url:[20,64],urls_neg:64,urls_po:64,urls_unsup:64,usag:[9,10,11,16,17,20,22,52,62],use:[7,8,9,10,11,12,15,16,17,19,20,22,23,40,44,52,58,60,62,63,64,65],use_global_stat:[10,16],use_gpu:[5,27,33,46,47,49,51,52,57,58,60,62,63,64,65],use_jpeg:57,use_old_updat:[46,47],use_seq:[28,62],use_seq_or_not:62,used:[3,9,10,11,12,16,17,18,19,20,22,23,24,26,44,60,62,64],useful:[10,11,16,17],usegpu:41,user:[7,9,10,11,15,16,17,20,22,23,26,50,60,62],user_featur:62,user_head:62,user_id:[52,62],user_info:20,user_meta:62,user_nam:62,userinfo:20,usernam:40,uses:24,using:[7,8,10,15,16,20,23,24,26,32,63],usr:[27,29,30,32,45,47,52],usrdict:56,usrmodel:56,usual:[10,16,20,22,44],utc:61,util:[44,52,57,62],v28:[10,16],valid:26,valu:[3,5,7,9,10,12,15,16,18,19,20,22,24,41,49,52,58,60,63],value1:47,value2:47,value_rang:20,vanilla:39,variabl:[10,16,20,23,51],varianc:[10,16],vector:[10,11,16,17,20,23,60,62],verb:20,veri:[10,16,19,57],version:[10,11,16,17,30,32,44,46,47],versu:23,vertic:[10,16],vgg:[11,17,57],vgg_16_cifar:57,via:[24,26,32,40],view:[10,16],vision:57,visipedia:57,visual:[10,16],vocab:64,volum:[30,50,51,52],volumemount:[51,52],vutbr:20,wai:[10,11,16,17,23,65],wait:[12,18,24,52],wall:63,want:[3,10,11,16,17,23,26],warn:[10,16,27,52],warp:[10,16],watch:24,wbia:58,wei:[63,64],weight:[9,10,11,12,16,17,18,41,58],weight_act:[11,17],weightlist:41,weights_:41,weights_t:41,wether:[10,16],what:[7,10,11,12,15,16,17,18,40,60],when:[3,7,9,10,12,15,16,20,22,24,44],where:[10,11,12,16,17,18,23,24],whether:[9,10,11,16,17,26,64],which:[9,10,11,12,16,17,18,20,23,24,26,60,62],whole:[3,9,20,25],whole_cont:62,whose:[10,16,20,24],why:[11,17,44],width:[9,10,16,20,25,26,41,65],wiki:[10,16],wikipedia:[10,16,20],wilder:3,window:[10,16,19,20,50],wise:[10,16],with_avx:[29,30,32,40],with_doc:29,with_doubl:[29,32,41],with_dso:29,with_gpu:[29,30,32,40],with_predict_sdk:32,with_profil:44,with_python:[29,32],with_rdma:[29,32],with_style_check:[29,40],with_swig_pi:29,with_test:[29,30,40],with_tim:[29,32,44],within:[10,16],without:[9,10,16,26],wmt14:65,wmt14_data:65,wmt14_model:65,wmt_shrinked_data:20,woboq:30,won:36,wonder:3,word2vec:27,word:[3,9,10,16,20,27,36,38,60,63],word_dict:[36,60,63],word_dim:[36,60],word_id:[3,27],word_idx:20,word_slot:63,word_vector:60,word_vector_dim:[39,56],words_freq_sort:20,work:[20,23,24,26,30,36,40,51,52],workspac:45,would:[22,26,63],wrapper:[11,17,44],write:[20,23,24,26,63],writelin:28,writer:23,wrong:26,wsj:63,wuyi:50,www:[10,16,20,57,65],xarg:[27,30,41],xgbe0:47,xgbe1:47,xiaojun:64,xrang:[26,28,33,41],xxbow:64,xxx:[23,58,65],y_i:[10,16],y_predict:33,yaml:[51,52],yann:20,yapf:40,yeild:22,yield:[3,20,23,26,27,28,33,36,39,60,62,63],you:[3,7,10,11,12,15,16,17,32,58,64],your:[10,16,23,27],your_param_nam:27,your_repo:52,yuyang18:[11,17,20],zachari:64,zeng:64,zero:[7,10,12,15,16,18,20,24,47],zhou:[63,64],zip:[20,52,61],zoo:56},titles:["\u5173\u4e8ePaddlePaddle","API","DataProvider\u7684\u4ecb\u7ecd","PyDataProvider2\u7684\u4f7f\u7528","API\u4e2d\u6587\u624b\u518c","\u57fa\u4e8ePython\u7684\u9884\u6d4b","Activations","Parameter Attributes","DataSources","Evaluators","Layers","Networks","Optimizers","Poolings","Activation","Parameter Attribute","Layers","Networks","Optimizer","Pooling","Data Reader Interface and DataSets","Model Configuration","Training and Inference","PaddlePaddle Design Doc","Design Doc: Distributed Training","Paddle\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0","Python Data Reader Design Doc","FAQ","\u7ecf\u5178\u7684\u7ebf\u6027\u56de\u5f52\u4efb\u52a1","PaddlePaddle\u7684\u7f16\u8bd1\u9009\u9879","PaddlePaddle\u7684Docker\u5bb9\u5668\u4f7f\u7528\u65b9\u5f0f","\u5b89\u88c5\u4e0e\u7f16\u8bd1","Ubuntu\u90e8\u7f72PaddlePaddle","\u57fa\u672c\u4f7f\u7528\u6982\u5ff5","\u65b0\u624b\u5165\u95e8","\u652f\u6301\u53cc\u5c42\u5e8f\u5217\u4f5c\u4e3a\u8f93\u5165\u7684Layer","\u5355\u53cc\u5c42RNN API\u5bf9\u6bd4\u4ecb\u7ecd","RNN\u76f8\u5173\u6a21\u578b","Recurrent Group\u6559\u7a0b","RNN\u914d\u7f6e","\u5982\u4f55\u8d21\u732e\u4ee3\u7801","\u5b9e\u73b0\u65b0\u7684\u7f51\u7edc\u5c42","\u5982\u4f55\u8d21\u732e/\u4fee\u6539\u6587\u6863","\u8fdb\u9636\u6307\u5357","GPU\u6027\u80fd\u5206\u6790\u4e0e\u8c03\u4f18","\u8fd0\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3","\u53c2\u6570\u6982\u8ff0","\u7ec6\u8282\u63cf\u8ff0","\u8bbe\u7f6e\u547d\u4ee4\u884c\u53c2\u6570","\u4f7f\u7528\u6848\u4f8b","Kubernetes \u7b80\u4ecb","Kubernetes\u5355\u673a\u8bad\u7ec3","Kubernetes\u5206\u5e03\u5f0f\u8bad\u7ec3","<no title>","<no title>","PaddlePaddle \u6587\u6863","\u4e2d\u6587\u8bcd\u5411\u91cf\u6a21\u578b\u7684\u4f7f\u7528","\u56fe\u50cf\u5206\u7c7b\u6559\u7a0b","Model Zoo - ImageNet","\u5b8c\u6574\u6559\u7a0b","\u5feb\u901f\u5165\u95e8\u6559\u7a0b","MovieLens\u6570\u636e\u96c6","MovieLens\u6570\u636e\u96c6\u8bc4\u5206\u56de\u5f52\u6a21\u578b","\u8bed\u4e49\u89d2\u8272\u6807\u6ce8\u6559\u7a0b","\u60c5\u611f\u5206\u6790\u6559\u7a0b","\u6587\u672c\u751f\u6210\u6559\u7a0b"],titleterms:{"\u4e00\u4e9b\u7ec6\u8282\u7684\u8865\u5145":52,"\u4e0b\u8f7d\u4e0e\u89e3\u538b\u7f29":65,"\u4e0b\u8f7d\u548c\u6570\u636e\u62bd\u53d6":56,"\u4e0b\u8f7d\u5e76\u89e3\u538b\u6570\u636e\u96c6":62,"\u4e0b\u8f7d\u6570\u636e":51,"\u4e0d\u4f7f\u7528":25,"\u4e0d\u4f7f\u7528swig\u8fd9\u79cd\u4ee3\u7801\u751f\u6210\u5668":25,"\u4e0d\u5bfc\u51fapaddle\u5185\u90e8\u7684\u7ed3\u6784\u4f53":25,"\u4e0d\u5f15\u7528\u5176\u4ed6\u52a8\u6001\u5e93":25,"\u4e2d\u6587\u5b57\u5178":56,"\u4e2d\u6587\u77ed\u8bed\u6539\u5199\u7684\u4f8b\u5b50":56,"\u4e2d\u6587\u8bcd\u5411\u91cf\u6a21\u578b\u7684\u4f7f\u7528":56,"\u4e2d\u6587\u8bcd\u5411\u91cf\u7684\u9884\u8bad\u7ec3\u6a21\u578b":56,"\u4e3a\u4ec0\u4e48\u9700\u8981\u6027\u80fd\u5206\u6790":44,"\u4ec0\u4e48\u662f\u6027\u80fd\u5206\u6790":44,"\u4ec5\u4ec5\u4f7f\u7528void":25,"\u4ecb\u7ecd":[56,58],"\u4ee3\u7801\u8981\u6c42":40,"\u4efb\u52a1\u7b80\u4ecb":28,"\u4f18\u5316\u7b97\u6cd5":60,"\u4f7f\u7528":[40,51],"\u4f7f\u7528\u52a8\u6001\u5e93\u6765\u5206\u53d1paddl":25,"\u4f7f\u7528\u6848\u4f8b":49,"\u4f7f\u7528\u6982\u8ff0":60,"\u4f7f\u7528\u6a21\u578b\u521d\u59cb\u5316\u7f51\u7edc":49,"\u4f7f\u7528\u73af\u5883\u53d8\u91cf":52,"\u4f7f\u7528\u7528\u6237\u6307\u5b9a\u7684\u8bcd\u5411\u91cf\u5b57\u5178":56,"\u4f7f\u7528\u8bf4\u660e":43,"\u4f7f\u7528docker\u6784\u5efapaddlepaddle\u7684\u6587\u6863":42,"\u4f7f\u7528paddlepaddle\u751f\u6210\u6a21\u578b":65,"\u4f7f\u7528paddlepaddle\u8bad\u7ec3\u6a21\u578b":65,"\u4fdd\u6301\u672c\u5730\u4ed3\u5e93\u6700\u65b0":40,"\u4fee\u6539\u542f\u52a8\u811a\u672c":51,"\u4fee\u6539\u6587\u6863":42,"\u514b\u9686":40,"\u5173\u4e8epaddlepaddl":0,"\u5185\u5b58\u4e0d\u591f\u7528\u7684\u60c5\u51b5":3,"\u5185\u7f6e\u5b9a\u65f6\u5668":44,"\u5199\u68af\u5ea6\u68c0\u67e5\u5355\u5143\u6d4b\u8bd5":41,"\u51c6\u5907\u5de5\u4f5c\u7a7a\u95f4":45,"\u51c6\u5907\u5e8f\u5217\u6570\u636e":39,"\u51c6\u5907\u6570\u636e":[28,62],"\u51c6\u5907\u8bad\u7ec3\u6570\u636e":52,"\u51c6\u5907\u96c6\u7fa4\u4f5c\u4e1a\u914d\u7f6e":45,"\u51cf\u5c11\u6570\u636e\u8f7d\u5165\u7684\u8017\u65f6":27,"\u51cf\u5c11dataprovider\u7f13\u51b2\u6c60\u5185\u5b58":27,"\u5206\u5272\u8bad\u7ec3":62,"\u521b\u5efa\u672c\u5730\u5206\u652f":40,"\u521b\u5efajob":52,"\u521b\u5efapaddl":51,"\u5220\u9664\u672c\u5730\u5206\u652f":40,"\u5220\u9664\u8fdc\u7a0b\u5206\u652f":40,"\u5229\u7528\u66f4\u591a\u7684\u8ba1\u7b97\u8d44\u6e90":27,"\u5230\u8fdc\u7a0b\u4ed3\u5e93":40,"\u5236\u4f5c\u955c\u50cf":52,"\u5236\u4f5cdocker\u955c\u50cf":51,"\u524d\u63d0\u6761\u4ef6":45,"\u52a0\u8f7dpaddlepaddl":33,"\u52a0\u901f\u8bad\u7ec3\u901f\u5ea6":27,"\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165\u4efb\u4f55\u5176\u4ed6\u8bed\u8a00\u7684\u89e3\u91ca\u5668":25,"\u5355\u5143\u6d4b\u8bd5":47,"\u5355\u53cc\u5c42rnn":36,"\u5377\u79ef\u6a21\u578b":60,"\u5377\u79ef\u795e\u7ecf\u7f51\u7edc":57,"\u539f\u56e0":25,"\u539f\u56e0\u5217\u8868":25,"\u53c2\u6570\u4fe1\u606f":58,"\u53c2\u6570\u5185\u5b58":27,"\u53c2\u6570\u670d\u52a1\u5668\u548c\u5206\u5e03\u5f0f\u901a\u4fe1":47,"\u53c2\u6570\u6982\u8ff0":46,"\u53c2\u6570\u8bfb\u53d6":58,"\u53c2\u8003":3,"\u53c2\u8003\u6587\u6863":64,"\u53c2\u8003\u8d44\u6599":44,"\u53cc\u5411lstm":64,"\u53cc\u5c42rnn":36,"\u53cc\u5c42rnn\u4ecb\u7ecd":38,"\u53cc\u5c42rnn\u7684\u4f7f\u7528":38,"\u53ef\u80fd\u7684\u5185\u5b58\u6cc4\u9732\u95ee\u9898":3,"\u53ef\u80fd\u9047\u5230\u7684\u95ee\u9898":32,"\u53ef\u9009\u529f\u80fd":56,"\u5411\u7cfb\u7edf\u4f20\u9001\u6570\u636e":60,"\u5411\u91cf":47,"\u542f\u52a8\u4efb\u52a1":52,"\u542f\u52a8\u96c6\u7fa4\u4f5c\u4e1a":45,"\u547d\u4ee4\u884c\u53c2\u6570":60,"\u548c":35,"\u56fe\u50cf\u5206\u7c7b\u6559\u7a0b":57,"\u5728\u4e0d\u540c\u8bbe\u5907\u4e0a\u6307\u5b9a\u5c42":49,"\u5728paddlepaddle\u5e73\u53f0\u8bad\u7ec3\u6a21\u578b":56,"\u57fa\u4e8epython\u7684\u9884\u6d4b":5,"\u57fa\u672c\u4f7f\u7528\u6982\u5ff5":33,"\u57fa\u672c\u539f\u7406":38,"\u57fa\u672c\u8981\u6c42":25,"\u5982\u4f55\u4e66\u5199paddlepaddle\u7684\u6587\u6863":42,"\u5982\u4f55\u5171\u4eab\u53c2\u6570":27,"\u5982\u4f55\u51cf\u5c11\u5185\u5b58\u5360\u7528":27,"\u5982\u4f55\u521d\u59cb\u5316\u53c2\u6570":27,"\u5982\u4f55\u52a0\u901fpaddlepaddle\u7684\u8bad\u7ec3\u901f\u5ea6":27,"\u5982\u4f55\u6307\u5b9agpu\u8bbe\u5907":27,"\u5982\u4f55\u66f4\u65b0www":42,"\u5982\u4f55\u6784\u5efapaddlepaddle\u7684\u6587\u6863":42,"\u5982\u4f55\u8d21\u732e":42,"\u5982\u4f55\u8d21\u732e\u4ee3\u7801":40,"\u5982\u4f55\u8fdb\u884c\u6027\u80fd\u5206\u6790":44,"\u5982\u4f55\u9009\u62e9sgd\u7b97\u6cd5\u7684\u5b66\u4e60\u7387":27,"\u5b50\u5e8f\u5217\u95f4\u65e0memori":36,"\u5b50\u5e8f\u5217\u95f4\u6709memori":36,"\u5b57\u6bb5\u914d\u7f6e\u6587\u4ef6":62,"\u5b89\u88c5":[32,60],"\u5b89\u88c5\u4e0e\u7f16\u8bd1":31,"\u5b89\u88c5\u6d41\u7a0b":31,"\u5b89\u88c5kubectl":50,"\u5b8c\u6574\u6559\u7a0b":59,"\u5b9e\u73b0\u65b0\u7684\u7f51\u7edc\u5c42":41,"\u5b9e\u73b0c":41,"\u5b9e\u73b0python\u5c01\u88c5":41,"\u5bfc\u51fac":25,"\u5c06\u547d\u4ee4\u53c2\u6570\u4f20\u7ed9\u7f51\u7edc\u914d\u7f6e":49,"\u5c0f\u7ed3":3,"\u5de5\u5177":44,"\u5e38\u7528\u6a21\u578b":59,"\u5e76\u5b8c\u6210":40,"\u5efa\u7acb":40,"\u5f00\u53d1\u6807\u51c6":43,"\u5f00\u59cb\u5f00\u53d1":40,"\u5f02\u6b65\u968f\u673a\u68af\u5ea6\u4e0b\u964d":47,"\u5f15\u7528":63,"\u5feb\u901f\u5165\u95e8\u6559\u7a0b":60,"\u6027\u80fd\u4f18\u5316":43,"\u6027\u80fd\u5206\u6790\u5c0f\u6280\u5de7":44,"\u6027\u80fd\u5206\u6790\u5de5\u5177\u4ecb\u7ecd":44,"\u6027\u80fd\u8c03\u4f18":47,"\u603b\u4f53\u6548\u679c\u603b\u7ed3":60,"\u60c5\u611f\u5206\u6790\u6559\u7a0b":64,"\u6216\u8005\u662f":27,"\u627e\u5230\u7684pythonlibs\u548cpythoninterp\u7248\u672c\u4e0d\u4e00\u81f4":27,"\u6307\u9488\u4f5c\u4e3a\u7c7b\u578b\u7684\u53e5\u67c4":25,"\u63a5\u53e3":58,"\u63a8\u5bfc\u65b9\u7a0b":41,"\u63d0\u4ea4":40,"\u63d0\u4ea4\u955c\u50cf":51,"\u63d0\u53d6\u7535\u5f71\u6216\u7528\u6237\u7684\u7279\u5f81\u5e76\u751f\u6210python\u5bf9\u8c61":62,"\u642d\u5efa\u795e\u7ecf\u7f51\u7edc":33,"\u652f\u6301\u53cc\u5c42\u5e8f\u5217\u4f5c\u4e3a\u8f93\u5165\u7684layer":35,"\u6570\u636e\u51c6\u5907":[57,62,65],"\u6570\u636e\u63cf\u8ff0":63,"\u6570\u636e\u63d0\u4f9b":63,"\u6570\u636e\u63d0\u4f9b\u811a\u672c":62,"\u6570\u636e\u652f\u6301":47,"\u6570\u636e\u683c\u5f0f\u51c6\u5907":60,"\u6570\u636e\u7684\u51c6\u5907\u548c\u9884\u5904\u7406":56,"\u6570\u636e\u96c6\u7279\u5f81":61,"\u6570\u636e\u9884\u5904\u7406":65,"\u6570\u6910\u4ecb\u7ecd":64,"\u6570\u6910\u51c6\u5907":64,"\u6574\u4f53\u65b9\u6848":52,"\u6587\u672c\u751f\u6210":65,"\u6587\u672c\u751f\u6210\u6559\u7a0b":65,"\u6587\u6863":[30,55],"\u65b0\u624b\u5165\u95e8":34,"\u65f6\u5e8f\u6a21\u578b":60,"\u65f6\u5e8f\u6a21\u578b\u7684\u4f7f\u7528\u573a\u666f":3,"\u65f6\u95f4\u5e8f\u5217":36,"\u65f6\u95f4\u6b65":36,"\u672c\u5730\u6d4b\u8bd5":49,"\u672c\u5730\u8bad\u7ec3":49,"\u6784\u5efa\u548c\u6d4b\u8bd5":40,"\u67e5\u770b\u8bad\u7ec3\u7ed3\u679c":51,"\u67e5\u770b\u8f93\u51fa":52,"\u6837\u4f8b\u6570\u636e":3,"\u6848\u4f8b\u4e00":49,"\u6848\u4f8b\u4e8c":49,"\u68c0\u67e5\u6a21\u578b\u8f93\u51fa":45,"\u68c0\u67e5\u96c6\u7fa4\u8bad\u7ec3\u7ed3\u679c":45,"\u6982\u8ff0":[35,38],"\u6a21\u578b":58,"\u6a21\u578b\u4e0b\u8f7d":58,"\u6a21\u578b\u68c0\u9a8c":28,"\u6a21\u578b\u7f51\u7edc\u7ed3\u6784":60,"\u6a21\u578b\u8bad\u7ec3":[57,65],"\u6a21\u578b\u8bc4\u4f30\u548c\u9884\u6d4b":62,"\u6a21\u578b\u914d\u7f6e":[36,43],"\u6a21\u578b\u914d\u7f6e\u7684\u6a21\u578b\u914d\u7f6e":36,"\u6ce8\u610f\u4e8b\u9879":3,"\u6d4b\u8bd5":[47,63],"\u6d4b\u8bd5\u6587\u4ef6":62,"\u6d4b\u8bd5\u6a21\u578b":64,"\u7279\u5f81":63,"\u7279\u5f81\u63d0\u53d6":58,"\u751f\u6210\u5e8f\u5217":39,"\u751f\u6210\u6a21\u578b\u7684\u547d\u4ee4\u4e0e\u7ed3\u679c":65,"\u751f\u6210\u6d41\u7a0b\u7684\u4f7f\u7528\u65b9\u6cd5":38,"\u7528\u6237\u6587\u4ef6\u63cf\u8ff0":61,"\u7528\u6237\u81ea\u5b9a\u4e49\u6570\u636e\u96c6":65,"\u7528\u6237\u81ea\u5b9a\u4e49\u6570\u6910\u9884\u5904\u7406":64,"\u7535\u5f71\u6587\u4ef6\u63cf\u8ff0":61,"\u76f4\u63a5\u6784\u5efapaddlepaddle\u7684\u6587\u6863":42,"\u76f8\u5173\u6982\u5ff5":38,"\u77e9\u9635":47,"\u793a\u4f8b1":36,"\u793a\u4f8b2":36,"\u793a\u4f8b3":36,"\u793a\u4f8b4":36,"\u795e\u7ecf\u5143\u6fc0\u6d3b\u5185\u5b58":27,"\u795e\u7ecf\u7f51\u7edc\u7ed3\u6784\u914d\u7f6e":62,"\u795e\u7ecf\u7f51\u7edc\u914d\u7f6e":63,"\u7a00\u758f\u8bad\u7ec3":49,"\u7b26\u53f7":25,"\u7b80\u4ecb":[50,65],"\u7b80\u5355\u5b9e\u73b0":25,"\u7b80\u5355\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":39,"\u7c7b":[25,41],"\u7ebf\u6027\u56de\u5f52\u5b8c\u6574\u793a\u4f8b":33,"\u7ec3\u4e60":57,"\u7ec6\u8282\u63a2\u7a76":57,"\u7ec6\u8282\u63cf\u8ff0":47,"\u7ec8\u6b62\u96c6\u7fa4\u4f5c\u4e1a":45,"\u7ecf\u5178\u7684\u7ebf\u6027\u56de\u5f52\u4efb\u52a1":28,"\u7f16\u5199yaml\u6587\u4ef6":51,"\u7f16\u8bd1\u6d41\u7a0b":31,"\u7f16\u8bd1\u9009\u9879\u7684\u8bbe\u7f6e":29,"\u7f51\u7edc\u53ef\u89c6\u5316":58,"\u7f51\u7edc\u914d\u7f6e\u4e2d\u7684\u8c03\u7528":3,"\u800c\u662f\u624b\u5199\u591a\u8bed\u8a00\u7ed1\u5b9a":25,"\u80cc\u666f":25,"\u81ea\u7136\u8bed\u8a00\u5904\u7406":47,"\u81f4\u8c22":0,"\u89c2\u6d4b\u8bcd\u5411\u91cf":56,"\u8bad\u7ec3":[47,62,63],"\u8bad\u7ec3\u5668\u914d\u7f6e\u6587\u4ef6":62,"\u8bad\u7ec3\u56e0\u6b64\u9000\u51fa\u600e\u4e48\u529e":27,"\u8bad\u7ec3\u6a21\u578b":[28,33,60,64],"\u8bad\u7ec3\u6a21\u578b\u7684\u547d\u4ee4\u4e0e\u7ed3\u679c":65,"\u8bad\u7ec3\u6d41\u7a0b\u7684\u4f7f\u7528\u65b9\u6cd5":38,"\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u51fa\u73b0":27,"\u8bbe\u7f6e\u547d\u4ee4\u884c\u53c2\u6570":48,"\u8bc4\u5206\u6587\u4ef6\u63cf\u8ff0":61,"\u8bcd\u5411\u91cf\u6a21\u578b":60,"\u8bcd\u5411\u91cf\u6a21\u578b\u7684\u4fee\u6b63":56,"\u8bcd\u6c47\u8868":36,"\u8be6\u7ec6\u6559\u7a0b":44,"\u8bed\u4e49\u89d2\u8272\u6807\u6ce8\u6559\u7a0b":63,"\u8bfb\u53d6\u53cc\u5c42\u5e8f\u5217\u6570\u636e":36,"\u8f93\u5165":38,"\u8f93\u5165\u4e0d\u7b49\u957f":36,"\u8f93\u5165\u793a\u4f8b":38,"\u8f93\u51fa":38,"\u8f93\u51fa\u65e5\u5fd7":60,"\u8fd0\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3":45,"\u8fd0\u884c\u5bb9\u5668":51,"\u8fd0\u884cdocker":27,"\u8fd0\u884cpaddlepaddl":30,"\u8fd9\u4e2a\u52a8\u6001\u5e93\u4f7f\u7528c99\u6807\u51c6\u7684\u5934\u6587\u4ef6\u5bfc\u51fa\u4e00\u4e9b\u51fd\u6570":25,"\u8fdb\u884c\u8bad\u7ec3":51,"\u8fdb\u9636\u6307\u5357":43,"\u9009\u62e9\u5b58\u50a8\u65b9\u6848":50,"\u901a\u7528":47,"\u901a\u8fc7docker\u5bb9\u5668\u5f00\u53d1paddlepaddl":30,"\u903b\u8f91\u56de\u5f52\u6a21\u578b":60,"\u9047\u5230":27,"\u90e8\u7f72kubernetes\u96c6\u7fa4":50,"\u914d\u7f6e\u4e2d\u7684\u6570\u636e\u52a0\u8f7d\u5b9a\u4e49":60,"\u914d\u7f6e\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u67b6\u6784":39,"\u914d\u7f6e\u7f51\u7edc":33,"\u914d\u7f6ekubectl":50,"\u914d\u7f6ekubectl\u8bbf\u95ee\u4f60\u7684kubernetes\u96c6\u7fa4":50,"\u94a9\u5b50":40,"\u9644\u5f55":60,"\u968f\u673a\u6570":47,"\u96c6\u7fa4\u8bad\u7ec3":49,"\u975e\u6cd5\u6307\u4ee4":27,"\u9884\u5904\u7406":57,"\u9884\u5904\u7406\u547d\u4ee4\u548c\u7ed3\u679c":65,"\u9884\u5904\u7406\u5de5\u4f5c\u6d41\u7a0b":65,"\u9884\u6d4b":[57,58,60,63,64],"\u9884\u6d4b\u6d41\u7a0b":5,"\u9884\u6d4bdemo":5,"\u9884\u8bad\u7ec3\u7684\u6a21\u578b":65,"api\u4e2d\u6587\u624b\u518c":4,"api\u5bf9\u6bd4\u4ecb\u7ecd":36,"beam_search\u7684\u751f\u6210":36,"blas\u8def\u5f84\u76f8\u5173\u7684\u7f16\u8bd1\u9009\u9879":29,"bleu\u8bc4\u4f30":65,"bool\u578b\u7684\u7f16\u8bd1\u9009\u9879":29,"cmake\u6e90\u7801\u7f16\u8bd1":27,"cudnn\u7684\u7f16\u8bd1\u9009\u9879":29,"dataprovider\u7684\u4ecb\u7ecd":2,"dataprovider\u7684\u4f7f\u7528":3,"float":27,"gpu\u548ccpu\u6df7\u5408\u4f7f\u7528":49,"gpu\u6027\u80fd\u5206\u6790\u4e0e\u8c03\u4f18":44,"gpu\u955c\u50cf\u51fa\u73b0":27,"group\u6559\u7a0b":38,"kubernetes\u5206\u5e03\u5f0f\u8bad\u7ec3":52,"kubernetes\u5355\u673a\u8bad\u7ec3":51,"meta\u6587\u4ef6":62,"meta\u914d\u7f6e\u6587\u4ef6":62,"mnist\u7684\u4f7f\u7528\u573a\u666f":3,"movielens\u6570\u636e\u96c6":61,"movielens\u6570\u636e\u96c6\u8bc4\u5206\u56de\u5f52\u6a21\u578b":62,"org\u6587\u6863":42,"paddle\u52a8\u6001\u5e93\u4e2d":25,"paddle\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0":25,"paddlepaddle\u53d1\u5e03\u7684docker\u955c\u50cf\u4f7f\u7528\u8bf4\u660e":30,"paddlepaddle\u7684\u7f16\u8bd1\u9009\u9879":29,"paddlepaddle\u7684docker\u5bb9\u5668\u4f7f\u7528\u65b9\u5f0f":30,"pod\u95f4\u901a\u4fe1":52,"pydataprovider2\u7684\u4f7f\u7528":3,"python\u63a5\u53e3":58,"python\u76f8\u5173\u7684\u5355\u5143\u6d4b\u8bd5\u90fd\u8fc7\u4e0d\u4e86":27,"python\u811a\u672c\u8bfb\u53d6\u6570\u636e":60,"return":26,"rnn\u76f8\u5173\u6a21\u578b":37,"rnn\u914d\u7f6e":39,"so\u627e\u4e0d\u5230":32,"ubuntu\u90e8\u7f72paddlepaddl":32,Abs:14,absactiv:6,activ:[6,14],adadelta:18,adadeltaoptim:12,adagrad:18,adagradoptim:12,adam:18,adamax:18,adamaxoptim:12,adamoptim:12,addto:16,addto_lay:10,aggreg:[10,16],algorithm:24,api:[1,4],applic:4,argument:26,async:47,attent:39,attribut:[7,15],auc_evalu:9,avg:19,avgpool:13,base:[9,10],baseactiv:6,basepool:19,basepoolingtyp:13,basesgdoptim:12,batch:26,batch_norm:16,batch_norm_lay:10,batch_siz:26,beam_search:[10,16],becaus:27,between:23,bidirectional_lstm:[11,17],big:27,bilinear_interp:16,bilinear_interp_lay:10,bla:29,block_expand:16,block_expand_lay:10,book:30,brelu:14,breluactiv:6,cach:3,check:[10,16],chunk_evalu:9,cifar:20,classif:9,classification_error_evalu:9,classification_error_printer_evalu:9,clone:40,column_sum_evalu:9,commit:40,compos:26,concat:16,concat_lay:10,config:4,configur:21,conll05:20,connect:[10,16],content:[3,27,35,44],context_project:[10,16],conv:[10,16],conv_oper:[10,16],conv_project:[10,16],conv_shift:16,conv_shift_lay:10,cos_sim:[10,16],cost:[10,16],cp27mu:27,creat:26,creator:26,crf:16,crf_decod:16,crf_decoding_lay:10,crf_layer:10,cross_channel_norm:16,cross_entropi:10,cross_entropy_cost:16,cross_entropy_with_selfnorm:10,cross_entropy_with_selfnorm_cost:16,ctc:16,ctc_error_evalu:9,ctc_layer:10,cuda:[27,29],cudnn:29,cudnnavg:19,cudnnmax:19,custom:26,dat:61,data:[10,16,20,26],data_lay:10,datafeed:20,dataprovid:[4,47],dataset:[20,24],datasourc:8,datatyp:20,decayedadagrad:18,decayedadagradoptim:12,decor:26,design:[23,24,26],dictionari:26,distribut:[23,24],doc:[23,24,26],dotmul_oper:[10,16],dotmul_project:[10,16],driver:27,dropout_lay:[11,17],dynam:24,embed:16,embedding_lay:10,entri:26,eos:16,eos_lay:10,evalu:9,event:[22,23],exampl:23,except:27,exp:14,expactiv:6,expand:[16,35],expand_lay:10,faq:27,fault:24,fc_layer:10,first_seq:[10,16,35],fork:40,format:24,from:23,full_matrix_project:[10,16],fulli:[10,16],gate:39,get_output:16,get_output_lay:10,gpu:47,gradient_printer_evalu:9,group:[10,16],gru:[11,17,47],gru_group:[11,17],gru_step:16,gru_step_lay:10,gru_unit:[11,17],grumemori:[10,16],handler:[23,25],how:26,hsigmoid:[10,16],huber_cost:[10,16],ident:14,identity_project:[10,16],identityactiv:6,illeg:27,imag:[10,11,16,17],imagenet:58,imdb:[20,64],img_cmrnorm:16,img_cmrnorm_lay:10,img_conv:16,img_conv_bn_pool:[11,17],img_conv_group:[11,17],img_conv_lay:10,img_pool:16,img_pool_lay:10,imikolov:20,implement:26,infer:22,ingredi:23,init_hook:3,input_typ:3,instruct:27,insuffici:27,interfac:[20,24,26],interpol:16,interpolation_lay:10,isn:26,issu:40,job:[24,51],join:[10,16],kubernet:[50,51],lambda_cost:[10,16],last_seq:[10,16,35],layer:[10,16,23],layeroutput:10,layertyp:10,libcudart:32,libcudnn:32,linear:14,linear_comb:16,linear_comb_lay:10,linearactiv:6,linux_x86_64:27,list:26,log:14,logactiv:6,lstm:[11,17,47,63,64],lstm_step:16,lstm_step_lay:10,lstmemori:[10,16],lstmemory_group:[11,17],lstmemory_unit:[11,17],map:26,master:24,math:[10,16],max:19,maxframe_printer_evalu:9,maxid:16,maxid_lay:10,maxid_printer_evalu:9,maxout:16,maxout_lay:10,maxpool:13,memori:[10,16,36,38],messag:27,mini:26,minibatch:20,misc:[11,17],mix:[10,16],mixed_lay:10,mnist:20,model:[4,21,23,39,58],momentum:18,momentumoptim:12,movi:61,movielen:20,mse_cost:[10,16],multi_binary_label_cross_entropi:10,multi_binary_label_cross_entropy_cost:16,multipl:26,nce:16,nce_lay:10,need:26,network:[11,17,39],neural:39,nlp:[11,17,47],norm:[10,16],nvprof:44,nvvp:44,object:24,onli:26,optim:[12,18,24],output:11,pad:16,pad_lay:10,paddl:26,paddlepaddl:[23,42,55],parallel_nn:49,paramet:[7,15,22,23,24],perform:47,platform:27,pnpair_evalu:9,point:27,pool:[10,13,16,19,35],pooling_lay:10,power:16,power_lay:10,pre:40,precision_recall_evalu:9,prefetch:26,print:9,process:24,protocol:27,provid:[3,26],pull:40,push:40,python:26,queue:24,rank:9,rank_cost:[10,16],rate:61,reader:[20,23,26],recoveri:24,recurr:[10,11,16,17,38,39],recurrent_group:[10,16],recurrent_lay:10,refer:3,reject:27,relu:14,reluactiv:6,repeat:16,repeat_lay:10,request:40,reshap:[10,16],resnet:58,rmsprop:18,rmspropoptim:12,rnn:[36,47],rotat:16,rotate_lay:10,sampl:[10,16],sampling_id:16,sampling_id_lay:10,scale:[16,24],scaling_lay:10,scaling_project:[10,16],selective_fc:16,selective_fc_lay:10,sentiment:20,seq_concat:16,seq_concat_lay:10,seq_reshap:16,seq_reshape_lay:10,seqtext_printer_evalu:9,sequenc:39,sequence_conv_pool:[11,17],sequencesoftmax:14,sequencesoftmaxactiv:6,server:24,set:12,sgd:47,share:23,shuffl:26,sigmoid:14,sigmoidactiv:6,simple_attent:[11,17],simple_gru:[11,17],simple_img_conv_pool:[11,17],simple_lstm:[11,17],singl:26,slice:[10,16],slope_intercept:16,slope_intercept_lay:10,softmax:14,softmaxactiv:6,softrelu:14,softreluactiv:6,spp:16,spp_layer:10,squar:14,squareactiv:6,squarerootn:19,squarerootnpool:13,stack:64,stanh:14,stanhactiv:6,start:23,suffici:26,sum:19,sum_cost:[10,16],sum_evalu:9,sum_to_one_norm:16,sum_to_one_norm_lay:10,summar:23,sumpool:13,support:27,table_project:[10,16],take:26,tanh:14,tanhactiv:6,task:24,tensor:16,tensor_lay:10,text_conv_pool:[11,17],thi:27,toler:24,too:27,train:[22,23,24,26],trainer:[22,24],tran:16,trans_full_matrix_project:[10,16],trans_lay:10,tune:47,uci_h:20,updat:23,usag:26,use:26,user:[24,61],util:9,value_printer_evalu:9,version:27,vgg_16_network:[11,17],warp_ctc:16,warp_ctc_lay:10,wheel:27,whl:27,why:26,wmt14:20,zoo:58}}) \ No newline at end of file +Search.setIndex({docnames:["api/index_cn","api/v1/data_provider/dataprovider_cn","api/v1/data_provider/pydataprovider2_cn","api/v1/index_cn","api/v1/predict/swig_py_paddle_cn","api/v2/config/activation","api/v2/config/attr","api/v2/config/evaluators","api/v2/config/layer","api/v2/config/networks","api/v2/config/optimizer","api/v2/config/pooling","api/v2/data","api/v2/data/data_reader","api/v2/data/dataset","api/v2/data/image","api/v2/fluid","api/v2/fluid/data_feeder","api/v2/fluid/evaluator","api/v2/fluid/executor","api/v2/fluid/initializer","api/v2/fluid/layers","api/v2/fluid/nets","api/v2/fluid/optimizer","api/v2/fluid/param_attr","api/v2/fluid/profiler","api/v2/fluid/regularizer","api/v2/model_configs","api/v2/run_logic","design/api","design/auto_gradient_check","design/block","design/build_system/README","design/cluster_train/README","design/cluster_train/checkpointing","design/cluster_train/data_dispatch","design/cluster_train/large_model_dist_train","design/cluster_train/master_server","design/cluster_train/pserver_client","design/cluster_train/remote_parameter_updater","design/cluster_train/save_model","design/cluster_train/submit-job","design/evaluator","design/executor","design/file_manager/README","design/file_manager/pfs/pfsclient","design/float16","design/functions_operators_layers","design/gan_api","design/graph","design/graph_survey","design/if_else_op","design/infer_var_type","design/model_format","design/multi_language_interface/00.why_plain_c","design/multi_language_interface/01.inference_implementation","design/ops/rnn","design/ops/sequence_decoder","design/optimizer","design/parameter_average","design/parameters_in_cpp","design/program","design/prune","design/python_api","design/reader/README","design/refactor/distributed_architecture","design/refactor/parameter_server","design/refactor/session","design/refactorization","design/register_grad_op","design/regularization","design/releasing_process","design/scope","design/selected_rows","design/simple_op_design","design/tensor_array","design/var_desc","faq/build_and_install/index_cn","faq/cluster/index_cn","faq/index_cn","faq/local/index_cn","faq/model/index_cn","faq/parameter/index_cn","getstarted/build_and_install/build_from_source_cn","getstarted/build_and_install/docker_install_cn","getstarted/build_and_install/index_cn","getstarted/build_and_install/pip_install_cn","getstarted/concepts/use_concepts_cn","getstarted/index_cn","howto/deep_model/rnn/hierarchical_layer_cn","howto/deep_model/rnn/hrnn_rnn_api_compare_cn","howto/deep_model/rnn/index_cn","howto/deep_model/rnn/recurrent_group_cn","howto/deep_model/rnn/rnn_config_cn","howto/dev/build_cn","howto/dev/contribute_to_paddle_cn","howto/dev/new_layer_cn","howto/dev/new_op_cn","howto/dev/use_eigen_cn","howto/dev/write_docs_cn","howto/index_cn","howto/optimization/cpu_profiling","howto/optimization/cpu_profiling_cn","howto/optimization/gpu_profiling_cn","howto/usage/cluster/cluster_train_cn","howto/usage/cmd_parameter/arguments_cn","howto/usage/cmd_parameter/detail_introduction_cn","howto/usage/cmd_parameter/index_cn","howto/usage/cmd_parameter/use_case_cn","howto/usage/k8s/k8s_basis_cn","howto/usage/k8s/k8s_cn","howto/usage/k8s/k8s_distributed_cn","howto/usage/k8s/src/k8s_data/README","howto/usage/k8s/src/k8s_train/README","index_cn","mobile/cross_compiling_for_android_cn","mobile/cross_compiling_for_ios_cn","mobile/cross_compiling_for_raspberry_cn","mobile/index_cn","survey/cluster_bootstrapping_tools","v1_api_tutorials/README","v1_api_tutorials/embedding_model/index_cn","v1_api_tutorials/imagenet_model/resnet_model_cn","v1_api_tutorials/quick_start/index_cn"],envversion:50,filenames:["api/index_cn.rst","api/v1/data_provider/dataprovider_cn.rst","api/v1/data_provider/pydataprovider2_cn.rst","api/v1/index_cn.rst","api/v1/predict/swig_py_paddle_cn.rst","api/v2/config/activation.rst","api/v2/config/attr.rst","api/v2/config/evaluators.rst","api/v2/config/layer.rst","api/v2/config/networks.rst","api/v2/config/optimizer.rst","api/v2/config/pooling.rst","api/v2/data.rst","api/v2/data/data_reader.rst","api/v2/data/dataset.rst","api/v2/data/image.rst","api/v2/fluid.rst","api/v2/fluid/data_feeder.rst","api/v2/fluid/evaluator.rst","api/v2/fluid/executor.rst","api/v2/fluid/initializer.rst","api/v2/fluid/layers.rst","api/v2/fluid/nets.rst","api/v2/fluid/optimizer.rst","api/v2/fluid/param_attr.rst","api/v2/fluid/profiler.rst","api/v2/fluid/regularizer.rst","api/v2/model_configs.rst","api/v2/run_logic.rst","design/api.md","design/auto_gradient_check.md","design/block.md","design/build_system/README.md","design/cluster_train/README.md","design/cluster_train/checkpointing.md","design/cluster_train/data_dispatch.md","design/cluster_train/large_model_dist_train.md","design/cluster_train/master_server.md","design/cluster_train/pserver_client.md","design/cluster_train/remote_parameter_updater.md","design/cluster_train/save_model.md","design/cluster_train/submit-job.md","design/evaluator.md","design/executor.md","design/file_manager/README.md","design/file_manager/pfs/pfsclient.md","design/float16.md","design/functions_operators_layers.md","design/gan_api.md","design/graph.md","design/graph_survey.md","design/if_else_op.md","design/infer_var_type.md","design/model_format.md","design/multi_language_interface/00.why_plain_c.md","design/multi_language_interface/01.inference_implementation.md","design/ops/rnn.md","design/ops/sequence_decoder.md","design/optimizer.md","design/parameter_average.md","design/parameters_in_cpp.md","design/program.md","design/prune.md","design/python_api.md","design/reader/README.md","design/refactor/distributed_architecture.md","design/refactor/parameter_server.md","design/refactor/session.md","design/refactorization.md","design/register_grad_op.md","design/regularization.md","design/releasing_process.md","design/scope.md","design/selected_rows.md","design/simple_op_design.md","design/tensor_array.md","design/var_desc.md","faq/build_and_install/index_cn.rst","faq/cluster/index_cn.rst","faq/index_cn.rst","faq/local/index_cn.rst","faq/model/index_cn.rst","faq/parameter/index_cn.rst","getstarted/build_and_install/build_from_source_cn.rst","getstarted/build_and_install/docker_install_cn.rst","getstarted/build_and_install/index_cn.rst","getstarted/build_and_install/pip_install_cn.rst","getstarted/concepts/use_concepts_cn.rst","getstarted/index_cn.rst","howto/deep_model/rnn/hierarchical_layer_cn.rst","howto/deep_model/rnn/hrnn_rnn_api_compare_cn.rst","howto/deep_model/rnn/index_cn.rst","howto/deep_model/rnn/recurrent_group_cn.md","howto/deep_model/rnn/rnn_config_cn.rst","howto/dev/build_cn.md","howto/dev/contribute_to_paddle_cn.md","howto/dev/new_layer_cn.rst","howto/dev/new_op_cn.md","howto/dev/use_eigen_cn.md","howto/dev/write_docs_cn.rst","howto/index_cn.rst","howto/optimization/cpu_profiling.md","howto/optimization/cpu_profiling_cn.md","howto/optimization/gpu_profiling_cn.rst","howto/usage/cluster/cluster_train_cn.md","howto/usage/cmd_parameter/arguments_cn.md","howto/usage/cmd_parameter/detail_introduction_cn.md","howto/usage/cmd_parameter/index_cn.rst","howto/usage/cmd_parameter/use_case_cn.md","howto/usage/k8s/k8s_basis_cn.md","howto/usage/k8s/k8s_cn.md","howto/usage/k8s/k8s_distributed_cn.md","howto/usage/k8s/src/k8s_data/README.md","howto/usage/k8s/src/k8s_train/README.md","index_cn.rst","mobile/cross_compiling_for_android_cn.md","mobile/cross_compiling_for_ios_cn.md","mobile/cross_compiling_for_raspberry_cn.md","mobile/index_cn.rst","survey/cluster_bootstrapping_tools.md","v1_api_tutorials/README.md","v1_api_tutorials/embedding_model/index_cn.md","v1_api_tutorials/imagenet_model/resnet_model_cn.md","v1_api_tutorials/quick_start/index_cn.rst"],objects:{"paddle.v2":{image:[15,1,0,"-"]},"paddle.v2.fluid":{regularizer:[26,1,0,"-"]},"paddle.v2.fluid.evaluator.Evaluator":{metrics:[18,0,1,""],states:[18,0,1,""]},"paddle.v2.fluid.regularizer":{L1DecayRegularizer:[26,2,1,""]},"paddle.v2.image":{batch_images_from_tar:[15,3,1,""],center_crop:[15,3,1,""],left_right_flip:[15,3,1,""],load_and_transform:[15,3,1,""],load_image:[15,3,1,""],load_image_bytes:[15,3,1,""],random_crop:[15,3,1,""],resize_short:[15,3,1,""],simple_transform:[15,3,1,""],to_chw:[15,3,1,""]}},objnames:{"0":["py","attribute","Python \u5c5e\u6027"],"1":["py","module","Python \u6a21\u5757"],"2":["py","class","Python \u7c7b"],"3":["py","function","Python \u51fd\u6570"]},objtypes:{"0":"py:attribute","1":"py:module","2":"py:class","3":"py:function"},terms:{"000\u5e45\u56fe\u50cf\u4e0a\u6d4b\u8bd5\u4e86\u6a21\u578b\u7684\u5206\u7c7b\u9519\u8bef\u7387":122,"000\u5f20\u7070\u5ea6\u56fe\u7247\u7684\u6570\u5b57\u5206\u7c7b\u6570\u636e\u96c6":2,"00186201e":4,"00m":103,"01852v1":8,"03m":103,"0424m":103,"0473v3":9,"04\u4ee5\u4e0a":86,"04\u4ee5\u53camaco":88,"0630u":103,"06u":103,"08083v2":8,"0810u":103,"08823112e":4,"0957m":103,"0\u53f7\u8bad\u7ec3\u8282\u70b9\u662f\u4e3b\u8bad\u7ec3\u8282\u70b9":106,"0\u5c42\u5e8f\u5217":89,"0_cudnn5":83,"0_cudnn5_avx_mkl":86,"0_cudnn7_avx_mkl":86,"0ab":8,"0rc1":71,"0rc2":71,"0x10f256d50":50,"0x7ffe4de00110":50,"100m":80,"101\u5c42\u548c152\u5c42\u7684\u7f51\u7edc\u7ed3\u6784\u4e2d":122,"101\u5c42\u548c152\u5c42\u7684\u7f51\u7edc\u914d\u7f6e\u6587\u4ef6\u53ef\u53c2\u7167":122,"101\u5c42\u7f51\u7edc\u6a21\u578b":122,"10g":41,"1150u":103,"11\u5b9e\u73b0\u4e86c":55,"11e6":110,"12194102e":4,"124n":103,"128\u7ef4\u548c256\u7ef4":121,"12\u4ee5\u4e0a":86,"12\u64cd\u4f5c\u7cfb\u7edf":77,"13m":110,"1490u":103,"14\u7248\u672c\u4ee5\u4e0a\u7684":117,"14\u8fd9\u79cd\u5199\u6cd5\u5c06\u4f1a\u6d4b\u8bd5\u6a21\u578b":108,"152\u5c42\u7f51\u7edc\u6a21\u578b":122,"15501715e":4,"1550u":103,"15\u884c":90,"16\u5b57\u8282\u8868\u793a\u4fdd\u5b58\u7684\u53c2\u6570\u603b\u4e2a\u6570":82,"16u":103,"173m":122,"173n":103,"1770u":103,"18e457ce3d362ff5f3febf8e7f85ffec852f70f3b629add10aed84f930a68750":110,"197u":103,"1\u4e4b\u540e\u7684\u4efb\u4f55\u4e00\u4e2a\u7248\u672c\u6765\u7f16\u8bd1\u8fd0\u884c":83,"1\u7684\u5c42\u4e4b\u5916":108,"1\u7a00\u758f\u6570\u636e":96,"1\u8f6e\u5b58\u50a8\u7684\u6240\u6709\u6a21\u578b":108,"210u":103,"211839e770f7b538e2d8":9,"215n":103,"228u":103,"234m":122,"2520u":103,"25639710e":4,"25k":123,"2680u":103,"26\u884c":90,"27787406e":4,"279n":103,"27m":103,"285m":103,"2863m":103,"28\u7684\u56fe\u7247\u50cf\u7d20\u7070\u5ea6\u503c":2,"28\u7ef4\u7684\u7a20\u5bc6\u6d6e\u70b9\u6570\u5411\u91cf\u548c\u4e00\u4e2a":2,"28m":103,"2977m":103,"2\u4e09\u7c7b\u7684\u6bd4\u4f8b\u4e3a":82,"2\u5206\u522b\u4ee3\u88683\u4e2a\u8282\u70b9\u7684trainer":111,"2\u610f\u5473\u77400\u53f7\u548c1\u53f7gpu\u5c06\u4f1a\u4f7f\u7528\u6570\u636e\u5e76\u884c\u6765\u8ba1\u7b97fc1\u548cfc2\u5c42":108,"2\u8fd9\u51e0\u4e2a\u76ee\u5f55\u8868\u793apaddlepaddle\u8282\u70b9\u4e0etrain":111,"302n":103,"30u":103,"3206326\u4e2a\u8bcd\u548c4\u4e2a\u7279\u6b8a\u6807\u8bb0":121,"32777140e":4,"328n":103,"32\u7ef4":121,"32u":103,"32x32":14,"331n":103,"3320u":103,"36540484e":4,"36u":103,"3710m":103,"3768m":103,"387u":103,"38u":103,"3920u":103,"39u":103,"3\u4ee5\u4e0a\u7684\u7b26\u53f7":86,"3\u53f7gpu":80,"4035m":103,"4090u":103,"4096mb":106,"4279m":103,"43630644e":4,"43u":103,"448a5b355b84":110,"4560u":103,"4563m":103,"45u":103,"4650u":103,"4726m":103,"473m":110,"48565123e":4,"48684503e":4,"49316648e":4,"4\u5b57\u8282\u8868\u793apaddlepaddle\u7248\u672c\u4fe1\u606f":82,"4gb":106,"4yf":8,"500m":80,"50\u5c42":122,"50\u5c42\u7f51\u7edc\u6a21\u578b":122,"51111044e":4,"514u":103,"525n":103,"526u":103,"53018653e":4,"536u":103,"5460u":103,"5470u":103,"54u":103,"5690m":103,"573u":103,"578n":103,"5798m":103,"586u":103,"58s":110,"5969m":103,"5\u4e2a\u6d4b\u8bd5\u6837\u4f8b\u548c2\u4e2a\u751f\u6210\u5f0f\u6837\u4f8b":121,"5\u4f5c\u4e3a\u7f16\u8bd1\u73af\u5883":86,"5\u5373\u5c06\u505c\u6b62\u7ef4\u62a4":86,"5_cudnn5_avx_mkl":86,"5delta":8,"6080u":103,"6082v4":8,"6140u":103,"6305m":103,"639u":103,"64\u5e73\u53f0\u4e3a\u4f8b":115,"64\u7ef4":121,"64m":53,"655u":103,"6780u":103,"6810u":103,"682u":103,"6970u":103,"6\u4e07\u4ebf\u6b21\u6d6e\u70b9\u8fd0\u7b97\u6bcf\u79d2":103,"6\u4ee5\u4e0a":[86,88],"6\u4f5c\u4e3a\u6807\u51c6\u7f16\u8bd1\u73af\u5883":86,"6\u5143\u4e0a\u4e0b\u6587\u4f5c\u4e3a\u8f93\u5165\u5c42":121,"704u":103,"70634608e":4,"7090u":103,"72296313e":4,"72u":103,"73u":103,"75u":103,"760u":103,"767u":103,"783n":103,"784u":103,"78m":103,"7\u4ee5\u4e0a":115,"7\u4ee5\u4e0a\u7684\u7b26\u53f7":86,"7\u4ee5\u4e0b":115,"7\u548cpip":77,"7\u7248\u672c\u5f00\u59cb":115,"7\u7cfb\u5217":86,"7eamaa":14,"7kb":110,"8000\u5c31\u53ef\u4ee5\u5728\u7f51\u9875\u4e0a\u751f\u6210\u9700\u8981\u7684\u6587\u6863":99,"8250u":103,"8300u":103,"830n":103,"849m":103,"85625684e":4,"861u":103,"8661m":103,"877\u4e2a\u88ab\u5411\u91cf\u5316\u7684\u8bcd":121,"877\u884c":121,"892m":103,"8\u5b57\u8282\u8868\u793a\u6bcf\u4e2a\u53c2\u6570\u5360\u7528\u7684\u5b57\u8282\u6570":82,"901n":103,"90u":103,"918u":103,"9247m":103,"924n":103,"9261m":103,"93137714e":4,"9330m":103,"94u":103,"9530m":103,"96644767e":4,"983m":103,"988u":103,"997u":103,"99982715e":4,"99m":122,"99u":103,"9\u4e2d\u7684\u4e00\u4e2a\u6570\u5b57":2,"9a235":116,"9f18":110,"\u4e00":90,"\u4e00\u4e2a0\u5c42\u5e8f\u5217":89,"\u4e00\u4e2a0\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u6269\u5c55\u6210\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":89,"\u4e00\u4e2a\u4e0d\u5171\u4eab\u7684\u4f8b\u5b50\u662f":97,"\u4e00\u4e2a\u5178\u578b\u7684chunk\u5982\u4e0b\u6240\u793a":44,"\u4e00\u4e2a\u5206\u5e03\u5f0f\u7684\u5b58\u50a8\u7cfb\u7edf":109,"\u4e00\u4e2a\u5206\u5e03\u5f0fpaddle\u8bad\u7ec3\u4efb\u52a1\u4e2d\u7684\u6bcf\u4e2a\u8fdb\u7a0b\u90fd\u53ef\u4ee5\u4ececeph\u8bfb\u53d6\u6570\u636e":110,"\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":89,"\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217\u6216\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":89,"\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u6269\u5c55\u6210\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":89,"\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217\u8fdb\u5165":92,"\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":89,"\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217\u6216\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":89,"\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u53d8\u6210\u4e00\u4e2a0\u5c42\u5e8f\u5217":89,"\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u53d8\u6210\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":89,"\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217\u8fdb\u5165":92,"\u4e00\u4e2a\u53cc\u5c42rnn\u7531\u591a\u4e2a\u5355\u5c42rnn\u7ec4\u6210":92,"\u4e00\u4e2a\u53ef\u8c03\u7528\u7684\u51fd\u6570":92,"\u4e00\u4e2a\u5e38\u7528\u7684cmake\u914d\u7f6e\u5982\u4e0b":117,"\u4e00\u4e2a\u6216\u591a\u4e2a":109,"\u4e00\u4e2a\u6570\u636e\u96c6\u5927\u90e8\u5206\u5e8f\u5217\u957f\u5ea6\u662f100":80,"\u4e00\u4e2a\u6587\u4ef6":2,"\u4e00\u4e2a\u662f\u6d6e\u70b9\u8ba1\u7b97\u91cf":103,"\u4e00\u4e2a\u72ec\u7acb\u7684\u5143\u7d20":89,"\u4e00\u4e2a\u72ec\u7acb\u7684\u8bcd\u8bed":89,"\u4e00\u4e2a\u795e\u7ecf\u7f51\u7edc\u7684\u6a21\u578b\u7531\u5927\u91cf\u7684\u53c2\u6570\u7ec4\u6210":34,"\u4e00\u4e2a\u7f51\u7edc\u5c42\u7684\u524d\u5411\u4f20\u64ad\u90e8\u5206\u628a\u8f93\u5165\u8f6c\u5316\u4e3a\u76f8\u5e94\u7684\u8f93\u51fa":96,"\u4e00\u4e2a\u7f51\u7edc\u5c42\u7684\u53c2\u6570\u662f\u5728":96,"\u4e00\u4e2a\u7f51\u7edc\u5c42\u7684c":96,"\u4e00\u4e2a\u8f93\u51fa\u7ec4\u6210":97,"\u4e00\u4e2a\u91cd\u8981\u7684\u95ee\u9898\u662f\u9009\u62e9\u6b63\u786e\u7684learning_r":82,"\u4e00\u4e2achunk\u7531\u6240\u5728\u7684\u6587\u4ef6\u504f\u79fb":44,"\u4e00\u4e2agpu\u8bbe\u5907\u4e0a\u4e0d\u5141\u8bb8\u914d\u7f6e\u591a\u4e2a\u6a21\u578b":106,"\u4e00\u4e2alabel":90,"\u4e00\u4e2alogging\u5bf9\u8c61":2,"\u4e00\u4e2amemory\u5305\u542b":93,"\u4e00\u4e2apass\u8868\u793a\u8fc7\u4e00\u904d\u6240\u6709\u8bad\u7ec3\u6837\u672c":123,"\u4e00\u4e2apod\u4e2d\u7684\u6240\u6709\u5bb9\u5668\u4f1a\u88ab\u8c03\u5ea6\u5230\u540c\u4e00\u4e2anode\u4e0a":109,"\u4e00\u4e2aposix\u517c\u5bb9\u7684\u6587\u4ef6\u7cfb\u7edf":44,"\u4e00\u4eba":90,"\u4e00\u53e5\u8bdd\u662f\u7531\u8bcd\u8bed\u6784\u6210\u7684\u5e8f\u5217":92,"\u4e00\u53f0\u7535\u8111":94,"\u4e00\u65e9":90,"\u4e00\u662fbatch":80,"\u4e00\u6761\u6837\u672c":2,"\u4e00\u6837\u7684\u65b9\u5f0f":94,"\u4e00\u6b21\u4f5c\u4e1a\u79f0\u4e3a\u4e00\u4e2ajob":109,"\u4e00\u6b21\u6027\u676f\u5b50":90,"\u4e00\u6b21yield\u8c03\u7528":2,"\u4e00\u81f4":[89,90],"\u4e00\u822c\u4e0d\u5141\u8bb8\u518d\u4ece":71,"\u4e00\u822c\u4ece":95,"\u4e00\u822c\u5728paddlepaddle\u4e2d":90,"\u4e00\u822c\u60c5\u51b5\u4e0b":1,"\u4e00\u822c\u63a8\u8350\u8bbe\u7f6e\u6210true":2,"\u4e00\u822c\u662f\u7531\u4e8e\u76f4\u63a5\u4f20\u9012\u5927\u5b57\u5178\u5bfc\u81f4\u7684":82,"\u4e00\u822c\u6765\u8bf4":93,"\u4e00\u822c\u8868\u793a":90,"\u4e00\u822c\u8bbe\u7f6e":82,"\u4e00\u884c\u4e3a\u4e00\u4e2a\u6837\u672c":123,"\u4e09\u79cd\u5e8f\u5217\u6a21\u5f0f":[2,87],"\u4e0a":95,"\u4e0a\u4ea4\u53c9\u7f16\u8bd1raspberri":117,"\u4e0a\u4f20\u5230cloud\u6216\u8005\u4e0b\u8f7d\u5230\u672c\u5730\u7684\u65f6\u95f4\u53ef\u80fd\u6bd4\u8f83\u957f":44,"\u4e0a\u4f20\u65b9\u6cd5":71,"\u4e0a\u4f20\u8ba1\u7b97\u5f97\u51fa\u7684\u68af\u5ea6":104,"\u4e0a\u56fe\u4e2d\u865a\u7ebf\u7684\u8fde\u63a5":90,"\u4e0a\u56fe\u63cf\u8ff0\u4e86\u4e00\u4e2a3\u8282\u70b9\u7684\u5206\u5e03\u5f0f\u8bad\u7ec3\u573a\u666f":111,"\u4e0a\u6ce8\u518c\u4e00\u4e0b":44,"\u4e0a\u7f16\u8bd1\u5f88\u6162":94,"\u4e0a\u7f51":90,"\u4e0a\u8fd0\u884c":115,"\u4e0a\u8ff0\u4ee3\u7801\u5c06bias\u5168\u90e8\u521d\u59cb\u5316\u4e3a1":82,"\u4e0a\u8ff0\u547d\u4ee4\u4e2d":84,"\u4e0a\u8ff0\u547d\u4ee4\u7f16\u8bd1\u51fa\u4e00\u4e2a":94,"\u4e0a\u8ff0\u7684":81,"\u4e0a\u8ff0\u7684\u4ee3\u7801\u7247\u6bb5\u5305\u542b\u4e86\u4e24\u79cd\u65b9\u6cd5":103,"\u4e0a\u9762\u7684\u4ee3\u7801\u5728":97,"\u4e0a\u9762\u7684\u4ee3\u7801\u9996\u5148\u5bfc\u5165\u4f9d\u8d56\u7684\u5305":97,"\u4e0b":[97,99],"\u4e0b\u540c":82,"\u4e0b\u56fe\u4e2d\u5c31\u5c55\u793a\u4e86\u4e00\u4e9b\u5173\u4e8e\u5185\u5b58\u6570\u636e\u8fc1\u5f99\u548c\u8ba1\u7b97\u8d44\u6e90\u5229\u7528\u7387\u7684\u5efa\u8bae":103,"\u4e0b\u56fe\u5c55\u793a\u7684\u662f\u57fa\u4e8e\u6b8b\u5dee\u7684\u8fde\u63a5\u65b9\u5f0f":122,"\u4e0b\u56fe\u662f\u4e00\u4e2a\u5168\u8fde\u63a5\u5c42\u7684\u793a\u610f\u56fe":96,"\u4e0b\u5b58\u653e\u516c\u5171\u6570\u636e\u96c6\u5408":35,"\u4e0b\u6587\u4ee5nlp\u4efb\u52a1\u4e3a\u4f8b":92,"\u4e0b\u6587\u4f7f\u7528":111,"\u4e0b\u6587\u5c31\u662f\u7528job\u7c7b\u578b\u7684\u8d44\u6e90\u6765\u8fdb\u884c\u8bad\u7ec3":110,"\u4e0b\u6b21":90,"\u4e0b\u7684":111,"\u4e0b\u8868\u5c55\u793a\u4e86batch":122,"\u4e0b\u8f7d":44,"\u4e0b\u8f7d\u5230\u672c\u5730":44,"\u4e0b\u8f7d\u5b8c\u6570\u636e\u540e":110,"\u4e0b\u8f7d\u6307\u5b9a\u7248\u672c\u7684docker\u955c\u50cf":84,"\u4e0b\u8f7dgpu\u7248\u672c\u7684docker\u955c\u50cf":84,"\u4e0b\u9762\u4e3e\u4e2a\u7b80\u5355\u7684\u4f8b\u5b50":103,"\u4e0b\u9762\u4ecb\u7ecd\u4ecb\u7ecd":97,"\u4e0b\u9762\u4ee5":104,"\u4e0b\u9762\u4ee5\u77e9\u9635\u4e58\u64cd\u4f5c":97,"\u4e0b\u9762\u4ee5addop\u4e3a\u4f8b\u8bf4\u660etensor\u7684\u4f7f\u7528\u8fc7\u7a0b":98,"\u4e0b\u9762\u5148\u7b80\u8981\u4ecb\u7ecd\u4e00\u4e0b\u672c\u6587\u7528\u5230\u7684\u51e0\u4e2akubernetes\u6982\u5ff5":109,"\u4e0b\u9762\u5206\u522b\u4ecb\u7ecd\u67d0\u4e00\u7c7b\u6587\u4ef6\u7684\u5b9e\u73b0\u65b9\u5f0f":55,"\u4e0b\u9762\u5217\u51fa\u4e86":93,"\u4e0b\u9762\u5217\u51fa\u4e86\u5168\u8fde\u63a5\u5c42\u7684\u68af\u5ea6\u68c0\u67e5\u5355\u5143\u6d4b\u8bd5":96,"\u4e0b\u9762\u5c31\u6839\u636e\u8fd9\u51e0\u4e2a\u6b65\u9aa4\u5206\u522b\u4ecb\u7ecd":111,"\u4e0b\u9762\u662f":97,"\u4e0b\u9762\u662f\u5bf9":97,"\u4e0b\u9762\u7684\u4ee3\u7801\u5c06\u968f\u673a\u751f\u6210\u7684\u77e9\u9635\u8f6c\u5316\u4e3a\u53ef\u4ee5\u88abpaddlepaddle\u52a0\u8f7d\u7684\u6a21\u578b\u53c2\u6570":82,"\u4e0b\u9762\u7684\u4ee3\u7801\u7247\u6bb5\u5b9e\u73b0\u4e86":96,"\u4e0b\u9762\u7684\u4f8b\u5b50\u4f7f\u7528\u4e86":122,"\u4e0b\u9762\u7684\u4f8b\u5b50\u540c\u6837\u4f7f\u7528\u4e86":122,"\u4e0b\u9762\u7684\u70b9\u5b9e\u73b0\u4e86mulop\u7684\u5b9a\u4e49":97,"\u4e0b\u9762\u7ed9\u51fa\u4e86\u4e00\u4e2a\u4f8b\u5b50":96,"\u4e0b\u9762\u7ed9\u51fa\u5728\u4e09\u7ef4\u7a7a\u95f4\u4e2d\u4f7f\u7528\u7ebf\u6027\u56de\u5f52\u62df\u5408\u4e00\u6761\u76f4\u7ebf\u7684\u4f8b\u5b50":87,"\u4e0b\u9762\u89e3\u91ca\u4ee3\u7801\u4e2d\u4e00\u4e9b\u5173\u952e\u7684\u5730\u65b9":97,"\u4e0b\u9762\u8fd9\u4e9blayer\u80fd\u591f\u63a5\u53d7\u53cc\u5c42\u5e8f\u5217\u4f5c\u4e3a\u8f93\u5165":89,"\u4e0d":90,"\u4e0d\u4e00\u5b9a\u548c\u65f6\u95f4\u6709\u5173\u7cfb":2,"\u4e0d\u4e00\u81f4\u7684\u7531pfsclient\u4e0b\u8f7d\u6216\u8005\u4f20\u8f93chunk\u5b8c\u6210":44,"\u4e0d\u4f1a\u4fdd\u7559\u5728\u78c1\u76d8\u4e0a":94,"\u4e0d\u4f1a\u518d\u4ece":80,"\u4e0d\u4f1a\u865a\u62df\u4efb\u4f55\u786c\u4ef6":94,"\u4e0d\u4f7f\u7528\u9759\u6001\u5e93":54,"\u4e0d\u4f7f\u7528\u989d\u5916\u7a7a\u95f4":96,"\u4e0d\u4f7f\u7528c":54,"\u4e0d\u4f7f\u7528swig":54,"\u4e0d\u5141\u8bb8\u4e00\u4e2a\u6587\u4ef6\u4e2d\u5305\u542b\u591a\u4e2aop":97,"\u4e0d\u5171\u4eab\u5219\u4e0d\u52a0":97,"\u4e0d\u5171\u4eab\u7684\u4f8b\u5b50\u53ef\u4ee5\u53c2\u8003":97,"\u4e0d\u540c\u4e3b\u673a":109,"\u4e0d\u540c\u4e8e\u4e0a\u8ff0\u4ecb\u7ecd\u7684recurr":81,"\u4e0d\u540c\u4e8eop\u7684\u7f16\u8bd1\u6d4b\u8bd5":97,"\u4e0d\u540c\u53c2\u6570\u670d\u52a1\u5668\u4e0a\u6570\u636e\u5927\u5c0f\u7684\u6700\u5927\u503c\u4e0e\u6700\u5c0f\u503c\u7684\u6bd4\u7387":106,"\u4e0d\u540c\u5c42\u7684\u7279\u5f81\u7531\u5206\u53f7":122,"\u4e0d\u540c\u65f6\u95f4\u6b65\u7684\u8f93\u5165\u662f\u4e0d\u540c\u7684":93,"\u4e0d\u540c\u7248\u672c\u7684\u7f16\u8bd1\u5668\u4e4b\u95f4":54,"\u4e0d\u540c\u7684\u4f18\u5316\u7b97\u6cd5\u9700\u8981\u4f7f\u7528\u4e0d\u540c\u5927\u5c0f\u7684\u5185\u5b58":80,"\u4e0d\u540c\u7684\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf":111,"\u4e0d\u540c\u7684\u6570\u636e\u7c7b\u578b\u548c\u5e8f\u5217\u6a21\u5f0f\u8fd4\u56de\u7684\u683c\u5f0f\u4e0d\u540c":[2,87],"\u4e0d\u540c\u7a7a\u95f4\u7684\u8d44\u6e90\u540d\u53ef\u4ee5\u91cd\u590d":109,"\u4e0d\u540c\u8bbe\u5907":97,"\u4e0d\u540c\u8bed\u8a00\u7684\u63a5\u53e3\u9002\u5e94\u4e0d\u540c\u8bed\u8a00\u7684\u7279\u6027":54,"\u4e0d\u540c\u8f93\u5165\u542b\u6709\u7684\u5b50\u53e5":92,"\u4e0d\u540c\u8f93\u5165\u5e8f\u5217\u542b\u6709\u7684\u8bcd\u8bed\u6570\u5fc5\u987b\u4e25\u683c\u76f8\u7b49":92,"\u4e0d\u540cdataprovider\u5bf9\u6bd4\u5982\u4e0b":90,"\u4e0d\u540cpod\u4e4b\u95f4\u53ef\u4ee5\u901a\u8fc7ip\u5730\u5740\u8bbf\u95ee":109,"\u4e0d\u540crank\u7684tensor\u662f\u4e0d\u540c\u7c7b\u578b":98,"\u4e0d\u5728":55,"\u4e0d\u5bb9\u6613\u51fa\u9519":44,"\u4e0d\u5c11":90,"\u4e0d\u5d4c\u5165\u5176\u4ed6\u8bed\u8a00\u89e3\u91ca\u5668":54,"\u4e0d\u5d4c\u5165python\u89e3\u91ca\u5668":54,"\u4e0d\u5e94\u8be5\u88ab\u62c6\u89e3":92,"\u4e0d\u6307\u5b9a\u65f6":92,"\u4e0d\u63d0\u4f9b\u5206\u5e03\u5f0f\u5b58\u50a8":109,"\u4e0d\u662f\u4e00\u6761\u5e8f\u5217":[2,87],"\u4e0d\u662f\u771f\u6b63\u7684layer":81,"\u4e0d\u662f\u901a\u8fc7\u4e00\u822c\u7684\u65b9\u5f0f\u6765\u5b9e\u73b0\u5bf9\u8f93\u51fa\u7684\u6fc0\u6d3b":81,"\u4e0d\u663e\u793a\u7684\u5199\u6bcf\u4e2a\u7c7b\u5177\u4f53\u5305\u542b\u4ec0\u4e48":54,"\u4e0d\u6ee1\u8db3\u94a9\u5b50\u7684":95,"\u4e0d\u7528mount\u7684\u65b9\u5f0f\u6765\u8bbf\u95ee\u6570\u636e":35,"\u4e0d\u7f13\u5b58\u4efb\u4f55\u6570\u636e":2,"\u4e0d\u80fd\u4fee\u6539op\u7684\u6210\u5458\u53d8\u91cf":97,"\u4e0d\u80fd\u592a\u968f\u610f":95,"\u4e0d\u80fd\u88ab\u63d0\u4ea4\u5230":95,"\u4e0d\u8fc7":90,"\u4e0d\u8fc7\u5b9e\u9645\u4e0a\u662f\u8fd0\u884c\u5728\u4e00\u4e2a":94,"\u4e0d\u8fdc":90,"\u4e0d\u9519":90,"\u4e0d\u9700\u8981\u4f9d\u8d56\u5176\u4ed6\u4efb\u4f55\u8f6f\u4ef6\u4e86":94,"\u4e0d\u9700\u8981\u8bbe\u7f6e":115,"\u4e0e":[97,102,111,121],"\u4e0e\u4e4b\u76f8\u5bf9\u7684\u662flocal":44,"\u4e0e\u529f\u80fd\u5206\u652f\u4e0d\u540c\u7684\u662f":71,"\u4e0e\u5355\u5c42rnn\u7684\u914d\u7f6e\u7c7b\u4f3c":90,"\u4e0e\u53ef\u80fd\u6709\u7684":71,"\u4e0e\u540c\u6b65sgd\u76f8\u6bd4":104,"\u4e0e\u5bfb\u627epython\u4ee3\u7801\u7684\u6027\u80fd\u74f6\u9888\u7c7b\u4f3c":102,"\u4e0e\u5f53\u524d\u7684\u8870\u51cf\u56e0\u5b50\u7684\u4e58\u79ef":82,"\u4e0e\u672c\u5730\u8bad\u7ec3\u76f8\u540c":104,"\u4e0e\u6b64\u4e0d\u540c\u7684\u662f":111,"\u4e0e\u8c03\u4f18":102,"\u4e0e\u8fd9\u4e2a\u8bad\u7ec3\u6570\u636e\u4ea4\u4e92\u7684layer":80,"\u4e0ejob":111,"\u4e0eoperator\u524d\u5411\u8ba1\u7b97\u7684\u8f93\u51fa\u8fdb\u884c\u5bf9\u6bd4":97,"\u4e0eoperator\u6ce8\u518c\u65f6\u6ce8\u518c\u7684\u7c7b\u578b\u4e00\u81f4":97,"\u4e0epython\u4e0d\u540c":102,"\u4e14":90,"\u4e14\u4e0d\u6392\u9664commit\u4e4b\u95f4\u7684\u4fee\u6539\u5b58\u5728\u76f8\u4e92\u8986\u76d6\u7684\u60c5\u51b5":95,"\u4e14\u589e\u52a0\u4e00\u4e2a\u7b2c\u4e09\u65b9\u8bed\u8a00":54,"\u4e14\u5c55\u793a\u6548\u679c\u66f4\u597d":102,"\u4e14\u5e8f\u5217\u7684\u6bcf\u4e00\u4e2a\u5143\u7d20\u8fd8\u662f\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217":[2,87],"\u4e14\u652f\u6301\u90e8\u7f72\u5230":109,"\u4e14\u6bcf\u4e2a\u53e5\u5b50\u8868\u793a\u4e3a\u5bf9\u5e94\u7684\u8bcd\u8868\u7d22\u5f15\u6570\u7ec4":90,"\u4e14\u8c03\u7528\u65f6\u4e0d\u80fd\u629b\u51fa\u5f02\u5e38\u6216\u51fa\u73b0\u8fd0\u884c\u65f6\u9519\u8bef":55,"\u4e14c99\u652f\u6301bool\u7c7b\u578b\u548c\u5b9a\u957f\u6574\u6570":54,"\u4e14c99\u76f8\u5bf9\u4e8ec11\u4f7f\u7528\u66f4\u52a0\u5e7f\u6cdb":54,"\u4e24":90,"\u4e24\u4e2a\u5b50\u76ee\u5f55\u4e0b":99,"\u4e24\u4e2a\u5d4c\u5957\u7684":92,"\u4e24\u4e2a\u64cd\u4f5c":103,"\u4e24\u4e2a\u8f93\u5165\u7684\u5b50\u5e8f\u5217\u957f\u5ea6\u4e5f\u5e76\u4e0d\u76f8\u540c":90,"\u4e24\u4e2a\u90e8\u5206":99,"\u4e24\u79cd\u65b9\u6cd5\u7684\u533a\u522b":80,"\u4e24\u79cd\u7c7b\u522b":123,"\u4e24\u79cdblas\u5e93":83,"\u4e24\u8005\u5747\u4e3a\u7eaf\u6587\u672c\u6587\u4ef6":1,"\u4e24\u8005\u90fd\u662f\u5bf9\u68af\u5ea6\u7684\u622a\u65ad":80,"\u4e25\u683c\u7684\u547d\u540d\u89c4\u8303pep":71,"\u4e2a":123,"\u4e2a\u5185\u5b58\u6c60\u5b9e\u9645\u4e0a\u51b3\u5b9a\u4e86shuffle\u7684\u7c92\u5ea6":80,"\u4e2a\u6027\u5316\u63a8\u8350":71,"\u4e2a\u6279\u6b21\u7684\u53c2\u6570\u5e73\u5747\u503c\u8fdb\u884c\u6d4b\u8bd5":106,"\u4e2a\u6a21\u578b\u6d4b\u8bd5\u6570\u636e":106,"\u4e2d":[54,55,80,96,97,98,102,111,123],"\u4e2d\u4e0d\u8981\u6dfb\u52a0\u5927\u6587\u4ef6\u7b49":95,"\u4e2d\u4ecb\u7ecd\u7684\u65b9\u6cd5":121,"\u4e2d\u4f1a\u4f7f\u7528\u5230\u7684\u5b57\u5178\u6570\u636e\u6587\u4ef6":104,"\u4e2d\u4f20\u5165\u53c2\u6570":104,"\u4e2d\u4f20\u5165\u7684\u53c2\u6570":104,"\u4e2d\u5143\u7d20\u7684\u4e2a\u6570\u7b49\u4e8e\u7f51\u7edc\u4e2d\u8f93\u51fa\u5c42\u7684\u4e2a\u6570":80,"\u4e2d\u5173\u4e8e\u65f6\u95f4\u9012\u5f52\u795e\u7ecf\u7f51\u7edc\u7684\u4ecb\u7ecd":90,"\u4e2d\u5199\u5165json\u5185\u5bb9":34,"\u4e2d\u5305\u542b\u4e00\u4e2araspberri":117,"\u4e2d\u5305\u542b\u4e86\u8bad\u7ec3\u6a21\u578b\u7684\u57fa\u672c\u547d\u4ee4":123,"\u4e2d\u5305\u542b\u6240\u4f9d\u8d56\u7684\u6240\u6709\u7b2c\u4e09\u65b9\u5e93":115,"\u4e2d\u5305\u542b\u82e5\u5e72\u4e2a\u4e0d\u540candroid":115,"\u4e2d\u5305\u542bc":[115,117],"\u4e2d\u5355\u5143\u6d4b\u8bd5\u7684\u4e00\u90e8\u5206":95,"\u4e2d\u5355\u5143\u6d4b\u8bd5\u80fd\u987a\u5229\u901a\u8fc7":95,"\u4e2d\u5b8c\u5168\u4e00\u81f4":54,"\u4e2d\u5b9a\u4e49":93,"\u4e2d\u5b9a\u4e49\u4f7f\u7528\u54ea\u79cddataprovid":1,"\u4e2d\u5b9a\u4e49\u548c\u4f7f\u7528":92,"\u4e2d\u5b9e\u73b0\u7684\u7ed3\u6784\u4f53":55,"\u4e2d\u5c55\u793a\u4e86\u5982\u4f55\u4f7f\u7528python\u6765\u63d0\u53d6\u7279\u5f81":122,"\u4e2d\u6307\u5b9a":106,"\u4e2d\u6307\u5b9a\u7684\u540d\u5b57":108,"\u4e2d\u6307\u5b9a\u7684\u5c42\u987a\u5e8f\u4e00\u81f4":122,"\u4e2d\u63d0\u4f9b\u4e86\u4e00\u4e9b\u5168\u5c40\u51fd\u6570\u7528\u6765\u5b9e\u73b0paddl":98,"\u4e2d\u63d0\u51fa\u7684resnet\u7f51\u7edc\u7ed3\u6784\u57282015\u5e74imagenet\u5927\u89c4\u6a21\u89c6\u89c9\u8bc6\u522b\u7ade\u8d5b":122,"\u4e2d\u641c\u7d22\u8fd9\u51e0\u4e2a\u5e93":83,"\u4e2d\u6587\u6587\u6863":99,"\u4e2d\u6587\u6587\u6863\u76ee\u5f55":99,"\u4e2d\u6587\u7ef4\u57fa\u767e\u79d1\u9875\u9762":90,"\u4e2d\u6bcf\u4e2a\u5143\u7d20\u662f\u4e00\u4e2alayer\u7684\u8f93\u51fa\u7ed3\u679c\u77e9\u9635":80,"\u4e2d\u6bcf\u4e2apod\u7684ip\u5730\u5740":111,"\u4e2d\u6bcf\u5c42\u7684\u6570\u503c\u7edf\u8ba1":106,"\u4e2d\u6dfb\u52a0\u4e24\u4e2a\u8f93\u5165":97,"\u4e2d\u7684":[98,122],"\u4e2d\u7684\u4e00\u884c":[2,95],"\u4e2d\u7684\u4ee3\u7801\u4f5c\u4e3a\u5b9e\u4f8b":104,"\u4e2d\u7684\u5bf9\u5e94\u5206\u652f\u5373\u53ef":95,"\u4e2d\u7684\u6570\u636e":122,"\u4e2d\u7684\u6570\u636e\u8fdb\u884c\u9884\u6d4b":122,"\u4e2d\u7684\u7248\u672c\u4fe1\u606f":71,"\u4e2d\u7684\u7528\u6237\u8bc1\u4e66":109,"\u4e2d\u7684\u8bf4\u660e":2,"\u4e2d\u83b7\u53d6":111,"\u4e2d\u8bbe\u7f6e\u7684\u6240\u6709\u8282\u70b9":104,"\u4e2d\u8be6\u7ec6\u4ecb\u7ecd":96,"\u4e2d\u8bfb\u53d6":2,"\u4e2d\u8c03\u7528":97,"\u4e2d\u8fd0\u884c\u4efb\u52a1\u7684\u89d2\u5ea6":35,"\u4e2d\u914d\u7f6e\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":93,"\u4e2d\u914d\u7f6e\u7684\u6548\u679c\u4e00\u81f4":2,"\u4e34\u65f6\u53d8\u91cf\u7b49\u7b49":80,"\u4e3a":[2,93,97,115,116,117],"\u4e3a0":2,"\u4e3a\u4e86\u4f7f":97,"\u4e3a\u4e86\u4f7f\u8bc4\u5ba1\u4eba\u5728\u8bc4\u5ba1\u4ee3\u7801\u65f6\u66f4\u597d\u5730\u4e13\u6ce8\u4e8e\u4ee3\u7801\u672c\u8eab":95,"\u4e3a\u4e86\u4fdd\u8bc1\u6548\u7387":96,"\u4e3a\u4e86\u4fdd\u8bc1gpu\u9a71\u52a8\u80fd\u591f\u5728\u955c\u50cf\u91cc\u9762\u6b63\u5e38\u8fd0\u884c":84,"\u4e3a\u4e86\u5b8c\u6210\u5206\u5e03\u5f0f\u673a\u5668\u5b66\u4e60\u8bad\u7ec3\u4efb\u52a1":109,"\u4e3a\u4e86\u5c01\u88c5\u80fd\u591f\u6b63\u786e\u5de5\u4f5c":96,"\u4e3a\u4e86\u5e94\u5bf9\u4ee5\u4e0a\u7684\u95ee\u9898":44,"\u4e3a\u4e86\u5f00\u53d1paddlepaddl":94,"\u4e3a\u4e86\u63cf\u8ff0\u65b9\u4fbf":92,"\u4e3a\u4e86\u65b9\u4fbf\u5927\u5bb6":95,"\u4e3a\u4e86\u66b4\u9732\u7684\u63a5\u53e3\u5c3d\u91cf\u7b80\u5355":55,"\u4e3a\u4e86\u751f\u6210\u66f4\u53ef\u8bfb\u7684\u6027\u80fd\u5206\u6790\u7ed3\u679c":102,"\u4e3a\u4e86\u7b80\u5316cmake\u914d\u7f6e":115,"\u4e3a\u4e86\u8fbe\u5230\u6027\u80fd\u6700\u4f18":103,"\u4e3a\u4e86\u8fbe\u5230\u6700\u5feb\u7684\u8ba1\u7b97\u901f\u5ea6":[115,116],"\u4e3a\u4ec0\u4e48\u7528":94,"\u4e3a\u4f8b":[81,97,123],"\u4e3a\u4f8b\u6765\u4ecb\u7ecd\u5982\u4f55\u5199\u5e26kernel\u7684oper":97,"\u4e3a\u4f8b\u8fdb\u884c\u9884\u6d4b":123,"\u4e3a\u53c2\u6570\u77e9\u9635\u7684\u5bbd\u5ea6":82,"\u4e3a\u5bb9\u5668\u5185\u6267\u884c\u7684\u547d\u4ee4":84,"\u4e3a\u60a8\u505a\u6027\u80fd\u8c03\u4f18\u63d0\u4f9b\u4e86\u65b9\u5411":103,"\u4e3a\u60f3\u4fee\u6b63\u8bcd\u5411\u91cf\u6a21\u578b\u7684\u7528\u6237\u63d0\u4f9b\u4e86\u5c06\u6587\u672c\u8bcd\u5411\u91cf\u6a21\u578b\u8f6c\u6362\u4e3a\u4e8c\u8fdb\u5236\u6a21\u578b\u7684\u547d\u4ee4":121,"\u4e3a\u65b9\u4fbf\u4f5c\u4e1a\u542f\u52a8\u63d0\u4f9b\u4e86\u4e24\u4e2a\u72ec\u7279\u7684\u547d\u4ee4\u9009\u9879":104,"\u4e3a\u6b64":110,"\u4e3a\u6bcf\u4e2aop\u521b\u5efa\u5355\u72ec\u7684":97,"\u4e3a\u8f93\u51fa\u5206\u914d\u5185\u5b58":96,"\u4e3aconst\u51fd\u6570":97,"\u4e3aoutput_\u7533\u8bf7\u5185\u5b58":96,"\u4e3b\u8981\u4e3a\u5f00\u53d1\u8005\u4f7f\u7528":106,"\u4e3b\u8981\u529f\u80fd\u5305\u62ec":44,"\u4e3b\u8981\u5305\u62ec\u4ee5\u4e0b\u4e94\u4e2a\u6b65\u9aa4":4,"\u4e3b\u8981\u5305\u62ec\u56db\u79cd\u7c7b\u578b":87,"\u4e3b\u8981\u539f\u56e0":90,"\u4e3b\u8981\u539f\u56e0\u5305\u62ec\u4e24\u4e2a\u65b9\u9762":80,"\u4e3b\u8981\u539f\u56e0\u662f\u589e\u52a0\u4e86\u521d\u59cb\u5316\u673a\u5236":2,"\u4e3b\u8981\u7528\u4e8epython":97,"\u4e3b\u8981\u804c\u8d23\u5728\u4e8e\u5c06\u8bad\u7ec3\u6570\u636e\u4f20\u5165\u5185\u5b58\u6216\u8005\u663e\u5b58":123,"\u4e3e\u4e00\u4e2a\u4f8b\u5b50":82,"\u4e3e\u4f8b":80,"\u4e3e\u4f8b\u8bf4\u660e":90,"\u4e4b\u524d":95,"\u4e4b\u524d\u914d\u7f6e\u6587\u4ef6\u4e2d":123,"\u4e4b\u540e":[87,96],"\u4e4b\u540e\u4f7f\u7528":96,"\u4e4b\u540e\u4f7f\u7528\u77e9\u9635\u8fd0\u7b97\u51fd\u6570\u6765\u8ba1\u7b97":96,"\u4e4b\u540e\u518d\u7528\u7f51\u9875\u8fde\u5230http":99,"\u4e4b\u540e\u521d\u59cb\u5316\u6240\u6709\u7684\u6743\u91cd\u77e9\u9635":96,"\u4e4b\u540e\u624d\u80fd\u5f00\u59cb\u7f16\u8bd1\u7684\u6b65\u9aa4":83,"\u4e4b\u5916\u7684\u6240\u6709\u5934\u6587\u4ef6":55,"\u4e4b\u7c7b\u7684\u7a0b\u5e8f\u6765\u7f16\u8bd1\u6e90\u7801":94,"\u4e4b\u95f4\u7684\u8fd0\u7b97\u662f\u72ec\u7acb\u7684":92,"\u4e58\u4e0a\u8f93\u51fa\u7684\u68af\u5ea6":96,"\u4e58\u6cd5\u548c\u4e58\u6cd5\u68af\u5ea6\u7684\u8ba1\u7b97\u5360\u75282":102,"\u4e58\u9664\u7b49\u65f6\u5019":80,"\u4e5f":90,"\u4e5f\u4e0d\u4f7f\u7528\u5176\u4ed6\u52a8\u6001\u5e93":54,"\u4e5f\u4e0d\u5b58\u5728\u4e00\u4e2asubseq\u76f4\u63a5\u751f\u6210\u4e0b\u4e00\u4e2asubseq\u7684\u60c5\u51b5":92,"\u4e5f\u4e0d\u5e94\u8be5\u62a5\u9519":55,"\u4e5f\u4e0d\u751f\u6210":55,"\u4e5f\u4e0d\u80fd\u63a5\u6536\u5e8f\u5217\u6570\u636e\u4f5c\u4e3a\u8f93\u5165":81,"\u4e5f\u4f1a\u5360\u7528\u78c1\u76d8":94,"\u4e5f\u53ef\u4ee5\u4f7f\u7528":95,"\u4e5f\u53ef\u4ee5\u5229\u7528paddlepaddl":99,"\u4e5f\u53ef\u4ee5\u53bb\u6389\u8fd9\u4e9b\u8bc1\u4e66\u7684\u914d\u7f6e":109,"\u4e5f\u53ef\u4ee5\u662f\u4e00\u4e2a\u8bcd\u8bed":92,"\u4e5f\u53ef\u4ee5\u662f\u5728\u4efb\u52a1\u542f\u52a8\u524d\u4e0b\u8f7d\u5230\u672c\u5730\u7684":104,"\u4e5f\u53ef\u4ee5\u76f4\u63a5\u5728\u7f51\u9875\u9884\u89c8\u6587\u6863":99,"\u4e5f\u53ef\u4ee5\u8bf4\u662f\u67d0\u4e9b\u7279\u5b9a\u6307\u4ee4\u7684\u4f7f\u7528\u60c5\u51b5":103,"\u4e5f\u53ef\u4ee5\u901a\u8fc7\u4fee\u6539":111,"\u4e5f\u53ef\u4ee5\u901a\u8fc7saving_period_by_batches\u8bbe\u7f6e\u6bcf\u9694\u591a\u5c11batch\u4fdd\u5b58\u4e00\u6b21\u6a21\u578b":123,"\u4e5f\u53ef\u4ee5\u914d\u7f6e\u4e0d\u540c\u7684\u91cd\u8bd5\u673a\u5236":109,"\u4e5f\u53ef\u5199\u6210":97,"\u4e5f\u53ef\u81ea\u884c\u524d\u5f80\u5b98\u7f51\u4e0b\u8f7d":116,"\u4e5f\u53ef\u901a\u8fc7\u4ee5\u4e0b\u547d\u4ee4\u83b7\u53d6":115,"\u4e5f\u5c31\u662f":95,"\u4e5f\u5c31\u662f\u5c06\u8bcd\u5411\u91cf\u6a21\u578b\u8fdb\u4e00\u6b65\u6f14\u5316\u4e3a\u4e09\u4e2a\u65b0\u6b65\u9aa4":123,"\u4e5f\u5c31\u662f\u672c\u5730\u7684\u6e90\u7801\u6811\u6839\u76ee\u5f55\u91cc\u7684":94,"\u4e5f\u5c31\u662f\u81ea\u5df1\u7528\u6237\u540d\u4e0b\u7684":95,"\u4e5f\u5c31\u662f\u8bf4":[106,108,121],"\u4e5f\u63cf\u8ff0\u4e86\u5bb9\u5668\u9700\u8981\u4f7f\u7528\u7684\u5b58\u50a8\u5377\u6302\u8f7d\u7684\u60c5\u51b5":111,"\u4e5f\u652f\u6301cpu\u7684\u6027\u80fd\u5206\u6790":103,"\u4e5f\u662f\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217":90,"\u4e5f\u662f\u5bb9\u5668\u4e0enode\u4e4b\u95f4\u5171\u4eab\u6587\u4ef6\u7684\u65b9\u5f0f":109,"\u4e5f\u662fdecoder\u5faa\u73af\u5c55\u5f00\u7684\u4f9d\u636e":92,"\u4e5f\u662fpaddlepaddle\u6240\u80fd\u591f\u4fdd\u8bc1\u7684shuffle\u7c92\u5ea6":2,"\u4e5f\u6ca1\u7528":77,"\u4e5f\u79f0\u4e3arnn\u6a21\u578b":123,"\u4e5f\u9700\u8981\u4e24\u6b21\u968f\u673a\u9009\u62e9\u5230\u76f8\u540cgenerator\u7684\u65f6\u5019":2,"\u4e66\u5199":54,"\u4e7e":90,"\u4e86":[90,94],"\u4e86\u89e3\u5176\u8c03\u7528\u5173\u7cfb":102,"\u4e86\u89e3\u60a8\u7684\u786c\u4ef6":103,"\u4e86\u89e3\u66f4\u591a\u7ec6\u8282":93,"\u4e86\u89e3\u66f4\u591a\u8be6\u7ec6\u4fe1\u606f":93,"\u4e8c\u7ef4\u77e9\u9635":122,"\u4e8c\u8005\u8bed\u610f\u4e0a\u5b8c\u5168\u4e00\u81f4":90,"\u4e8c\u8fdb\u5236":121,"\u4e8e\u662f\u6211\u4eec\u53ef\u4ee5\u70b9\u51fb":102,"\u4e8e\u662f\u8fd9\u91cc\u4f7f\u7528":102,"\u4e92\u76f8\u901a\u4fe1":109,"\u4e94\u661f\u7ea7":90,"\u4ea4\u4e92\u7684\u65b9\u6cd5":102,"\u4ea4\u53c9\u7f16\u8bd1\u5de5\u5177\u94fe\u4e3a":115,"\u4ea4\u53c9\u7f16\u8bd1android\u5e73\u53f0\u4e0a\u9002\u7528\u7684paddlepaddle\u5e93":115,"\u4ea4\u53c9\u7f16\u8bd1android\u7248\u672c\u7684paddlepaddle\u5e93\u65f6":115,"\u4ea4\u53c9\u7f16\u8bd1ios\u5e73\u53f0\u4e0a\u9002\u7528\u7684paddlepaddle\u5e93":116,"\u4ea4\u53c9\u7f16\u8bd1ios\u7248\u672c\u7684paddlepaddle\u5e93\u65f6":116,"\u4ea4\u53c9\u7f16\u8bd1raspberri":117,"\u4ea4\u7531cmake\u7cfb\u7edf\u672c\u8eab\u6765\u5904\u7406":115,"\u4ea4\u901a":90,"\u4ea4\u901a\u4fbf\u5229":90,"\u4ea6\u53ef\u4ee5\u901a\u8fc7\u624b\u52a8\u8bbe\u7f6e":[115,116],"\u4eab\u53d7\u60a8\u7684\u65c5\u7a0b":84,"\u4eba\u8138\u8bc6\u522b":35,"\u4ec0\u4e48\u662f":94,"\u4ec5\u4ec5\u4f7f\u7528":54,"\u4ec5\u5728\u8fdc\u7a0b\u7a00\u758f\u8bad\u7ec3\u65f6\u6709\u6548":96,"\u4ec5\u5bf9\u7a00\u758f\u6570\u636e\u6709\u6548":96,"\u4ec5\u9700\u8981\u77e5\u9053\u5982\u4f55\u4ece":2,"\u4ecb\u7ecd\u4e86\u4e00\u79cd\u901a\u8fc7ssh\u8fdc\u7a0b\u5206\u53d1\u4efb\u52a1":111,"\u4ecb\u7ecd\u4ea4\u53c9\u7f16\u8bd1android\u5e73\u53f0\u4e0a\u9002\u7528\u7684paddlepaddle\u5e93\u7684\u65b9\u6cd5\u548c\u6b65\u9aa4":115,"\u4ecb\u7ecd\u4f7f\u7528paddlepaddl":104,"\u4ecb\u7ecd\u5206\u5e03\u5f0f\u8bad\u7ec3\u4e4b\u524d":109,"\u4ecb\u7ecdpaddlepaddle\u7684\u57fa\u672c\u4f7f\u7528\u65b9\u6cd5":123,"\u4ece":[71,78,103],"\u4ece0\u5230num":106,"\u4ece0\u5f00\u59cb\u7684\u6574\u6570":104,"\u4ece\u4e00\u4e2aword\u751f\u6210\u4e0b\u4e00\u4e2aword":92,"\u4ece\u5185\u6838\u51fd\u6570\u7684\u89d2\u5ea6":103,"\u4ece\u6a21\u578b\u6587\u4ef6\u5c06\u9884\u8bad\u7ec3\u53c2\u6570\u8f7d\u5165":82,"\u4ece\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65\u6765\u770b":90,"\u4ece\u6bcf\u4e2a\u5355\u8bcd\u5de6\u53f3\u4e24\u7aef\u5206\u522b\u83b7\u53d6k\u4e2a\u76f8\u90bb\u7684\u5355\u8bcd":123,"\u4ece\u6e90\u7801\u4ea4\u53c9\u7f16\u8bd1ios\u5e73\u53f0\u4e0a\u9002\u7528\u7684paddlepaddle\u5e93":116,"\u4ece\u6e90\u7801\u4ea4\u53c9\u7f16\u8bd1paddlepaddl":115,"\u4ece\u6e90\u7801\u7f16\u8bd1":85,"\u4ece\u78c1\u76d8\u6587\u4ef6\u4e2d\u52a0\u8f7duuid\u6587\u4ef6\u540d\u7684\u68c0\u67e5\u70b9\u5feb\u7167\u6587\u4ef6":34,"\u4ece\u800c\u53ef\u4ee5\u505a\u4e00\u4e9b\u4e0e\u8ba1\u7b97\u91cd\u53e0\u7684\u5de5\u4f5c":96,"\u4ece\u800c\u5f15\u53d1\u5176\u4ed6\u8282\u70b9\u65e0\u6cd5\u8fde\u63a5\u5bfc\u81f4":78,"\u4ece\u800c\u751f\u6210\u591a\u4e2agener":2,"\u4ece\u800c\u80fd\u591f\u88abpaddlepaddl":123,"\u4ece\u800c\u9632\u6b62\u8fc7\u62df\u5408":1,"\u4ece\u8bed\u4e49\u4e0a\u770b":92,"\u4ece\u8d77\u59cb\u7aef\u53e3\u76d1\u542c\u591a\u4e2a\u7aef\u53e3\u7528\u4e8e\u901a\u4fe1":104,"\u4ece\u8f93\u5165\u6570\u636e\u4e0a\u770b":90,"\u4ece\u9884\u8bad\u7ec3\u6a21\u578b\u4e2d":121,"\u4ececmake":115,"\u4eceetcd\u4e2d\u8bfb\u53d6\u8282\u70b9":34,"\u4ecestart":106,"\u4ed3\u5e93\u7684\u8fdc\u7a0b\u4e3b\u673a":95,"\u4ed4\u7ec6\u89c2\u5bdf":122,"\u4ed6\u4e3b\u8981\u5305\u542b\u4e86\u5b9e\u9645\u66b4\u9732\u7684\u7c7b\u578b\u7ed3\u6784":55,"\u4ed6\u4eec\u5206\u522b\u662f":90,"\u4ed6\u4eec\u5728\u81ea\u5df1\u7684":94,"\u4ed6\u4eec\u5728paddle\u7684\u6587\u6863\u548capi\u4e2d\u662f\u4e00\u4e2a\u6982\u5ff5":90,"\u4ed6\u4eec\u63d0\u51fa\u6b8b\u5dee\u5b66\u4e60\u7684\u6846\u67b6\u6765\u7b80\u5316\u7f51\u7edc\u7684\u8bad\u7ec3":122,"\u4ed6\u662f\u5c06":55,"\u4ed6\u7684\u76ee\u6807\u662f\u4f7f\u7528c":54,"\u4ee3\u66ff":111,"\u4ee3\u7801\u4e2d9":90,"\u4ee3\u7801\u53c2\u8003":104,"\u4ee3\u7801\u5982\u4e0b":[80,81,82,93],"\u4ee3\u7801\u6ce8\u91ca\u8bf7\u9075\u5b88":95,"\u4ee3\u7801\u751f\u6210\u7684\u7b26\u53f7\u53ef\u80fd\u4e0d\u4e00\u81f4":54,"\u4ee3\u7801\u7684\u6027\u80fd\u5206\u6790":102,"\u4ee3\u7801\u793a\u4f8b\u5982\u4e0b":97,"\u4ee3\u8868\u5bbf\u4e3b\u673a\u76ee\u5f55":111,"\u4ee3\u8868\u8fd9\u4e2ashard\u7684\u6700\u5927index":35,"\u4ee3\u8868shard\u7684index":35,"\u4ee5":81,"\u4ee5\u4e0a":[95,115],"\u4ee5\u4e0a\u4ee3\u7801\u7684reader\u8f93\u51fa\u7684data":35,"\u4ee5\u4e0a\u547d\u4ee4\u4f1a\u5728\u5f53\u524d\u76ee\u5f55\u4e0b\u751f\u6210100\u4e2a\u6587\u4ef6":35,"\u4ee5\u4e0b":35,"\u4ee5\u4e0b\u4ee3\u7801\u7247\u6bb5\u5b9a\u4e49":93,"\u4ee5\u4e0b\u547d\u4ee4\u542f\u52a8\u4e00\u4e2a":94,"\u4ee5\u4e0b\u6307\u4ee4\u80fd\u68c0\u67e5linux\u7535\u8111\u662f\u5426\u652f\u6301avx":84,"\u4ee5\u4e0b\u64cd\u4f5c\u5747\u5728head\u8282\u70b9\u4e2d\u6267\u884c":104,"\u4ee5\u4e0b\u6559\u7a0b\u5c06\u6307\u5bfc\u60a8\u63d0\u4ea4\u4ee3\u7801":95,"\u4ee5\u4e0b\u662f\u5bf9\u4e0a\u8ff0\u6570\u636e\u52a0\u8f7d\u7684\u89e3\u91ca":123,"\u4ee5\u4e0b\u793a\u8303\u5982\u4f55\u4f7f\u7528\u9884\u8bad\u7ec3\u7684\u4e2d\u6587\u5b57\u5178\u548c\u8bcd\u5411\u91cf\u8fdb\u884c\u77ed\u8bed\u6539\u5199":121,"\u4ee5\u4ea4\u4e92\u5f0f\u7684\u65b9\u5f0f\u6267\u884c\u6216\u8c03\u8bd5\u60a8\u7684\u4ee3\u7801":84,"\u4ee5\u4f7f\u7528":115,"\u4ee5\u4f7f\u7528adam\u7b97\u6cd5\u4e3a\u4f8b":82,"\u4ee5\u4fbf\u6211\u4eec\u53ef\u4ee5\u628a\u66f4\u591a\u7684\u7cbe\u529b\u653e\u5230\u903b\u8f91\u672c\u8eab\u4e0a":44,"\u4ee5\u4fbf\u83b7\u5f97\u8bad\u7ec3\u6570\u636e\u7684\u4f4d\u7f6e\u548c\u83b7\u53d6\u73af\u5883\u53d8\u91cf\u914d\u7f6e":104,"\u4ee5\u4fdd\u8bc1\u68af\u5ea6\u7684\u6b63\u786e\u8ba1\u7b97":96,"\u4ee5\u4fdd\u8bc1\u68af\u5ea6\u8ba1\u7b97\u7684\u6b63\u786e\u6027":96,"\u4ee5\u4fdd\u8bc1\u7f16\u8bd1\u9ad8\u6548":94,"\u4ee5\u53ca":[94,96],"\u4ee5\u53ca\u4f7f\u7528\u5b50\u5e8f\u5217\u6765\u5b9a\u4e49\u5206\u7ea7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u67b6\u6784":93,"\u4ee5\u53ca\u5207\u6362\u673a\u5668\u65f6\u9700\u8981\u65b0\u5b89\u88c5\u7684\u8f9b\u82e6":94,"\u4ee5\u53ca\u53cc\u5c42\u5e8f\u5217":89,"\u4ee5\u53ca\u76ee\u6807\u673a\u7248openblas\u5e93":117,"\u4ee5\u53ca\u76f8\u5173\u7684\u5c5e\u6027\u53c2\u6570":97,"\u4ee5\u53ca\u8ba1\u7b97\u903b\u8f91\u5728\u5e8f\u5217\u4e0a\u7684\u5faa\u73af\u5c55\u5f00":92,"\u4ee5\u53ca\u8f93\u5165\u7684\u68af\u5ea6":96,"\u4ee5\u53caandroid":115,"\u4ee5\u53caandroid\u6240\u9700":115,"\u4ee5\u53cagcc":83,"\u4ee5\u53canumpi":35,"\u4ee5\u53carelu":96,"\u4ee5\u63d0\u4f9b\u4e00\u4e9b\u9ed8\u8ba4\u7684\u7f16\u8bd1\u5668\u548c\u7f16\u8bd1\u53c2\u6570\u76f8\u5173\u914d\u7f6e":115,"\u4ee5\u63d0\u4f9b\u4e00\u4e9b\u9ed8\u8ba4\u7684\u7f16\u8bd1\u5668\u548c\u7f16\u8bd1\u53c2\u6570\u914d\u7f6e":116,"\u4ee5\u76f8\u5bf9\u8def\u5f84\u5f15\u7528":1,"\u4ee5\u786e\u4fdd\u6240\u6709\u7684\u7b2c\u4e09\u65b9\u4f9d\u8d56\u5e93\u548cpaddlepaddle\u4ee3\u7801\u90fd\u662f\u9488\u5bf9\u65b0\u7684cmake\u914d\u7f6e\u91cd\u65b0\u7f16\u8bd1\u7684":[115,116,117],"\u4ee5\u8f93\u51fa":80,"\u4ee5\u9017\u53f7":121,"\u4ee5\u9017\u53f7\u95f4\u9694":106,"\u4ee5eigentensor\u4e3a\u4f8b":98,"\u4ee5embedding\u5c42\u4e3a\u4f8b":82,"\u4ee5lstm\u4e3a\u4f8b":81,"\u4ef7\u683c":90,"\u4efb\u52a1\u6765\u7ec8\u6b62\u96c6\u7fa4\u4f5c\u4e1a":104,"\u4efb\u52a1\u88ab\u8c03\u5ea6\u5728\u96c6\u7fa4\u4e2d\u65f6":104,"\u4efb\u610f\u5c06\u4e00\u4e9b\u6570\u636e\u7ec4\u5408\u6210\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217":90,"\u4efb\u610f\u65f6\u523b\u53ea\u53ef\u80fd\u540c\u65f6\u6709\u4e00\u53f0\u670d\u52a1\u5668\u6545\u969c":34,"\u4f18\u5316\u5668\u5219\u7528\u94fe\u5f0f\u6cd5\u5219\u6765\u5bf9\u6bcf\u4e2a\u53c2\u6570\u8ba1\u7b97\u635f\u5931\u51fd\u6570\u7684\u68af\u5ea6":96,"\u4f1a\u4ea7\u751f\u5f53\u524dpython\u4e8c\u8fdb\u5236\u7684\u5b8c\u6574\u8def\u5f84":102,"\u4f1a\u5148\u8fdb\u884c\u53c2\u6570\u7684\u521d\u59cb\u5316\u4e0e\u89e3\u6790":111,"\u4f1a\u5171\u4eab\u53c2\u6570":82,"\u4f1a\u5173\u8054\u53c2\u6570":81,"\u4f1a\u5206\u522b\u4ecb\u7ecd\u96c6\u7fa4\u4f5c\u4e1a\u7684\u542f\u52a8\u548c\u505c\u6b62\u65b9\u6cd5":104,"\u4f1a\u52a0\u8f7d\u4e0a\u4e00\u8f6e\u7684\u53c2\u6570":106,"\u4f1a\u53d8\u6210\u8bcd\u8868\u4e2d\u7684\u4f4d\u7f6e":90,"\u4f1a\u542f\u52a8pserver\u4e0etrainer\u8fdb\u7a0b":111,"\u4f1a\u5728":99,"\u4f1a\u5728\u5f53\u524d\u76ee\u5f55\u751f\u6210\u4e24\u4e2a\u5b50\u76ee\u5f55":99,"\u4f1a\u5927\u4e0d\u76f8\u540c":104,"\u4f1a\u5bf9\u6bcf\u4e00\u4e2a\u6fc0\u6d3b\u6682\u5b58\u4e00\u4e9b\u6570\u636e":80,"\u4f1a\u5bf9\u8fd9\u7c7b\u8f93\u5165\u8fdb\u884c\u62c6\u89e3":92,"\u4f1a\u5bfc\u81f4\u4e0d\u540c\u7248\u672cpython\u5728\u4e00\u4e2a\u8fdb\u7a0b\u91cc\u7684bug":54,"\u4f1a\u5c06\u6bcf\u4e2a\u65f6\u95f4\u6b65\u7684\u8f93\u51fa\u62fc\u63a5":92,"\u4f1a\u5c06\u7b2c\u4e00\u4e2a":80,"\u4f1a\u6210\u4e3astep\u51fd\u6570\u7684\u8f93\u5165":92,"\u4f1a\u6253\u5370\u5230\u6807\u51c6\u8f93\u51fa":102,"\u4f1a\u6254\u5230\u8fd9\u6761\u6570\u636e":2,"\u4f1a\u628a\u8bad\u7ec3\u96c6\u548c\u6d4b\u8bd5\u96c6\u5206\u522b\u5206\u5272\u6210\u591a\u4e2a\u6587\u4ef6":104,"\u4f1a\u62a5\u5982\u4e0b\u7684\u9519\u8bef":80,"\u4f1a\u62a5\u9519":92,"\u4f1a\u6839\u636e\u547d\u4ee4\u884c\u53c2\u6570\u6307\u5b9a\u7684\u6d4b\u8bd5\u65b9\u5f0f":1,"\u4f1a\u6839\u636einput_types\u68c0\u67e5\u6570\u636e\u7684\u5408\u6cd5\u6027":2,"\u4f1a\u751f\u6210\u6027\u80fd\u5206\u6790\u7ed3\u679c\u6587\u4ef6":102,"\u4f1a\u76f4\u63a5\u62a5\u9519\u9000\u51fa":54,"\u4f1a\u76f8\u5e94\u5730\u6539\u53d8\u8f93\u51fa\u7684\u5c3a\u5bf8":96,"\u4f1a\u81ea\u52a8\u5173\u95ed\u5bf9\u5e94\u7684issu":95,"\u4f1a\u81ea\u52a8\u5728\u7f16\u8bd1\u65f6\u4e0b\u8f7d":83,"\u4f1a\u83b7\u53d6\u5f53\u524dnamespace\u4e0b\u7684\u6240\u6709pod":111,"\u4f1a\u88ab":104,"\u4f1a\u88ab\u62c6\u89e3\u4e3a\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":92,"\u4f1a\u88ab\u62c6\u89e3\u4e3a\u975e\u5e8f\u5217":92,"\u4f1a\u88abpickle\u5e8f\u5217\u5316\u6210\u5b57\u7b26\u4e32":35,"\u4f1a\u9020\u6210\u90ae\u4ef6\u707e\u96be":95,"\u4f20\u5165":[2,35],"\u4f20\u5165\u4e0a\u4e00\u6b65\u89e3\u6790\u51fa\u6765\u7684\u6a21\u578b\u914d\u7f6e\u5c31\u53ef\u4ee5\u521b\u5efa\u4e00\u4e2a":4,"\u4f20\u5165\u9884\u6d4b\u6570\u636e":4,"\u4f20\u7ed9dataprovider\u7684\u67d0\u4e00\u4e2aargs\u8fc7\u5927":82,"\u4f20\u9012\u7ed9\u914d\u7f6e\u6587\u4ef6\u7684\u53c2\u6570":106,"\u4f46":55,"\u4f46\u4e0d\u66b4\u9732":55,"\u4f46\u4e0d\u7528\u4e8e\u8ba1\u7b97\u68af\u5ea6":96,"\u4f46\u4e0d\u9700\u8981\u63d0\u524d\u521b\u5efa":106,"\u4f46\u4e8e\u53cc\u5c42\u5e8f\u5217\u7684lstm\u6765\u8bf4":90,"\u4f46\u548c\u5355\u5c42rnn\u4e0d\u540c":90,"\u4f46\u5b50\u53e5\u542b\u6709\u7684\u8bcd\u8bed\u6570\u53ef\u4ee5\u4e0d\u76f8\u7b49":92,"\u4f46\u5c3d\u91cf\u8bf7\u4fdd\u6301\u7f16\u8bd1\u548c\u8fd0\u884c\u4f7f\u7528\u7684cudnn\u662f\u540c\u4e00\u4e2a\u7248\u672c":83,"\u4f46\u5e76\u6ca1\u6709\u7ecf\u8fc7\u56de\u5f52\u6d4b\u8bd5":71,"\u4f46\u5e8f\u5217\u8f93\u51fa\u65f6":90,"\u4f46\u5f53\u8c03\u7528\u8fc7\u4e00\u6b21\u540e":2,"\u4f46\u622a\u65ad\u65f6\u673a\u4e0d\u540c":80,"\u4f46\u6240\u6709fork\u7684\u7248\u672c\u5e93\u7684\u6240\u6709\u5206\u652f\u90fd\u76f8\u5f53\u4e8e\u7279\u6027\u5206\u652f":71,"\u4f46\u662f":[80,90],"\u4f46\u662f\u53c8\u8fc7\u4e8e\u7410\u788e":55,"\u4f46\u662f\u5927\u90e8\u5206\u53c2\u6570\u662f\u4e3a\u5f00\u53d1\u8005\u63d0\u4f9b\u7684":105,"\u4f46\u662f\u5b50\u5e8f\u5217\u7684\u6570\u76ee\u5fc5\u987b\u4e00\u6837":90,"\u4f46\u662f\u5e76\u4e0d\u80fd\u4fdd\u8bc1\u53c2\u6570\u540c\u6b65\u66f4\u65b0":104,"\u4f46\u662f\u652f\u6301avx\u6307\u4ee4\u96c6":95,"\u4f46\u662f\u6bcf\u4e2a\u6837\u672c\u4ec5\u5305\u542b\u51e0\u4e2a\u8bcd":108,"\u4f46\u662f\u7a81\u7136\u6709\u4e00\u4e2a10000\u957f\u7684\u5e8f\u5217":80,"\u4f46\u662f\u865a\u62df\u7684\u4e0d\u4ec5\u4ec5\u662f":94,"\u4f46\u662f\u89e3\u91ca\u6027\u8bed\u8a00":54,"\u4f46\u662f\u8c03\u8bd5python\u4e2d\u4f7f\u7528\u7684\u52a8\u6001\u94fe\u63a5\u5e93\u4e0e\u76f4\u63a5\u8c03\u8bd5\u539f\u59cb\u4e8c\u8fdb\u5236\u76f8\u6bd4\u589e\u52a0\u4e86\u5f88\u591a\u590d\u6742\u5ea6":102,"\u4f46\u662fbatch":80,"\u4f46\u6709\u503c\u7684\u5730\u65b9\u5fc5\u987b\u4e3a1":[2,87],"\u4f46\u6709\u503c\u7684\u90e8\u5206\u53ef\u4ee5\u662f\u4efb\u4f55\u6d6e\u70b9\u6570":[2,87],"\u4f46\u7531\u4e8ecuda\u5e93\u901a\u5e38\u9700\u8981cento":86,"\u4f46\u8fd9\u4e2a\u5173\u7cfb\u53ef\u80fd\u4e0d\u6b63\u786e":2,"\u4f46\u9700\u6ce8\u610f\u53cd\u5411op\u6ca1\u6709":97,"\u4f46eigen":98,"\u4f4d\u7f6e":90,"\u4f4f":90,"\u4f5c\u4e3a\u4e0b\u4e00\u4e2a\u5b50\u53e5memory\u7684\u521d\u59cb\u72b6\u6001":90,"\u4f5c\u4e3a\u4f8b\u5b50\u6f14\u793a\u5982\u4f55\u914d\u7f6e\u590d\u6742\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u6a21\u578b":93,"\u4f5c\u4e3a\u53c2\u6570\u5c5e\u6027":97,"\u4f5c\u4e3a\u53c2\u6570\u7684id":82,"\u4f5c\u4e3a\u5b58\u50a8\u7cfb\u7edf":35,"\u4f5c\u4e3a\u5f53\u524d\u65f6\u523b\u8f93\u5165":92,"\u4f5c\u4e3a\u7c7b\u53e5\u67c4":54,"\u4f5c\u4e3a\u7f16\u8bd1\u5de5\u5177":83,"\u4f5c\u4e3a\u8f93\u51fa":93,"\u4f5c\u4e3aboot_layer\u4f20\u7ed9\u4e0b\u4e00\u4e2a\u5b50\u53e5\u7684memori":90,"\u4f5c\u7528":89,"\u4f60\u4e5f\u53ef\u4ee5\u4f7f\u7528\u8fd9\u4e09\u4e2a\u503c":122,"\u4f60\u4e5f\u53ef\u4ee5\u5148\u8df3\u8fc7\u672c\u6587\u7684\u89e3\u91ca\u73af\u8282":123,"\u4f60\u4e5f\u53ef\u4ee5\u7b80\u5355\u7684\u8fd0\u884c\u4ee5\u4e0b\u7684\u547d\u4ee4":121,"\u4f60\u4e5f\u53ef\u4ee5\u901a\u8fc7\u5728\u547d\u4ee4\u884c\u53c2\u6570\u4e2d\u589e\u52a0\u4e00\u4e2a\u53c2\u6570\u5982":122,"\u4f60\u53ea\u9700\u8981\u5728\u547d\u4ee4\u884c\u8f93\u5165\u4ee5\u4e0b\u547d\u4ee4":123,"\u4f60\u53ef\u4ee5\u4f7f\u7528":122,"\u4f60\u53ef\u4ee5\u5c06\u7f51\u7edc\u914d\u7f6e\u6210\u67d0\u4e9b\u5c42\u4f7f\u7528gpu\u8ba1\u7b97":108,"\u4f60\u53ef\u4ee5\u6267\u884c\u4e0a\u8ff0\u547d\u4ee4\u6765\u4e0b\u8f7d\u6240\u6709\u7684\u6a21\u578b\u548c\u5747\u503c\u6587\u4ef6":122,"\u4f60\u53ef\u4ee5\u901a\u8fc7\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u6765\u5f97\u5230resnet\u7f51\u7edc\u7684\u7ed3\u6784\u53ef\u89c6\u5316\u56fe":122,"\u4f60\u5c06\u4f1a\u770b\u5230\u4ee5\u4e0b\u7684\u6a21\u578b\u7ed3\u6784":121,"\u4f60\u5c06\u4f1a\u770b\u5230\u5982\u4e0b\u7ed3\u679c":122,"\u4f60\u5c06\u4f1a\u770b\u5230\u7279\u5f81\u5b58\u50a8\u5728":122,"\u4f60\u8fd8\u53ef\u4ee5\u901a\u8fc7\u8fd0\u884cdjango\u6846\u67b6\u76f4\u63a5\u6fc0\u6d3b\u5de5\u5177\u7684\u670d\u52a1\u5668":99,"\u4f60\u9700\u8981\u4e00\u4e9b\u66f4\u590d\u6742\u7684\u5355\u5143\u6d4b\u8bd5\u6765\u4fdd\u8bc1\u4f60\u5b9e\u73b0\u7684\u7f51\u7edc\u5c42\u662f\u6b63\u786e\u7684":96,"\u4f60\u9700\u8981\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u6307\u5b9a\u8bbe\u5907\u7684id\u53f7":108,"\u4f60\u9700\u8981\u5728\u914d\u7f6ecmake\u65f6\u5c06":96,"\u4f60\u9700\u8981\u628a\u8be5\u6587\u4ef6\u52a0\u5165":96,"\u4f7f\u5176\u8f6c\u53d8\u4e3a\u7ef4\u5ea6\u4e3ahidden_dim\u7684\u65b0\u5411\u91cf":123,"\u4f7f\u5f97\u5355\u5143\u6d4b\u8bd5\u6709\u4e00\u4e2a\u5e72\u51c0\u7684\u73af\u5883":77,"\u4f7f\u5f97\u642d\u6a21\u578b\u65f6\u66f4\u65b9\u4fbf":96,"\u4f7f\u68af\u5ea6\u7684\u63d0\u4ea4\u548c\u53c2\u6570\u7684\u66f4\u65b0\u6309\u7167\u987a\u5e8f\u65b9\u5f0f\u6267\u884c":104,"\u4f7f\u7528":[55,71,80,81,82,90,92,93,96,102,103,106,115,123],"\u4f7f\u75280\u53f7\u548c1\u53f7gpu\u8ba1\u7b97fc2\u5c42":108,"\u4f7f\u75280\u53f7gpu\u8ba1\u7b97fc2\u5c42":108,"\u4f7f\u752810\u4e2a\u88c1\u526a\u56fe\u50cf\u5757":122,"\u4f7f\u75281\u53f7gpu\u8ba1\u7b97fc3\u5c42":108,"\u4f7f\u75282\u53f7\u548c3\u53f7gpu\u8ba1\u7b97fc3\u5c42":108,"\u4f7f\u7528\u4e00\u4e2a\u5c3a\u5ea6\u4e3a":96,"\u4f7f\u7528\u4e00\u4e2a\u8bcd\u524d\u4e24\u4e2a\u8bcd\u548c\u540e\u4e24\u4e2a\u8bcd":80,"\u4f7f\u7528\u4e0a\u6587\u521b\u5efa\u7684yaml\u6587\u4ef6\u521b\u5efakubernet":110,"\u4f7f\u7528\u4e0b\u9762\u547d\u4ee4":35,"\u4f7f\u7528\u4e0b\u9762\u7684\u547d\u4ee4\u6765\u8fd0\u884c\u5b83":99,"\u4f7f\u7528\u4e86\u540c\u6837\u7684parameter\u548cbia":82,"\u4f7f\u7528\u4ee5\u4e0b\u547d\u4ee4\u8fdb\u884c\u6a21\u578b\u8bad\u7ec3":121,"\u4f7f\u7528\u52a8\u6001\u5e93":54,"\u4f7f\u7528\u53c2\u6570":[83,104],"\u4f7f\u7528\u540c\u6837\u7684\u8bad\u7ec3\u6570\u636eblock":34,"\u4f7f\u7528\u57fa\u4e8edocker\u5bb9\u5668\u7684\u7f16\u8bd1\u65b9\u5f0f":115,"\u4f7f\u7528\u591a\u5757\u663e\u5361\u8bad\u7ec3":80,"\u4f7f\u7528\u591a\u7ebf\u7a0b\u8bad\u7ec3":80,"\u4f7f\u7528\u5982\u4e0b\u547d\u4ee4":121,"\u4f7f\u7528\u5b66\u4e60\u5b8c\u6210\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u6a21\u578b\u751f\u6210\u5e8f\u5217":93,"\u4f7f\u7528\u5b83\u4f1a\u5f00\u542f\u4e00\u4e2ahttp\u670d\u52a1":102,"\u4f7f\u7528\u5bb9\u5668\u65b9\u5f0f\u8fd0\u884c\u8bad\u7ec3\u4efb\u52a1\u7684kubernet":111,"\u4f7f\u7528\u6211\u4eec\u4e4b\u524d\u6784\u9020\u7684\u955c\u50cf":110,"\u4f7f\u7528\u6570\u503c\u6cd5\u68c0\u6d4b\u68af\u5ea6\u6b63\u786e\u6027\u548c\u7a33\u5b9a\u6027":97,"\u4f7f\u7528\u6587\u6863":97,"\u4f7f\u7528\u663e\u5361\u8bad\u7ec3":80,"\u4f7f\u7528\u667a\u80fd\u6307\u9488\u7684\u539f\u56e0\u662f":55,"\u4f7f\u7528\u6848\u4f8b":107,"\u4f7f\u7528\u73af\u5883\u53d8\u91cf":104,"\u4f7f\u7528\u76f8\u5bf9\u8def\u5f84\u7684\u5f15\u7528\u65b9\u5f0f":55,"\u4f7f\u7528\u8005\u4e0d\u9700\u8981\u5173\u5fc3":106,"\u4f7f\u7528\u8005\u53ea\u9700\u8981\u5173\u6ce8\u4e8e\u8bbe\u8ba1rnn\u5728\u4e00\u4e2a\u65f6\u95f4\u6b65\u4e4b\u5185\u5b8c\u6210\u7684\u8ba1\u7b97":92,"\u4f7f\u7528\u8005\u53ef\u4ee5\u4f7f\u7528\u4e0b\u9762\u7684python\u811a\u672c\u6765\u8bfb\u53d6\u53c2\u6570\u503c":122,"\u4f7f\u7528\u8005\u65e0\u9700\u5173\u5fc3\u8fd9\u4e2a\u53c2\u6570":106,"\u4f7f\u7528\u8005\u901a\u5e38\u65e0\u9700\u5173\u5fc3":106,"\u4f7f\u7528\u8be5learning_rate_schedule\u65f6":82,"\u4f7f\u7528\u8fd9\u4e2a\u795e\u7ecf\u7f51\u7edc\u53ef\u4ee5\u5b8c\u6210\u5bf9\u65b0\u6570\u636e\u7684\u9884\u6d4b":34,"\u4f7f\u7528\u8fd9\u79cd\u65b9\u5f0f":90,"\u4f7f\u7528\u8fdc\u7a0b\u7a00\u758f\u65b9\u5f0f\u8bad\u7ec3\u65f6":96,"\u4f7f\u7528\u9759\u6001\u5e93\u548c\u52a8\u6001\u5e93\u96be\u5ea6\u5dee\u4e0d\u591a":54,"\u4f7f\u7528\u9884\u8bad\u7ec3\u7684\u6807\u51c6\u683c\u5f0f\u8bcd\u5411\u91cf\u6a21\u578b":121,"\u4f7f\u7528args\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u8bbe\u7f6e":2,"\u4f7f\u7528c":55,"\u4f7f\u7528c99\u505a\u63a5\u53e3":54,"\u4f7f\u7528c99\u800c\u4e0d\u4f7f\u7528c11\u7684\u539f\u56e0\u662f":54,"\u4f7f\u7528c99\u800c\u4e0d\u4f7f\u7528c89":54,"\u4f7f\u7528checkgrad\u6a21\u5f0f\u65f6\u7684\u53c2\u6570\u53d8\u5316\u5927\u5c0f":106,"\u4f7f\u7528cmake\u7684\u8bdd":102,"\u4f7f\u7528cpu\u4e24\u7ebf\u7a0b\u8ba1\u7b97fc4\u5c42":108,"\u4f7f\u7528cpu\u8ba1\u7b97fc4\u5c42":108,"\u4f7f\u7528docker":84,"\u4f7f\u7528docker\u5b89\u88c5\u548c\u8fd0\u884cpaddlepaddle\u53ef\u4ee5\u65e0\u9700\u8003\u8651":84,"\u4f7f\u7528docker\u5b89\u88c5\u8fd0\u884c":85,"\u4f7f\u7528docker\u5c31\u4e0d\u7528\u914d\u7f6e\u4ea4\u53c9\u7f16\u8bd1\u73af\u5883\u4e86":94,"\u4f7f\u7528docker\u6784\u5efapaddlepaddle\u7684\u6587\u6863":99,"\u4f7f\u7528docker\u7684\u60c5\u51b5\u4e0b":83,"\u4f7f\u7528eigen\u8fdb\u884c\u77e9\u9635\u8ba1\u7b97":115,"\u4f7f\u7528init":108,"\u4f7f\u7528lstm\u4f5c\u4e3aencod":90,"\u4f7f\u7528memory\u7684rnn\u5b9e\u73b0\u4fbf\u5982\u4e0b\u56fe\u6240\u793a":90,"\u4f7f\u7528model":108,"\u4f7f\u7528openblas\u7684\u955c\u50cf":84,"\u4f7f\u7528openblas\u8fdb\u884c\u77e9\u9635\u8ba1\u7b97":115,"\u4f7f\u7528paddlepaddl":123,"\u4f7f\u7528pip\u5b89\u88c5":85,"\u4f7f\u7528rdma\u8fd8\u662ftcp\u4f20\u8f93\u534f\u8bae":106,"\u4f7f\u7528regress":71,"\u4f7f\u7528swig\u53ea\u652f\u6301cpython\u89e3\u91ca\u5668":54,"\u4f7f\u7528swig\u9700\u8981\u591a\u8bed\u8a00\u7ed1\u5b9a\u7684\u5f00\u53d1\u4eba\u5458\u719f\u7ec3\u638c\u63e1swig\u914d\u7f6e":54,"\u4f7f\u7528void":54,"\u4f7f\u8be5\u5c42\u7684\u53c2\u6570\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u4fdd\u6301\u4e0d\u53d8":82,"\u4f86":90,"\u4f8b\u5982":[2,35,54,55,71,80,81,83,87,90,93,96,103,105,106,108,111,122,123],"\u4f8b\u5982\u4e0a\u6587\u7684pod":109,"\u4f8b\u5982\u4e0b\u56fe\u4e2d":102,"\u4f8b\u5982\u4f7f\u7528":80,"\u4f8b\u5982\u5bf9\u4e8ejava\u6216\u8005python":54,"\u4f8b\u5982\u5bf9\u4e8ejava\u6765\u8bf4":54,"\u4f8b\u5982\u5bf9\u4e8epython":54,"\u4f8b\u5982\u5c06\u7b2c\u4e00\u6761\u6570\u636e\u8f6c\u5316\u4e3a":90,"\u4f8b\u5982\u6587\u672c\u5206\u7c7b\u4e2d":90,"\u4f8b\u5982\u672c\u4f8b\u4e2d\u7684\u4e24\u4e2a\u7279\u5f81":90,"\u4f8b\u5982\u673a\u5668\u4e0a\u67094\u5757gpu":80,"\u4f8b\u5982c":54,"\u4f8b\u5982hostpath":109,"\u4f8b\u5982java\u4e0epython\u7684\u9519\u8bef\u5904\u7406\u662f\u76f4\u63a5\u6254\u51fa\u6765except":54,"\u4f8b\u5982output\u76ee\u5f55\u4e0b\u5c31\u5b58\u653e\u4e86\u8f93\u51fa\u7ed3\u679c":111,"\u4f8b\u5982python\u53ef\u4ee5\u4f7f\u7528":54,"\u4f8b\u5982python\u7684":54,"\u4f8b\u5982sigmoid":96,"\u4f8b\u5982sigmoid\u53d8\u6362":123,"\u4f8b\u5b50\u4e2d\u4e3a3\u4e2a":104,"\u4f8b\u5b50\u4e2d\u662f":96,"\u4f8b\u5b50\u4e2d\u662f0":96,"\u4f8b\u5b50\u4e2d\u662f100":96,"\u4f8b\u5b50\u4e2d\u662f4096":96,"\u4f8b\u5b50\u4e2d\u662f8192":96,"\u4f8b\u5b50\u4e2d\u662ffc":96,"\u4f8b\u5b50\u4e2d\u662fsoftmax":96,"\u4f8b\u5b50\u4f7f\u7528":109,"\u4f9bpaddlepaddle\u52a0\u8f7d":106,"\u4f9d\u636e\u662f\u5426\u5305\u542bkernel":97,"\u4f9d\u6b21\u7c7b\u63a8":71,"\u4f9d\u8d56":[83,86],"\u4f9d\u8d56\u73af\u5883\u5373\u53ef\u8fd0\u884c":84,"\u4f9d\u8d56libpython2":83,"\u4fbf\u4e8e\u5b58\u50a8\u8d44\u6e90\u7ba1\u7406\u548cpod\u5f15\u7528":109,"\u4fbf\u4e8e\u672c\u5730\u9a8c\u8bc1\u548c\u6d4b\u8bd5":109,"\u4fbf\u5229":90,"\u4fbf\u548c\u5355\u5c42rnn\u914d\u7f6e\u4e2d\u7684":90,"\u4fbf\u5b9c":90,"\u4fbf\u662f\u5c06\u9759\u6001\u5e93\u52a0\u5165jvm\u4e2d":54,"\u4fdd\u5b58\u6a21\u578b\u53c2\u6570\u7684\u76ee\u5f55":106,"\u4fdd\u5b58\u7684\u53c2\u6570\u4e5f\u662ffloat\u7c7b\u578b":82,"\u4fdd\u5b58\u7f51\u7edc\u5c42\u8f93\u51fa\u7ed3\u679c\u7684\u76ee\u5f55":106,"\u4fdd\u5b58\u9884\u6d4b\u7ed3\u679c\u7684\u6587\u4ef6\u540d":106,"\u4fdd\u6301\u5bbd\u9ad8\u6bd4\u7f29\u653e\u5230\u77ed\u8fb9\u4e3a256":122,"\u4fdd\u6301\u5c3d\u91cf\u5c11\u7684commit":95,"\u4fe1\u53f7\u6765\u81ea\u52a8\u7ec8\u6b62\u5b83\u542f\u52a8\u7684\u6240\u6709\u8fdb\u7a0b":104,"\u4fee\u590d\u6240\u6709bug\u540e":71,"\u4fee\u590ddocker\u7f16\u8bd1\u955c\u50cf\u95ee\u9898":71,"\u4fee\u590dubuntu":71,"\u4fee\u6539":[71,109,110],"\u4fee\u6539\u542f\u52a8\u811a\u672c\u540e":110,"\u4fee\u6539\u6210":71,"\u4fee\u6539\u6210\u66f4\u5feb\u7684\u7248\u672c":103,"\u4fee\u6539\u6587\u6863":100,"\u503c\u5f97\u6ce8\u610f\u7684\u662f":[90,95],"\u503c\u5f97\u6df1\u5165\u5206\u6790":103,"\u503c\u7c7b\u578b":108,"\u5047\u5982\u6211\u4eec\u662f\u4e09\u5206\u7c7b\u95ee\u9898":82,"\u5047\u8bbe":96,"\u5047\u8bbe\u60a8\u5df2\u7ecf\u5728\u5f53\u524d\u76ee\u5f55":84,"\u5047\u8bbe\u635f\u5931\u51fd\u6570\u662f":96,"\u5047\u8bbe\u7b2c\u4e00\u4e2alayer\u7684\u8f93\u51faa\u662f\u4e00\u4e2a":80,"\u5047\u8bbe\u8bcd\u5411\u91cf\u7ef4\u5ea6\u4e3a32":121,"\u504f\u7f6e\u53c2\u6570":122,"\u504f\u7f6e\u53c2\u6570\u7684\u5927\u5c0f":96,"\u505a\u4e00\u4e2a\u4ecb\u7ecd":98,"\u505a\u53ea\u8bfb\u6302\u8f7d":35,"\u505a\u5982\u4e0b\u51e0\u4e2a\u64cd\u4f5c":71,"\u505a\u63a5\u53e3":54,"\u505a\u68af\u5ea6\u68c0\u6d4b":97,"\u505a\u68c0\u67e5":97,"\u505c\u6b62\u4fdd\u5b58\u68c0\u67e5\u70b9\u7684\u7ebf\u7a0b":34,"\u505c\u6b62\u52a0\u8f7d\u6570\u636e":106,"\u505c\u7535":90,"\u5141\u8bb8\u5916\u7f51\u8bbf\u95ee\u8fd9\u4e2ahttp\u670d\u52a1":102,"\u5143\u7d20":89,"\u5143\u7d20\u4e4b\u95f4\u7684\u987a\u5e8f\u662f\u91cd\u8981\u7684\u8f93\u5165\u4fe1\u606f":89,"\u5148\u4ece\u5355\u7ebf\u7a0b\u5f00\u59cb":102,"\u5148\u5b9e\u73b0\u6a21\u578b\u63a8\u65ad\u7684api":55,"\u5148\u627e\u51fa\u53c2\u6570":81,"\u5148\u67e5\u770b\u4e00\u4e0b\u662f\u5426\u66fe\u7ecf\u5b89\u88c5\u8fc7paddl":77,"\u5148\u68c0\u67e5\u5173\u952e\u8def\u5f84\u7684\u6027\u80fd\u95ee\u9898":102,"\u5148\u8c03\u7528initializer\u51fd\u6570":123,"\u514b\u9686\u4e0b\u9762":117,"\u5168\u5bb6":90,"\u5168\u8fde\u63a5\u5c42":121,"\u5168\u8fde\u63a5\u5c42\u4ee5\u4e00\u4e2a\u7ef4\u5ea6\u4e3a":96,"\u5168\u8fde\u63a5\u5c42\u6743\u91cd":122,"\u5168\u8fde\u63a5\u5c42\u6ca1\u6709\u7f51\u7edc\u5c42\u914d\u7f6e\u7684\u8d85\u53c2\u6570":96,"\u5168\u8fde\u63a5\u5c42\u7684\u5b9e\u73b0\u4f4d\u4e8e":96,"\u5168\u8fde\u63a5\u5c42\u7684\u6bcf\u4e2a\u8f93\u51fa\u90fd\u8fde\u63a5\u5230\u4e0a\u4e00\u5c42\u7684\u6240\u6709\u7684\u795e\u7ecf\u5143\u4e0a":96,"\u5168\u8fde\u63a5\u5c42python\u5c01\u88c5\u7684\u4f8b\u5b50\u4e2d\u5305\u542b\u4e0b\u9762\u51e0\u6b65":96,"\u516c\u5f0f":84,"\u5171\u4eab\u4e00\u4e2aop\u5b9a\u4e49":97,"\u5171\u4eab\u540c\u4e00\u4e2akernel\u65f6":97,"\u5171\u4eab\u5b58\u50a8\u6302\u5728\u7684\u8def\u5f84":111,"\u5171\u670932":121,"\u5173\u4e8e\u5728paddlepaddle\u4e2d\u5982\u4f55\u4f7f\u7528eigen\u5e93":97,"\u5173\u4e8e\u65f6\u95f4\u5e8f\u5217":90,"\u5173\u4e8e\u6784\u5efa\u548c\u6d4b\u8bd5\u7684\u66f4\u591a\u4fe1\u606f":95,"\u5173\u4e8eavx":84,"\u5173\u4e8eeigen":98,"\u5173\u4e8elstm":81,"\u5173\u4e8epaddlepaddle\u7684\u5206\u5e03\u5f0f\u8bad\u7ec3":111,"\u5173\u4e8epaddlepaddle\u7684\u66f4\u591a\u4f7f\u7528\u65b9\u6cd5\u8bf7\u53c2\u8003":87,"\u5173\u4e8eunbound":92,"\u5173\u952e\u8bcd\u5305\u62ec":95,"\u5176\u4e2d":[2,54,71,80,82,87,93,96,102,115,117,121,122],"\u5176\u4e2d156\u548c285\u662f\u8fd9\u4e9b\u56fe\u50cf\u7684\u5206\u7c7b\u6807\u7b7e":122,"\u5176\u4e2d\u5305\u542b\u4e86\u7528\u6237\u7684\u8bad\u7ec3\u7a0b\u5e8f":104,"\u5176\u4e2d\u5305\u542b\u6240\u4f9d\u8d56\u7684\u6240\u6709\u7b2c\u4e09\u65b9\u5e93":116,"\u5176\u4e2d\u5305\u542b\u6240\u6709c":116,"\u5176\u4e2d\u5305\u542bpaddlepaddle\u7684c":116,"\u5176\u4e2d\u6587\u672c\u8f93\u5165\u7c7b\u578b\u5b9a\u4e49\u4e3a\u6574\u6570\u65f6\u5e8f\u7c7b\u578binteger_value_sequ":123,"\u5176\u4e2d\u6bcf\u4e00\u884c\u5bf9\u5e94\u4e00\u4e2a\u6570\u636e\u6587\u4ef6\u5730\u5740":1,"\u5176\u4e2d\u6bcf\u4e2a\u5143\u7d20\u662f\u53cc\u5c42\u5e8f\u5217\u4e2d\u6bcf\u4e2asubseq\u6700\u540e\u4e00\u4e2a":89,"\u5176\u4e2d\u6bcf\u884c\u6570\u636e\u4ee3\u8868\u4e00\u5f20\u56fe\u7247":2,"\u5176\u4e2d\u8f93\u5165\u56fe\u50cf\u7684\u989c\u8272\u901a\u9053\u987a\u5e8f\u4e3a":122,"\u5176\u4e2dcheckgrad\u4e3b\u8981\u4e3a\u5f00\u53d1\u8005\u4f7f\u7528":106,"\u5176\u4e2dmean\u548cstd\u662f\u8bad\u7ec3\u914d\u7f6e\u4e2d\u7684\u53c2\u6570":106,"\u5176\u4e2dvalue\u5373\u4e3asoftmax\u5c42\u7684\u8f93\u51fa":4,"\u5176\u4e2dx\u8868\u793a\u8f93\u5165\u6570\u636e\u662f\u4e00\u4e2a\u7ef4\u5ea6\u4e3a2\u7684\u7a20\u5bc6\u5411\u91cf":87,"\u5176\u4e3b\u8981\u63a5\u53e3\u5982\u4e0b":98,"\u5176\u4ed6\u4eba\u53ef\u4ee5\u590d\u73b0\u95ee\u9898\u4ee5\u4fbf\u5e2e\u52a9":94,"\u5176\u4ed6\u5185\u5b58\u6742\u9879":80,"\u5176\u4ed6\u5185\u5b58\u6742\u9879\u662f\u6307paddlepaddle\u672c\u8eab\u6240\u7528\u7684\u4e00\u4e9b\u5185\u5b58":80,"\u5176\u4ed6\u51fd\u6570\u5747\u8fd4\u56de":55,"\u5176\u4ed6\u53c2\u6570\u4f7f\u7528":2,"\u5176\u4ed6\u53c2\u6570\u8bf7\u53c2\u8003":123,"\u5176\u4ed6\u6240\u6709\u5c42\u90fd\u4f1a\u4f7f\u7528gpu\u8ba1\u7b97":108,"\u5176\u4ed6\u7528\u6237\u7684fork\u7248\u672c\u5e93\u5e76\u4e0d\u9700\u8981\u4e25\u683c\u9075\u5b88":71,"\u5176\u4ed6\u7684\u4f9d\u8d56\u8f6f\u4ef6":83,"\u5176\u4ed6\u914d\u7f6e\u53c2\u6570":[115,116],"\u5176\u4ed6\u9ad8\u7ea7\u529f\u80fd\u5305\u62ec\u5b9a\u4e49\u591a\u4e2amemori":93,"\u5176\u4f1a\u81ea\u52a8\u88ab\u52a0\u5165\u7f16\u8bd1\u5217\u8868":96,"\u5176\u4f59\u884c\u662f":121,"\u5176\u4f5c\u7528\u662f\u5c06\u6570\u636e\u4f20\u5165\u5185\u5b58\u6216\u663e\u5b58":1,"\u5176\u5185\u90e8\u7684\u6587\u4ef6\u4e5f\u4f1a\u968f\u4e4b\u6d88\u5931":109,"\u5176\u5305\u62ec\u4e24\u4e2a\u51fd\u6570":123,"\u5176\u53c2\u6570\u5982\u4e0b":2,"\u5176\u547d\u4ee4\u5982\u4e0b":102,"\u5176\u5b83\u90e8\u5206\u548c\u903b\u8f91\u56de\u5f52\u7f51\u7edc\u7ed3\u6784\u4e00\u81f4":123,"\u5176\u5b83layer\u7684\u8f93\u51fa":92,"\u5176\u5b9e\u4e5f\u662f\u548c\u6bcf\u4e2amini":80,"\u5176\u63d0\u4f9b\u5e94\u7528\u90e8\u7f72":109,"\u5176\u6b21":[2,90,123],"\u5176\u8bf4\u660e\u5982\u4e0b":90,"\u5176\u8f93\u51fa\u88ab\u7528\u4f5cmemory\u7684\u521d\u59cb\u503c":93,"\u5176name\u7531\u53c2\u6570":81,"\u5177\u4f53\u4f7f\u7528\u65b9\u6cd5\u4e3a":[55,80],"\u5177\u4f53\u505a\u6cd5\u8bf7\u53c2\u8003":94,"\u5177\u4f53\u539f\u56e0\u53c2\u8003":55,"\u5177\u4f53\u53ef\u4ee5\u53c2\u8003":[2,80,96],"\u5177\u4f53\u53ef\u53c2\u8003\u6587\u6863":92,"\u5177\u4f53\u60c5\u51b5\u56e0\u4eba\u800c\u5f02":103,"\u5177\u4f53\u64cd\u4f5c\u5982\u4e0b":77,"\u5177\u4f53\u6d41\u7a0b\u5982\u4e0b":123,"\u5177\u4f53\u7684\u683c\u5f0f\u8bf4\u660e":2,"\u5177\u4f53\u7684\u89e3\u51b3\u65b9\u6cd5\u662f":77,"\u5177\u4f53\u8bf7\u53c2\u7167\u793a\u4f8b":122,"\u5177\u4f53\u8bf7\u53c2\u8003":[2,55,95],"\u5177\u4f53\u8bf7\u89c1":95,"\u5177\u6709\u76f8\u540c\u7684\u7ed3\u679c\u4e86":90,"\u5185":93,"\u5185\u5b58":103,"\u5185\u5b58\u4e0d\u8db3":78,"\u5185\u5b58\u5bb9\u9650\u9608\u503c":106,"\u5185\u5bb9":[97,123],"\u5185\u5bb9\u5982\u4e0b":110,"\u5185\u5c42inner_step\u7684recurrent_group\u548c\u5355\u5c42\u5e8f\u5217\u7684\u51e0\u4e4e\u4e00\u6837":90,"\u5185\u5df2\u7ecf\u5305\u542bpaddlepaddle\u7684\u6267\u884c\u7a0b\u5e8f\u4f46\u662f\u8fd8\u6ca1\u4e0a\u8ff0\u529f\u80fd":111,"\u5185\u90e8":111,"\u5185\u90e8\u9a71\u52a8python\u89e3\u91ca\u5668\u8fdb\u884c\u6a21\u578b\u914d\u7f6e\u89e3\u6790\u548c\u6570\u636e\u8bfb\u53d6":54,"\u518d\u4ee5":97,"\u518d\u5199\u5165\u7f51\u7edc\u53c2\u6570":82,"\u518d\u5728\u6bcf\u4e00\u4e2aapi\u4e2d\u81ea\u5df1\u68c0\u67e5\u7c7b\u578b":54,"\u518d\u57fa\u4e8e":71,"\u518d\u5b89\u88c5":[77,86],"\u518d\u5bf9\u6bcf\u4e00\u4e2a\u5355\u5c42\u65f6\u95f4\u5e8f\u5217\u8fdb\u884c\u5904\u7406":90,"\u518d\u5bf9\u6bcf\u4e00\u53e5\u8bdd\u7684\u7f16\u7801\u5411\u91cf\u7528lstm\u7f16\u7801\u6210\u4e00\u4e2a\u6bb5\u843d\u7684\u5411\u91cf":90,"\u518d\u5bf9\u8fd9\u4e2a\u6bb5\u843d\u5411\u91cf\u8fdb\u884c\u5206\u7c7b":90,"\u518d\u5c06\u66f4\u65b0\u540e\u7684\u53c2\u6570\u4e0b\u53d1\u5230\u6bcf\u4e2a\u8ba1\u7b97\u8282\u70b9":104,"\u518d\u5f00\u542f\u591a\u7ebf\u7a0b":102,"\u518d\u6307\u5b9a":83,"\u518d\u68c0\u67e5\u5176\u4ed6\u90e8\u5206\u7684\u6027\u80fd\u95ee\u9898":102,"\u518d\u6b21\u5bf9\u4ee3\u7801\u8fdb\u884c\u6027\u80fd\u5206\u6790":103,"\u518d\u6b21\u8fdb\u884c\u6027\u80fd\u5206\u6790":102,"\u518d\u7528\u8fd9\u4e2a\u68af\u5ea6\u53bb\u548c":96,"\u518d\u901a\u8fc7\u51fd\u6570":111,"\u5197\u4f59\u7b49\u529f\u80fd":109,"\u5199\u4ee3\u7801":54,"\u5199\u5165\u5feb\u7167\u6570\u636e":34,"\u5199\u68af\u5ea6\u68c0\u67e5\u5355\u5143\u6d4b\u8bd5\u662f\u4e00\u4e2a\u9a8c\u8bc1\u65b0\u5b9e\u73b0\u7684\u5c42\u662f\u5426\u6b63\u786e\u7684\u76f8\u5bf9\u7b80\u5355\u7684\u529e\u6cd5":96,"\u5199\u7684":102,"\u51c6\u5907":90,"\u51c6\u5907\u60a8\u7684\u8ba1\u7b97\u96c6\u7fa4":104,"\u51c6\u5907\u8bad\u7ec3\u6570\u636e":104,"\u51c6\u5907\u8bad\u7ec3\u6570\u636e\u548c\u9a8c\u8bc1\u6570\u636e\u96c6":104,"\u51c6\u5907\u9884\u6d4b\u6570\u636e":4,"\u51cf\u5c0f\u5e8f\u5217\u7684\u957f\u5ea6":80,"\u51cf\u5c0f\u8fd9\u4e2a\u5185\u5b58\u6c60\u5373\u53ef\u51cf\u5c0f\u5185\u5b58\u5360\u7528":80,"\u51cf\u5c0fbatch":80,"\u51e0\u53f0\u5230\u51e0\u5343\u53f0\u89c4\u6a21":104,"\u51fa\u53bb\u73a9":90,"\u51fa\u5dee":90,"\u51fa\u6765":90,"\u51fa\u73b0":77,"\u51fa\u73b0\u4ee5\u4e0b\u9519\u8bef":82,"\u51fa\u73b0\u8be5\u9519\u8bef\u7684\u539f\u56e0\u4e00\u822c\u662f\u7528\u6237\u5bf9\u4e0d\u540clayer\u7684\u53c2\u6570":81,"\u51fa\u73b0\u8fd9\u4e2a\u95ee\u9898\u7684\u4e3b\u8981\u539f\u56e0\u662f":[77,86],"\u51fd\u6570":[2,93,96,102,103],"\u51fd\u6570\u4e2d\u4f7f\u7528":2,"\u51fd\u6570\u4e2d\u64cd\u4f5c\u7684\u91cd\u8981\u53d8\u91cf\u7684\u8be6\u7ec6\u89e3\u91ca":97,"\u51fd\u6570\u5047\u8bbe":93,"\u51fd\u6570\u52a0\u5230\u4ee3\u7801\u4e2d":103,"\u51fd\u6570\u5373\u53ef\u5b8c\u6210\u8f6c\u6362":35,"\u51fd\u6570\u53ea\u5173\u6ce8\u4e8ernn\u4e00\u4e2a\u65f6\u95f4\u6b65\u4e4b\u5185\u7684\u8ba1\u7b97":92,"\u51fd\u6570\u540d":102,"\u51fd\u6570\u540d\u4e3a":55,"\u51fd\u6570\u547d\u540d":54,"\u51fd\u6570\u5b9a\u4e49\u8f93\u5165":97,"\u51fd\u6570\u5b9e\u9645\u4f7f\u7528\u7684\u603b\u65f6\u95f4":102,"\u51fd\u6570\u5c31\u662f\u6839\u636e\u8be5\u673a\u5236\u914d\u7f6e\u7684":2,"\u51fd\u6570\u5f97\u5230\u7684\u68af\u5ea6\u53bb\u5bf9\u6bd4":96,"\u51fd\u6570\u5fc5\u987b\u5148\u8c03\u7528\u57fa\u7c7b\u4e2d\u7684\u51fd\u6570":96,"\u51fd\u6570\u5fc5\u987b\u8fd4\u56de\u4e00\u4e2a\u6216\u591a\u4e2alayer\u7684\u8f93\u51fa":92,"\u51fd\u6570\u603b\u65f6\u95f4":102,"\u51fd\u6570\u6307\u51fa\u4e86\u5728\u8bad\u7ec3\u65f6\u9700\u8981\u4ece\u53c2\u6570\u670d\u52a1\u5668\u53d6\u51fa\u7684\u884c":96,"\u51fd\u6570\u6765\u5c06\u4fe1\u606f\u8f93\u51fa\u5230\u754c\u9762\u4e2d":103,"\u51fd\u6570\u67e5\u8be2\u8f6f\u4ef6\u5305\u76f8\u5173api\u8bf4\u660e":4,"\u51fd\u6570\u7684":2,"\u51fd\u6570\u7684\u5b9e\u73b0\u662f\u6b63\u786e\u7684":96,"\u51fd\u6570\u7684\u5f00\u5934\u5fc5\u987b\u8c03\u7528":96,"\u51fd\u6570\u7684\u603b\u5171\u8017\u65f6\u5f88\u957f":102,"\u51fd\u6570\u7684\u8c03\u7528\u6b21\u6570":102,"\u51fd\u6570\u91cc\u5b9e\u73b0":97,"\u5206\u4e3a\u597d\u8bc4":123,"\u5206\u522b\u4e3a":121,"\u5206\u522b\u4ece\u8bcd\u8bed\u548c\u53e5\u5b50\u7ea7\u522b\u7f16\u7801\u8f93\u5165\u6570\u636e":92,"\u5206\u522b\u4f7f\u7528\u5355\u53cc\u5c42rnn\u4f5c\u4e3a\u7f51\u7edc\u914d\u7f6e\u7684\u6a21\u578b":90,"\u5206\u522b\u5b9a\u4e49\u5b50\u53e5\u7ea7\u522b\u548c\u8bcd\u8bed\u7ea7\u522b\u4e0a\u9700\u8981\u5b8c\u6210\u7684\u8fd0\u7b97":92,"\u5206\u522b\u662f":89,"\u5206\u522b\u662frnn\u72b6\u6001\u548c\u8f93\u5165\u7684\u53d8\u6362\u77e9\u9635":93,"\u5206\u522b\u662fsentences\u548clabel":90,"\u5206\u522b\u662fwords\u548clabel":90,"\u5206\u522b\u8ba1\u7b97\u6bcf\u4e2a\u53c2\u6570\u7684\u68af\u5ea6":96,"\u5206\u522b\u8fdb\u884c\u5e8f\u5217\u64cd\u4f5c":90,"\u5206\u5e03\u5f0f\u5b58\u50a8\u670d\u52a1":34,"\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf":109,"\u5206\u5e03\u5f0f\u8ba1\u7b97\u5e73\u53f0\u901a\u5e38\u4f1a\u901a\u8fc7api\u6216\u8005\u73af\u5883\u53d8\u91cf\u63d0\u4f9b\u4efb\u52a1\u8fd0\u884c\u9700\u8981\u7684\u53c2\u6570":104,"\u5206\u5e03\u5f0f\u8bad\u7ec3\u67b6\u6784\u5982\u4e0b\u56fe\u6240\u793a":104,"\u5206\u6210\u4e24\u90e8\u5206":2,"\u5206\u652f":[71,95],"\u5206\u652f\u4e00\u65e6\u5efa\u7acb":71,"\u5206\u652f\u4e0a":95,"\u5206\u652f\u4e0a\u521b\u5efa\u65b0\u5206\u652f":95,"\u5206\u652f\u4e2d":71,"\u5206\u652f\u4e3a\u5f00\u53d1":71,"\u5206\u652f\u4e3a\u6bcf\u4e00\u6b21release\u65f6\u5efa\u7acb\u7684\u4e34\u65f6\u5206\u652f":71,"\u5206\u652f\u4e3a\u7a33\u5b9a":71,"\u5206\u652f\u529f\u80fd\u7684\u5c01\u95ed":71,"\u5206\u652f\u5408\u5165":71,"\u5206\u652f\u5408\u5165master\u5206\u652f":71,"\u5206\u652f\u540c\u6b65\u4e3b\u7248\u672c\u5e93\u7684":71,"\u5206\u652f\u540d":95,"\u5206\u652f\u540d\u4e3a":71,"\u5206\u652f\u5b58\u5728\u7684\u65f6\u5019":71,"\u5206\u652f\u6d3e\u751f\u51fa\u65b0\u7684\u5206\u652f":71,"\u5206\u652f\u7528\u6765\u6d4b\u8bd5\u53ea\u9700\u8981\u8ba1\u7b97\u4e00\u4e2a\u8f93\u5165\u68af\u5ea6\u7684\u60c5\u51b5":97,"\u5206\u652f\u7684\u7248\u672c\u90fd\u662f\u7ecf\u8fc7\u5355\u5143\u6d4b\u8bd5\u548c\u56de\u5f52\u6d4b\u8bd5\u7684\u7248\u672c":71,"\u5206\u652f\u7684\u7248\u672c\u90fd\u7ecf\u8fc7\u5355\u5143\u6d4b\u8bd5":71,"\u5206\u652f\u7684\u76ee\u6807\u673a\u7248openblas\u5e93":115,"\u5206\u652f\u89c4\u8303":95,"\u5206\u6790\u5f97\u5230\u7684\u4fe1\u606f\u7528\u4e8e\u534f\u52a9\u8fdb\u884c\u7a0b\u5e8f\u7684\u4f18\u5316":103,"\u5206\u7247":34,"\u5206\u7c7b\u6210\u6b63\u9762\u60c5\u7eea\u548c\u8d1f\u9762\u60c5\u7eea\u4e24\u7c7b":2,"\u5206\u7c7b\u9519\u8bef\u7387\u548c\u6a21\u578b\u5927\u5c0f\u7531\u4e0b\u8868\u7ed9\u51fa":122,"\u5206\u8bcd\u5e8f\u5217\u7684\u5f00\u59cb":121,"\u5206\u8bcd\u5e8f\u5217\u7684\u7ed3\u675f":121,"\u5206\u8bcd\u98ce\u683c\u5982\u4e0b":121,"\u5206\u914d\u5230\u5f53\u524d\u6570\u636e\u5757\u6837\u672c\u6570\u7684\u56db\u5206\u4e4b\u4e00":106,"\u5206\u9694":121,"\u5207\u6362\u5230":95,"\u5207\u6362\u5230\u6240\u5efa\u5206\u652f":95,"\u5217\u540d":102,"\u5217\u8868\u5982\u4e0b":[2,87],"\u5219\u4e0d\u5728\u4e4e\u5185\u5b58\u6682\u5b58\u591a\u5c11\u6761\u6570\u636e":2,"\u5219\u4e0d\u9700\u8981\u91cd\u5199\u8be5\u51fd\u6570":96,"\u5219\u4f1a\u4f7f\u7528openblas\u4f5c\u4e3ablas\u5e93":83,"\u5219\u4f1a\u9884\u5148\u8bfb\u53d6\u5168\u90e8\u6570\u636e\u5230\u5185\u5b58\u4e2d":2,"\u5219\u4f7f\u7528":116,"\u5219\u4f7f\u7528\u540c\u6b65\u8bad\u7ec3":106,"\u5219\u4f7f\u7528\u542f\u52a8\u53c2\u6570\u5b9a\u4e49\u7684\u521d\u59cb\u5316\u65b9\u6cd5\u521d\u59cb\u5316\u53c2\u6570":34,"\u5219\u4f7f\u7528\u8be5\u53c2\u6570\u4f5c\u4e3a\u9ed8\u8ba4\u503c":106,"\u5219\u53ef\u8bbe\u7f6e":[115,116,117],"\u5219\u5b57\u4e0e\u5b57\u4e4b\u95f4\u7528\u7a7a\u683c\u5206\u9694":123,"\u5219\u5e76\u4e0d\u4f1a\u7b49\u5f85\u6240\u6709trainer\u63d0\u4ea4\u68af\u5ea6\u624d\u66f4\u65b0\u53c2\u6570":104,"\u5219\u5ffd\u7565":34,"\u5219\u603b\u4f1a\u663e\u793a\u963b\u9694\u6458\u8981\u4fe1\u606f":106,"\u5219\u628a\u53e6\u4e00\u4e2a\u6162\u901f\u7684kill\u6389":34,"\u5219\u63a8\u8350\u5927\u4e8e\u8bad\u7ec3\u65f6batch":2,"\u5219\u662f\u5e26gui\u7684nvidia\u53ef\u89c6\u5316\u6027\u80fd\u5206\u6790\u5de5\u5177":103,"\u5219\u663e\u793a\u963b\u9694\u6027\u80fd\u7684\u6458\u8981\u4fe1\u606f":106,"\u5219\u76f4\u63a5\u5f15\u5165\u53e6\u4e00\u79cd\u7c7b\u578b\u7684\u5934\u6587\u4ef6":55,"\u5219\u8bbe\u7f6e\u6210":[115,117],"\u5219\u9700\u8981\u4f7f\u7528\u7b49\u4e8e\u6743\u91cd\u53c2\u6570\u89c4\u6a21\u5927\u7ea65\u500d\u7684\u5185\u5b58":80,"\u5219\u9700\u8981\u5206\u522b\u7f16\u8bd1\u771f\u673a\u548c\u6a21\u62df\u5668\u7248\u672c":116,"\u5219\u9700\u8981\u56de\u6eda\u5230\u4e0a\u4e00\u4e2a\u68c0\u67e5\u70b9":34,"\u5219\u9700\u8981\u5728\u672c\u673a\u5b89\u88c5\u4e0b\u9762\u7ae0\u8282\u5217\u51fa\u7684":83,"\u5219\u9700\u8981\u624b\u52a8\u62f7\u8d1d\u5c5e\u4e8e\u6bcf\u4e2atrainer\u8282\u70b9\u7684\u8bad\u7ec3\u6570\u636e\u5230\u5bf9\u5e94\u7684\u8282\u70b9\u4e0a":104,"\u5219\u9700\u8981\u914d\u7f6e":109,"\u521b\u5efa\u4e00\u4e2a":88,"\u521b\u5efa\u4e00\u4e2akubernet":111,"\u521b\u5efa\u53cd\u5411op\u5bf9\u5e94\u7684\u524d\u5411op":97,"\u521b\u5efa\u5e76\u5207\u6362\u5230\u65b0\u5206\u652f":95,"\u521b\u5efa\u6210\u529f\u540e":111,"\u521b\u5efa\u65e5\u5fd7\u76ee\u5f55":104,"\u521b\u5efagener":2,"\u521d\u59cb\u5316\u4e4b\u540e":4,"\u521d\u59cb\u5316\u504f\u7f6e\u5411\u91cf":96,"\u521d\u59cb\u5316\u65f6\u8c03\u7528\u7684\u51fd\u6570":2,"\u521d\u59cb\u5316\u6743\u91cd\u8868":96,"\u521d\u59cb\u5316\u6a21\u578b\u7684\u8def\u5f84":106,"\u521d\u59cb\u5316\u6a21\u578b\u7684\u8def\u5f84\u914d\u7f6e\u4e3a":121,"\u521d\u59cb\u5316\u7236\u7c7b":96,"\u521d\u59cb\u5316biases_":96,"\u521d\u59cb\u5316paddlepaddle\u73af\u5883":4,"\u521d\u59cb\u72b6\u6001":92,"\u5220\u9664":95,"\u5220\u9664\u78c1\u76d8\u76ee\u5f55\u4e2d\u4e0d\u662f\u5f53\u524duuid\u7684\u5feb\u7167\u6587\u4ef6":34,"\u5224\u65ad\u662f\u5426\u5b89\u88c5\u6210\u529f":116,"\u5229\u7528\u5206\u5e03\u5f0f\u8bad\u7ec3\u9a7e\u9a6d\u66f4\u591a\u7684\u8ba1\u7b97\u8d44\u6e90":80,"\u5229\u7528\u5355\u8bcdid\u67e5\u627e\u8be5\u5355\u8bcd\u5bf9\u5e94\u7684\u8fde\u7eed\u5411\u91cf":123,"\u5229\u7528\u66f4\u591a\u7684\u8ba1\u7b97\u8d44\u6e90\u53ef\u4ee5\u5206\u4e3a\u4ee5\u4e0b\u51e0\u4e2a\u65b9\u5f0f\u6765\u8fdb\u884c":80,"\u5229\u7528\u8fd9\u79cd\u7279\u6027":92,"\u5229\u7528\u903b\u8f91\u56de\u5f52\u6a21\u578b\u5bf9\u8be5\u5411\u91cf\u8fdb\u884c\u5206\u7c7b":123,"\u5229\u7528kubernetes\u80fd\u65b9\u4fbf\u5730\u7ba1\u7406\u8de8\u673a\u5668\u8fd0\u884c\u5bb9\u5668\u5316\u7684\u5e94\u7528":109,"\u5229\u843d":90,"\u522b\u4eba\u5e2e\u4e86\u5fd9":95,"\u522b\u5fd8\u4e86":94,"\u5230":[34,77,93],"\u5230\u672c\u5730":95,"\u5230\u6b64":97,"\u5236\u4f5c\u65b0\u955c\u50cf\u6765\u5b8c\u6210\u4ee5\u4e0a\u7684\u5de5\u4f5c":111,"\u5236\u4f5cpaddlepaddle\u955c\u50cf":111,"\u5237\u7259":90,"\u524d\u4e00\u7bc7\u6587\u7ae0\u4ecb\u7ecd\u4e86\u5982\u4f55\u5728kubernetes\u96c6\u7fa4\u4e0a\u542f\u52a8\u4e00\u4e2a\u5355\u673apaddlepaddle\u8bad\u7ec3\u4f5c\u4e1a":111,"\u524d\u53f0":90,"\u524d\u5411\u4f20\u64ad":96,"\u524d\u5411\u4f20\u64ad\u7ed9\u5b9a\u8f93\u5165":96,"\u524d\u5411\u548c\u540e\u5411":96,"\u524d\u5411op\u5355\u5143\u6d4b\u8bd5\u7ee7\u627f\u81ea":97,"\u524d\u5411op\u5b9e\u73b0\u5b8c\u6210":97,"\u524d\u5411operator\u5355\u6d4b":97,"\u524d\u8005\u5728":80,"\u524d\u8005\u5b58\u50a8op\u7684\u8f93\u5165\u8f93\u51fa\u548c\u53c2\u6570\u5c5e\u6027":97,"\u524d\u8005\u622a\u65ad\u53ef\u5b66\u4e60\u53c2\u6570\u7684\u68af\u5ea6":80,"\u524d\u8005op\u7684\u5b9a\u4e49\u7ee7\u627f\u81ea":97,"\u524d\u81ea\u52a8\u68c0\u67e5\u4e00\u4e9b\u57fa\u672c\u4e8b\u5b9c":95,"\u524d\u9700\u8981\u5b89\u88c5":102,"\u524d\u9988":104,"\u5269\u4e0b\u7684pass\u4f1a\u76f4\u63a5\u4ece\u5185\u5b58\u91cc":2,"\u529f\u80fd":44,"\u529f\u80fd\u7684\u6b63\u786e\u6027\u5305\u62ec\u9a8c\u8bc1paddlepaddle\u76ee\u524d\u7684":71,"\u52a0\u4e0a\u504f\u7f6e\u5411\u91cf":96,"\u52a0\u4e86l2\u6b63\u5219\u548c\u68af\u5ea6\u622a\u65ad":123,"\u52a0\u5165":103,"\u52a0\u6743\u548c\u7528\u6765\u751f\u6210":93,"\u52a0\u6743\u7f16\u7801\u5411\u91cf":93,"\u52a0\u8f7d\u5177\u4f53\u7f51\u7edc\u53c2\u6570":82,"\u52a0\u8f7d\u9884\u8bad\u7ec3\u53c2\u6570":82,"\u52a0\u8f7dtest":106,"\u52a0\u901f\u7f16\u8bd1":83,"\u52a0\u901fpaddlepaddle\u8bad\u7ec3\u53ef\u4ee5\u8003\u8651\u4ece\u4ee5\u4e0b\u51e0\u4e2a\u65b9\u9762":80,"\u52a8\u6001\u5e93":54,"\u52a9\u624b":96,"\u5305":102,"\u5305\u542b20\u4e2a\u8bad\u7ec3\u6837\u4f8b":121,"\u5305\u542b3\u4e2a\u5c5e\u6027":121,"\u5305\u542b\u4e86\u67d0\u79cd\u7c7b\u578b\u7684\u7c7b\u578b\u5b9a\u4e49\u548c\u66b4\u9732\u7684\u5168\u90e8\u51fd\u6570":55,"\u5305\u542b\u4f46\u4e0d\u9650\u4e8e":83,"\u5305\u542b\u6d4b\u8bd5\u6570\u636e\u96c6\u7684\u76ee\u5f55":104,"\u5305\u542b\u8bad\u7ec3\u6570\u636e\u7684\u76ee\u5f55":104,"\u5305\u542b\u8fd9\u4e2a\u51fd\u6570\u8c03\u7528\u5176\u4ed6\u51fd\u6570\u7684\u65f6\u95f4":102,"\u5305\u542bkernel\u7684op\u548c\u4e0d\u5305\u542bkernel\u7684op":97,"\u5305\u62ec":[35,104,106,123],"\u5305\u62ec\u4ee5\u4e0b\u4e24\u79cd":2,"\u5305\u62ec\u5b57\u7b26\u4e32\u5206\u914d":80,"\u5305\u62ec\u6743\u91cdw\u548c\u504f\u7f6eb":34,"\u5305\u62ec\u751f\u6210cpu":83,"\u5305\u62ec\u795e\u7ecf\u7f51\u7edc\u62d3\u6251\u7ed3\u6784":87,"\u5305\u62ec\u7b80\u5355\u7684":123,"\u5305\u62ecbool":108,"\u5305\u62eclinux":115,"\u5305\u7684\u65b9\u6cd5\u662f":77,"\u533a\u522b\u662f\u540c\u65f6\u5904\u7406\u4e86\u4e24\u4e2a\u8f93\u5165":90,"\u533a\u522b\u662frnn\u4f7f\u7528\u4e24\u5c42\u5e8f\u5217\u6a21\u578b":90,"\u5341\u4e00":90,"\u534e\u6da6\u4e07\u5bb6":90,"\u534f\u540c\u5b8c\u6210releas":71,"\u5355\u4e2a\u503c":35,"\u5355\u4f4d\u662fmb":106,"\u5355\u5143\u6d4b\u8bd5":[94,98],"\u5355\u5143\u6d4b\u8bd5\u4f1a\u5f15\u7528site":77,"\u5355\u5143\u6d4b\u8bd5\u4f1a\u88ab\u81ea\u52a8\u52a0\u5165\u5de5\u7a0b\u8fdb\u884c\u7f16\u8bd1":97,"\u5355\u5143\u6d4b\u8bd5\u5728\u5185\u7684\u6240\u6709\u5355\u5143\u6d4b\u8bd5":94,"\u5355\u5143\u6d4b\u8bd5checkgrad_ep":105,"\u5355\u53cc\u5c42\u5e8f\u5217\u7684\u53e5\u5b50\u662f\u4e00\u6837\u7684":90,"\u5355\u53cc\u5c42rnn":91,"\u5355\u5c42":92,"\u5355\u5c42\u4e0d\u7b49\u957frnn":90,"\u5355\u5c42\u548c\u53cc\u5c42\u5e8f\u5217\u7684\u4f7f\u7528\u548c\u793a\u4f8b2\u4e2d\u7684\u793a\u4f8b\u7c7b\u4f3c":90,"\u5355\u5c42\u5e8f\u5217":89,"\u5355\u5c42\u5e8f\u5217\u7684\u6bcf\u4e2a\u5143\u7d20":89,"\u5355\u5c42\u5e8f\u5217\u7b2ci\u4e2a\u5143\u7d20":89,"\u5355\u5c42\u6216\u53cc\u5c42":89,"\u5355\u5c42\u65f6\u95f4\u5e8f\u5217":90,"\u5355\u5c42rnn":[90,92],"\u5355\u5c42rnn\u548c\u53cc\u5c42rnn\u7684\u7f51\u7edc\u914d\u7f6e":90,"\u5355\u673acpu\u8bad\u7ec3":80,"\u5355\u673agpu\u8bad\u7ec3":80,"\u5355\u6b65\u51fd\u6570":93,"\u5355\u6b65\u51fd\u6570\u548c\u8f93\u51fa\u51fd\u6570\u5728":93,"\u5355\u6b65\u51fd\u6570\u548c\u8f93\u51fa\u51fd\u6570\u90fd\u975e\u5e38\u7b80\u5355":93,"\u5355\u6b65\u51fd\u6570\u7684\u5b9e\u73b0\u5982\u4e0b\u6240\u793a":93,"\u5355\u6d4b\u5305\u62ec\u5bf9\u6bd4\u524d\u5411op\u4e0d\u540c\u8bbe\u5907":97,"\u5355\u70b9\u6545\u969c":34,"\u5355\u7eaf\u7684":102,"\u5355\u8fdb\u5355\u51fa":92,"\u5360\u4f4d\u7b26":121,"\u5360\u7528\u4e8617":102,"\u536b\u751f":90,"\u5373":[55,80,81,97,99,111,123],"\u5373\u4e00\u4e2a\u5c06\u5355\u8bcd\u5b57\u7b26\u4e32\u6620\u5c04\u5230\u5355\u8bcdid\u7684\u5b57\u5178":2,"\u5373\u4e0a\u8ff0\u4ee3\u7801\u4e2d\u7684\u7b2c19\u884c":90,"\u5373\u4e0d\u5141\u8bb8\u5728":97,"\u5373\u4e0d\u8981\u5c06\u6bcf\u4e00\u4e2a\u6837\u672c\u90fd\u653e\u5165train":2,"\u5373\u4e0d\u9700\u8981\u4f7f\u7528memori":90,"\u5373\u4e3a\u4e00\u4e2a\u65f6\u95f4\u6b65":90,"\u5373\u4e3a\u5355\u5c42rnn\u5e8f\u5217\u7684\u4f7f\u7528\u4ee3\u7801":90,"\u5373\u4e3a\u65f6\u95f4\u5e8f\u5217\u7684\u8f93\u5165":90,"\u5373\u4e3a\u8fd9\u4e2a\u53cc\u5c42rnn\u7684\u7f51\u7edc\u7ed3\u6784":90,"\u5373\u4e3a\u8fd9\u4e2a\u6570\u636e\u6587\u4ef6\u7684\u540d\u5b57":2,"\u5373\u4e8c\u7ef4\u6570\u7ec4":90,"\u5373\u4f7f\u7528":[55,81],"\u5373\u4f7f\u7528\u6237\u76f4\u63a5\u5f15\u7528\u67d0\u79cd\u7c7b\u578b\u7684\u5934\u6587\u4ef6":55,"\u5373\u4f7f\u95f4\u9694\u5f88\u5c0f":106,"\u5373\u4f7fc":55,"\u5373\u4f7fprocess\u51fd\u6570\u91cc\u9762\u53ea\u6709\u4e00\u4e2ayield":2,"\u5373\u4f8b\u5982":55,"\u5373\u4fbf\u662f":94,"\u5373\u4fbf\u8bbe\u7f6e":77,"\u5373\u4fbfpaddl":55,"\u5373\u521d\u59cb\u72b6\u6001\u4e3a0":92,"\u5373\u5305\u542b\u65f6\u95f4\u6b65\u4fe1\u606f":2,"\u5373\u5355\u65f6\u95f4\u6b65\u6267\u884c\u7684\u51fd\u6570":93,"\u5373\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217":90,"\u5373\u53cc\u5c42rnn\u7684\u6bcf\u4e2a\u72b6\u6001":92,"\u5373\u53ef":95,"\u5373\u53ef\u4ee5\u6781\u5927\u7684\u52a0\u901f\u6570\u636e\u8f7d\u5165\u6d41\u7a0b":80,"\u5373\u53ef\u4f7f\u7528\u5f00\u53d1\u955c\u50cf\u6765\u7f16\u8bd1android\u7248paddlepaddl":115,"\u5373\u53ef\u5728":117,"\u5373\u53ef\u5f00\u59cb\u4e0b\u8f7d":86,"\u5373\u53ef\u5f00\u59cb\u4e0b\u9762\u7684\u6b65\u9aa4":84,"\u5373\u53ef\u663e\u793a\u6027\u80fd\u5206\u6790\u7684\u7ed3\u679c":102,"\u5373\u53ef\u68c0\u67e5\u6211\u4eec\u8c03\u4f18\u540e\u7684\u4fee\u6b63\u662f\u5426\u80fd\u591f\u6539\u5584\u7a0b\u5e8f\u7684\u6027\u80fd":102,"\u5373\u5728\u53cc\u5c42\u5e8f\u5217\u7684\u539f\u59cb\u6570\u636e\u4e2d":90,"\u5373\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d":80,"\u5373\u5927\u90e8\u5206\u503c\u4e3a0":[2,87],"\u5373\u5b8c\u6210\u67d0\u4e00\u4e2a\u4efb\u52a1\u7684\u6700\u5c11\u51fd\u6570":55,"\u5373\u5bf9\u7b2c\u4e09\u6b65\u8fdb\u884c\u66ff\u6362":123,"\u5373\u5c06\u4e00\u6bb5\u82f1\u6587\u6587\u672c\u6570\u636e":2,"\u5373\u5c06\u4e00\u6bb5\u8bdd\u8fdb\u884c\u5206\u7c7b":90,"\u5373\u5f53\u524d\u65f6\u95f4\u6b65\u4e0b\u7684\u795e\u7ecf\u7f51\u7edc\u4f9d\u8d56\u524d\u4e00\u4e2a\u65f6\u95f4\u6b65\u795e\u7ecf\u7f51\u7edc\u4e2d\u67d0\u4e00\u4e2a\u795e\u7ecf\u5143\u8f93\u51fa":90,"\u5373\u6211\u4eec\u53ef\u4ee5\u5148\u5b9a\u4e49\u4e00\u4e2atensor":98,"\u5373\u628a\u5355\u5c42rnn\u751f\u6210\u540e\u7684subseq\u7ed9\u62fc\u63a5\u6210\u4e00\u4e2a\u65b0\u7684\u53cc\u5c42seq":92,"\u5373\u6574\u4e2a\u53cc\u5c42group\u662f\u5c06\u524d\u4e00\u4e2a\u5b50\u53e5\u7684\u6700\u540e\u4e00\u4e2a\u5411\u91cf":90,"\u5373\u6574\u4e2a\u8f93\u5165\u5e8f\u5217":89,"\u5373\u6574\u6570\u6570\u7ec4":90,"\u5373\u65f6\u95f4\u9012\u5f52\u795e\u7ecf\u7f51\u7edc":90,"\u5373\u662f\u8de8\u8d8a\u65f6\u95f4\u6b65\u7684\u7f51\u7edc\u8fde\u63a5":90,"\u5373\u66b4\u9732":55,"\u5373\u7279\u5f81\u7684\u6570\u7ec4":90,"\u5373\u7f51\u5361\u540d":111,"\u5373\u82e5\u5e72\u6570\u636e\u6587\u4ef6\u8def\u5f84\u7684\u67d0\u4e00\u4e2a":2,"\u5373\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u51fa\u73b0nan\u6216\u8005inf":80,"\u5373\u8bbe\u7f6e":80,"\u5373\u8fd0\u884c\u8bad\u7ec3\u7a0b\u5e8f":84,"\u5373\u8fd9\u4e2a\u52a8\u6001\u5e93\u662f\u4e0d\u4f9d\u8d56\u4e8e\u5176\u4ed6\u4efb\u4f55\u6587\u4ef6\u7684":54,"\u5373define_py_data_sources2\u5e94\u6539\u4e3a":82,"\u5373input":92,"\u5373rnn\u4e4b\u95f4\u6709\u4e00\u6b21\u5d4c\u5957\u5173\u7cfb":90,"\u5377\u79ef\u5c42\u6743\u91cd":122,"\u5377\u79ef\u7f51\u7edc\u662f\u4e00\u79cd\u7279\u6b8a\u7684\u4ece\u8bcd\u5411\u91cf\u8868\u793a\u5230\u53e5\u5b50\u8868\u793a\u7684\u65b9\u6cd5":123,"\u5378\u8f7dpaddlepaddle\u5305":77,"\u538b\u6241\u6210\u4e3aeigen\u7684\u4e00\u7ef4tensor":98,"\u538b\u7f29\u6210\u4e00\u4e2a\u5411\u91cf":90,"\u539f\u56e0":95,"\u539f\u56e0\u5728\u4e8e\u6ca1\u6709\u628a\u673a\u5668\u4e0acuda\u76f8\u5173\u7684\u9a71\u52a8\u548c\u5e93\u6620\u5c04\u5230\u5bb9\u5668\u5185\u90e8":77,"\u539f\u56e0\u662f\u6bcf\u4e2a\u56de\u590d\u90fd\u4f1a\u53d1\u9001\u4e00\u5c01\u90ae\u4ef6":95,"\u53bb\u8fc7":90,"\u53c2\u6570":[2,6,7,8,9,10,11,13,14,15,18,21,23,25,28,54,80,94,96,104,105,111,121,122],"\u53c2\u6570\u4e3a":97,"\u53c2\u6570\u5171\u4eab\u7684\u914d\u7f6e\u793a\u4f8b\u4e3a":82,"\u53c2\u6570\u540d":122,"\u53c2\u6570\u548c\u73af\u5883\u53d8\u91cf":104,"\u53c2\u6570\u6570\u91cf":123,"\u53c2\u6570\u670d\u52a1\u5668":[104,105],"\u53c2\u6570\u670d\u52a1\u5668\u4e4b\u95f4\u4e0d\u76f8\u4e92\u4f9d\u8d56":104,"\u53c2\u6570\u670d\u52a1\u5668\u4e5f\u4e0d\u4f1a\u7b49\u5f85\u8ba1\u7b97\u8282\u70b9\u5168\u90e8\u90fd\u63d0\u4ea4\u68af\u5ea6\u4e4b\u540e\u624d\u5f00\u59cb\u4e0b\u4e00\u6b65":104,"\u53c2\u6570\u670d\u52a1\u5668\u63a5\u6536\u4ece\u8ba1\u7b97\u8282\u70b9\u4e0a\u4f20\u7684\u68af\u5ea6":104,"\u53c2\u6570\u670d\u52a1\u5668\u7684\u53c2\u6570\u5206\u5757\u5927\u5c0f":106,"\u53c2\u6570\u670d\u52a1\u5668\u7684\u76d1\u542c\u7aef\u53e3":106,"\u53c2\u6570\u670d\u52a1\u5668\u7684\u7f51\u7edc\u8bbe\u5907\u540d\u79f0":106,"\u53c2\u6570\u670d\u52a1\u5668\u7684ip\u5730\u5740":106,"\u53c2\u6570\u670d\u52a1\u5668\u7a00\u758f\u66f4\u65b0\u7684\u53c2\u6570\u5206\u5757\u5927\u5c0f":106,"\u53c2\u6570\u6765\u63a7\u5236\u7f13\u5b58\u65b9\u6cd5":80,"\u53c2\u6570\u6982\u8ff0":107,"\u53c2\u6570\u7684\u89e3\u6790":111,"\u53c2\u6570\u7ef4\u5ea6":121,"\u53c2\u6570\u884c":121,"\u53c2\u6570\u8bbe\u7f6e":79,"\u53c2\u6570\u8bbe\u7f6e\u4e86\u5916\u5c42":90,"\u53c2\u6570\u8bf4\u660e\u5bb9\u5668\u5df2\u4ea4\u4e92\u5f0f\u8fd0\u884c":84,"\u53c2\u6570\u8f93\u5165":80,"\u53c2\u6570\u9700\u8981\u5b9e\u73b0":93,"\u53c2\u7167\u4e0a\u8ff0\u6b65\u9aa4\u66f4\u65b0":95,"\u53c2\u8003":[44,54,84,109],"\u53c2\u8003\u5f3a\u8c03\u90e8\u5206":103,"\u53c2\u8003\u65f6\u95f4\u5e8f\u5217":90,"\u53c2\u8003\u6837\u4f8b\u6570\u636e\u51c6\u5907\u811a\u672c":104,"\u53c2\u8003\u955c\u50cf\u7684":111,"\u53c8":90,"\u53c8\u662f\u4e00\u4e2a\u5355\u5c42\u7684\u5e8f\u5217":89,"\u53c8\u8981\u4fdd\u8bc1\u6570\u636e\u662f\u968f\u673a\u7684":80,"\u53ca":96,"\u53cc\u5411\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u9690\u85cf\u72b6\u6001":93,"\u53cc\u5411\u9a8c\u8bc1":44,"\u53cc\u5c42":92,"\u53cc\u5c42\u4e0d\u7b49\u957frnn":90,"\u53cc\u5c42\u5e8f\u5217":89,"\u53cc\u5c42\u5e8f\u5217\u6216\u5355\u5c42\u5e8f\u5217":89,"\u53cc\u5c42\u5e8f\u5217\u6570\u636e\u4e00\u5171\u67094\u4e2a\u6837\u672c":90,"\u53cc\u5c42\u5e8f\u5217\u662f\u4e00\u4e2a\u5d4c\u5957\u7684\u5e8f\u5217":89,"\u53cc\u5c42\u5e8f\u5217\u662fpaddlepaddle\u652f\u6301\u7684\u4e00\u79cd\u975e\u5e38\u7075\u6d3b\u7684\u6570\u636e\u7ec4\u7ec7\u65b9\u5f0f":92,"\u53cc\u5c42\u5e8f\u5217\u6bcf\u4e2asubseq\u4e2d\u6bcf\u4e2a\u5143\u7d20":89,"\u53cc\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u53d8\u6210\u4e00\u4e2a0\u5c42\u5e8f\u5217":89,"\u53cc\u5c42\u6216\u8005\u5355\u5c42":89,"\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217\u7684dataprovider\u7684\u4ee3\u7801":90,"\u53cc\u5c42rnn":92,"\u53cc\u5c42rnn\u6570\u636e\u968f\u610f\u52a0\u4e86\u4e00\u4e9b\u9694\u65ad":90,"\u53cc\u5c42rnn\u987e\u540d\u601d\u4e49":90,"\u53cc\u8fdb\u5355\u51fa":92,"\u53cc\u8fdb\u53cc\u51fa":92,"\u53cd\u5411\u4f20\u64ad":96,"\u53cd\u5411\u4f20\u64ad\u6839\u636e\u8f93\u51fa\u7684\u68af\u5ea6":96,"\u53cd\u5411\u5355\u5143\u6d4b\u8bd5\u51fd\u6570\u9700\u8981\u4ee5":97,"\u53cd\u5411op\u5355\u5143\u6d4b\u8bd5\u7ee7\u627f\u81ea":97,"\u53cd\u5411op\u7684\u68af\u5ea6\u6d4b\u8bd5":97,"\u53cd\u5411op\u7c7b":97,"\u53cd\u5411op\u7c7b\u7684\u5b9a\u4e49":97,"\u53cd\u5411operator\u5355\u6d4b":97,"\u53cd\u5411opkernel\u7684\u5b9a\u4e49\u4e0e\u524d\u5411op\u7c7b\u4f3c":97,"\u53d1\u5e03\u5230dockerhub":71,"\u53d1\u5e03\u5230github":71,"\u53d1\u6563\u5230\u4e86\u4e00\u4e2a\u6570\u503c\u7279\u522b\u5927\u7684\u5730\u65b9":80,"\u53d1\u884c\u548c\u7ef4\u62a4":95,"\u53d1\u9001\u53c2\u6570\u7684\u7aef\u53e3\u53f7":106,"\u53d6\u503c\u76f8\u540c\u7684layer":81,"\u53d6\u51b3\u4e8e":97,"\u53d8\u6362\u77e9\u9635":96,"\u53d8\u91cf\u6765\u8bbe\u7f6e\u5185\u5b58\u4e2d\u6682\u5b58\u7684\u6570\u636e\u6761":2,"\u53e3\u5934":90,"\u53e5\u5b50\u8868\u793a\u7684\u8ba1\u7b97\u66f4\u65b0\u4e3a\u4e24\u6b65":123,"\u53e6\u4e00\u4e2a\u65b9\u6cd5\u662f\u4ea4\u53c9\u7f16\u8bd1":117,"\u53e6\u4e00\u4e2a\u662f\u5185\u5b58\u64cd\u4f5c\u91cf":103,"\u53e6\u4e00\u4e2a\u662f\u6bcf\u6761\u5e8f\u5217":80,"\u53e6\u4e00\u79cd\u65b9\u5f0f\u662f\u5c06\u7f51\u7edc\u5c42\u5212\u5206\u5230\u4e0d\u540c\u7684gpu\u4e0a\u53bb\u8ba1\u7b97":108,"\u53e6\u5916":[90,94],"\u53e6\u5916\u4e24\u4e2a\u5206\u522b\u662f\u6ed1\u52a8\u5747\u503c\u548c\u65b9\u5dee":122,"\u53e6\u5916\u6700\u65b0\u7684pip\u5b98\u65b9\u6e90\u4e2d\u7684\u5b89\u88c5\u5305\u9ed8\u8ba4\u662fmanylinux1\u6807\u51c6":86,"\u53ea\u4f5c\u4e3aread":92,"\u53ea\u4fdd\u5b58\u6700\u540e\u4e00\u8f6e\u7684\u53c2\u6570":106,"\u53ea\u5728\u7b2c\u4e00\u6b21cmake\u7684\u65f6\u5019\u6709\u6548":83,"\u53ea\u5bf9\u7279\u6b8a\u5728\u7ebf\u7cfb\u7edf\u8003\u8651\u4e24\u53f0\u4ee5\u4e0a\u540c\u65f6\u6545\u969c\u7684\u5bb9\u707e":34,"\u53ea\u622a\u53d6\u4e2d\u5fc3\u65b9\u5f62\u7684\u56fe\u50cf\u533a\u57df":122,"\u53ea\u662f\u53cc\u5c42\u5e8f\u5217\u5c06\u5176\u53c8\u505a\u4e86\u5b50\u5e8f\u5217\u5212\u5206":90,"\u53ea\u662f\u5c06\u53e5\u5b50\u7528\u8fde\u7eed\u5411\u91cf\u8868\u793a\u66ff\u6362\u4e3a\u7528\u7a00\u758f\u5411\u91cf\u8868\u793a":123,"\u53ea\u662f\u8bf4\u660e\u6570\u636e\u7684\u987a\u5e8f\u662f\u91cd\u8981\u7684":2,"\u53ea\u66b4\u9732\u6982\u5ff5\u7684\u63a5\u53e3":55,"\u53ea\u6709":90,"\u53ea\u67092\u4e2a\u914d\u7f6e\u4e0d\u4e00\u6837":121,"\u53ea\u6709\u5728\u9047\u5230\u9700\u8981":85,"\u53ea\u6709\u5f53\u8bbe\u7f6e\u4e86spars":106,"\u53ea\u7528\u4e8e\u5728\u5e8f\u5217\u751f\u6210\u4efb\u52a1\u4e2d\u6307\u5b9a\u8f93\u5165\u6570\u636e":92,"\u53ea\u80fd\u5728recurrent_group\u4e2d\u4f5c\u4e3astep":81,"\u53ea\u80fd\u6d4b\u8bd5\u5355\u4e2a\u6a21\u578b":108,"\u53ea\u80fd\u8bbf\u95ee\u5b83\u4eec\u7684\u8f93\u51fa\u503c":81,"\u53ea\u80fd\u8c03\u7528paddle\u7684\u52a8\u6001\u5e93":54,"\u53ea\u8981\u4e00\u7cfb\u5217\u7279\u5f81\u6570\u636e\u4e2d\u7684":90,"\u53ea\u8981\u51fa\u73b0\u6d6e\u70b9\u6570\u5f02\u5e38":80,"\u53ea\u8bfbmemory\u8f93\u5165":92,"\u53ea\u9488\u5bf9\u5185\u5b58":80,"\u53ea\u9700\u4e2d\u65ad":104,"\u53ea\u9700\u7528\u60a8\u5b9a\u4e49\u7684\u76ee\u5f55\u4fee\u6539":104,"\u53ea\u9700\u77e5\u9053\u8fd9\u662f\u4e00\u4e2a\u6807\u8bb0\u5c5e\u6027\u7684\u65b9\u6cd5\u5c31\u53ef\u4ee5\u4e86":2,"\u53ea\u9700\u8981":93,"\u53ea\u9700\u8981\u4e00\u884c\u4ee3\u7801\u5c31\u53ef\u4ee5\u8c03\u7528\u8fd9\u4e2apydataprovider2":2,"\u53ea\u9700\u8981\u5728\u51fd\u6570\u4e2d\u8c03\u7528\u591a\u6b21yield\u5373\u53ef":2,"\u53ea\u9700\u8981\u6062\u590d\u8fd9\u53f0\u8282\u70b9":34,"\u53ef\u4ee5":[84,90,95,99],"\u53ef\u4ee5\u4ece":84,"\u53ef\u4ee5\u4ece\u6211\u4eec\u7684ci\u7cfb\u7edf\u4e2d\u4e0b\u8f7d\u6700\u65b0\u7684whl\u5b89\u88c5\u5305\u548cc":86,"\u53ef\u4ee5\u4f20\u5165\u4e00\u4e2a\u51fd\u6570":2,"\u53ef\u4ee5\u4f30\u8ba1\u51fa\u5982\u679c\u6a21\u578b\u91c7\u7528\u4e0d\u53d8\u7684\u8f93\u51fa\u6700\u5c0f\u7684cost0\u662f\u591a\u5c11":82,"\u53ef\u4ee5\u4f7f\u7528":[82,104],"\u53ef\u4ee5\u4f7f\u7528\u4e0b\u9762\u7684\u547d\u4ee4\u66f4\u65b0\u60a8\u7684pip":86,"\u53ef\u4ee5\u4f7f\u7528\u5982\u4e0b\u4ee3\u7801":82,"\u53ef\u4ee5\u4f7f\u7528\u76f8\u5e94\u6570\u636e\u7c7b\u578b\u7684":82,"\u53ef\u4ee5\u4f7f\u7528\u8be5\u53c2\u6570":106,"\u53ef\u4ee5\u4f7f\u7528kubernetes\u7684\u547d\u4ee4\u884c\u5de5\u5177\u521b\u5efajob":111,"\u53ef\u4ee5\u4f7f\u7528python\u7684":4,"\u53ef\u4ee5\u51cf\u5c0f\u7cfb\u7edf\u590d\u6742\u6027":34,"\u53ef\u4ee5\u51cf\u5c11\u7f13\u5b58\u6c60\u7684\u5927\u5c0f":80,"\u53ef\u4ee5\u521b\u5efa\u4e00\u4e2a":110,"\u53ef\u4ee5\u521b\u5efa\u975e":97,"\u53ef\u4ee5\u52a0\u901fpaddlepaddle\u7684\u8ba1\u7b97":84,"\u53ef\u4ee5\u53c2\u8003":[84,90,93,94,95,109,111],"\u53ef\u4ee5\u53c2\u8003\u4e0b\u9762\u7684\u6b65\u9aa4\u6392\u67e5":78,"\u53ef\u4ee5\u53c2\u8003\u4fdd\u5b58\u5728":121,"\u53ef\u4ee5\u53c2\u8003paddlepaddl":87,"\u53ef\u4ee5\u540c\u65f6\u5728cpu":98,"\u53ef\u4ee5\u544a\u8bc9\u60a8\u67d0\u4e2a\u64cd\u4f5c\u5230\u5e95\u82b1\u4e86\u591a\u957f\u65f6\u95f4":103,"\u53ef\u4ee5\u5728":[83,104],"\u53ef\u4ee5\u5728\u4efb\u4f55\u673a\u5668\u4e0a\u6267\u884c\u7684":54,"\u53ef\u4ee5\u5728\u5171\u4eab\u5b58\u50a8\u4e0a\u67e5\u770b\u8f93\u51fa\u7684\u65e5\u5fd7\u548c\u6a21\u578b":111,"\u53ef\u4ee5\u5728\u8fd9\u4e2a":95,"\u53ef\u4ee5\u5728kubernetes\u4e2d\u6309\u7167":109,"\u53ef\u4ee5\u5b8c\u6210\u795e\u7ecf\u7f51\u7edc\u7684sgd\u65b9\u6cd5\u7684\u8bad\u7ec3":104,"\u53ef\u4ee5\u5b9e\u73b0\u4ecepaddl":98,"\u53ef\u4ee5\u5c06\u67d0\u4e00\u4e2a\u51fd\u6570\u6807\u8bb0\u6210\u4e00\u4e2apydataprovider2":2,"\u53ef\u4ee5\u5c06\u78c1\u76d8\u4e0a\u67d0\u4e2a\u76ee\u5f55\u5171\u4eab\u7ed9\u7f51\u7edc\u4e2d\u5176\u4ed6\u673a\u5668\u8bbf\u95ee":109,"\u53ef\u4ee5\u5c06cmake":102,"\u53ef\u4ee5\u5c06memory\u7406\u89e3\u4e3a\u4e00\u4e2a\u65f6\u5ef6\u64cd\u4f5c":92,"\u53ef\u4ee5\u5c06op\u5206\u4e3a\u4e24\u79cd":97,"\u53ef\u4ee5\u5c1d\u8bd5\u4ee5\u4e0b\u7684\u65b9\u6cd5":84,"\u53ef\u4ee5\u5e2e\u60a8\u63d0\u4f9b\u4e00\u4e9b\u5b9a\u4f4d\u6027\u80fd\u74f6\u9888\u7684\u5efa\u8bae":103,"\u53ef\u4ee5\u5e76\u884c\u7f16\u8bd1\u5417":94,"\u53ef\u4ee5\u5feb\u901f\u5728\u672c\u5730\u542f\u52a8\u4e00\u4e2a\u5305\u542b\u4e86paddlepaddle\u5b98\u65b9book\u6559\u7a0b\u7684jupyt":84,"\u53ef\u4ee5\u6267\u884c":[77,86],"\u53ef\u4ee5\u6267\u884c\u4ee5\u4e0b\u547d\u4ee4\u7f16\u8bd1\u751f\u6210\u6587\u6863":99,"\u53ef\u4ee5\u6267\u884cctest\u547d\u4ee4\u5373\u53ef":83,"\u53ef\u4ee5\u628a\u5b83\u60f3\u8c61\u4e3a\u4e00\u4e2a\u7c7b\u4f3c":94,"\u53ef\u4ee5\u628a\u672c\u5730\u7684\u6570\u636e\u4e0a\u4f20\u5230\u5b58\u50a8\u96c6\u7fa4\u4e2d":35,"\u53ef\u4ee5\u6307\u5b9a\u540c\u65f6\u6267\u884cgpu\u4e0a\u7684\u5355\u5143\u6d4b\u8bd5":83,"\u53ef\u4ee5\u6307\u5b9a\u54ea\u4e00\u4e2a\u8f93\u5165\u548c\u8f93\u51fa\u5e8f\u5217\u4fe1\u606f\u4e00\u81f4":90,"\u53ef\u4ee5\u6307\u5b9a\u5f00\u542f\u81ea\u52a8\u68c0\u6d4bsm\u67b6\u6784":83,"\u53ef\u4ee5\u6309\u7167\u4e0b\u9762\u7684\u65b9\u6cd5":83,"\u53ef\u4ee5\u662f\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":[89,92],"\u53ef\u4ee5\u662f\u4e00\u4e2a\u975e\u5e8f\u5217":92,"\u53ef\u4ee5\u662f\u4ece\u5206\u5e03\u5f0f\u5b58\u50a8\u6302\u8f7d\u8fc7\u6765\u7684":104,"\u53ef\u4ee5\u662f\u4ee5\u4e0b\u51e0\u79cd":96,"\u53ef\u4ee5\u663e\u793a\u5730\u6307\u5b9a\u4e00\u4e2alayer\u7684\u8f93\u51fa\u7528\u4e8e\u521d\u59cb\u5316memori":92,"\u53ef\u4ee5\u66f4\u6709\u6b21\u5e8f\u7684\u5b8c\u6210\u6027\u80fd\u7684\u4f18\u5316":102,"\u53ef\u4ee5\u6709\u4ee5\u4e0b\u4e24\u79cd":92,"\u53ef\u4ee5\u6709\u6548\u51cf\u5c0f\u7f51\u7edc\u7684\u963b\u585e":106,"\u53ef\u4ee5\u6709\u6548\u7684\u907f\u514dparamet":34,"\u53ef\u4ee5\u67e5\u770b":111,"\u53ef\u4ee5\u67e5\u770b\u6b64pod\u8fd0\u884c\u7684\u5bbf\u4e3b\u673a":110,"\u53ef\u4ee5\u6d4b\u8bd5\u591a\u4e2a\u6a21\u578b":108,"\u53ef\u4ee5\u7528":[44,94],"\u53ef\u4ee5\u7528\u4e8e\u5c0f\u91cf\u6570\u636e\u7684\u9a8c\u8bc1":109,"\u53ef\u4ee5\u7528\u4e8e\u63a5\u6536\u548cpydataprovider2\u4e00\u6837\u7684\u8f93\u5165\u6570\u636e\u5e76\u8f6c\u6362\u6210\u9884\u6d4b\u63a5\u53e3\u6240\u9700\u7684\u6570\u636e\u7c7b\u578b":4,"\u53ef\u4ee5\u7528\u4ee5\u4e0b\u6307\u4ee4":35,"\u53ef\u4ee5\u7528\u5982\u4e0b\u547d\u4ee4":95,"\u53ef\u4ee5\u7528\u6765\u8ba1\u7b97cpu\u51fd\u6570\u6216cuda\u5185\u6838\u7684\u65f6\u95f4\u6d88\u8017":103,"\u53ef\u4ee5\u770b\u4f5c\u662f\u4e00\u4e2a\u975e\u5e8f\u5217\u8f93\u5165":89,"\u53ef\u4ee5\u770b\u51fa":104,"\u53ef\u4ee5\u770b\u5230\u6700\u8017\u65f6\u7684\u51fd\u6570\u662fc":102,"\u53ef\u4ee5\u7cbe\u786e\u8bf4\u660e\u4e00\u4e2a\u957f\u8017\u65f6\u64cd\u4f5c\u7684\u5177\u4f53\u539f\u56e0":103,"\u53ef\u4ee5\u7ee7\u7eed\u5728\u81ea\u5df1\u7684\u529f\u80fd\u5206\u652f\u63d0\u4ea4\u4ee3\u7801":71,"\u53ef\u4ee5\u8003\u8651\u4f7f\u7528\u4e00\u4e9b\u4f18\u5316\u7b97\u6cd5":80,"\u53ef\u4ee5\u8054\u7cfbop":78,"\u53ef\u4ee5\u8054\u7cfbop\u662f\u5426\u53ef\u4ee5\u66f4\u6362\u96c6\u7fa4\u6216\u5347\u7ea7\u5f53\u524d\u96c6\u7fa4":78,"\u53ef\u4ee5\u88c5\u7684\u662f":94,"\u53ef\u4ee5\u8bbe\u7f6e":[102,115,116,117],"\u53ef\u4ee5\u8bbf\u95ee\u7531recurr":81,"\u53ef\u4ee5\u8c03\u7528resize\u63a5\u53e3\u8fdb\u884c\u6539\u53d8":98,"\u53ef\u4ee5\u8f7b\u677e\u5730\u5b8c\u6210\u795e\u7ecf\u7f51\u7edc\u914d\u7f6e":87,"\u53ef\u4ee5\u8fd0\u884c":104,"\u53ef\u4ee5\u9009\u5728\u5728\u5f53\u524d\u673a\u5668\u5b89\u88c5\u4e5f\u53ef\u4ee5\u62f7\u8d1d\u5230\u76ee\u6807\u673a\u5668\u5b89\u88c5":83,"\u53ef\u4ee5\u9009\u62e9\u662f\u5426\u4f7f\u7528\u53c2\u6570":108,"\u53ef\u4ee5\u901a\u8fc7":[95,109],"\u53ef\u4ee5\u901a\u8fc7\u4fee\u6539\u8fd9\u4e24\u4e2a\u51fd\u6570\u6765\u5b9e\u73b0\u590d\u6742\u7684\u7f51\u7edc\u914d\u7f6e":93,"\u53ef\u4ee5\u901a\u8fc7\u7f51\u9875\u6d4f\u89c8":84,"\u53ef\u4ee5\u901a\u8fc7\u8c03\u7528":4,"\u53ef\u4ee5\u901a\u8fc7\u9636\u6bb5\u6027\u7684\u4fdd\u5b58\u6bcf\u4e2aparamet":34,"\u53ef\u4ee5\u901a\u8fc7show_parameter_stats_period\u8bbe\u7f6e\u6253\u5370\u53c2\u6570\u4fe1\u606f\u7b49":123,"\u53ef\u4ee5\u91c7\u53d6\u4e0b\u9762\u51e0\u70b9\u63aa\u65bd":102,"\u53ef\u4ee5\u91cd\u547d\u540d\u8fd9\u4e2awhl\u5305\u4e3a":[77,86],"\u53ef\u5728\u547d\u4ee4\u884c\u6267\u884c":116,"\u53ef\u663e\u5f0f\u6307\u5b9a\u4e3a":116,"\u53ef\u7528\u4e8e\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u89e3\u6790\u8fd9\u4e9b\u53c2\u6570":108,"\u53ef\u7528\u5728\u6d4b\u8bd5\u6216\u8bad\u7ec3\u65f6\u6307\u5b9a\u521d\u59cb\u5316\u6a21\u578b":123,"\u53ef\u80fd\u4f1a\u5bfc\u81f4\u51fa\u9519":111,"\u53ef\u80fd\u4f1a\u9020\u6210\u7f51\u7edc\u62e5\u585e":34,"\u53ef\u80fd\u7684\u4ee3\u7801\u4e3a":80,"\u53ef\u80fd\u7684\u539f\u56e0\u662f":82,"\u53ef\u80fd\u7684\u60c5\u51b5\u4e0b":103,"\u53ef\u80fd\u9700\u8981\u6ce8\u610f\u7ed9\u8fd9\u4e2a\u865a\u62df\u673a\u591a\u5206\u914d\u4e00\u4e9b":94,"\u53ef\u89c1\u8be5\u8ba1\u7b97\u7531\u4e24\u4e2a\u8f93\u5165":97,"\u53ef\u8bbe\u7f6e":[115,116],"\u53ef\u8bbe\u7f6e\u4e3a":116,"\u53ef\u8bbe\u7f6e\u7684\u76ee\u6807\u67b6\u6784\u5982\u4e0b\u8868\u6240\u793a":116,"\u53ef\u9009":[2,83,96,104],"\u53ef\u9009\u7684\u4e0d\u540c\u7f16\u8bd1\u73af\u5883docker\u955c\u50cf":83,"\u53ef\u91c7\u7528\u7b2c\u4e8c\u79cd\u65b9\u5f0f":81,"\u53f3\u8fb9\u662f":122,"\u5403":90,"\u5403\u996d":90,"\u5404\u65b9\u9762":90,"\u5404\u9879\u66f4\u52a0\u5177\u4f53\u7684\u5355\u5143\u6d4b\u8bd5\u5728":97,"\u5408":90,"\u5408\u7406":90,"\u540c\u65f6":[77,80,103],"\u540c\u65f6\u4e5f\u4f1a\u8bfb\u53d6\u76f8\u5173\u8def\u5f84\u53d8\u91cf\u6765\u8fdb\u884c\u641c\u7d22":83,"\u540c\u65f6\u4e5f\u53ef\u4ee5\u52a0\u901f\u5f00\u59cb\u8bad\u7ec3\u524d\u6570\u636e\u8f7d\u5165\u7684\u8fc7\u7a0b":80,"\u540c\u65f6\u4e5f\u53ef\u4ee5\u901a\u8fc7":95,"\u540c\u65f6\u4e5f\u80fd\u591f\u5f15\u5165\u66f4\u52a0\u590d\u6742\u7684\u8bb0\u5fc6\u673a\u5236":92,"\u540c\u65f6\u4f1a\u8ba1\u7b97\u5206\u7c7b\u51c6\u786e\u7387":123,"\u540c\u65f6\u4f60\u53ef\u4ee5\u4f7f\u7528":122,"\u540c\u65f6\u4f7f\u7528\u4e86l2\u6b63\u5219":123,"\u540c\u65f6\u5176\u5185\u90e8\u5b9e\u73b0\u53ef\u4ee5\u907f\u514d\u7eafcpu\u7248\u672cpaddlepaddle\u5728\u6267\u884c\u672c\u8bed\u53e5\u65f6\u53d1\u751f\u5d29\u6e83":103,"\u540c\u65f6\u518d\u5c06":71,"\u540c\u65f6\u53ef\u4ee5\u4f7f\u7528\u6237\u53ea\u5173\u6ce8\u5982\u4f55\u4ece\u6587\u4ef6\u4e2d\u8bfb\u53d6\u6bcf\u4e00\u6761\u6570\u636e":2,"\u540c\u65f6\u5728\u5185\u5b58\u91cc\u76f4\u63a5\u968f\u5373\u9009\u53d6\u6570\u636e\u6765\u505ashuffl":80,"\u540c\u65f6\u5c06\u53c2\u6570\u521d\u59cb\u5316\u4e3a":82,"\u540c\u65f6\u628a\u5f53\u524d\u76ee\u5f55":94,"\u540c\u65f6\u63d0\u8d77":71,"\u540c\u65f6\u7528\u6237\u9700\u8981\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u6307\u5b9a":108,"\u540c\u65f6\u8bbe\u7f6e\u5185\u5b58\u7f13\u5b58\u529f\u80fd":80,"\u540c\u65f6\u8bbe\u7f6e\u5b83\u7684input_types\u5c5e\u6027":2,"\u540c\u65f6\u8f93\u51fa\u5e8f\u5217\u5c42\u548c\u975e\u5e8f\u5217\u5c42":80,"\u540c\u65f6\u9884\u6d4b\u7f51\u7edc\u901a\u5e38\u76f4\u63a5\u8f93\u51fa\u6700\u540e\u4e00\u5c42\u7684\u7ed3\u679c\u800c\u4e0d\u662f\u50cf\u8bad\u7ec3\u7f51\u7edc\u4e00\u6837\u518d\u63a5\u4e00\u5c42cost":4,"\u540c\u6837":87,"\u540c\u6837\u4e5f\u53ef\u4ee5\u5728\u6d4b\u8bd5\u6a21\u5f0f\u4e2d\u6307\u5b9a\u6a21\u578b\u8def\u5f84":106,"\u540c\u6837\u53ef\u4ee5\u6269\u5c55\u5230\u53cc\u5c42\u5e8f\u5217\u7684\u5904\u7406\u4e0a":92,"\u540c\u6837\u53ef\u83b7\u53d6\u5230\u8f93\u5165\u8f93\u51fa\u548c\u5c5e\u6027\u53c2\u6570":97,"\u540c\u6b65\u6267\u884c\u64cd\u4f5c\u7684\u7ebf\u7a0b\u6570":106,"\u540c\u7406":97,"\u540d\u5b57\u4fee\u9970":54,"\u540d\u79f0":123,"\u540e":[82,83,95,111,115,116,117],"\u540e\u5411\u4f20\u64ad":96,"\u540e\u5411\u4f20\u64ad\u7ed9\u5b9a\u8f93\u51fa\u7684\u68af\u5ea6":96,"\u540e\u7f00\u4e3a":104,"\u540e\u8005\u5728\u6fc0\u6d3b\u51fd\u6570\u53cd\u5411\u8ba1\u7b97\u65f6\u88ab\u8c03\u7528":80,"\u540e\u8005\u622a\u65ad\u56de\u4f20\u7ed9\u524d\u5c42\u7684\u68af\u5ea6":80,"\u540e\u8005\u7528\u4e8e\u68c0\u67e5\u53c2\u6570\u5c5e\u6027\u7684\u5408\u6cd5\u6027":97,"\u540e\u8005\u7ee7\u627f\u81ea":97,"\u540e\u9988":104,"\u5411\u6307\u5b9a\u7684\u76ee\u5f55\u4e2d\u4e00\u4e2a\u65b0\u7684\u6587\u4ef6":34,"\u5411\u91cfenable_parallel_vector":105,"\u5411paddlepaddle\u7684\u4e3b\u7248\u672c\u5e93\u63d0\u4ea4":71,"\u5417":94,"\u5426\u5219":[1,97,115,117],"\u5426\u5219\u4f1a\u628a":95,"\u5426\u5219\u4f7f\u7528\u591a\u673a\u8bad\u7ec3":106,"\u5426\u5219\u4f7f\u7528cpu\u6a21\u5f0f":106,"\u5426\u5219\u4f7f\u7528gpu":108,"\u5426\u5219\u5b83\u4ee5\u4e00\u4e2a\u5e8f\u5217\u8f93\u5165":93,"\u5426\u5219\u5f97\u628apaddle\u9759\u6001\u5e93\u94fe\u63a5\u5230\u89e3\u91ca\u5668\u91cc":54,"\u5426\u5219\u9891\u7e41\u7684\u591a\u8282\u70b9\u5de5\u4f5c\u7a7a\u95f4\u90e8\u7f72\u53ef\u80fd\u4f1a\u5f88\u9ebb\u70e6":104,"\u542b\u4e49":[102,122],"\u542b\u6709\u5e8f\u5217\u4fe1\u606f\u548c\u5b50\u5e8f\u5217\u4fe1\u606f\u7684\u7a20\u5bc6\u5411\u91cf":96,"\u542b\u6709\u5e8f\u5217\u4fe1\u606f\u7684\u6574\u6570":96,"\u542b\u6709\u5e8f\u5217\u4fe1\u606f\u7684\u7a20\u5bc6\u5411\u91cf":96,"\u542f\u52a8\u4e00\u4e2a\u65b0\u7684\u7ebf\u7a0b\u5f00\u59cb\u4fdd\u5b58\u68c0\u67e5\u70b9":34,"\u542f\u52a8\u4e00\u4e2a\u6d4b\u8bd5\u96c6\u7fa4":104,"\u542f\u52a8\u5bb9\u5668\u5f00\u59cb\u8bad\u7ec3":111,"\u542f\u52a8\u5e76\u884c\u5411\u91cf\u7684\u9608\u503c":106,"\u542f\u52a8\u5feb\u901f\u5e94\u7b54":106,"\u542f\u52a8\u8bad\u7ec3\u4efb\u52a1":104,"\u542f\u7528\u68af\u5ea6\u53c2\u6570\u7684\u9608\u503c":106,"\u5440":90,"\u5468\u56f4":90,"\u547d\u4ee4\u4e3a":[77,110],"\u547d\u4ee4\u521b\u5efa\u65b0\u955c\u50cf":110,"\u547d\u4ee4\u5220\u9664":[115,116,117],"\u547d\u4ee4\u53ef\u4ee5\u8bbe\u7f6e":83,"\u547d\u4ee4\u6307\u5b9a\u7684\u53c2\u6570\u4f1a\u4f20\u5165\u7f51\u7edc\u914d\u7f6e\u4e2d":123,"\u547d\u4ee4\u65f6":115,"\u547d\u4ee4\u6709\u65f6\u5019\u4f1a\u4ea7\u751f\u4e00\u4e9b\u4e2d\u95f4\u7ed3\u679c":94,"\u547d\u4ee4\u770b\u5230\u505c\u6b62\u540e\u4f46\u662f\u6ca1\u6709\u5220\u9664\u7684":94,"\u547d\u4ee4\u7f16\u8bd1\u6e90\u7801\u5373\u53ef":94,"\u547d\u4ee4\u884c\u4e2d\u7684":102,"\u547d\u4ee4\u884c\u53c2\u6570\u6587\u6863":123,"\u547d\u4ee4\u8bbe\u7f6e\u8be5\u7c7b\u7f16\u8bd1\u9009\u9879":83,"\u547d\u4ee4\u9009\u9879\u5e76\u4e14":104,"\u547d\u4ee4\u91cc\u90fd\u7528\u4e86":94,"\u547d\u540d\u4e3a":95,"\u547d\u540d\u7a7a\u95f4":109,"\u547d\u540d\u7a7a\u95f4\u4e3b\u8981\u4e3a\u4e86\u5bf9\u8c61\u8fdb\u884c\u903b\u8f91\u4e0a\u7684\u5206\u7ec4\u4fbf\u4e8e\u7ba1\u7406":109,"\u547d\u540d\u89c4\u8303":97,"\u547d\u540d\u8bf7\u9075\u5b88":97,"\u548c":[35,54,55,71,80,81,82,83,90,93,94,95,96,97,98,99,102,103,104,108,109,115,117,121,123],"\u548c\u4e00\u4e2a\u5df2\u7ecf\u5206\u8bcd\u540e\u7684\u53e5\u5b50":90,"\u548c\u4e09\u79cd\u5e8f\u5217\u6a21\u5f0f":[2,87],"\u548c\u4e0b\u9762\u5c06\u8981\u4ecb\u7ecd\u7684\u6ce8\u518c\u51fd\u6570\u4e00\u8d77\u653e\u5728":97,"\u548c\u4e2d\u6587\u6587\u6863":99,"\u548c\u4e4b\u524d\u51cf\u5c0f\u901a\u8fc7\u51cf\u5c0f\u7f13\u5b58\u6c60\u6765\u51cf\u5c0f\u5185\u5b58\u5360\u7528\u7684\u539f\u7406\u4e00\u81f4":80,"\u548c\u504f\u7f6e\u5411\u91cf":96,"\u548c\u5185\u5b58":94,"\u548c\u53cc\u5c42\u5e8f\u5217\u542b\u6709subseq":89,"\u548c\u5728":2,"\u548c\u5bf9\u5e94\u884c\u7684\u4ee3\u7801":102,"\u548c\u5bf9\u8c61\u5b58\u50a8api":109,"\u548c\u5dee\u8bc4":123,"\u548c\u5e8f\u5217\u4e2d\u542b\u6709\u5143\u7d20\u7684\u6570\u76ee\u540c":89,"\u548c\u5f02\u6b65\u968f\u673a\u68af\u5ea6\u4e0b\u964d":104,"\u548c\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u8f93\u5165":93,"\u548c\u64cd\u4f5c\u7cfb\u7edf\u4e0a\u76f4\u63a5\u8fd0\u884c\u7684":94,"\u548c\u68af\u5ea6\u622a\u65ad":123,"\u548c\u793a\u4f8b2\u4e2d\u7684\u914d\u7f6e\u7c7b\u4f3c":90,"\u548c\u79bb\u7ebf\u6570\u636e\u7684\u65b9\u5f0f":35,"\u548c\u90e8\u5206layer":92,"\u548cpool":89,"\u548cpserver\u4e4b\u95f4\u7528\u4e8e\u7a00\u758f\u7c7b\u578b\u53c2\u6570\u901a\u4fe1\u7684\u7aef\u53e3\u4e2a\u6570":104,"\u548cpython\u63a5\u53e3\u6765\u63d0\u53d6\u7279\u5f81":122,"\u54c1\u8d28":90,"\u54ea\u4e2atrainer\u5148\u5b8c\u6210block\u7684\u8bad\u7ec3":34,"\u54ea\u4e9b\u4e0d\u662f":90,"\u5546\u52a1":90,"\u554a":90,"\u5668":123,"\u56db\u79cd\u6570\u636e\u7c7b\u578b":[2,87],"\u56e0\u4e3a\u5168\u8fde\u63a5\u5c42\u7684\u6fc0\u6d3b\u53ef\u4ee5\u662fsoftmax":96,"\u56e0\u4e3a\u53c2\u6570":108,"\u56e0\u4e3a\u5b83\u4eec\u7684\u8ba1\u7b97\u6548\u7387\u6bd4":93,"\u56e0\u4e3a\u5b83\u6bd4":93,"\u56e0\u4e3a\u5b98\u65b9\u955c\u50cf":111,"\u56e0\u4e3a\u5bb9\u5668\u5185\u7684\u6587\u4ef6\u90fd\u662f\u6682\u65f6\u5b58\u5728\u7684":109,"\u56e0\u4e3a\u6211\u4eec\u4f1a\u628a\u6240\u6709\u7f16\u8bd1\u5de5\u5177\u90fd\u5b89\u88c5\u8fdb\u4e00\u4e2a":94,"\u56e0\u4e3a\u6e90\u7801\u5c31\u5728\u672c\u673a\u4e0a":94,"\u56e0\u4e3a\u8fd9\u4e2a\u5de5\u5177\u5177\u6709web\u670d\u52a1\u754c\u9762":102,"\u56e0\u4e3a\u8fd9\u6837\u505a\u4e5f\u6ca1\u6cd5\u4fdd\u8bc1\u6d88\u9664\u968f\u673a\u6027":34,"\u56e0\u4e3ac":102,"\u56e0\u4e3apython\u7684\u641c\u7d22\u8def\u5f84\u662f\u4f18\u5148\u5df2\u7ecf\u5b89\u88c5\u7684python\u5305":77,"\u56e0\u4e3aswig\u5728\u7b2c\u4e09\u65b9\u8bed\u8a00\u4e2d\u66b4\u9732\u7684\u51fd\u6570\u540d":54,"\u56e0\u6b64":[1,2,90,92,96,97,115],"\u56e0\u6b64\u4f7f\u7528":2,"\u56e0\u6b64\u53cc\u5c42\u5e8f\u5217\u7684\u914d\u7f6e\u4e2d":90,"\u56e0\u6b64\u53ef\u4ee5\u4f7f\u7528\u8be5\u9009\u9879":121,"\u56e0\u6b64\u53ef\u80fd\u6d4b\u8bd5\u4e0d\u591f\u5b8c\u5907":98,"\u56e0\u6b64\u5728\u8f6c\u6362\u65f6\u9700\u8981\u663e\u793a\u7684\u6307\u5b9a":98,"\u56e0\u6b64\u5b83\u662finteger_value_sub_sequ":90,"\u56e0\u6b64\u5f53":115,"\u56e0\u6b64\u6211\u4eec\u91c7\u7528\u8f93\u51fa\u7684\u52a0\u6743\u548c":96,"\u56e0\u6b64\u6709\u4e24\u79cd\u89e3\u51b3\u65b9\u6848":2,"\u56e0\u6b64\u7528\u6237\u5e76\u4e0d\u9700\u8981\u5173\u5fc3\u5b83\u4eec":105,"\u56e0\u6b64\u8be5\u5c42\u4e2d\u6ca1\u6709\u504f\u7f6e":122,"\u56e0\u6b64\u9519\u8bef\u7684\u4f7f\u7528\u4e8c\u8fdb\u5236\u53d1\u884c\u7248\u53ef\u80fd\u4f1a\u5bfc\u81f4\u8fd9\u79cd\u9519\u8bef":77,"\u56e0\u6b64init_hook\u5c3d\u91cf\u4f7f\u7528":2,"\u56fe":122,"\u56fe\u50cf\u5206\u7c7b":71,"\u56fe\u50cf\u5927\u5c0f\u4e3a3":122,"\u56fe\u8868":84,"\u5728":[2,55,71,89,90,93,94,95,97,102,104,116,122,123],"\u5728\u4e00\u4e2a\u4e0d\u53ef\u4e2d\u65ad\u5e76\u7f3a\u5c11\u5907\u4efd\u7684\u8bad\u7ec3\u4efb\u52a1\u4e2d":34,"\u5728\u4e00\u4e2a\u529f\u80fd\u9f50\u5168\u7684kubernetes\u673a\u7fa4\u91cc":110,"\u5728\u4e00\u4e2a\u53c2\u6570\u7684\u68af\u5ea6\u88ab\u66f4\u65b0\u540e":96,"\u5728\u4e00\u4e9b\u5206\u5e03\u5f0f\u7cfb\u7edf\u4e2d":104,"\u5728\u4e00\u6b21\u8bad\u7ec3\u4e2d":102,"\u5728\u4e00\u8f6e\u4e2d\u6bcfsave":106,"\u5728\u4e0a\u56fe\u4e2d\u663e\u793a\u4e86\u5728\u4e00\u4e2a\u5b9e\u9645\u751f\u4ea7\u73af\u5883\u4e2d\u7684\u5e94\u7528":35,"\u5728\u4e0a\u9762\u4ee3\u7801\u4e2d":90,"\u5728\u4e0a\u9762\u7684\u4ee3\u7801\u4e2d":97,"\u5728\u4e0b\u4e00\u7bc7\u4e2d":110,"\u5728\u4e0b\u9762\u4f8b\u5b50\u91cc":123,"\u5728\u4e0d\u540c\u64cd\u4f5c\u7cfb\u7edf":109,"\u5728\u4e4b\u540e\u7684":80,"\u5728\u4e86\u89e3docker\u7684\u57fa\u672c\u4f7f\u7528\u65b9\u6cd5\u4e4b\u540e":84,"\u5728\u4efb\u610f\u65f6\u95f4\u67d0\u4e00\u53f0\u53c2\u6570\u670d\u52a1\u5668\u4e0a\u4fdd\u5b58\u7684\u53c2\u6570\u53ef\u80fd\u6bd4\u53e6\u4e00\u53f0\u8981\u66f4\u65b0":104,"\u5728\u4f7f\u7528\u4e0d\u540c\u7684\u5206\u5e03\u5f0f\u8ba1\u7b97\u5e73\u53f0\u65f6":104,"\u5728\u4f7f\u7528\u5206\u5e03\u5f0f\u8ba1\u7b97\u5e73\u53f0\u8fdb\u884c\u8bad\u7ec3\u65f6":104,"\u5728\u4f7f\u7528\u540c\u6b65sgd\u8bad\u7ec3\u795e\u7ecf\u7f51\u7edc\u65f6":104,"\u5728\u4f7f\u7528\u8be5\u6587\u6863\u4e4b\u524d":87,"\u5728\u4f7f\u7528twine\u4e0a\u4f20\u4e4b\u524d":71,"\u5728\u5168\u8fde\u63a5\u5c42\u4e2d":96,"\u5728\u5177\u4f53\u7684\u8ba1\u7b97\u4e2d":98,"\u5728\u51c6\u5907\u53d1\u8d77":95,"\u5728\u51fa\u73b0\u5355\u70b9\u6545\u969c\u65f6":34,"\u5728\u51fd\u6570":111,"\u5728\u5206\u5e03\u5f0f\u73af\u5883\u4e2d\u6d4b\u8bd5":106,"\u5728\u5206\u5e03\u5f0f\u8bad\u7ec3\u4e2d":106,"\u5728\u521b\u5efaparameters\u540e":82,"\u5728\u5355\u5c42\u6570\u636e\u7684\u57fa\u7840\u4e0a":90,"\u5728\u53c2\u6570\u670d\u52a1\u5668\u4e0a\u52a0\u8f7d\u548c\u4fdd\u5b58\u53c2\u6570":106,"\u5728\u53c2\u6570\u670d\u52a1\u5668\u7ec8\u7aef\u6bcflog":106,"\u5728\u53cc\u5c42rnn\u4e2d\u7684\u7ecf\u5178\u60c5\u51b5\u662f\u5c06\u5185\u5c42\u7684\u6bcf\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217\u6570\u636e":90,"\u5728\u53cd\u5411\u4f20\u9012\u7684\u65f6\u5019":80,"\u5728\u53d8\u6362\u65f6\u9700\u8981\u5c06\u8f93\u5165\u5e8f\u5217\u4f20\u5165":90,"\u5728\u540c\u4e00\u4e2a\u547d\u540d\u7a7a\u95f4\u4e2d":109,"\u5728\u542f\u52a8job\u4e4b\u524d":111,"\u5728\u547d\u4ee4\u884c\u663e\u793a\u5206\u6790\u7ed3\u679c":102,"\u5728\u56de\u590d\u8bc4\u5ba1\u4eba\u610f\u89c1\u65f6":95,"\u5728\u58f0\u660edataprovider\u7684\u65f6\u5019\u4f20\u5165dictionary\u4f5c\u4e3a\u53c2\u6570":2,"\u5728\u591acpu\u8bad\u7ec3\u65f6\u5171\u4eab\u8be5\u53c2\u6570":106,"\u5728\u5b8c\u6210\u4e00\u5b9a\u91cf\u6570\u636e\u7684\u8bad\u7ec3\u540e":104,"\u5728\u5b8c\u6210\u795e\u7ecf\u7f51\u7edc\u7684\u642d\u5efa\u4e4b\u540e":87,"\u5728\u5b9a\u4e49\u8f93\u5165layer\u4e4b\u540e":87,"\u5728\u5b9e\u73b0\u8fc7\u7a0b\u4e2d":55,"\u5728\u5b9e\u9645\u5e94\u7528\u4e2d":81,"\u5728\u5bb9\u5668\u4e2d\u7f16\u8f91\u4ee3\u7801":84,"\u5728\u5bb9\u5668\u521b\u5efa\u540e":111,"\u5728\u5bf9\u5bb9\u5668\u7684\u63cf\u8ff0":111,"\u5728\u5c42\u4e2d\u6307\u5b9a":108,"\u5728\u5e8f\u5217\u751f\u6210\u4efb\u52a1\u4e2d":92,"\u5728\u5f00\u59cb\u8bad\u7ec3\u4e4b\u524d":35,"\u5728\u5f02\u6784\u96c6\u7fa4\u4e2d":34,"\u5728\u5f02\u6b65sgd\u4e2d":104,"\u5728\u5f15\u5165\u5176\u4ed6\u7c7b\u578b\u7684\u5934\u6587\u4ef6\u65f6":55,"\u5728\u5f53\u524d\u7684\u5b9e\u73b0\u65b9\u5f0f\u4e0b":96,"\u5728\u5f97\u5230":111,"\u5728\u5feb\u7167\u5199\u5165\u5b8c\u6210\u540e":34,"\u5728\u60a8\u7684\u5b9e\u9645\u73af\u5883\u4e2d":34,"\u5728\u6211\u4eec\u7684\u4f8b\u5b50\u4e2d":93,"\u5728\u6267\u884c\u65f6":98,"\u5728\u63d0\u4ea4":95,"\u5728\u642d\u5efa\u795e\u7ecf\u7f51\u7edc\u7684\u8fc7\u7a0b\u4e2d":87,"\u5728\u6570\u636e\u52a0\u8f7d\u548c\u7f51\u7edc\u914d\u7f6e\u5b8c\u6210\u4e4b\u540e":123,"\u5728\u672c\u4f8b\u4e2d":[90,95,108],"\u5728\u672c\u4f8b\u4e2d\u6ca1\u6709\u4f7f\u7528":2,"\u5728\u672c\u6559\u7a0b\u4e2d":93,"\u5728\u672c\u6587\u6863\u4e2d":44,"\u5728\u672c\u793a\u4f8b\u4e2d":90,"\u5728\u672c\u8282\u4e2d":93,"\u5728\u673a\u7fa4\u4e0a\u8fd0\u884c\u8f6c\u6362\u7a0b\u5e8f":35,"\u5728\u6811\u7684\u6bcf\u4e00\u5c42\u4e0a":106,"\u5728\u6837\u4f8b\u4e2d":55,"\u5728\u6a21\u578b\u914d\u7f6e\u4e2d\u901a\u8fc7":123,"\u5728\u6b64":[105,108],"\u5728\u6b64\u4e3a\u65b9\u4fbf\u5bf9\u6bd4\u4e0d\u540c\u7f51\u7edc\u7ed3\u6784":123,"\u5728\u6b64\u611f\u8c22":121,"\u5728\u6bcf\u4e2a\u65f6\u95f4\u6b65\u4e2d":93,"\u5728\u6bcf\u4e2a\u65f6\u95f4\u6b65\u7684\u5b50\u5e8f\u5217\u957f\u5ea6\u53ef\u4ee5\u4e0d\u76f8\u7b49":90,"\u5728\u6bcf\u4e2a\u65f6\u95f4\u6b65\u957f":93,"\u5728\u6bcf\u4e2apod\u4e0a\u90fd\u901a\u8fc7volume\u65b9\u5f0f\u6302\u8f7d\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf\u7684\u4e00\u4e2a\u76ee\u5f55\u7528\u4e8e\u4fdd\u5b58\u8bad\u7ec3\u6570\u636e\u548c\u8f93\u51fa\u7ed3\u679c":111,"\u5728\u6d4b\u8bd5\u9636\u6bb5":106,"\u5728\u6d4b\u8bd5\u9636\u6bb5\u5b83\u4eec\u5c06\u4f1a\u88ab\u52a0\u8f7d\u5230\u6a21\u578b\u4e2d":122,"\u5728\u6e90\u7801\u76ee\u5f55\u6811\u7684\u6839\u76ee\u5f55\u4e2d\u8fd0\u884c":95,"\u5728\u7269\u7406\u673a\u4e0a\u624b\u52a8\u90e8\u7f72":109,"\u5728\u751f\u6210\u65f6":93,"\u5728\u7528\u6237\u4f7f\u7528c":55,"\u5728\u76f8\u5e94\u7684\u4f18\u5316\u7b97\u6cd5\u91cc\u8bbe\u7f6elearning_rate_schedule\u53ca\u76f8\u5173\u53c2\u6570":82,"\u5728\u76f8\u5e94layer\u7684":81,"\u5728\u7a0b\u5e8f\u5f00\u59cb\u9636\u6bb5":4,"\u5728\u7ebf\u4e0a\u7cfb\u7edf\u4e2d":104,"\u5728\u7ebf\u6a21\u578b\u9884\u6d4b\u670d\u52a1":35,"\u5728\u7ec4\u5408\u65f6":87,"\u5728\u7f16\u8bd1\u5bbf\u4e3b\u673a\u7248protoc\u53ef\u6267\u884c\u6587\u4ef6\u548c\u76ee\u6807\u673a\u7248openblas\u5e93\u65f6\u9700\u8981\u7528\u5230":[115,117],"\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d":96,"\u5728\u7f51\u7edc\u914d\u7f6e\u91cc":2,"\u5728\u81ea\u7136\u8bed\u8a00\u5904\u7406\u4efb\u52a1\u4e2d":89,"\u5728\u8bad\u7ec3\u4e2d":93,"\u5728\u8bad\u7ec3\u4e4b\u524d":111,"\u5728\u8bad\u7ec3\u6570\u96c6\u4e0a\u8bad\u7ec3\u751f\u6210\u8bcd\u5411\u91cf\u5b57\u5178":121,"\u5728\u8bad\u7ec3\u65f6":110,"\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d":111,"\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u6bcfshow":106,"\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u8fdb\u884c\u6d4b\u8bd5":1,"\u5728\u8bbe\u7f6e":[115,116,117],"\u5728\u8bc4\u5ba1\u8fc7\u7a0b\u4e2d":71,"\u5728\u8be5\u793a\u4f8b\u4e2d":82,"\u5728\u8be5\u914d\u7f6e\u76847":90,"\u5728\u8d2d\u7269\u7f51\u7ad9\u4e0a":123,"\u5728\u8f6f\u4ef6\u5de5\u7a0b\u7684\u8303\u7574\u91cc":103,"\u5728\u8f93\u51fa\u7684\u8fc7\u7a0b\u4e2d":92,"\u5728\u8fd0\u884c\u5b8c\u6027\u80fd\u5206\u6790\u540e":102,"\u5728\u8fd0\u884c\u795e\u7ecf\u7f51\u7edc\u8ba1\u7b97\u56fe\u65f6":98,"\u5728\u8fd9\u4e2a":71,"\u5728\u8fd9\u4e2a\u4f8b\u5b50\u91cc":[96,110],"\u5728\u8fd9\u4e2a\u51fd\u6570\u4e2d":90,"\u5728\u8fd9\u4e2a\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165\u4efb\u4f55\u5176\u4ed6\u8bed\u8a00\u7684\u89e3\u91ca\u5668":54,"\u5728\u8fd9\u4e2a\u6559\u7a0b\u4e2d":103,"\u5728\u8fd9\u4e2a\u6a21\u578b\u4e2d":93,"\u5728\u8fd9\u4e2a\u9636\u6bb5\u7684\u4ee3\u7801\u6b63\u5728\u7ecf\u5386\u56de\u5f52\u6d4b\u8bd5":71,"\u5728\u8fd9\u4e9b\u5934\u6587\u4ef6\u4e2d":55,"\u5728\u8fd9\u4e9b\u6587\u4ef6\u4e2d":55,"\u5728\u8fd9\u4e9blayer\u4e2d":90,"\u5728\u8fd9\u65f6\u771f\u6b63\u7684\u5206\u914d\u5185\u5b58":98,"\u5728\u8fd9\u6bb5\u4ee3\u7801\u4e2d":98,"\u5728\u8fd9\u79cd\u60c5\u51b5\u4e0b":[93,96],"\u5728\u8fd9\u79cd\u7ed3\u6784\u4e2d":92,"\u5728\u8fd9\u7bc7\u6587\u6863\u91cc":110,"\u5728\u8fd9\u7bc7\u6587\u7ae0\u91cc":111,"\u5728\u8fd9\u91cc":92,"\u5728\u8fd9\u91cc\u6211\u4eec\u4f7f\u7528\u5168\u8fde\u63a5\u5c42\u4f5c\u4e3a\u4f8b\u5b50\u6765\u5c55\u793a\u5b9e\u73b0\u65b0\u7f51\u7edc\u5c42\u6240\u9700\u8981\u7684\u56db\u4e2a\u6b65\u9aa4":96,"\u5728\u8fd9\u91cc\u7528eigenvector\u6765\u8868\u793a":98,"\u5728\u8fd9\u91cc\u9700\u8981\u6ce8\u610f\u7684\u662f":98,"\u5728\u8fdb\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3\u65f6":104,"\u5728\u8fdb\u884c\u7f51\u7edc\u914d\u7f6e\u4e4b\u524d":87,"\u5728\u914d\u7f6e\u4e2d\u9700\u8981\u8bfb\u53d6\u5916\u90e8\u5b57\u5178":2,"\u5728\u914d\u7f6e\u6587\u4ef6\u4e2d\u7684":122,"\u5728\u91c7\u7528sgd":82,"\u5728\u96c6\u7fa4\u4e0a\u8bad\u7ec3\u4e00\u4e2a\u7a00\u758f\u6a21\u578b\u9700\u8981\u52a0\u4e0a\u4e0b\u9762\u7684\u53c2\u6570":108,"\u5728\u975e\u5e8f\u5217\u8f93\u5165\u65f6":80,"\u5728android\u5e73\u53f0\u4e0a\u4e0d\u652f\u6301\u901a\u8fc7swig\u8c03\u7528\u6765\u8bad\u7ec3\u6216\u8005\u9884\u6d4b":115,"\u5728android\u5e73\u53f0\u4e0a\u53ea\u652f\u6301\u4f7f\u7528c":115,"\u5728aws\u4e0a\u5feb\u901f\u90e8\u7f72\u96c6\u7fa4":109,"\u5728build\u76ee\u5f55\u4e0b\u6267\u884c":77,"\u5728c":54,"\u5728c\u7684\u5934\u6587\u4ef6":54,"\u5728cmake\u53c2\u6570\u914d\u7f6e\u4e0a":[115,116],"\u5728cmake\u7684\u547d\u4ee4\u884c\u4e2d":83,"\u5728eigen\u4e2d":98,"\u5728generator\u7684\u4e0a\u4e0b\u6587\u4e2d\u5c3d\u91cf\u7559\u4e0b\u975e\u5e38\u5c11\u7684\u53d8\u91cf\u5f15\u7528":2,"\u5728ios\u5e73\u53f0\u4e0a\u4e0d\u652f\u6301\u901a\u8fc7swig\u8c03\u7528\u6765\u8bad\u7ec3\u6216\u8005\u9884\u6d4b":116,"\u5728ios\u5e73\u53f0\u4e0a\u53ea\u652f\u6301\u4f7f\u7528c":116,"\u5728kubernetes\u4e2d\u521b\u5efa\u7684\u6240\u6709\u8d44\u6e90\u5bf9\u8c61":109,"\u5728main":102,"\u5728paddl":111,"\u5728paddle\u4e2d":108,"\u5728paddle\u4e4b\u4e0a\u8fd0\u884c\u7684\u6df1\u5ea6\u5b66\u4e60\u8bad\u7ec3\u8f93\u51fa\u7684\u6a21\u578b\u4f1a\u63d0\u4f9b\u7ed9\u5728\u7ebf\u4eba\u8138\u8bc6\u522b\u7684\u5e94\u7528\u4f7f\u7528":35,"\u5728paddlepaddle\u4e2d":[87,92],"\u5728paddlepaddle\u4e2d\u4f7f\u7528dropout\u6709\u4e24\u79cd\u65b9\u5f0f":81,"\u5728paddlepaddle\u4e2d\u5305\u542b\u4ee5\u4e0b":81,"\u5728paddlepaddle\u7684\u6587\u6863\u4e2d":90,"\u5728paramet":34,"\u5728python\u811a\u672c\u4e2d\u5b9e\u73b0\u4e0e\u524d\u5411operator\u76f8\u540c\u7684\u8ba1\u7b97\u903b\u8f91":97,"\u5728step\u51fd\u6570\u4e2d\u5b9a\u4e49":92,"\u5728step\u51fd\u6570\u4e2d\u5b9a\u4e49memori":92,"\u5728trainer":108,"\u5728trainer\u4e2d\u53ef\u4ee5\u4f7f\u7528\u4e0b\u9762\u53d6\u6a21\u7684\u65b9\u6cd5\u4e3a\u6bcf\u4e2atrainer\u5206\u914d\u8bad\u7ec3\u6570\u636e\u6587\u4ef6":104,"\u5730\u5740\u4e5f\u53ef\u4ee5\u4e3ahdfs\u6587\u4ef6\u8def\u5f84":1,"\u5730\u6bb5":90,"\u5730\u7406\u4f4d\u7f6e":90,"\u5730\u94c1\u7ad9":90,"\u5747\u4f1a\u88ab\u5b89\u88c5\u5230includ":55,"\u5747\u503c\u56fe\u50cf\u6587\u4ef6":122,"\u5747\u5300\u5206\u5e03":82,"\u5747\u5300\u5206\u5e03\u7684\u8303\u56f4\u662f":106,"\u5747\u662f\u5728":55,"\u5747\u6709\u4e09\u4e2a\u5b50\u5e8f\u5217":90,"\u5747\u6709\u4e24\u7ec4\u7279\u5f81":90,"\u57fa\u4e8e\u53cc\u5c42\u5e8f\u5217\u8f93\u5165":92,"\u57fa\u4e8e\u7c98\u6027\u4f1a\u8bdd\u7684\u8d1f\u8f7d\u5747\u8861\u529f\u80fd":44,"\u57fa\u4e8epython\u7684\u6a21\u578b\u9884\u6d4b":4,"\u57fa\u4e8epython\u7684\u9884\u6d4b":[3,123],"\u57fa\u672c\u4e0a\u548cmnist\u6837\u4f8b\u4e00\u81f4":2,"\u57fa\u672c\u4f7f\u7528\u6982\u5ff5":88,"\u57fa\u672c\u76f8\u540c":121,"\u57fa\u7c7b":97,"\u586b\u5199":95,"\u589e\u52a0":97,"\u589e\u52a0\u4e86\u4e00\u6761cd\u547d\u4ee4":110,"\u589e\u52a0\u4e86\u8bbe\u5907\u7c7b\u578b":97,"\u589e\u52a0\u5982\u4e0b\u53c2\u6570":108,"\u589e\u52a0\u68af\u5ea6\u68c0\u6d4b\u7684\u5355\u5143\u6d4b\u8bd5":96,"\u5904\u7406\u5668\u6709\u4e24\u4e2a\u5173\u952e\u6027\u80fd\u9650\u5236":103,"\u5904\u7406\u6570\u636e\u7684python\u811a\u672c\u6587\u4ef6":123,"\u5904\u7406\u7684\u8f93\u5165\u5e8f\u5217\u4e3b\u8981\u5206\u4e3a\u4ee5\u4e0b\u4e09\u79cd\u7c7b\u578b":92,"\u5907\u6ce8":103,"\u590d\u6742\u5ea6\u6216\u65f6\u95f4\u590d\u6742\u5ea6":103,"\u5916\u5c42memory\u662f\u4e00\u4e2a\u5143\u7d20":90,"\u5916\u5c42outer_step\u4e2d":90,"\u591a\u4e2a\u503c":35,"\u591a\u4e2a\u5c42\u7684\u8f93\u51fa\u77e9\u9635\u7684\u9ad8\u5ea6\u4e0d\u4e00\u81f4\u5bfc\u81f4\u62fc\u63a5\u5931\u8d25":80,"\u591a\u4e2a\u8f93\u51fa\u5c42\u5904\u7406\u591a\u4e2a\u4e0d\u540c\u957f\u5ea6\u7684\u5e8f\u5217":80,"\u591a\u4e2ainput\u4ee5list\u65b9\u5f0f\u8f93\u5165":123,"\u591a\u4e2aip\u4f7f\u7528":104,"\u591a\u4e2aparamet":34,"\u591a\u53e5\u8bdd\u8fdb\u4e00\u6b65\u6784\u6210\u4e86\u6bb5\u843d":92,"\u591a\u673a\u8bad\u7ec3":80,"\u591a\u7ebf\u7a0b\u7684\u6570\u636e\u8bfb\u53d6":2,"\u591a\u8f6e\u5bf9\u8bdd\u7b49\u66f4\u4e3a\u590d\u6742\u7684\u8bed\u8a00\u6570\u636e":92,"\u5927\u591a\u6570\u5c42\u4e0d\u9700\u8981\u8fdc\u7a0b\u7a00\u758f\u8bad\u7ec3\u51fd\u6570":96,"\u5927\u591a\u6570\u5c42\u9700\u8981\u8bbe\u7f6e\u4e3a":96,"\u5927\u591a\u6570\u7f51\u7edc\u5c42\u4e0d\u9700\u8981\u652f\u6301\u8fdc\u7a0b\u7a00\u758f\u66f4\u65b0":96,"\u5927\u591a\u6570\u8bed\u8a00\u90fd\u652f\u6301\u4f7f\u7528c\u8bed\u8a00api":54,"\u5927\u5bb6\u53ef\u4ee5\u7528\u628a\u5f00\u53d1\u5de5\u5177\u5b89\u88c5\u8fdb\u5165":94,"\u5927\u5bb6\u53ef\u4ee5\u901a\u8fc7\u5b83\u9605\u8bfb\u6559\u7a0b":84,"\u5927\u5c0f\u4e0d\u4e00\u6837\u65f6":80,"\u5927\u6982\u82b1\u5341\u5206\u949f\u770b\u4e00\u4e0b":94,"\u5927\u90e8\u5206":102,"\u5929":90,"\u5929\u4e00\u5e7f\u573a":90,"\u5929\u4e00\u9601":90,"\u5934\u4fe1\u606f\u4e2d":82,"\u5934\u6587\u4ef6\u4e2d\u628a\u53c2\u6570\u5b9a\u4e49\u4e3a\u7c7b\u7684\u6210\u5458\u53d8\u91cf":96,"\u5934\u6587\u4ef6\u5982\u4e0b":96,"\u597d":90,"\u597d\u5403":90,"\u597d\u8bc4":123,"\u5982":[2,93,95,97,108],"\u5982\u4e0a\u4e00\u5c0f\u8282\u6240\u793a":98,"\u5982\u4e0b":[2,104],"\u5982\u4e0b\u56fe\u6240\u793a":[90,103],"\u5982\u4e0b\u6240\u793a":[108,122],"\u5982\u4e0b\u662f\u4e00\u6bb5\u4f7f\u7528mnist":4,"\u5982\u4e0b\u8868\u683c":123,"\u5982\u4f55\u5b58\u50a8\u7b49\u7b49":2,"\u5982\u4f55\u89e3\u6790\u8be5\u5730\u5740\u4e5f\u662f\u7528\u6237\u81ea\u5b9a\u4e49dataprovider\u65f6\u9700\u8981\u8003\u8651\u7684\u5730\u65b9":1,"\u5982\u4f55\u8d21\u732e":100,"\u5982\u4f55\u8d21\u732e\u4ee3\u7801":100,"\u5982\u4f55\u8fdb\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3":123,"\u5982\u56fe\u4e2dtrainer":34,"\u5982\u6709":97,"\u5982\u672c\u4f8b\u4e2d":2,"\u5982\u672c\u4f8b\u7684":2,"\u5982\u679c\u4e00\u4e2a\u7f51\u7edc\u5c42\u9700\u8981\u914d\u7f6e\u7684\u8bdd":96,"\u5982\u679c\u4e0a\u9762\u4e24\u6b65\u51fa\u73b0\u9519\u8bef":34,"\u5982\u679c\u4e0b\u8f7d\u6210\u529f":122,"\u5982\u679c\u4e0d\u4e3a0":106,"\u5982\u679c\u4e0d\u4e86\u89e3":2,"\u5982\u679c\u4e0d\u4f7f\u7528\u5206\u5e03\u5f0f\u5b58\u50a8":104,"\u5982\u679c\u4e0d\u4f7f\u7528docker":83,"\u5982\u679c\u4e0d\u4f7f\u7528docker\u7f16\u8bd1\u73af\u5883":83,"\u5982\u679c\u4e0d\u5207\u8bcd":123,"\u5982\u679c\u4e0d\u60f3\u4f7f\u7528":99,"\u5982\u679c\u4e0d\u6307\u5b9a\u8fd9\u4e2a\u6587\u4ef6":102,"\u5982\u679c\u4e0d\u6536\u655b":82,"\u5982\u679c\u4e3a":2,"\u5982\u679c\u4e3a0":106,"\u5982\u679c\u4e3a\u5426\u5219\u662f\u7528openbla":83,"\u5982\u679c\u4e3afals":106,"\u5982\u679c\u4e3atrue":[2,106],"\u5982\u679c\u4e4b\u540e\u60f3\u8981\u91cd\u65b0\u8bbe\u7f6e":83,"\u5982\u679c\u4ec5\u4ec5\u4fee\u6539\u4e00\u4e2a\u6587\u4ef6\u4f46\u63d0\u4ea4\u4e86\u5341\u51e0\u4e2acommit":95,"\u5982\u679c\u4ecd\u7136\u5b58\u5728\u95ee\u9898":86,"\u5982\u679c\u4ed4\u7ec6\u8bbe\u7f6e\u7684\u8bdd":106,"\u5982\u679c\u4f60\u53ea\u9700\u8981\u4f7f\u7528\u7b80\u5355\u7684rnn":93,"\u5982\u679c\u4f60\u60f3\u4f7f\u7528\u8fd9\u4e9b\u7279\u6027":108,"\u5982\u679c\u4f60\u60f3\u8981\u4fdd\u5b58\u67d0\u4e9b\u5c42\u7684\u7279\u5f81\u56fe":106,"\u5982\u679c\u4f60\u66fe\u5728\u6e90\u7801\u76ee\u5f55\u4e0b\u7f16\u8bd1\u8fc7\u5176\u4ed6\u5e73\u53f0\u7684paddlepaddle\u5e93":116,"\u5982\u679c\u4f60\u66fe\u7ecf\u5728\u6e90\u7801\u76ee\u5f55\u4e0b\u7f16\u8bd1\u8fc7\u5176\u4ed6\u5e73\u53f0\u7684paddlepaddle\u5e93":[115,117],"\u5982\u679c\u4f60\u6b63\u5728\u5904\u7406\u5e8f\u5217\u6807\u8bb0\u4efb\u52a1":93,"\u5982\u679c\u4f60\u8981\u4e3a\u4e86\u6d4b\u8bd5\u800c\u589e\u52a0\u65b0\u7684\u6587\u4ef6":96,"\u5982\u679c\u4f7f\u7528":121,"\u5982\u679c\u4f7f\u7528docker\u7f16\u8bd1\u73af\u5883":83,"\u5982\u679c\u4f7f\u7528mkl\u5e76\u4e14\u673a\u5668\u542b\u6709avx2\u6307\u4ee4\u96c6":83,"\u5982\u679c\u4f7f\u7528ssl\u8ba4\u8bc1":109,"\u5982\u679c\u4f7f\u7528swig\u6211\u4eec\u9700\u8981\u5c06\u5728interface\u6587\u4ef6\u91cc":54,"\u5982\u679c\u5173\u95edmkl":83,"\u5982\u679c\u51fa\u73b0\u4ee5\u4e0bpython\u76f8\u5173\u7684\u5355\u5143\u6d4b\u8bd5\u90fd\u8fc7\u4e0d\u4e86\u7684\u60c5\u51b5":77,"\u5982\u679c\u53c2\u6570\u4fdd\u5b58\u4e0b\u6765\u7684\u6a21\u578b\u76ee\u5f55":80,"\u5982\u679c\u53c2\u6570\u6a21\u578b\u6587\u4ef6\u7f3a\u5931":121,"\u5982\u679c\u53d1\u73b0\u6700\u65e9\u7684\u62a5\u9519\u5c31\u662f\u7f51\u7edc\u901a\u4fe1\u7684\u95ee\u9898":78,"\u5982\u679c\u540c\u65f6\u4f7f\u7528":104,"\u5982\u679c\u5728\u70b9\u51fb\u4e0b\u9762\u94fe\u63a5\u65f6\u51fa\u73b0\u5982\u4e0b\u767b\u9646\u754c\u9762":86,"\u5982\u679c\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u672a\u8bbe\u7f6easync":106,"\u5982\u679c\u5728\u8bad\u7ec3\u671f\u95f4\u540c\u65f6\u53d1\u8d77\u53e6\u5916\u4e00\u4e2a\u8fdb\u7a0b\u8fdb\u884c\u6d4b\u8bd5":106,"\u5982\u679c\u5728\u8bad\u7ec3\u914d\u7f6e\u4e2d\u8bbe\u7f6ebatch":106,"\u5982\u679c\u5728\u8bad\u7ec3nlp\u76f8\u5173\u6a21\u578b\u65f6":82,"\u5982\u679c\u591a\u4e2aop\u4f9d\u8d56\u4e00\u4e9b\u5171\u7528\u7684\u51fd\u6570":97,"\u5982\u679c\u5931\u8d25":71,"\u5982\u679c\u5b58\u5728\u67d0\u4e9btrainer\u6267\u884c\u901f\u5ea6\u8fc7\u6162\u4f1a\u5f71\u54cd\u6574\u4f53\u96c6\u7fa4\u7684\u901f\u5ea6":34,"\u5982\u679c\u5c06\u8fd9\u4e2a\u5185\u5b58\u6c60\u51cf\u5c0f":80,"\u5982\u679c\u5df2\u7ecf\u6709pod\u8fd0\u884c":111,"\u5982\u679c\u5df2\u7ecf\u6b63\u5728\u6267\u884c\u4fdd\u5b58\u68c0\u67e5\u70b9\u7684\u7ebf\u7a0b":34,"\u5982\u679c\u5e0c\u671b\u53ef\u4ee5\u5728\u540e\u53f0\u8fd0\u884cpserver\u7a0b\u5e8f":104,"\u5982\u679c\u5f53\u524dmpi\u96c6\u7fa4\u5e76\u4e0d\u652f\u6301\u4efb\u52a1\u72ec\u5360\u6a21\u5f0f":78,"\u5982\u679c\u60a8\u5728\u4f7f\u7528window":84,"\u5982\u679c\u60a8\u60f3\u8981\u66f4\u6df1\u5165\u4e86\u89e3deep":84,"\u5982\u679c\u60a8\u671f\u671b\u5728\u7f16\u8bd1\u5b8c\u6210\u540e\u7acb\u5373\u6267\u884c\u6240\u6709\u7684\u5355\u5143\u6d4b\u8bd5":83,"\u5982\u679c\u60a8\u6ca1\u6709\u542c\u8bf4":94,"\u5982\u679c\u60a8\u7684\u7535\u8111\u4e0d\u652f\u6301avx":84,"\u5982\u679c\u60a8\u7684gpu\u7406\u8bba\u53ef\u4ee5\u8fbe\u52306":103,"\u5982\u679c\u60a8\u9009\u62e9\u4e0d\u4f7f\u7528docker\u955c\u50cf":83,"\u5982\u679c\u60f3\u4e3a\u4e00\u4e2a\u6570\u636e\u6587\u4ef6\u8fd4\u56de\u591a\u6761\u6837\u672c":2,"\u5982\u679c\u60f3\u4f7f\u7528\u53ef\u89c6\u5316\u7684\u5206\u6790\u5668":103,"\u5982\u679c\u60f3\u5f88\u597d\u7684\u7406\u89e3\u7a0b\u5e8f\u7684\u884c\u4e3a":103,"\u5982\u679c\u60f3\u6539\u53d8\u539f\u6709tensor\u7684shape\u4fe1\u606f":98,"\u5982\u679c\u60f3\u8981\u4e86\u89e3\u53cc\u5c42rnn\u5728\u5177\u4f53\u95ee\u9898\u4e2d\u7684\u4f7f\u7528":90,"\u5982\u679c\u60f3\u8981\u542f\u7528paddlepaddle\u7684\u5185\u7f6e\u5b9a\u65f6\u5668":103,"\u5982\u679c\u6211\u4eec\u53ea\u9700\u8981\u7f16\u8bd1\u4e00\u4e2a\u53ea\u652f\u6301":94,"\u5982\u679c\u6211\u77e5\u9053\u5185\u6838\u82b1\u4e8610ms\u6765\u79fb\u52a81gb\u6570\u636e":103,"\u5982\u679c\u6267\u884c\u5931\u8d25":109,"\u5982\u679c\u6267\u884c\u6210\u529f":122,"\u5982\u679c\u6307\u5b9a\u4e862\u4e2alayer\u4f5c\u4e3a\u8f93\u51fa\u5c42":80,"\u5982\u679c\u63d0\u793a\u6b63\u786e":99,"\u5982\u679c\u652f\u6301\u589e\u52a0\u6b64\u53c2\u6570\u63d0\u4ea4":78,"\u5982\u679c\u6570\u636e\u6587\u4ef6\u5b58\u4e8e\u672c\u5730\u78c1\u76d8":1,"\u5982\u679c\u662f\u4f7f\u7528\u975essl\u65b9\u5f0f\u8bbf\u95ee":109,"\u5982\u679c\u662f\u5176\u5b83\u7c7b\u578b":35,"\u5982\u679c\u6709\u591a\u4e2a\u8f93\u5165":92,"\u5982\u679c\u6709\u591a\u4e2a\u8f93\u5165\u5e8f\u5217":92,"\u5982\u679c\u6709\u66f4\u590d\u6742\u7684\u4f7f\u7528":1,"\u5982\u679c\u6709\u9700\u8981\u4fee\u6539\u7684\u5730\u65b9":95,"\u5982\u679c\u6709bugfix\u7684\u884c\u4e3a":71,"\u5982\u679c\u672a\u8bbe\u7f6e":106,"\u5982\u679c\u672a\u8bbe\u7f6egpu":108,"\u5982\u679c\u67d0\u4e00\u4e2a\u7c7b\u578b\u9700\u8981\u5f15\u7528\u53e6\u4e00\u4e2a\u7c7b\u578b":55,"\u5982\u679c\u67d0\u4e00\u4e2apaddl":55,"\u5982\u679c\u67d0\u4e00\u4e2apaddle\u6982\u5ff5\u5fc5\u987b\u8981\u66b4\u9732":55,"\u5982\u679c\u67d0\u4e00\u5757\u6839\u672c\u5c31\u4e0d\u600e\u4e48\u8017\u65f6":103,"\u5982\u679c\u68c0\u67e5\u5230\u5206\u914d\u5728\u4e0d\u540c\u53c2\u6570\u670d\u52a1\u5668\u4e0a\u7684\u53c2\u6570\u7684\u5206\u5e03\u4e0d\u5747\u5300\u6b21\u6570\u5927\u4e8echeck":106,"\u5982\u679c\u6ca1\u6709\u5b89\u88c5nvidia":84,"\u5982\u679c\u6ca1\u6709\u5b9a\u4e49memori":92,"\u5982\u679c\u6ca1\u6709\u8bbe\u7f6etest":1,"\u5982\u679c\u6ca1\u8fc7":95,"\u5982\u679c\u6d88\u606f\u6570\u636e\u592a\u5c0f":106,"\u5982\u679c\u6ee1\u8db3\u6761\u4ef6":34,"\u5982\u679c\u7528\u516c\u7528\u7684\u7535\u8111\u5f00\u53d1":94,"\u5982\u679c\u7528\u6237\u4e0d\u663e\u793a\u6307\u5b9a\u8fd4\u56de\u6570\u636e\u7684\u5bf9\u5e94\u5173\u7cfb":2,"\u5982\u679c\u7528\u6237\u4e0d\u9700\u8981\u8bbf\u95eelstm\u7684\u4e2d\u95f4\u53d8\u91cf":81,"\u5982\u679c\u7528\u6237\u60f3\u8981\u4e86\u89e3\u8be6\u7ec6\u7684\u6570\u636e\u96c6\u7684\u683c\u5f0f":121,"\u5982\u679c\u7528\u6237\u60f3\u8981\u81ea\u5b9a\u4e49\u521d\u59cb\u5316\u65b9\u5f0f":82,"\u5982\u679c\u7528\u6237\u8981\u628apaddle\u7684\u9759\u6001\u5e93":54,"\u5982\u679c\u7528\u81ea\u5df1\u7684\u7535\u8111\u5f00\u53d1":94,"\u5982\u679c\u771f\u60f3\u6316\u6398\u5185\u6838\u6df1\u5904\u7684\u67d0\u4e2a\u79d8\u5bc6":103,"\u5982\u679c\u7a0b\u5e8f\u5d29\u6e83\u4f60\u4e5f\u53ef\u4ee5\u624b\u52a8\u7ec8\u6b62":104,"\u5982\u679c\u7cfb\u7edf\u5b89\u88c5\u4e86\u591a\u4e2apython\u7248\u672c":77,"\u5982\u679c\u7cfb\u7edf\u652f\u6301":[77,86],"\u5982\u679c\u7cfb\u7edf\u652f\u6301\u7684\u662f":[77,86],"\u5982\u679c\u7f16\u8bd1\u7684\u65f6\u5019\u6211\u4eec\u7528\u4e86":94,"\u5982\u679c\u7f51\u7edc\u5c42\u4e0d\u9700\u8981\u8fdc\u7a0b\u7a00\u758f\u66f4\u65b0":96,"\u5982\u679c\u7f51\u7edc\u67b6\u6784\u7b80\u5355":93,"\u5982\u679c\u8981\u4e0a\u4f20gpu\u7248\u672c\u7684\u5305":71,"\u5982\u679c\u8981\u542f\u7528gpu":104,"\u5982\u679c\u8981\u8fd0\u884c\u6240\u6709\u7684\u5355\u5143\u6d4b\u8bd5":95,"\u5982\u679c\u89e3\u51b3\u4e86\u67d0\u4e2aissue\u7684\u95ee\u9898":95,"\u5982\u679c\u8bad\u7ec3\u4e00\u4e2apass":82,"\u5982\u679c\u8bad\u7ec3\u8fc7\u7a0b\u7684\u7684cost\u660e\u663e\u9ad8\u4e8e\u8fd9\u4e2a\u5e38\u6570\u8f93\u51fa\u7684cost":82,"\u5982\u679c\u8bbe\u7f6e":2,"\u5982\u679c\u8bbe\u7f6e\u8be5\u53c2\u6570":106,"\u5982\u679c\u8bc4\u5ba1\u610f\u89c1\u6bd4\u8f83\u591a":95,"\u5982\u679c\u8c03\u7528\u9759\u6001\u5e93\u53ea\u80fd\u5c06\u9759\u6001\u5e93\u4e0e\u89e3\u91ca\u5668\u94fe\u63a5":54,"\u5982\u679c\u8f93\u51fa\u662fno":84,"\u5982\u679c\u8fd0\u884c":77,"\u5982\u679c\u8fd0\u884c\u6210\u529f":122,"\u5982\u679c\u8fd8\u4e0d\u884c":77,"\u5982\u679c\u9700\u8981\u5b89\u88c5\u652f\u6301gpu\u7684\u7248\u672c":[86,88],"\u5982\u679c\u9700\u8981\u6269\u5927\u77e9\u9635":96,"\u5982\u679c\u9700\u8981\u7f29\u51cf\u77e9\u9635":96,"\u5982\u679c\u9700\u8981\u83b7\u53d6":86,"\u5982\u679cgpu":97,"\u5982\u679clearning_rate\u592a\u5927":82,"\u5982\u679clearning_rate\u592a\u5c0f":82,"\u5982\u679cop\u6ca1\u6709\u5b9e\u73b0gpu":97,"\u5982\u679cop\u7684\u67d0\u4e2a\u8f93\u5165\u4e0d\u53c2\u4e0e\u53cd\u5411\u68af\u5ea6\u7684\u8ba1\u7b97":97,"\u5982\u679cpaddlepaddle\u5305\u5df2\u7ecf\u5728python\u7684sit":77,"\u5982\u679cpaddlepaddle\u5e93\u9700\u8981\u540c\u65f6\u652f\u6301\u771f\u673a\u548c\u6a21\u62df\u5668":116,"\u5982\u679cparamet":34,"\u5982\u6bcf\u4e2a\u6587\u4ef6\u53ea\u6709\u4e00\u4e2a":95,"\u5982\u795e\u7ecf\u5143\u6fc0\u6d3b\u503c\u7b49":80,"\u5982\u8981build\u8fd9\u4e2a\u5f00\u53d1\u955c\u50cf":95,"\u5982\u9ad8\u4eae\u90e8\u5206":103,"\u5982train":104,"\u5b50":90,"\u5b50\u53e5":92,"\u5b50\u53e5\u7684\u5355\u8bcd\u6570\u548c\u6307\u5b9a\u7684\u4e00\u4e2a\u8f93\u5165\u5e8f\u5217\u4e00\u81f4":92,"\u5b50\u76ee\u5f55":94,"\u5b57\u5178\u5171\u5305\u542b":121,"\u5b57\u5178\u91c7\u7528utf8\u7f16\u7801":121,"\u5b57\u6bb5\u4e2d":111,"\u5b57\u6bb5\u4e3a\u4f8b":80,"\u5b57\u6bb5\u8868\u793a\u5bb9\u5668\u7684\u73af\u5883\u53d8\u91cf":111,"\u5b57\u6bb5\u8868\u793a\u8fd9\u4e2ajob\u4f1a\u540c\u65f6\u5f00\u542f3\u4e2apaddlepaddle\u8282\u70b9":111,"\u5b57\u6bb5\u8bbe\u4e3a":71,"\u5b57\u7b26\u4e32":35,"\u5b58\u50a8":35,"\u5b58\u50a8\u5377":109,"\u5b58\u5165settings\u5bf9\u8c61":2,"\u5b66\u4e60":94,"\u5b66\u4e60\u6210\u672c\u9ad8":54,"\u5b66\u4e60\u7387\u4e3a":82,"\u5b81\u6ce2":90,"\u5b83\u4eec\u7684\u6587\u4ef6\u540d\u662f":35,"\u5b83\u4f1a\u5728dataprovider\u521b\u5efa\u7684\u65f6\u5019\u6267\u884c":2,"\u5b83\u4f7f\u752850\u5c42\u7684resnet\u6a21\u578b\u6765\u5bf9":122,"\u5b83\u5305\u542b\u4ee5\u4e0b\u51e0\u6b65":96,"\u5b83\u5305\u542b\u4ee5\u4e0b\u53c2\u6570":96,"\u5b83\u5305\u542b\u7684\u5c5e\u6027\u53c2\u6570\u5982\u4e0b":2,"\u5b83\u53eb\u505a":93,"\u5b83\u53ef\u4ee5\u5e2e\u52a9\u51cf\u5c11\u5206\u53d1\u5ef6\u8fdf":104,"\u5b83\u53ef\u4ee5\u5e2e\u52a9\u6211\u4eec\u683c\u5f0f\u5316\u6e90\u4ee3\u7801":95,"\u5b83\u53ef\u4ee5\u6307\u6d4b\u91cf\u4e00\u4e2a\u7a0b\u5e8f\u7684\u7a7a\u95f4":103,"\u5b83\u53ef\u80fd\u6709\u4e0d\u6b62\u4e00\u4e2a\u6743\u91cd":96,"\u5b83\u548c\u6570\u636e\u4f20\u5165\u51fd\u6570\u7684\u7b2c\u4e00\u4e2a\u53c2\u6570":2,"\u5b83\u5b9a\u4e49\u4e86":93,"\u5b83\u5b9a\u4e49\u89e3\u7801\u7f51\u7edc\u7684":93,"\u5b83\u5c06\u88ab\u5206\u53d1\u5230":104,"\u5b83\u5c06\u8fd4\u56de\u5982\u4e0b\u7684\u5b57\u5178":122,"\u5b83\u5e76\u4e0d\u662f\u4e00\u4e2a\u5b8c\u6574\u7684recurr":81,"\u5b83\u5e94\u8be5\u6253\u5370\u51fa\u9884\u6d4b\u4f4f\u623f\u6570\u636e\u7684\u6e05\u5355":88,"\u5b83\u652f\u6301\u591a\u7ebf\u7a0b\u66f4\u65b0":96,"\u5b83\u662finteger_value\u7c7b\u578b\u7684":90,"\u5b83\u662finteger_value_sequence\u7c7b\u578b\u7684":90,"\u5b83\u6709\u52a9\u4e8e\u5e2e\u52a9\u9891\u7e41\u4fee\u6539\u548c\u8bbf\u95ee\u5de5\u4f5c\u533a\u6587\u4ef6\u7684\u7528\u6237\u51cf\u5c11\u8d1f\u62c5":104,"\u5b83\u7684":93,"\u5b83\u7684\u529f\u80fd\u662f":97,"\u5b83\u7684\u6bcf\u4e00\u4e2a\u5143\u7d20":89,"\u5b83\u7684\u8f93\u5165\u4e0e\u7ecf\u8fc7\u5b66\u4e60\u7684\u53c2\u6570\u505a\u5185\u79ef\u5e76\u52a0\u4e0a\u504f\u7f6e":96,"\u5b83\u8868\u793a":94,"\u5b83\u9996\u5148\u8c03\u7528\u57fa\u6784\u9020\u51fd\u6570":96,"\u5b89\u6392":90,"\u5b89\u88c5":102,"\u5b89\u88c5\u4e0e\u7f16\u8bd1":88,"\u5b89\u88c5\u540e":84,"\u5b89\u88c5\u540e\u7684\u76ee\u5f55\u7ed3\u6784\u4e3a":55,"\u5b89\u88c5\u597ddocker\u4e4b\u540e\u53ca\u53ef\u7528\u4ee5\u4e0b\u547d\u4ee4\u542f\u52a8\u5de5\u5177":99,"\u5b89\u88c5\u597ddocker\u4e4b\u540e\u53ef\u4ee5\u4f7f\u7528\u6e90\u7801\u76ee\u5f55\u4e0b\u7684\u811a\u672c\u6784\u5efa\u6587\u6863":99,"\u5b89\u88c5\u5b8c\u6210\u4e4b\u540e":[104,116],"\u5b89\u88c5\u5b8c\u6bd5\u540e":102,"\u5b89\u88c5\u5f00\u53d1\u5de5\u5177\u5230":94,"\u5b89\u88c5\u6587\u6863":87,"\u5b89\u88c5\u65b9\u5f0f\u6765\u5feb\u901f\u5b89\u88c5paddlepaddl":104,"\u5b89\u88c5\u6d41\u7a0b":123,"\u5b89\u88c5\u8be5\u8f6f\u4ef6\u5305\u5c31\u53ef\u4ee5\u5728python\u73af\u5883\u4e0b\u5b9e\u73b0\u6a21\u578b\u9884\u6d4b":4,"\u5b89\u88c5paddlepaddl":123,"\u5b89\u9759":90,"\u5b8c\u6210":95,"\u5b8c\u6210\u4e00\u4e2a\u4f20\u8f93\u52a8\u4f5c\u5b8c\u6210\u7684\u65f6\u95f4\u4e5f\u6bd4\u8f83\u77ed":44,"\u5b8c\u6210\u4efb\u610f\u7684\u8fd0\u7b97\u903b\u8f91":92,"\u5b8c\u6210\u540evolume\u4e2d\u7684\u6587\u4ef6\u5185\u5bb9\u5927\u81f4\u5982\u4e0b":111,"\u5b8c\u6210\u5728windows\u4e0a\u5b89\u88c5\u548c\u4f7f\u7528dock":84,"\u5b8c\u6210\u5b89\u88c5":86,"\u5b8c\u6210\u6570\u636e\u7684\u9884\u5904\u7406":35,"\u5b8c\u6210\u76f8\u5e94\u7684\u8ba1\u7b97":89,"\u5b8c\u6210paddlepaddle\u7684\u5b89\u88c5":87,"\u5b8c\u6574\u4ee3\u7801\u53ef\u4ee5\u53c2\u8003\u793a\u4f8b":80,"\u5b8c\u6574\u6e90\u7801\u53ef\u53c2\u8003":82,"\u5b8c\u6574\u7684\u4ee3\u7801\u89c1":4,"\u5b8c\u6574\u7684\u53c2\u6570\u77e9\u9635\u88ab\u5206\u5e03\u5728\u4e0d\u540c\u7684\u53c2\u6570\u670d\u52a1\u5668\u4e0a":96,"\u5b8c\u6574\u7684\u914d\u7f6e\u6587\u4ef6\u5728":93,"\u5b98\u65b9\u6587\u6863":83,"\u5b9a\u4e49\u4e00\u4e2a\u65f6\u95f4\u6b65\u4e4b\u5185rnn\u5355\u5143\u5b8c\u6210\u7684\u8ba1\u7b97":92,"\u5b9a\u4e49\u4e00\u4e2apython\u7684":2,"\u5b9a\u4e49\u4e86\u4e00\u4e2a\u53ea\u8bfb\u7684memori":92,"\u5b9a\u4e49\u4e86lstm\u5355\u5143\u5728\u4e00\u4e2a\u65f6\u95f4\u6b65\u5185\u7684\u8ba1\u7b97\u8fc7\u7a0b":81,"\u5b9a\u4e49\u4f4d\u7f6e":97,"\u5b9a\u4e49\u5728\u5916\u5c42":92,"\u5b9a\u4e49\u5f02\u6b65\u8bad\u7ec3\u7684\u957f\u5ea6":106,"\u5b9a\u4e49\u6e90\u8bed\u53e5\u7684\u6570\u636e\u5c42":93,"\u5b9a\u4e49\u7c7b\u578b":97,"\u5b9a\u4e49\u89e3\u7801\u5668\u7684memori":93,"\u5b9a\u4e49\u8f93\u5165":97,"\u5b9a\u4e49\u8f93\u51fa":97,"\u5b9a\u4e49\u8f93\u51fa\u51fd\u6570":93,"\u5b9a\u4e49\u95e8\u63a7\u5faa\u73af\u5355\u5143\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u5355\u6b65\u51fd\u6570":93,"\u5b9d\u5854\u7684\u5e95\u7aef\u9700\u8981\u575a\u5b9e\u7684\u57fa\u5ea7\u6765\u652f\u6491":87,"\u5b9e\u73b0\u4e24\u4e2a\u5b8c\u5168\u7b49\u4ef7\u7684\u5168\u8fde\u63a5rnn":90,"\u5b9e\u73b0\u524d\u5411\u4f20\u64ad\u7684\u90e8\u5206\u6709\u4e0b\u9762\u51e0\u4e2a\u6b65\u9aa4":96,"\u5b9e\u73b0\u5355\u6b65\u51fd\u6570":93,"\u5b9e\u73b0\u540e\u5411\u4f20\u64ad\u7684\u90e8\u5206\u6709\u4e0b\u9762\u51e0\u4e2a\u6b65\u9aa4":96,"\u5b9e\u73b0\u5728":97,"\u5b9e\u73b0\u5bf9":98,"\u5b9e\u73b0\u6570\u636e\u8f93\u5165\u51fd\u6570":2,"\u5b9e\u73b0\u65b0\u7684op\u90fd\u6dfb\u52a0\u81f3\u76ee\u5f55":97,"\u5b9e\u73b0\u6784\u9020\u51fd\u6570":96,"\u5b9e\u73b0\u7684":81,"\u5b9e\u73b0\u7b80\u5355":54,"\u5b9e\u73b0\u7ec6\u8282":96,"\u5b9e\u73b0\u7f51\u7edc\u5c42\u7684\u524d\u5411\u4f20\u64ad":96,"\u5b9e\u73b0\u7f51\u7edc\u5c42\u7684\u540e\u5411\u4f20\u64ad":96,"\u5b9e\u73b0\u8bcd\u8bed\u548c\u53e5\u5b50\u4e24\u4e2a\u7ea7\u522b\u7684\u53cc\u5c42rnn\u7ed3\u6784":92,"\u5b9e\u73b0\u8be5\u5c42\u7684c":96,"\u5b9e\u9645\u4e0a\u4f7f\u7528\u4e86":81,"\u5b9e\u9645\u4e0a\u53ea\u6709":122,"\u5b9e\u9645\u4e0a\u9700\u8981\u7684\u8f93\u51fa\u7ed3\u679c\u662f\u4e24\u4e2a\u77e9\u9635":80,"\u5ba2\u6237":90,"\u5bb6":90,"\u5bb9\u5668":109,"\u5bb9\u5668\u4e0d\u4f1a\u4fdd\u7559\u5728\u8fd0\u884c\u65f6\u751f\u6210\u7684\u6570\u636e":109,"\u5bb9\u5668\u8fd0\u884c\u90fd\u8fd0\u884c":111,"\u5bb9\u5668\u9ed8\u8ba4\u6267\u884c":115,"\u5bbd\u5ea6\u7b49\u4e8e\u914d\u7f6e\u4e2dlayer\u7684s":80,"\u5bbf\u4e3b\u673a\u7684c":[115,116,117],"\u5bbf\u4e3b\u673a\u76ee\u5f55":109,"\u5bc4\u5b58\u5668\u4f7f\u7528\u60c5\u51b5\u548c\u5171\u4eab\u5185\u5b58\u4f7f\u7528\u60c5\u51b5\u80fd\u8ba9\u6211\u4eec\u5bf9gpu\u7684\u6574\u4f53\u4f7f\u7528\u6709\u66f4\u597d\u7684\u7406\u89e3":103,"\u5bf9":90,"\u5bf9\u4e00\u4e2a5\u7ef4\u975e\u5e8f\u5217\u7684\u7a00\u758f01\u5411\u91cf":[2,87],"\u5bf9\u4e00\u4e2a5\u7ef4\u975e\u5e8f\u5217\u7684\u7a00\u758f\u6d6e\u70b9\u5411\u91cf":[2,87],"\u5bf9\u4e8e":93,"\u5bf9\u4e8e\u4e0d\u540c\u7684\u8bad\u7ec3\u4efb\u52a1":104,"\u5bf9\u4e8e\u4e0d\u540c\u7684\u96c6\u7fa4\u5e73\u53f0":104,"\u5bf9\u4e8e\u4e0d\u540c\u8bed\u8a00":54,"\u5bf9\u4e8e\u4e24\u79cd\u4e0d\u540c\u7684\u8f93\u5165\u6570\u636e\u7c7b\u578b":90,"\u5bf9\u4e8e\u4e60\u60ef\u4f7f\u7528windows\u548cmacos\u7684\u5f00\u53d1\u8005\u6765\u8bf4":94,"\u5bf9\u4e8e\u5185\u5b58\u8f83\u5c0f\u7684\u673a\u5668":2,"\u5bf9\u4e8e\u5355\u5c42rnn":90,"\u5bf9\u4e8e\u5355\u5c42rnn\u7684\u6570\u636e\u4e00\u5171\u6709\u4e24\u4e2a\u6837\u672c":90,"\u5bf9\u4e8e\u53cc\u5c42rnn":90,"\u5bf9\u4e8e\u540c\u4e00\u6bb5c":54,"\u5bf9\u4e8e\u540c\u6837\u7684\u6570\u636e":90,"\u5bf9\u4e8e\u56fd\u5185\u7528\u6237":84,"\u5bf9\u4e8e\u591a\u8bed\u8a00\u63a5\u53e3":54,"\u5bf9\u4e8e\u5927\u591a\u6570\u8bed\u8a00":54,"\u5bf9\u4e8e\u6027\u80fd\u7684\u5173\u952e\u8def\u5f84\u90fd\u505a\u51fa\u4e86\u7ea2\u8272\u6807\u8bb0":102,"\u5bf9\u4e8e\u6211\u4eec\u652f\u6301\u7684\u5168\u90e8\u77e9\u9635\u64cd\u4f5c":96,"\u5bf9\u4e8e\u672c\u6837\u4f8b\u4ee3\u7801":104,"\u5bf9\u4e8e\u6bb5\u843d\u7684\u6587\u672c\u5206\u7c7b":90,"\u5bf9\u4e8e\u6bcf\u4e00\u4e2a\u5355\u5c42rnn\u7684\u6570\u636e":90,"\u5bf9\u4e8e\u6bcf\u79cd\u7c7b\u578b":55,"\u5bf9\u4e8e\u6bcf\u79cdc":55,"\u5bf9\u4e8e\u7ed9\u5b9a\u7684\u4e00\u6761\u6587\u672c":123,"\u5bf9\u4e8e\u914d\u5907\u6709\u6ce8\u610f\u529b\u673a\u5236\u7684\u89e3\u7801\u5668":93,"\u5bf9\u4e8eamazon":123,"\u5bf9\u4ee3\u7801\u8fdb\u884c\u6027\u80fd\u5206\u6790":103,"\u5bf9\u5168\u8fde\u63a5\u5c42\u6765\u8bf4":96,"\u5bf9\u52a0\u8f7d\u9884\u8bad\u7ec3\u53c2\u6570\u7684\u5c42":82,"\u5bf9\u5df2\u7ecfpush\u5230\u8fdc\u7a0b\u4ed3\u5e93\u7684\u591a\u4e2acommit":95,"\u5bf9\u5e94":116,"\u5bf9\u5e94\u4e00\u4e2a\u5b50\u53e5":92,"\u5bf9\u5e94\u4e00\u4e2a\u8bcd":92,"\u5bf9\u5e94\u4e8e\u5b57\u5178":121,"\u5bf9\u5e94\u7684":2,"\u5bf9\u5e94\u7684\u68af\u5ea6op\u8ba1\u7b97\u4e4b\u4e2d":97,"\u5bf9\u6574\u4e2a\u65b0\u5411\u91cf\u96c6\u5408\u7684\u6bcf\u4e00\u4e2a\u7ef4\u5ea6\u53d6\u6700\u5927\u503c\u6765\u8868\u793a\u6700\u540e\u7684\u53e5\u5b50":123,"\u5bf9\u6bcf\u4e2a\u8f93\u5165":96,"\u5bf9\u6bcf\u4e2a\u8f93\u5165\u4e58\u4e0a\u53d8\u6362\u77e9\u9635":96,"\u5bf9\u6bd4":54,"\u5bf9\u6bd4\u53cd\u5411op\u4e0d\u540c\u8bbe\u5907":97,"\u5bf9\u6fc0\u6d3b\u6c42\u5bfc":96,"\u5bf9\u7528\u6237\u6765\u8bf4":2,"\u5bf9\u8bad\u7ec3\u6570\u636e\u8fdb\u884cshuffl":2,"\u5bf9\u8bc4\u5ba1\u610f\u89c1\u4e0d\u540c\u610f\u7684":95,"\u5bf9\u8bc4\u5ba1\u610f\u89c1\u540c\u610f\u4e14\u6309\u5176\u4fee\u6539\u5b8c\u7684":95,"\u5bf9\u8be5\u5411\u91cf\u8fdb\u884c\u975e\u7ebf\u6027\u53d8\u6362":123,"\u5bf9\u8c61":82,"\u5bf9\u8f93\u5165\u53c2\u6570\u7684\u5b89\u5168\u6027\u8fdb\u884c\u4e86\u5fc5\u8981\u7684\u5224\u65ad":55,"\u5bf9\u8f93\u51fa\u7684\u5408\u5e76":92,"\u5bf9\u8fd9\u4e2a\u7248\u672c\u7684\u63d0\u4ea4":71,"\u5bf9\u9762":90,"\u5bf9check":2,"\u5bf9sparse_binary_vector\u548csparse_float_vector":[2,87],"\u5bfb\u627epython\u4e0ec":102,"\u5bfc\u51fa\u8fd9\u4e9b\u63a5\u53e3":55,"\u5bfc\u81f4\u4e86\u6d6e\u70b9\u6570\u6ea2\u51fa":80,"\u5bfc\u81f4\u53c2\u6570\u6536\u655b\u5230\u4e86\u4e00\u4e9b\u5947\u5f02\u7684\u60c5\u51b5":80,"\u5bfc\u81f4\u53c2\u6570\u7d2f\u52a0":80,"\u5bfc\u81f4\u7f16\u8bd1paddlepaddle\u5931\u8d25":77,"\u5bfc\u81f4\u8bad\u7ec3\u65f6\u95f4\u8fc7\u957f":82,"\u5c01\u88c5\u4e86":103,"\u5c01\u88c5\u8be5\u5c42\u7684python\u63a5\u53e3":96,"\u5c06":[2,71,82,103,123],"\u5c06\u4e00\u4e2a\u795e\u7ecf\u7f51\u7edc\u53c2\u6570\u62c6\u5206\u6210\u591a\u4efd":34,"\u5c06\u4e0a\u4e00\u65f6\u95f4\u6b65\u6240\u751f\u6210\u7684\u8bcd\u7684\u5411\u91cf\u6765\u4f5c\u4e3a\u5f53\u524d\u65f6\u95f4\u6b65\u7684\u8f93\u5165":93,"\u5c06\u4f1a\u4f18\u5148\u4f7f\u7528":104,"\u5c06\u4f1a\u59cb\u7ec8\u4f7f\u7528":115,"\u5c06\u4f1a\u5c06\u7528\u6237\u4f20\u8fdb\u6765\u7684\u914d\u7f6e\u53c2\u6570\u4f20\u9012cmake\u7cfb\u7edf":115,"\u5c06\u4f1a\u81ea\u52a8\u8ba1\u7b97\u51fa\u4e00\u4e2a\u5408\u9002\u7684\u503c":106,"\u5c06\u5176\u8bbe\u7f6e\u6210":80,"\u5c06\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217\u6570\u636e\u5148\u53d8\u6362\u6210\u5355\u5c42\u65f6\u95f4\u5e8f\u5217\u6570\u636e":90,"\u5c06\u542b\u6709\u5b50\u53e5":92,"\u5c06\u542b\u6709\u8bcd\u8bed\u7684\u53e5\u5b50\u5b9a\u4e49\u4e3a\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":92,"\u5c06\u5916\u90e8\u7684\u5b58\u50a8\u670d\u52a1\u5728kubernetes\u4e2d\u63cf\u8ff0\u6210\u4e3a\u7edf\u4e00\u7684\u8d44\u6e90\u5f62\u5f0f":109,"\u5c06\u591a\u53e5\u8bdd\u770b\u6210\u4e00\u4e2a\u6574\u4f53\u540c\u65f6\u4f7f\u7528encoder\u538b\u7f29":90,"\u5c06\u591a\u53f0\u673a\u5668\u7684\u6d4b\u8bd5\u7ed3\u679c\u5408\u5e76":106,"\u5c06\u5927\u91cf\u7684":54,"\u5c06\u5b57\u5178\u7684\u5730\u5740\u4f5c\u4e3aargs\u4f20\u7ed9dataprovid":82,"\u5c06\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u524d\u5411\u548c\u53cd\u5411\u90e8\u5206\u6df7\u5408\u5728\u4e00\u8d77":93,"\u5c06\u6027\u80fd\u5206\u6790\u7ed3\u679c\u4ee5\u7f51\u9875\u7684\u5f62\u5f0f\u5c55\u793a\u51fa\u6765":102,"\u5c06\u6027\u80fd\u5206\u6790\u7ed3\u679c\u6309\u7167tottime\u6392\u5e8f":102,"\u5c06\u6570\u636e\u5207\u5206\u6210\u591a\u4efd":104,"\u5c06\u6570\u636e\u5904\u7406\u6210\u89c4\u8303\u683c\u5f0f":121,"\u5c06\u6570\u636e\u7ec4\u5408\u6210batch\u8fdb\u884c\u8bad\u7ec3":2,"\u5c06\u65b0\u5206\u652f\u7684\u7248\u672c\u6253\u4e0atag":71,"\u5c06\u65b0\u5efa\u7684\u6743\u91cd\u52a0\u5165\u6743\u91cd\u8868":96,"\u5c06\u660e\u6587\u53c2\u6570\u8f6c\u5316\u4e3apaddlepaddle\u53ef\u52a0\u8f7d\u7684\u6a21\u578b\u53c2\u6570\u65f6":82,"\u5c06\u672c\u5730\u7684\u4fee\u6539\u63a8\u9001\u5230":95,"\u5c06\u6837\u672c\u7684\u5730\u5740\u653e\u5165\u53e6\u4e00\u4e2a\u6587\u672c\u6587\u4ef6":2,"\u5c06\u6b64\u76ee\u5f55\u6302\u8f7d\u4e3a\u5bb9\u5668\u7684":111,"\u5c06\u73af\u5883\u53d8\u91cf\u8f6c\u6362\u6210paddle\u7684\u547d\u4ee4\u884c\u53c2\u6570":111,"\u5c06\u7528\u4e8epython":97,"\u5c06\u7ed3\u679c\u4fdd\u5b58\u5230\u6b64\u76ee\u5f55\u91cc":111,"\u5c06\u884c\u4e2d\u7684\u6570\u636e\u8f6c\u6362\u6210\u4e0einput_types\u4e00\u81f4\u7684\u683c\u5f0f":2,"\u5c06\u88ab\u5206\u4e3a":121,"\u5c06\u8bad\u7ec3\u6587\u4ef6\u4e0e\u5207\u5206\u597d\u7684\u6570\u636e\u4e0a\u4f20\u5230\u5171\u4eab\u5b58\u50a8":111,"\u5c06\u8be5\u53e5\u8bdd\u5305\u542b\u7684\u6240\u6709\u5355\u8bcd\u5411\u91cf\u6c42\u5e73\u5747":123,"\u5c06\u8df3\u8fc7\u5206\u53d1\u9636\u6bb5\u76f4\u63a5\u542f\u52a8\u6240\u6709\u8282\u70b9\u7684\u96c6\u7fa4\u4f5c\u4e1a":104,"\u5c06\u8fd9\u79cd\u8de8\u8d8a\u65f6\u95f4\u6b65\u7684\u8fde\u63a5\u7528\u4e00\u4e2a\u7279\u6b8a\u7684\u795e\u7ecf\u7f51\u7edc\u5355\u5143\u5b9e\u73b0":90,"\u5c06\u8fdc\u7a0b\u4ed3\u5e93":95,"\u5c06\u900f\u660e":104,"\u5c06\u9700\u8981\u8f93\u51fa\u7684\u5c42\u4f5c\u4e3a":80,"\u5c06cuda\u5e93\u548clinux\u8bbe\u5907\u6302\u8f7d\u5230docker\u5bb9\u5668\u5185":84,"\u5c06ip\u6392\u5e8f\u751f\u6210\u7684\u5e8f\u53f7\u4f5c\u4e3atrain":111,"\u5c06master\u5206\u652f\u7684\u5408\u5165commit\u6253\u4e0atag":71,"\u5c06node\u8282\u70b9\u7684ip\u5730\u5740\u4fdd\u5b58\u5230machines\u6587\u4ef6\u4e2d":104,"\u5c06paddlepaddle\u4fdd\u5b58\u7684\u6a21\u578b\u53c2\u6570\u8fd8\u539f\u56de\u660e\u6587\u65f6":82,"\u5c06recurr":81,"\u5c1a\u53ef":90,"\u5c31":90,"\u5c31\u4f1a\u5728\u5b8c\u6210\u7f16\u8bd1\u4e4b\u540e":83,"\u5c31\u4f1a\u751f\u6210\u975e\u5e38\u591a\u7684gener":2,"\u5c31\u53ef\u4ee5\u4e86\u89e3\u5230\u95ee\u9898\u4ee3\u7801\u5728\u54ea\u91cc":102,"\u5c31\u53ef\u4ee5\u4f7f\u7528\u4e0b\u9762\u7684\u547d\u4ee4\u5f00\u59cb\u6267\u884c\u8bad\u7ec3":84,"\u5c31\u53ef\u4ee5\u5c06\u6570\u636e\u4f20\u9001\u7ed9paddlepaddle\u4e86":2,"\u5c31\u53ef\u4ee5\u5c06\u8fd9\u4e9b\u6587\u4ef6\u6301\u4e45\u5316\u5b58\u50a8":109,"\u5c31\u53ef\u4ee5\u6309":94,"\u5c31\u5f88\u5bb9\u6613\u5bfc\u81f4\u5185\u5b58\u8d85\u9650":80,"\u5c31\u662f":90,"\u5c31\u662f\u7528\u4e8e\u5c55\u793a\u4e0a\u8ff0\u5206\u6790\u5de5\u5177\u7684\u7528\u6cd5":103,"\u5c31\u80fd\u591f\u5f88\u65b9\u4fbf\u7684\u5b8c\u6210\u6570\u636e\u4e0b\u8f7d\u548c\u76f8\u5e94\u7684\u9884\u5904\u7406\u5de5\u4f5c":123,"\u5c31\u8fd9\u4e48\u7b80\u5355":84,"\u5c31\u901a\u5e38\u7684gpu\u6027\u80fd\u5206\u6790\u6765\u8bf4":103,"\u5c31\u9700\u8981\u5bf9\u8fd9\u4e2a\u7b2c\u4e09\u65b9\u8bed\u8a00\u589e\u52a0\u4e00\u4e9b\u5b9a\u4e49":54,"\u5c31\u9700\u8981\u9009\u62e9\u4f7f\u7528no":84,"\u5c3a\u5bf8":122,"\u5c3d\u65e9\u62a5\u9519":97,"\u5c42\u548c\u8f93\u5165\u7684\u914d\u7f6e":96,"\u5c42\u6743\u91cd":122,"\u5c42\u6b21\u5316\u7684rnn":92,"\u5c42\u7279\u5f81":122,"\u5c42\u7684\u540d\u79f0\u4e0e":93,"\u5c42\u7684\u5927\u5c0f":96,"\u5c42\u7684\u7279\u5f81":122,"\u5c42\u7684\u7c7b\u578b":96,"\u5c42\u7684\u8f93\u51fa\u88ab\u7528\u4f5c\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684":93,"\u5c45\u7136":90,"\u5c55\u793a\u4e86\u4e0a\u8ff0\u7f51\u7edc\u6a21\u578b\u7684\u8bad\u7ec3\u6548\u679c":123,"\u5c55\u793a\u7684\u8c03\u7528\u56fe\u4e5f\u53ef\u4ee5\u5e2e\u52a9\u6211\u4eec\u53d1\u73b0\u6027\u80fd\u4e2d\u7684\u95ee\u9898":102,"\u5c5e\u4e8e\u8fd9\u4e00\u7c7b\u7684\u5b9e\u73b0":81,"\u5c5e\u6027":97,"\u5de5\u4f5c\u6a21\u5f0f":106,"\u5de5\u4f5c\u7a7a\u95f4\u4e2d\u7684":104,"\u5de5\u4f5c\u7a7a\u95f4\u5e94\u5982\u4e0b\u6240\u793a":104,"\u5de5\u5177\u5408\u5e76fat\u5e93":116,"\u5de5\u5177\u670d\u52a1\u5668\u5c06\u8bfb\u53d6\u73af\u5883\u53d8\u91cf":99,"\u5de5\u5177\u6765\u7ba1\u7406":95,"\u5de5\u5177\u6765\u7f16\u8bd1\u6587\u6863":99,"\u5de5\u5177\u94fe":115,"\u5de5\u5177\u94fe\u7684android":115,"\u5de6\u53f3\u7684\u8ba1\u7b97\u65f6\u95f4":102,"\u5de6\u56fe\u6784\u9020\u7f51\u7edc\u6a21\u5757\u7684\u65b9\u5f0f\u88ab\u7528\u4e8e34\u5c42\u7684\u7f51\u7edc\u4e2d":122,"\u5de6\u8fb9\u662f":122,"\u5dee\u8bc4":123,"\u5df2\u6253\u5f00":95,"\u5df2\u7ecf\u5728\u96c6\u7fa4\u63d0\u4ea4\u73af\u5883\u4e2d\u5b8c\u6210\u8bbe\u7f6e":106,"\u5e02\u9762\u4e0a\u5df2\u7ecf\u6709nvidia\u6216\u7b2c\u4e09\u65b9\u63d0\u4f9b\u7684\u4f17\u591a\u5de5\u5177":103,"\u5e26\u6709\u4e0b\u9762\u4e24\u4e2a\u6a21\u677f\u53c2\u6570":97,"\u5e2e\u52a9\u6211\u4eec\u5b8c\u6210\u5bf9\u8f93\u5165\u5e8f\u5217\u7684\u62c6\u5206":92,"\u5e2e\u52a9\u6211\u4eec\u66f4\u597d\u5730\u63cf\u8ff0\u6bb5\u843d":92,"\u5e2e\u52a9\u6211\u4eec\u6784\u9020\u4e00\u4e9b\u590d\u6742\u7684\u8f93\u5165\u4fe1\u606f":89,"\u5e38\u5e38\u51fa\u73b0":77,"\u5e38\u7528\u4f18\u5316\u7b97\u6cd5\u5305\u62ecmomentum":123,"\u5e38\u7528\u7684cmake\u914d\u7f6e\u5982\u4e0b":[115,116],"\u5e38\u89c1\u7684\u5305\u62ec":102,"\u5e38\u89c1\u7684\u53ef\u9009\u5b58\u50a8\u670d\u52a1\u5305\u62ec":109,"\u5e72\u51c0":90,"\u5e73\u53f0\u4e3a\u60f3\u89c2\u6d4b\u8bcd\u5411\u91cf\u7684\u7528\u6237\u63d0\u4f9b\u4e86\u5c06\u4e8c\u8fdb\u5236\u8bcd\u5411\u91cf\u6a21\u578b\u8f6c\u6362\u4e3a\u6587\u672c\u6a21\u578b\u7684\u529f\u80fd":121,"\u5e73\u5747\u6545\u969c\u4fee\u590d\u65f6\u95f4":34,"\u5e73\u5747\u6545\u969c\u7387":34,"\u5e76\u4e0d\u4fdd\u8bc1":96,"\u5e76\u4e0d\u662f\u4f7f\u7528\u53cc\u5c42rnn\u89e3\u51b3\u5b9e\u9645\u7684\u95ee\u9898":90,"\u5e76\u4e0d\u662fkubernetes\u4e2d\u7684node\u6982\u5ff5":111,"\u5e76\u4e0d\u771f\u6b63\u7684\u548c":90,"\u5e76\u4e0d\u96be":94,"\u5e76\u4e14":[2,93],"\u5e76\u4e14\u4e5f\u53ef\u4ee5\u5728windows\u7684docker\u4e2d\u8fd0\u884c":84,"\u5e76\u4e14\u4e66\u5199\u4e00\u4efd\u4ee3\u7801":98,"\u5e76\u4e14\u4f1a\u6839\u636e":115,"\u5e76\u4e14\u4f7f\u7528":55,"\u5e76\u4e14\u5185\u5c42\u7684\u5e8f\u5217\u64cd\u4f5c\u4e4b\u95f4\u72ec\u7acb\u65e0\u4f9d\u8d56":90,"\u5e76\u4e14\u52a0\u4e0a\u4e0b\u9762\u7684\u547d\u4ee4\u884c\u53c2\u6570":108,"\u5e76\u4e14\u5305\u62ecunit":95,"\u5e76\u4e14\u53ea\u6709\u4e00\u4e2a\u6743\u91cd":122,"\u5e76\u4e14\u53ef\u80fd\u4f1a\u52a0\u901f\u8bad\u7ec3\u8fc7\u7a0b":80,"\u5e76\u4e14\u542f\u52a8\u8bad\u7ec3":111,"\u5e76\u4e14\u5728\u5185\u5b58\u8db3\u591f\u7684\u60c5\u51b5\u4e0b\u8d8a\u5927\u8d8a\u597d":2,"\u5e76\u4e14\u5728\u5e38\u89c1\u7684\u5e73\u53f0\u4e0a":54,"\u5e76\u4e14\u5728\u968f\u540e\u7684\u8bfb\u53d6\u6570\u636e\u8fc7\u7a0b\u4e2d\u586b\u5145\u8bcd\u8868":123,"\u5e76\u4e14\u5728dataprovider\u4e2d\u5b9e\u73b0\u5982\u4f55\u8bbf\u95ee\u8bad\u7ec3\u6587\u4ef6\u5217\u8868":1,"\u5e76\u4e14\u5b83\u4eec\u7684\u987a\u5e8f\u4e0e":122,"\u5e76\u4e14\u5c55\u793a\u4e86\u5982\u4f55\u5229\u7528paddlepaddle\u6765\u89e3\u51b3\u4e00\u4e2a\u7ecf\u5178\u7684\u7ebf\u6027\u56de\u5f52\u95ee\u9898":87,"\u5e76\u4e14\u5f3a\u5236\u8bbe\u7f6e\u4e00\u4e9bpaddlepaddle\u53c2\u6570\u7684\u503c":116,"\u5e76\u4e14\u628a\u5404\u79cd\u5f00\u53d1\u5de5\u5177\u5b89\u88c5\u8fdb\u53bb":94,"\u5e76\u4e14\u628a\u7cfb\u7edf\u751f\u6210\u7684ca":44,"\u5e76\u4e14\u628a\u7ed3\u679c\u8fd4\u56depfsclient\u7aef":44,"\u5e76\u4e14\u67e5\u8be2paddlepaddle\u5355\u5143\u6d4b\u8bd5\u7684\u65e5\u5fd7":77,"\u5e76\u4e14\u7f16\u8bd1\u65f6\u9700\u8981\u6253\u5f00":97,"\u5e76\u4e14\u7f16\u8bd1\u80fd\u901a\u8fc7\u4ee3\u7801\u6837\u5f0f\u68c0\u67e5":95,"\u5e76\u4e14\u8ba9\u63a5\u53e3\u8131\u79bb\u5b9e\u73b0\u7ec6\u8282":54,"\u5e76\u4e14\u8bbe\u7f6e\u9ed8\u8ba4\u503c\u4e3a1":97,"\u5e76\u4e14\u8f93\u51fa\u4e00\u4e2a":95,"\u5e76\u4e14\u8fd0\u884c":94,"\u5e76\u4e14\u9700\u8981\u91cd\u5199\u57fa\u7c7b\u4e2d\u7684\u4ee5\u4e0b\u51e0\u4e2a\u865a\u51fd\u6570":96,"\u5e76\u4e14cpu":97,"\u5e76\u4e14softmax\u5c42\u7684\u4e24\u4e2a\u8f93\u5165\u4e5f\u4f7f\u7528\u4e86\u540c\u6837\u7684\u53c2\u6570":82,"\u5e76\u4f20\u5165\u76f8\u5e94\u7684\u547d\u4ee4\u884c\u53c2\u6570\u521d\u59cb\u5316paddlepaddl":4,"\u5e76\u4f7f\u7528":104,"\u5e76\u4f7f\u7528\u4e86dropout":123,"\u5e76\u4fdd\u5b58\u8f93\u51fa\u5230\u4e00\u4e2a\u65e5\u5fd7\u6587\u4ef6":104,"\u5e76\u521b\u5efa\u4e86\u4e00\u4e2a\u65b0\u6587\u4ef6":95,"\u5e76\u521b\u5efaoptim":87,"\u5e76\u521d\u59cb\u5316":97,"\u5e76\u5220\u9664":71,"\u5e76\u5220\u9664\u66f4\u65e9\u7684\u5feb\u7167":34,"\u5e76\u52a0\u8f7d\u5176\u4e2d\u7684\u53c2\u6570":34,"\u5e76\u53d1\u5e03\u5230pypi":71,"\u5e76\u53ef\u4ee5\u5728\u5927\u591a\u6570\u4e3b\u6d41\u7684linux\u64cd\u4f5c\u7cfb\u7edf\u4ee5\u53camacos\u4e0a\u6267\u884c":86,"\u5e76\u548c\u53c2\u6570\u670d\u52a1\u5668\u901a\u4fe1":104,"\u5e76\u5728\u4e58\u79ef\u7ed3\u679c\u4e0a\u518d\u52a0\u4e0a\u7ef4\u5ea6\u4e3a":96,"\u5e76\u5728\u6700\u5f00\u59cb\u521d\u59cb\u5316\u4e3a\u8d77\u59cb\u8bcd":93,"\u5e76\u5728\u7c7b\u6784\u5efa\u51fd\u6570\u4e2d\u628a\u5b83\u653e\u5165\u4e00\u4e2a\u7c7b\u6210\u5458\u53d8\u91cf\u91cc":96,"\u5e76\u5728\u8be5layer\u91cc\u91c7\u7528\u7b2c\u4e00\u79cd\u65b9\u5f0f\u8bbe\u7f6e":81,"\u5e76\u5728\u96c6\u7fa4\u4e2d\u8fd0\u884c\u591a\u4e2a\u5206\u5e03\u5f0f\u6570\u636e\u5904\u7406\u4efb\u52a1":35,"\u5e76\u5728python\u811a\u672c\u4e2d\u5b8c\u6210\u4e0eoperator\u540c\u6837\u7684\u8ba1\u7b97\u903b\u8f91":97,"\u5e76\u5b89\u88c5\u4e86python":77,"\u5e76\u5b89\u88c5\u6700\u65b0":86,"\u5e76\u5b89\u88c5\u6709python2":88,"\u5e76\u5b8c\u6210\u53c2\u6570\u4f18\u5316\u66f4\u65b0":104,"\u5e76\u5b9a\u4e49\u5143\u7c7b":97,"\u5e76\u5bf9\u6bd4\u662f\u5426\u548c\u6b63\u5728\u5b89\u88c5\u7684\u540e\u7f00\u4e00\u81f4":77,"\u5e76\u5bf9\u76f8\u5e94\u7684\u53c2\u6570\u8c03\u7528":96,"\u5e76\u5c06":71,"\u5e76\u5c06\u5176\u6295\u5c04\u5230":93,"\u5e76\u5c06\u8be5layer\u4e0a\u4e00\u65f6\u95f4\u6b65\u7684\u8f93\u51fa\u4f5c\u4e3a\u81ea\u8eab\u5f53\u524d\u65f6\u95f4\u6b65\u7684\u8f93\u51fa":81,"\u5e76\u5c06c":55,"\u5e76\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4":84,"\u5e76\u628a\u5feb\u7167\u4fdd\u5b58\u5230\u8fd9\u4e2a\u76ee\u5f55\u4e0b":34,"\u5e76\u63d0\u4f9b\u4e86\u7b80\u5355\u7684cache\u529f\u80fd":2,"\u5e76\u66f4\u6362job":78,"\u5e76\u6839\u636e\u5206\u5e03\u5f0f\u8bad\u7ec3\u5e76\u53d1\u6570":104,"\u5e76\u68c0\u67e5\u548c\u9700\u5b89\u88c5\u7684\u5305\u662f\u5426\u5339\u914d":86,"\u5e76\u6ca1\u6709paddle\u7279\u522b\u9700\u8981\u7684\u7279\u6027":54,"\u5e76\u6dfb\u52a0\u6ce8\u91ca":97,"\u5e76\u7279\u5316\u6a21\u677f\u53c2\u6570\u4e3a":97,"\u5e76\u7c98\u8d34\u6b64python\u4ee3\u7801":88,"\u5e76\u7ed9\u51fa\u7684\u76f8\u5173\u6a21\u578b\u683c\u5f0f\u7684\u5b9a\u4e49":121,"\u5e76\u81ea\u52a8\u4e0b\u8f7d\u5b89\u88c5\u4f9d\u8d56\u8f6f\u4ef6":86,"\u5e76\u81ea\u52a8\u7f16\u8bd1\u5bbf\u4e3b\u673a\u7248protoc\u53ef\u6267\u884c\u6587\u4ef6":[115,117],"\u5e76\u884c\u5730\u6267\u884c\u6a21\u578b\u7684\u8bad\u7ec3":104,"\u5e76\u884c\u5730\u63a5\u6536\u68af\u5ea6\u548c\u66f4\u65b0\u53c2\u6570":104,"\u5e76\u88ab\u5b58\u50a8\u5728\u8bf8\u5982hadoop":35,"\u5e76\u89c2\u5bdf\u7ed3\u679c":103,"\u5e76\u89e3\u91ca\u4e86\u5404\u81ea\u542b\u4e49":97,"\u5e76\u8bb0\u5f55\u5b83\u7684\u7f16\u53f7":95,"\u5e76\u8fdb\u884c\u521d\u59cb\u5316\u64cd\u4f5c":87,"\u5e76\u9002\u5e94github\u7684\u7279\u6027\u505a\u4e86\u4e00\u4e9b\u533a\u522b":71,"\u5e76\u9010\u6e10\u5c55\u793a\u66f4\u52a0\u6df1\u5165\u7684\u529f\u80fd":123,"\u5e76\u91cd\u65b0\u6253\u5305wheel\u5305":71,"\u5e76\u94fe\u63a5\u5230\u751f\u6210\u7684lib\u5e93\u4e2d":97,"\u5e78\u800cpython\u7684\u4e00\u4e2a\u7b2c\u4e09\u65b9\u5e93":102,"\u5e8a\u4e0a\u7528\u54c1":90,"\u5e8a\u57ab":90,"\u5e8f\u5217\u4e2d\u542b\u6709\u5143\u7d20\u7684\u6570\u76ee\u540c":89,"\u5e8f\u5217\u6570\u636e\u662f\u81ea\u7136\u8bed\u8a00\u5904\u7406\u4efb\u52a1\u9762\u5bf9\u7684\u4e00\u79cd\u4e3b\u8981\u8f93\u5165\u6570\u636e\u7c7b\u578b":92,"\u5e8f\u5217\u662f\u4e00\u79cd\u5e38\u89c1\u7684\u6570\u636e\u7c7b\u578b":89,"\u5e8f\u5217\u751f\u6210\u4efb\u52a1\u5927\u591a\u9075\u5faaencod":92,"\u5e8f\u5217\u751f\u6210\u4efb\u52a1\u7684\u8f93\u5165":92,"\u5e8f\u5217\u7684\u6bcf\u4e2a\u5143\u7d20\u662f\u539f\u6765\u53cc\u5c42\u5e8f\u5217\u6bcf\u4e2asubseq\u5143\u7d20\u7684\u5e73\u5747\u503c":89,"\u5e8f\u5217\u8f93\u5165\u65f6\u7b49\u4e8e":80,"\u5e94\u7528\u524d\u5411\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":93,"\u5e94\u7528\u53cd\u5411\u9012\u5f52\u795e\u7ecf\u7f51\u7edc":93,"\u5e94\u7528\u6a21\u578b":123,"\u5e94\u80fd\u53cd\u6620\u5f53\u524dcommit\u7684\u5185\u5bb9":95,"\u5e94\u8be5":90,"\u5e94\u8be5\u4e0e\u5b83\u7684memory\u540d\u5b57\u76f8\u540c":93,"\u5e94\u8be5\u8bf4\u8c22\u8c22":95,"\u5e94\u8be5\u8bfb\u53d6\u5f53\u524d\u76ee\u5f55\u4e0b\u7684":94,"\u5e94\u8be5\u964d\u4f4e\u5b66\u4e60\u7387":82,"\u5e95\u5c42\u8fdb\u7a0b":104,"\u5efa\u7acb\u4e00\u4e2a":95,"\u5efa\u8bae":[71,85,95],"\u5efa\u8bae\u5c06\u8be5\u53c2\u6570\u8bbe\u4e3atrue":106,"\u5f00\u53d1\u4e86\u6a21\u578b\u9884\u6d4b\u7684\u6837\u4f8b\u4ee3\u7801":55,"\u5f00\u53d1\u4eba\u5458\u4f7f\u7528":95,"\u5f00\u53d1\u5206\u652f":86,"\u5f00\u53d1\u8005\u4f7f\u7528":94,"\u5f00\u53d1\u8005\u4fee\u6539\u81ea\u5df1\u7684\u4ee3\u7801":71,"\u5f00\u53d1\u8005fork\u7684\u7248\u672c\u5e93\u4e2d":71,"\u5f00\u53d1\u8005fork\u7684\u7248\u672c\u5e93\u4f7f\u7528":71,"\u5f00\u53d1\u955c\u50cf":95,"\u5f00\u542f":83,"\u5f00\u5934":97,"\u5f00\u5934\u7684\u90e8\u5206":104,"\u5f00\u5934\u90e8\u5206\u6307\u5b9a":104,"\u5f00\u59cb\u63d0\u4f9b\u670d\u52a1":34,"\u5f00\u59cb\u6807\u8bb0":93,"\u5f00\u59cb\u795e\u7ecf\u7f51\u7edc\u7684":104,"\u5f00\u59cb\u8bad\u7ec3\u6a21\u578b":123,"\u5f00\u59cb\u9636\u6bb5":103,"\u5f02\u6b65\u968f\u673a\u68af\u5ea6\u4e0b\u964d":105,"\u5f15\u5165\u4e86\u7c7b\u578b\u7684\u5934\u6587\u4ef6":55,"\u5f15\u5165paddlepaddle\u7684pydataprovider2\u5305":2,"\u5f15\u53d1":13,"\u5f15\u5bfc\u5c42":93,"\u5f15\u7528memory\u5f97\u5230\u8fd9layer\u4e0a\u4e00\u65f6\u523b\u8f93\u51fa":92,"\u5f3a\u70c8\u63a8\u8350":90,"\u5f52\u4e00\u5316\u6982\u7387\u5411\u91cf":93,"\u5f53":108,"\u5f53\u4f60\u6267\u884c\u547d\u4ee4":96,"\u5f53\u4fdd\u5b58\u7684\u7f51\u7edc\u53c2\u6570\u4e3afloat\u7c7b\u578b\u65f6\u4e3a4":82,"\u5f53\u51fd\u6570\u8fd4\u56de\u7684\u65f6\u5019":2,"\u5f53\u524d\u65f6\u95f4\u6b65\u5904\u7684memory\u7684\u8f93\u51fa\u4f5c\u4e3a\u4e0b\u4e00\u65f6\u95f4\u6b65memory\u7684\u8f93\u5165":93,"\u5f53\u524d\u7684\u5b66\u4e60\u7387\u4e3a\u6240\u8bbe\u7f6e":82,"\u5f53\u524d\u7684\u5b9e\u73b0\u65b9\u5f0f\u4e0b":96,"\u5f53\u524d\u7684\u8f93\u5165y\u548c\u4e0a\u4e00\u4e2a\u65f6\u95f4\u6b65\u7684\u8f93\u51farnn_state\u505a\u4e86\u4e00\u4e2a\u5168\u94fe\u63a5":90,"\u5f53\u524d\u8bad\u7ec3\u4efb\u52a1\u542f\u52a8\u7684pserver\u7684ip\u5217\u8868":104,"\u5f53\u524d\u8bad\u7ec3\u4efb\u52a1pserver\u603b\u6570":104,"\u5f53\u524d\u8bad\u7ec3\u4efb\u52a1trainer\u603b\u4e2a\u6570":104,"\u5f53\u524dlog_period\u4e2abatch\u6240\u6709\u6837\u672c\u7684\u5e73\u5747\u5206\u7c7b\u9519\u8bef\u7387":123,"\u5f53\u524dlog_period\u4e2abatch\u6240\u6709\u6837\u672c\u7684\u5e73\u5747cost":123,"\u5f53\u529f\u80fd\u5206\u652f\u5f00\u53d1\u5b8c\u6bd5\u540e":71,"\u5f53\u5728\u7f51\u7edc\u5c42\u914d\u7f6e\u4e2d\u8bbe\u7f6e":106,"\u5f53\u5728\u8bad\u7ec3\u914d\u7f6e\u4e2d\u8bbe\u7f6e":106,"\u5f53\u5bb9\u5668\u56e0\u4e3a\u5404\u79cd\u539f\u56e0\u88ab\u9500\u6bc1\u65f6":109,"\u5f53\u5df2\u8bad\u7ec3\u6837\u672c\u6570\u5927\u4e8e1000\u5c0f\u4e8e\u7b49\u4e8e2000\u65f6":82,"\u5f53\u5df2\u8bad\u7ec3\u6837\u672c\u6570\u5927\u4e8e2000\u65f6":82,"\u5f53\u5df2\u8bad\u7ec3\u6837\u672c\u6570\u5c0f\u4e8e\u7b49\u4e8e1000\u65f6":82,"\u5f53\u5df2\u8bad\u7ec3pass\u6570\u5927\u4e8e1\u5c0f\u4e8e\u7b49\u4e8e2\u65f6":82,"\u5f53\u5df2\u8bad\u7ec3pass\u6570\u5927\u4e8e2\u65f6":82,"\u5f53\u5df2\u8bad\u7ec3pass\u6570\u5c0f\u4e8e\u7b49\u4e8e1\u65f6":82,"\u5f53\u6211\u4eec\u505a\u51fa\u6027\u80fd\u4fee\u6b63\u540e":102,"\u5f53\u6240\u6709pod\u90fd\u5904\u4e8erunning\u72b6\u6001":111,"\u5f53\u6a21\u578b\u53c2\u6570\u4e0d\u5b58\u5728\u65f6":106,"\u5f53\u6a21\u578b\u8bad\u7ec3\u597d\u4e86\u4e4b\u540e":123,"\u5f53\u6a21\u5f0f\u4e3a":106,"\u5f53\u7136":[84,103],"\u5f53\u7136\u53ef\u4ee5":94,"\u5f53\u7528\u6237\u4f7f\u7528\u5b8c\u8fd9\u4e2a\u53c2\u6570\u540e":55,"\u5f53\u7528\u6237\u6ca1\u6709\u663e\u5f0f\u8bbe\u5b9a\u65f6":81,"\u5f53\u7f51\u7edc\u5c42\u7528\u4e00\u4e2a\u6279\u6b21\u505a\u8bad\u7ec3\u65f6":96,"\u5f53\u89e3\u8bfb\u6bcf\u4e00\u4e2a":93,"\u5f53\u8bad\u7ec3\u6570\u636e\u975e\u5e38\u591a\u65f6":2,"\u5f53\u8d85\u8fc7\u8be5\u9608\u503c\u65f6":106,"\u5f53\u8f93\u5165\u662f\u7ef4\u5ea6\u5f88\u9ad8\u7684\u7a00\u758f\u6570\u636e\u65f6":108,"\u5f53\u9700\u8981\u5b8c\u6210\u8ba1\u7b97\u65f6":98,"\u5f53destination\u6587\u4ef6\u4e0d\u5b58\u5728\u6216\u8005\u5927\u5c0f\u548csource\u6587\u4ef6\u4e0d\u4e00\u81f4\u65f6":44,"\u5f53n1":80,"\u5f62\u6210recurr":92,"\u5f62\u6210recurrent\u8fde\u63a5":92,"\u5f62\u72b6":122,"\u5f88":[90,123],"\u5f88\u591a":[90,94],"\u5f88\u5b89\u9759":90,"\u5f88\u5e72\u51c0":90,"\u5f88\u65b9\u4fbf":90,"\u5f88\u6709\u53ef\u80fd\u5b9e\u9645\u5e94\u7528\u5c31\u662f\u6ca1\u6709\u6309\u7167\u60a8\u7684\u9884\u671f\u60c5\u51b5\u8fd0\u884c":103,"\u5f88\u6709\u53ef\u80fd\u662f\u975e\u72ec\u5360\u65b9\u5f0f\u6267\u884c\u5bfc\u81f4\u7684\u7aef\u53e3\u51b2\u7a81":78,"\u5f88\u96be\u4fdd\u8bc1\u591a\u8bed\u8a00\u4ee3\u7801\u98ce\u683c\u7684\u4e00\u81f4\u6027":54,"\u5f88\u96be\u6574\u4f53\u4fee\u6b63":2,"\u5f97":90,"\u5f97\u4f7f\u7528":54,"\u5f97\u5230\u53e5\u5b50\u7684\u8868\u793a":123,"\u5f97\u5230\u8f93\u51fa\u503c":97,"\u5faa\u73af\u5c55\u5f00\u7684\u6bcf\u4e2a\u65f6\u95f4\u6b65\u603b\u662f\u80fd\u591f\u5f15\u7528\u6240\u6709\u8f93\u5165":92,"\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u4e2d":93,"\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u4f5c\u4e3a\u4f7f\u7528":93,"\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u548c":93,"\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u5728\u6bcf\u4e2a\u65f6\u95f4\u6b65\u9aa4\u987a\u5e8f\u5730\u5904\u7406\u5e8f\u5217":93,"\u5faa\u73af\u7f51\u7edc\u4ece":93,"\u5fc5\u8981":55,"\u5fc5\u9009":104,"\u5fc5\u987b":96,"\u5fc5\u987b\u4e00\u81f4":2,"\u5fc5\u987b\u4f7f\u7528python\u5173\u952e\u8bcd":2,"\u5fc5\u987b\u5c06\u524d\u4e00\u4e2a\u5b50\u53e5\u7684\u6700\u540e\u4e00\u4e2a\u5143\u7d20":90,"\u5fc5\u987b\u6307\u5411\u4e00\u4e2apaddlepaddle\u5b9a\u4e49\u7684lay":92,"\u5fc5\u987b\u662f\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":92,"\u5fc5\u987b\u662f\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":89,"\u5fc5\u987b\u7531\u53ea\u8bfbmemory\u7684":93,"\u5fc5\u987b\u8bbe\u7f6e\u4e3a":[115,116],"\u5fc5\u987b\u8bbe\u7f6e\u4e3aon":116,"\u5fc5\u987b\u914d\u7f6e\u4e3a":117,"\u5feb":90,"\u5feb\u901f\u5728\u672c\u5730\u542f\u52a8\u4e00\u4e2a\u5355\u673a\u7684kubernetes\u670d\u52a1\u5668":109,"\u5feb\u901f\u90e8\u7f72\u96c6\u7fa4":109,"\u6027\u4ef7\u6bd4":90,"\u6027\u80fd\u4f18\u5316\u7684\u8fc7\u7a0b\u901a\u5e38\u662f\u4e0d\u65ad\u91cd\u590d\u5730":102,"\u6027\u80fd\u5206\u6790":103,"\u6027\u80fd\u5206\u6790\u5de5\u5177\u662f\u7528\u4e8e\u7ed9\u5e94\u7528\u7a0b\u5e8f\u7684\u6027\u80fd\u505a\u5b9a\u91cf\u5206\u6790\u7684":103,"\u6027\u80fd\u5206\u6790\u662f\u6027\u80fd\u4f18\u5316\u7684\u5173\u952e\u4e00\u6b65":103,"\u6027\u80fd\u548c\u628a\u7f16\u8bd1\u5de5\u5177\u5b89\u88c5\u5728\u672c\u673a\u8fd0\u884c\u4e00\u6837":94,"\u6027\u80fd\u8c03\u4f18":105,"\u6027\u80fdtip":[115,116],"\u603b\u4f53\u6765\u8bf4":90,"\u603b\u7aef\u53e3\u4e2a\u6570":104,"\u603b\u8ba1\u7684\u53c2\u6570\u4e2a\u6570":121,"\u60a8\u4e5f\u53ef\u4ee5\u8fdb\u5165\u5230docker\u5bb9\u5668\u4e2d":84,"\u60a8\u4f1a\u5728\u63a5\u4e0b\u6765\u7684\u90e8\u5206\u4e2d\u83b7\u5f97\u66f4\u591a\u7684\u7ec6\u8282\u4ecb\u7ecd":103,"\u60a8\u53ef\u4ee5\u4ece\u4e0b\u9762\u7684\u8868\u683c\u4e2d\u627e\u5230\u9700\u8981\u7684\u7248\u672c":86,"\u60a8\u53ef\u4ee5\u4efb\u610f\u4f7f\u7528\u4e00\u4e2a\u6216\u4e24\u4e2a\u6765\u5bf9\u611f\u5174\u8da3\u7684\u4ee3\u7801\u6bb5\u505a\u6027\u80fd\u5206\u6790":103,"\u60a8\u53ef\u4ee5\u5728":84,"\u60a8\u53ef\u4ee5\u5728\u5bb9\u5668\u4e2d\u6267\u884c":84,"\u60a8\u53ef\u4ee5\u5bfc\u5165":103,"\u60a8\u53ef\u4ee5\u6309\u7167\u4e0b\u9762\u7684\u6b65\u9aa4\u5728openmpi\u96c6\u7fa4\u4e2d\u63d0\u4ea4paddle\u8bad\u7ec3\u4efb\u52a1":104,"\u60a8\u53ef\u4ee5\u91c7\u7528\u4e0b\u9762\u4e94\u4e2a\u6b65\u9aa4":103,"\u60a8\u53ef\u80fd\u9700\u8981\u4fee\u6539":104,"\u60a8\u5c06\u4e86\u89e3\u5982\u4f55":93,"\u60a8\u5c31\u80fd\u83b7\u5f97\u5982\u4e0b\u7684\u5206\u6790\u7ed3\u679c":103,"\u60a8\u6309\u5982\u4e0b\u6b65\u9aa4\u64cd\u4f5c\u5373\u53ef":103,"\u60a8\u6700\u597d\u5148\u786e\u8ba4":103,"\u60a8\u9996\u5148\u9700\u8981\u5728\u76f8\u5173\u4ee3\u7801\u6bb5\u4e2d\u52a0\u5165":103,"\u60c5\u611f\u5206\u6790":71,"\u60f3\u4e86\u89e3\u66f4\u591apaddlepaddl":99,"\u610f\u5473\u7740\u4e0d\u540c\u65f6\u95f4\u6b65\u7684\u8f93\u5165\u90fd\u662f\u76f8\u540c\u7684\u503c":93,"\u610f\u601d\u662f\u4e0d\u4f7f\u7528\u5e73\u5747\u53c2\u6570\u6267\u884c\u6d4b\u8bd5":106,"\u610f\u601d\u662f\u4e0d\u4fdd\u5b58\u7ed3\u679c":106,"\u610f\u601d\u662f\u4f7f\u7528\u7b2ctest":106,"\u610f\u601d\u662f\u5728gpu\u6a21\u5f0f\u4e0b\u4f7f\u75284\u4e2agpu":106,"\u611f\u89c9":90,"\u6210\u529f\u8bad\u7ec3\u4e14\u9000\u51fa\u7684pod\u6570\u76ee\u4e3a3\u65f6":111,"\u6210\u5458":97,"\u6210\u719f\u7684\u9ad8\u6027\u80fd\u5e76\u884c\u8ba1\u7b97\u6846\u67b6":104,"\u6211\u4eec\u4e0d\u80fd\u901a\u8fc7\u5e38\u89c4\u7684\u68af\u5ea6\u68c0\u67e5\u7684\u65b9\u5f0f\u6765\u8ba1\u7b97\u68af\u5ea6":96,"\u6211\u4eec\u4e3b\u8981\u4f1a\u4ecb\u7ecdnvprof\u548cnvvp":103,"\u6211\u4eec\u4e5f\u53ef\u4ee5\u786e\u5b9a\u6bcf\u4e00\u4e2a\u53c2\u6570\u7684\u7c7b\u578b":55,"\u6211\u4eec\u4ec5\u4ec5\u5bf9\u795e\u7ecf\u7f51\u7edc\u7684\u8f93\u5165\u8fdb\u884c\u4e86\u63cf\u8ff0":87,"\u6211\u4eec\u4ec5\u6709\u4e00\u4e2a\u8f93\u5165":96,"\u6211\u4eec\u4ecb\u7ecd\u5982\u4f55\u5728":110,"\u6211\u4eec\u4ecb\u7ecd\u5982\u4f55\u5728kubernetes\u96c6\u7fa4\u4e0a\u8fdb\u884c\u5206\u5e03\u5f0fpaddlepaddle\u8bad\u7ec3\u4f5c\u4e1a":111,"\u6211\u4eec\u4ece\u63d0\u524d\u7ed9\u5b9a\u7684\u7c7b\u522b\u96c6\u5408\u4e2d\u9009\u62e9\u5176\u6240\u5c5e\u7c7b\u522b":123,"\u6211\u4eec\u4ee5mnist\u624b\u5199\u8bc6\u522b\u4e3a\u4f8b":2,"\u6211\u4eec\u4f1a\u5bf9\u6bcf\u4e2a\u8bad\u7ec3\u4efb\u52a1\u90fd\u4f1a\u5728\u6bcf\u4e2a\u8282\u70b9\u4e0a\u521b\u5efa\u4e00\u4e2a\u5de5\u4f5c\u7a7a\u95f4":104,"\u6211\u4eec\u4f1a\u7ee7\u7eed\u4f7f\u7528\u73b0\u6709\u7684\u5185\u5b58\u5757":96,"\u6211\u4eec\u4f1a\u91cd\u65b0\u5206\u914d\u5185\u5b58":96,"\u6211\u4eec\u4f7f\u7528":96,"\u6211\u4eec\u4f7f\u7528\u4e0d\u540c\u7684layer\u8fdb\u884c\u7ec4\u5408":87,"\u6211\u4eec\u4f7f\u7528\u4e86":90,"\u6211\u4eec\u4f7f\u7528\u52a8\u6001\u5e93\u6765\u5206\u53d1paddl":54,"\u6211\u4eec\u4f7f\u7528paddl":104,"\u6211\u4eec\u4f7f\u7528paddlepaddle\u5728ilsvrc\u7684\u9a8c\u8bc1\u96c6\u517150":122,"\u6211\u4eec\u5047\u8bbe\u4e00\u53f0\u673a\u5668\u4e0a\u67094\u4e2agpu":108,"\u6211\u4eec\u5148\u8c03\u7528\u6bcf\u4e2a":98,"\u6211\u4eec\u5373\u53ef\u5b8c\u6210\u795e\u7ecf\u7f51\u7edc\u7684\u642d\u5efa":87,"\u6211\u4eec\u53ea\u6f14\u793a\u4e00\u4e2a\u5355\u673a\u4f5c\u4e1a":110,"\u6211\u4eec\u53ea\u9700\u8981":94,"\u6211\u4eec\u53ea\u9700\u8981\u4f7f\u7528lstm":90,"\u6211\u4eec\u53ea\u9700\u8981\u8fd0\u884c":123,"\u6211\u4eec\u53ea\u9700\u8981\u8fd0\u884c\u4e0b\u9762\u547d\u4ee4\u628a\u7f16\u8bd1\u597d\u7684paddlepaddle\u6253\u5305\u6210\u4e00\u4e2a":95,"\u6211\u4eec\u53ea\u9700\u8981\u914d\u7f6e":94,"\u6211\u4eec\u53ef\u4ee5":94,"\u6211\u4eec\u53ef\u4ee5\u4f7f\u7528":102,"\u6211\u4eec\u53ef\u4ee5\u4f7f\u7528\u5176\u4ed6layer\u8fdb\u884c\u7ec4\u5408":87,"\u6211\u4eec\u53ef\u4ee5\u4f7f\u7528\u5b83\u6765\u751f\u6210\u5e8f\u5217":93,"\u6211\u4eec\u53ef\u4ee5\u521b\u5efatrainer\u6765\u5bf9\u7f51\u7edc\u8fdb\u884c\u8bad\u7ec3":87,"\u6211\u4eec\u53ef\u4ee5\u53c2\u8003tensorflow\u7684":98,"\u6211\u4eec\u53ef\u4ee5\u5728":95,"\u6211\u4eec\u53ef\u4ee5\u5728\u547d\u4ee4\u884c\u4e2d\u7b80\u5355\u7684\u770b\u4e00\u4e0b\u751f\u6210\u6548\u679c":102,"\u6211\u4eec\u53ef\u4ee5\u5b9a\u4e49\u5982\u4e0b\u7684layer\u7ec4\u5408":87,"\u6211\u4eec\u53ef\u4ee5\u5b9a\u4e49\u5982\u4e0blayer\u6765\u63cf\u8ff0\u795e\u7ecf\u7f51\u7edc\u7684\u8f93\u5165":87,"\u6211\u4eec\u53ef\u4ee5\u6309\u7167\u5982\u4e0b\u5c42\u6b21\u5b9a\u4e49\u975e\u5e8f\u5217":89,"\u6211\u4eec\u53ef\u4ee5\u67e5\u770b\u6027\u80fd\u5206\u6790\u7684\u7ed3\u679c":102,"\u6211\u4eec\u53ef\u4ee5\u8bbe\u8ba1\u642d\u5efa\u4e00\u4e2a\u7075\u6d3b\u7684":92,"\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7":102,"\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7\u65e5\u5fd7\u67e5\u770b\u5bb9\u5668\u8bad\u7ec3\u7684\u60c5\u51b5":111,"\u6211\u4eec\u5728":98,"\u6211\u4eec\u5728\u51fd\u6570\u7684\u7ed3\u5c3e\u8fd4\u56de":93,"\u6211\u4eec\u5728initialzier\u51fd\u6570\u91cc\u521d\u59cb\u5316\u8bcd\u8868":123,"\u6211\u4eec\u5bf9\u6a21\u578b\u8fdb\u884c\u4e86\u4ee5\u4e0b\u66f4\u6539":93,"\u6211\u4eec\u5c06":111,"\u6211\u4eec\u5c06\u4e00\u6bb5\u8bdd\u770b\u6210\u53e5\u5b50\u7684\u6570\u7ec4":90,"\u6211\u4eec\u5c06\u4ecb\u7ecd\u5982\u4f55\u542f\u52a8\u5206\u5e03\u5f0f\u8bad\u7ec3\u4f5c\u4e1a":110,"\u6211\u4eec\u5c06\u4ee5":123,"\u6211\u4eec\u5c06\u4ee5\u6700\u57fa\u672c\u7684\u903b\u8f91\u56de\u5f52\u7f51\u7edc\u4f5c\u4e3a\u8d77\u70b9":123,"\u6211\u4eec\u5c06\u4f7f\u7528":93,"\u6211\u4eec\u5c06\u4f7f\u7528\u7b80\u5355\u7684":93,"\u6211\u4eec\u5c06\u539f\u59cb\u6570\u636e\u7684\u6bcf\u4e00\u7ec4":90,"\u6211\u4eec\u5c06\u5728\u540e\u9762\u4ecb\u7ecd\u8bad\u7ec3\u548c\u9884\u6d4b\u6d41\u7a0b\u7684\u811a\u672c":123,"\u6211\u4eec\u5c06\u5b83\u4eec\u5212\u5206\u4e3a\u4e0d\u540c\u7684\u7c7b\u522b":105,"\u6211\u4eec\u5c06\u5bf9\u8fd9\u4e24\u4e2a\u6b65\u9aa4\u7ed9\u51fa\u4e86\u8be6\u7ec6\u7684\u89e3\u91ca":123,"\u6211\u4eec\u5c31\u53ef\u4ee5\u8bad\u7ec3\u6a21\u578b\u4e86":123,"\u6211\u4eec\u5c31\u53ef\u4ee5\u8fdb\u884c\u9884\u6d4b\u4e86":123,"\u6211\u4eec\u5c31\u5b8c\u6210\u4e86\u4e00\u6b21\u4ee3\u7801\u8d21\u732e\u7684\u8fc7\u7a0b":95,"\u6211\u4eec\u5df2\u7ecf\u5b9e\u73b0\u4e86\u5927\u591a\u6570\u5e38\u7528\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u67b6\u6784":93,"\u6211\u4eec\u5efa\u8bae\u4f60\u4e3a\u4f60\u7684python\u5c01\u88c5\u5b9e\u73b0\u4e00\u4e2a":96,"\u6211\u4eec\u5efa\u8bae\u4f60\u5728\u5199\u65b0\u7f51\u7edc\u5c42\u65f6\u628a\u6d4b\u8bd5\u4ee3\u7801\u653e\u5165\u65b0\u7684\u6587\u4ef6\u4e2d":96,"\u6211\u4eec\u5efa\u8bae\u4f7f\u7528\u7b2c\u4e8c\u7c7b\u5b9e\u73b0":81,"\u6211\u4eec\u603b\u7ed3\u4e86\u5404\u4e2a\u7f51\u7edc\u7684\u590d\u6742\u5ea6\u548c\u6548\u679c":123,"\u6211\u4eec\u628apaddlepaddle\u7684\u4ea4\u53c9\u7f16\u8bd1\u73af\u5883\u6253\u5305\u6210\u4e00\u4e2a\u955c\u50cf":115,"\u6211\u4eec\u63a8\u8350\u4f7f\u7528":[84,104],"\u6211\u4eec\u63a8\u8350\u4f7f\u7528\u6700\u65b0\u7248\u672c\u7684cudnn":83,"\u6211\u4eec\u63a8\u8350\u60a8\u4f7f\u7528paddlepaddl":83,"\u6211\u4eec\u63d0\u4f9b\u4e24\u4e2a\u8f6c\u6362\u65b9\u5f0f":35,"\u6211\u4eec\u63d0\u4f9b\u4e86\u4e00\u4e2a\u793a\u4f8b\u811a\u672c":122,"\u6211\u4eec\u63d0\u4f9b\u4e86\u52a0\u901f\u8bbf\u95ee\u7684\u955c\u50cf\u6e90":84,"\u6211\u4eec\u63d0\u4f9b\u4e86c":122,"\u6211\u4eec\u63d0\u4f9b\u53ef\u4ee5\u76f4\u63a5\u8fd0\u884cpaddlepaddl":84,"\u6211\u4eec\u63d0\u51fa\u4e86chunk\u7684\u6982\u5ff5":44,"\u6211\u4eec\u662f\u5bf9\u6bcf\u4e00\u4e2a\u5b50\u5e8f\u5217\u53d6\u6700\u540e\u4e00\u4e2a\u5143\u7d20":90,"\u6211\u4eec\u6700\u7ec8\u7684\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165python\u6216\u8005\u5176\u4ed6\u4efb\u4f55\u8bed\u8a00\u7684\u89e3\u91ca\u5668":54,"\u6211\u4eec\u6709\u4e00\u4e2a\u5e8f\u5217\u4f5c\u4e3a\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u72b6\u6001":93,"\u6211\u4eec\u7684":94,"\u6211\u4eec\u7684\u5b57\u5178\u4f7f\u7528\u5185\u90e8\u7684\u5206\u8bcd\u5de5\u5177\u5bf9\u767e\u5ea6\u77e5\u9053\u548c\u767e\u5ea6\u767e\u79d1\u7684\u8bed\u6599\u8fdb\u884c\u5206\u8bcd\u540e\u4ea7\u751f":121,"\u6211\u4eec\u7684\u6807\u51c6\u5f00\u53d1\u6d41\u7a0b\u662f\u628a\u8fd9\u4e9b\u5de5\u5177\u90fd\u88c5\u8fdb\u4e00\u4e2adocker":95,"\u6211\u4eec\u770b\u4e00\u4e0b\u5355\u5c42rnn\u7684\u914d\u7f6e":90,"\u6211\u4eec\u770b\u4e00\u4e0b\u8bed\u4e49\u76f8\u540c\u7684\u53cc\u5c42rnn\u7684\u7f51\u7edc\u914d\u7f6e":90,"\u6211\u4eec\u771f\u8bda\u5730\u611f\u8c22\u60a8\u7684\u8d21\u732e":95,"\u6211\u4eec\u79f0\u4e4b\u4e3a\u4e00\u4e2a0\u5c42\u7684\u5e8f\u5217":89,"\u6211\u4eec\u8bbe\u8ba1\u8bf4\u660e\u4e86\u540d\u4e3afilemanager\u7cfb\u7edf":44,"\u6211\u4eec\u8c03\u7528\u4e86eigenvector\u7684flatten\u63a5\u53e3":98,"\u6211\u4eec\u8fd8\u53ef\u4ee5\u767b\u5f55\u5230\u5bbf\u4e3b\u673a\u4e0a\u67e5\u770b\u8bad\u7ec3\u7ed3\u679c":110,"\u6211\u4eec\u8fd8\u5c06\u7f16\u7801\u5411\u91cf\u6295\u5c04\u5230":93,"\u6211\u4eec\u9009\u53d6\u5355\u53cc\u5c42\u5e8f\u5217\u914d\u7f6e\u4e2d\u7684\u4e0d\u540c\u90e8\u5206":90,"\u6211\u4eec\u9009\u62e9":35,"\u6211\u4eec\u901a\u5e38\u501f\u52a9":97,"\u6211\u4eec\u901a\u5e38\u5c06\u4e00\u53e5\u8bdd\u7406\u89e3\u6210\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217":90,"\u6211\u4eec\u901a\u8fc7\u8bfb\u53d6":111,"\u6211\u4eec\u90fd\u63d0\u4f9bpython\u7684\u8f6c\u6362\u5e93":35,"\u6211\u4eec\u91c7\u7528\u5355\u5c42lstm\u6a21\u578b":123,"\u6211\u4eec\u9700\u8981":94,"\u6211\u4eec\u9700\u8981\u5148\u628a\u8f93\u5165tensor\u548c\u8f93\u51fatensor\u8f6c\u6362\u4e3aeigen\u652f\u6301\u7684\u683c\u5f0f":98,"\u6211\u4eec\u9700\u8981\u5236\u4f5c\u4e00\u4e2a\u5305\u542b\u8bad\u7ec3\u6570\u636e\u7684paddle\u955c\u50cf":110,"\u6211\u4eec\u9700\u8981\u5728\u96c6\u7fa4\u7684\u6240\u6709\u8282\u70b9\u4e0a\u5b89\u88c5":104,"\u6211\u4eec\u9700\u8981\u7b49\u5f0f\u5de6\u8fb9\u7684eigentensor\u8c03\u7528device\u63a5\u53e3":98,"\u6211\u4eec\u9700\u8981\u8ba1\u7b97":96,"\u6211\u4eec\u9884\u8bad\u7ec3\u5f97\u52304\u79cd\u4e0d\u540c\u7ef4\u5ea6\u7684\u8bcd\u5411\u91cf":121,"\u6211\u4eec\u9996\u5148\u9700\u8981\u6839\u636e\u795e\u7ecf\u7f51\u7edc\u7ed3\u6784\u6765\u521b\u5efa\u6240\u9700\u8981\u4f18\u5316\u7684paramet":87,"\u6211\u5220\u9664\u4e86":95,"\u6211\u53ef\u4ee5\u7528":94,"\u6211\u53ef\u4ee5\u9009\u62e9\u4e0d\u7528docker\u5417":94,"\u6216":[2,103],"\u6216\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":89,"\u6216\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u53d8\u6210\u4e00\u4e2a0\u5c42\u5e8f\u5217":89,"\u6216\u4e00\u4e2a\u5411\u91cf":92,"\u6216\u5355\u5c42\u5e8f\u5217\u7ecf\u8fc7\u8fd0\u7b97\u53d8\u6210\u4e00\u4e2a0\u5c42\u5e8f\u5217":89,"\u6216\u6700\u5927\u503c":89,"\u6216\u6d4b\u8bd5\u6587\u4ef6\u5217\u8868":1,"\u6216\u7b2c\u4e00\u4e2a":89,"\u6216\u7b2c\u4e00\u4e2a\u5143\u7d20":89,"\u6216\u7f16\u5199\u7a0b\u5e8f\u65f6":104,"\u6216\u8005":[54,55,80,89,90,94,97,102,103],"\u6216\u8005\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":92,"\u6216\u8005\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":[89,92],"\u6216\u8005\u4ece\u5de5\u5177\u7684\u754c\u9762\u91cc\u8fd0\u884c\u60a8\u7684\u5e94\u7528":103,"\u6216\u8005\u5236\u4f5c\u548c\u5206\u4eab\u5e26\u6709\u4ee3\u7801":84,"\u6216\u8005\u53cd\u5411\u5730\u4ece":93,"\u6216\u8005\u53ef\u88abdns\u89e3\u6790\u7684\u4e3b\u673a\u540d":104,"\u6216\u8005\u5728cpu\u6a21\u5f0f\u4e0b\u4f7f\u75284\u4e2a\u7ebf\u7a0b":106,"\u6216\u8005\u5c06\u8fd9\u53f0\u8282\u70b9\u8fc1\u79fb\u5230\u53e6\u4e00\u4e2a\u8282\u70b9\u5e76\u542f\u52a8\u5373\u53ef\u6062\u590d\u8bad\u7ec3\u4efb\u52a1":34,"\u6216\u8005\u5df2\u7ecf\u5728\u96c6\u7fa4\u63d0\u4ea4\u73af\u5883\u4e2d\u81ea\u52a8\u8bbe\u7f6e":105,"\u6216\u8005\u6570\u636e\u5e93\u8fde\u63a5\u8def\u5f84\u7b49":1,"\u6216\u8005\u6570\u7ec4\u7684\u6570\u7ec4\u8fd9\u4e2a\u6982\u5ff5":90,"\u6216\u8005\u662f\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":89,"\u6216\u8005\u662f\u51fd\u6570\u8c03\u7528\u7684\u9891\u7387\u548c\u8017\u65f6\u7b49":103,"\u6216\u8005\u66f4\u65e9":82,"\u6216\u8005\u6bcf\u4e00\u4e2a\u7cfb\u5217\u91cc\u7684\u7279\u5f81\u6570\u636e":90,"\u6216\u8005\u7528tuple\u8868\u793a\u7684\u591a\u4e2a\u503c":35,"\u6216\u8005\u7531\u5b83\u4eec\u7ec4\u6210\u7684list":35,"\u6216\u8005\u76f4\u63a5\u4f7f\u7528\u4e0b\u9762\u7684shell\u547d\u4ee4":122,"\u6216\u8005\u76f4\u63a5\u6254\u6389\u975e\u5e38\u957f\u7684\u5e8f\u5217":80,"\u6216\u8005\u76f8\u5bf9\u4e8e\u6784\u5efa\u76ee\u5f55\u7684\u76f8\u5bf9\u8def\u5f84":[115,117],"\u6216\u8005\u8f93\u5165\u6570\u636e\u5c3a\u5ea6\u8fc7\u5927":80,"\u6216\u8005\u8fd0\u884c":77,"\u6216\u8005\u91c7\u7528\u5e76\u884c\u8ba1\u7b97\u6765\u52a0\u901f\u67d0\u4e9b\u5c42\u7684\u66f4\u65b0":108,"\u6216\u8005\u9700\u8981\u66f4\u9ad8\u7684\u6548\u7387":1,"\u6216\u8bbe\u7f6e\u4e3anone":1,"\u6216gpu":106,"\u622a\u65ad\u5bf9\u8c61\u4e0d\u540c":80,"\u623f":90,"\u623f\u95f4":90,"\u6240\u4ee5":[2,80,84,102],"\u6240\u4ee5\u4e00\u822c\u9700\u8981\u5bf9\u8bad\u7ec3\u7528\u7684\u6a21\u578b\u914d\u7f6e\u6587\u4ef6\u7a0d\u4f5c\u76f8\u5e94\u4fee\u6539\u624d\u80fd\u5728\u9884\u6d4b\u65f6\u4f7f\u7528":4,"\u6240\u4ee5\u4e0d\u80fd\u91c7\u7528\u7b2c\u4e00\u79cd\u65b9\u5f0f\u5728\u8fd9\u51e0\u4e2alayer\u91cc\u8bbe\u7f6e":81,"\u6240\u4ee5\u505a\u6cd5\u53ef\u4ee5\u6709\u4e24\u79cd":80,"\u6240\u4ee5\u53ef\u4ee5\u7b80\u5316\u5bf9\u73af\u5883\u7684\u8981\u6c42":110,"\u6240\u4ee5\u5728\u5199\u5165\u5feb\u7167\u7684\u8fc7\u7a0b\u4e2d":34,"\u6240\u4ee5\u5916\u5c42\u8f93\u51fa\u7684\u5e8f\u5217\u5f62\u72b6":90,"\u6240\u4ee5\u5b83\u4eec\u4f7f\u7528\u540c\u4e00\u4e2aip\u5730\u5740":109,"\u6240\u4ee5\u5bf9":90,"\u6240\u4ee5\u5f00\u53d1\u8005\u9700\u8981\u6839\u636e\u81ea\u5df1\u8bad\u7ec3\u4efb\u52a1\u7684\u5b9e\u9645\u573a\u666f\u5b8c\u6210\u8bad\u7ec3\u6570\u636e\u7684\u5206\u5272\u548c":104,"\u6240\u4ee5\u6027\u80fd\u4e5f\u5c31\u9010\u6b65\u53d8\u6210\u4e86\u6df1\u5ea6\u5b66\u4e60\u9886\u57df\u6700\u91cd\u8981\u7684\u6307\u6807":103,"\u6240\u4ee5\u6211\u4eec\u4f7f\u7528\u8fd9\u4e2a\u955c\u50cf\u6765\u4e0b\u8f7d\u8bad\u7ec3\u6570\u636e\u5230docker":110,"\u6240\u4ee5\u6211\u4eec\u53ef\u4ee5\u5728\u8fd9\u4e2a\u57fa\u7840\u4e0a":111,"\u6240\u4ee5\u6211\u4eec\u786e\u4fdd\u53d1\u5e03\u7684\u4e8c\u8fdb\u5236\u5305\u53ef\u4ee5\u652f\u6301\u4e3b\u6d41\u7684linux\u64cd\u4f5c\u7cfb\u7edf":86,"\u6240\u4ee5\u6211\u4eec\u9700\u8981\u5c06\u8f93\u5165\u6570\u636e\u6807\u8bb0\u6210":90,"\u6240\u4ee5\u6211\u4eec\u9ed8\u8ba4\u4f7f\u7528cento":86,"\u6240\u4ee5\u63a8\u8350\u4f7f\u7528\u663e\u5f0f\u6307\u5b9a\u7684\u65b9\u5f0f\u6765\u8bbe\u7f6einput_typ":2,"\u6240\u4ee5\u7528\u6237\u9700\u8981\u9996\u5148\u5728":44,"\u6240\u4ee5\u76f8\u6bd4\u4e8erecurr":81,"\u6240\u4ee5\u8f93\u51fa\u7684value\u5305\u542b\u4e24\u4e2a\u5411\u91cf":4,"\u6240\u4ee5\u8fd9\u4e00\u6b65\u662f\u5fc5\u8981\u7684":96,"\u6240\u4f7f\u7528":116,"\u6240\u4f9d\u8d56\u7684\u7b2c\u4e09\u65b9\u5e93\u540c\u65f6\u4e5f\u88ab\u5b89\u88c5\u5230":115,"\u6240\u5bf9\u5e94\u7684\u8bcd\u8868index\u6570\u7ec4":90,"\u6240\u6709\u4e0e\u7c7b\u578b\u76f8\u5173\u7684\u51fd\u6570":55,"\u6240\u6709\u4ee3\u7801\u5fc5\u987b\u5177\u6709\u5355\u5143\u6d4b\u8bd5":95,"\u6240\u6709\u53c2\u6570\u7f6e\u4e3a\u96f6":106,"\u6240\u6709\u547d\u4ee4\u884c\u9009\u9879\u53ef\u4ee5\u8bbe\u7f6e\u4e3a":104,"\u6240\u6709\u6587\u4ef6\u5217\u8868":2,"\u6240\u6709\u67b6\u6784":115,"\u6240\u6709\u7684":[95,96],"\u6240\u6709\u7684\u5355\u6d4b\u90fd\u4f1a\u88ab\u6267\u884c\u4e00\u6b21":96,"\u6240\u6709\u7684\u63a5\u53e3\u5747\u4e3ac\u63a5\u53e3":55,"\u6240\u6709\u7684\u64cd\u4f5c\u90fd\u662f\u9488\u5bf9\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65\u6765\u8fdb\u884c\u7684":90,"\u6240\u6709\u7684python\u5c01\u88c5\u90fd\u4f7f\u7528":96,"\u6240\u6709\u7684python\u5c01\u88c5\u90fd\u5728":96,"\u6240\u6709\u7c7b\u578b\u540d\u4e3a":55,"\u6240\u6709\u7f51\u7edc\u5c42\u7684\u68af\u5ea6\u68c0\u67e5\u5355\u6d4b\u90fd\u4f4d\u4e8e":96,"\u6240\u6709\u8f93\u5165\u5e8f\u5217\u5e94\u8be5\u6709\u76f8\u540c\u7684\u957f\u5ea6":93,"\u6240\u6709\u914d\u7f6e\u90fd\u80fd\u5728":123,"\u6240\u6784\u5efa\u7f51\u7edc\u7ed3\u6784\u7684\u7684\u6df1\u5ea6\u6bd4\u4e4b\u524d\u4f7f\u7528\u7684\u7f51\u7edc\u6709\u5927\u5e45\u5ea6\u7684\u63d0\u9ad8":122,"\u6240\u8c13\u65f6\u95f4\u6b65\u4fe1\u606f":2,"\u6240\u9700\u652f\u6301\u7684\u6700\u4f4eandroid":115,"\u6240\u9700\u7684\u5f00\u53d1\u5de5\u5177\u548c\u7b2c\u4e09\u65b9\u5e93\u53ef\u4ee5\u53c2\u8003":117,"\u624b\u5199\u591a\u8bed\u8a00\u7ed1\u5b9a":54,"\u624d\u4f1a\u91ca\u653e\u8be5\u6bb5\u5185\u5b58":2,"\u624d\u4f1astop":2,"\u624d\u53ef\u4ee5\u5b89\u88c5":86,"\u624d\u80fd\u4fdd\u8bc1\u548c\u5355\u5c42\u5e8f\u5217\u7684\u914d\u7f6e\u4e2d":90,"\u624d\u80fd\u53d1\u6325\u5176\u5168\u90e8\u80fd\u529b":103,"\u6253\u5f00":103,"\u6253\u5f00\u6587\u672c\u6587\u4ef6":2,"\u6253\u5f00\u6d4f\u89c8\u5668\u8bbf\u95ee\u5bf9\u5e94\u76ee\u5f55\u4e0b\u7684index":99,"\u6253\u5f00\u8fd9\u4e2a\u7f16\u8bd1\u9009\u9879":55,"\u6267\u884c":[88,104],"\u6267\u884c\u4e0a\u8ff0":115,"\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4":83,"\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u4ee5\u542f\u52a83\u4e2a\u8282\u70b9\u7684openmpi\u96c6\u7fa4\u548c\u4e00\u4e2a":104,"\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u5373\u53ef\u5728\u5f53\u524d\u673a\u5668\u4e0a\u5b89\u88c5paddlepaddle\u7684\u8fd0\u884c\u65f6\u73af\u5883":86,"\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u53ef\u4ee5\u67e5\u770b\u5df2\u7ecf\u5b89\u88c5\u7684\u7248\u672c":104,"\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u5b8c\u6210\u5feb\u901f\u5b89\u88c5":88,"\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u6765\u8fd0\u884c\u5355\u5143\u6d4b\u8bd5":97,"\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u7f16\u8bd1cpu":83,"\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4\u83b7\u53d6\u6700\u65b0\u7684paddlepaddl":84,"\u6267\u884c\u4ee5\u4e0b\u547d\u4ee4":[115,116,117],"\u6267\u884c\u4ee5\u4e0b\u547d\u4ee4\u542f\u52a8\u4f7f\u7528python\u7f16\u5199\u7684trainer\u7a0b\u5e8f":104,"\u6267\u884c\u4ee5\u4e0b\u64cd\u4f5c":93,"\u6267\u884c\u4ee5\u4e0b\u7684\u547d\u4ee4\u542f\u52a8\u4e00\u4e2a\u53c2\u6570\u670d\u52a1\u5668\u5e76\u7b49\u5f85\u548c\u8ba1\u7b97\u8282\u70b9\u7684\u6570\u636e\u4ea4\u4e92":104,"\u6267\u884c\u5b8c\u5b89\u88c5\u547d\u4ee4\u540e":[115,116,117],"\u6267\u884c\u60a8\u7684\u4ee3\u7801":103,"\u6267\u884c\u7684\u547d\u4ee4\u5982\u4e0b":122,"\u6269\u5c55\u673a\u5236\u7b49\u529f\u80fd":109,"\u627e\u5230":[83,93,104],"\u627e\u5230\u6700\u65e9\u62a5\u9519\u7684\u5730\u65b9":78,"\u627e\u5230\u8fd0\u884c\u6162\u7684\u539f\u56e0":103,"\u627e\u5230\u8fd0\u884c\u6162\u7684\u90e8\u5206":103,"\u628a":[35,96],"\u628a\u4e4b\u524d\u793a\u4f8b\u4e2d\u8f6c\u6362\u5b8c\u6bd5\u7684random":35,"\u628a\u4efb\u610f\u7ef4\u5ea6\u7684tensor\u8f6c\u4e3a\u4e86\u4e00\u7ef4\u7684eigenvector":98,"\u628a\u5de5\u5177\u548c\u914d\u7f6e\u90fd\u5b89\u88c5\u5728\u4e00\u4e2a":94,"\u628a\u8bad\u7ec3\u6570\u636e\u76f4\u63a5\u653e\u5728":110,"\u628a\u8fd9\u4e9b\u5de5\u5177\u5b89\u88c5\u5230\u672c\u673a":94,"\u6295\u5c04\u53cd\u5411rnn\u7684\u7b2c\u4e00\u4e2a\u5b9e\u4f8b\u5230":93,"\u6295\u5c04\u7f16\u7801\u5411\u91cf\u5230":93,"\u62bd\u53d6\u51fa\u7684\u65b0\u8bcd\u8868\u7684\u4fdd\u5b58\u8def\u5f84":121,"\u62bd\u53d6\u5bf9\u5e94\u7684\u8bcd\u5411\u91cf\u6784\u6210\u65b0\u7684\u8bcd\u8868":121,"\u62c6\u89e3":92,"\u62c6\u89e3\u6210\u7684\u6bcf\u4e00\u53e5\u8bdd\u518d\u901a\u8fc7\u4e00\u4e2alstm\u7f51\u7edc":90,"\u62f7\u8d1d\u5fc5\u8981\u7684\u6587\u4ef6\u5230head\u8282\u70b9":104,"\u62f7\u8d1d\u8bad\u7ec3\u6570\u636e\u5230\u5404\u81ea\u7684\u8282\u70b9":104,"\u62f7\u8d1d\u8bad\u7ec3\u6587\u4ef6\u5230\u5bb9\u5668\u5185":111,"\u62f7\u8d1d\u8bad\u7ec3\u7a0b\u5e8f\u548c\u5b57\u5178\u6587\u4ef6\u5230\u6bcf\u53f0mpi\u8282\u70b9":104,"\u62fc\u63a5":80,"\u62fc\u63a5\u6210\u4e00\u4e2a\u65b0\u7684\u5411\u91cf":123,"\u6302\u8f7d\u5230\u5bb9\u5668\u5185\u90e8\u7684":84,"\u6302\u8f7d\u6216\u4e0b\u8f7d\u7684\u8bad\u7ec3\u6570\u636e\u5206\u7247":104,"\u6307\u53d1\u73b0\u6027\u80fd\u74f6\u9888":102,"\u6307\u5411\u4e00\u4e2alayer":92,"\u6307\u5b9a":[80,81,92,93],"\u6307\u5b9a\u4e00\u53f0\u673a\u5668\u4e0a\u4f7f\u7528\u7684\u7ebf\u7a0b\u6570":106,"\u6307\u5b9a\u4f7f\u75282":80,"\u6307\u5b9a\u521d\u59cb\u5316\u6a21\u578b\u8def\u5f84":123,"\u6307\u5b9a\u524d\u5411\u7f51\u7edc\u6700\u7ec8\u7684\u8f93\u51fa\u76ee\u6807\u53d8\u91cf":97,"\u6307\u5b9a\u52a0\u8f7d\u7684\u65b9\u5f0f":106,"\u6307\u5b9a\u5728\u751f\u6210\u6027\u80fd\u5206\u6790\u6587\u4ef6\u4e4b\u540e":102,"\u6307\u5b9a\u5bf9\u8f93\u5165\u53d8\u91cf":97,"\u6307\u5b9a\u5c06\u5f53\u524d\u8def\u5f84":84,"\u6307\u5b9a\u5de5\u4f5c\u6a21\u578b\u8fdb\u884c\u9884\u6d4b":122,"\u6307\u5b9a\u5de5\u4f5c\u6a21\u5f0f\u6765\u63d0\u53d6\u7279\u5f81":122,"\u6307\u5b9a\u6267\u884c\u5176\u4e2d\u4e00\u4e2a\u5355\u5143\u6d4b\u8bd5":83,"\u6307\u5b9a\u63d0\u53d6\u7279\u5f81\u7684\u5c42":122,"\u6307\u5b9a\u662f\u5426\u4f7f\u7528gpu":122,"\u6307\u5b9a\u68c0\u6d4b\u68af\u5ea6\u65f6\u80fd\u5bb9\u5fcd\u7684\u6700\u5927\u9519\u8bef\u503c":97,"\u6307\u5b9a\u751f\u6210\u6570\u636e\u7684\u51fd\u6570":123,"\u6307\u5b9a\u7684\u5185\u5bb9\u5b58\u50a8\u5e93\u8fd0\u884c\u547d\u4ee4":99,"\u6307\u5b9a\u7684\u6570\u636e\u5c06\u4f1a\u88ab\u6d4b\u8bd5":123,"\u6307\u5b9a\u7684\u8f93\u5165\u4e0d\u4f1a\u88ab":92,"\u6307\u5b9a\u8981\u8f93\u51fa\u7684\u5b57\u6bb5\u8fdb\u884c\u8f93\u51fa":80,"\u6307\u5b9a\u8bad\u7ec3\u6570\u636e\u548c\u6d4b\u8bd5\u6570\u636e":123,"\u6307\u5b9a\u9700\u8981\u4f7f\u7528\u7684\u5bb9\u5668":84,"\u6307\u5b9acudnn\u7684\u6700\u5927\u5de5\u4f5c\u7a7a\u95f4\u5bb9\u9650":106,"\u6307\u5bf9\u4e8e\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217\u8f93\u5165\u6570\u636e":90,"\u6307\u5f00\u542fhttp\u670d\u52a1":102,"\u6307\u6d88\u9664\u74f6\u9888":102,"\u6307\u6df1\u5ea6\u5b66\u4e60\u8bad\u7ec3\u4e4b\u540e\u5f97\u5230\u7684\u6240\u6709\u53c2\u6570":34,"\u6307\u793a\u4f7f\u7528\u54ea\u4e2agpu\u6838":106,"\u6307\u793a\u5728\u7b80\u5355\u7684recurrentlayer\u5c42\u7684\u8ba1\u7b97\u4e2d\u662f\u5426\u4f7f\u7528\u6279\u5904\u7406\u65b9\u6cd5":106,"\u6307\u793a\u5f53\u6307\u5b9a\u8f6e\u7684\u6d4b\u8bd5\u6a21\u578b\u4e0d\u5b58\u5728\u65f6":106,"\u6307\u793a\u662f\u5426\u4f7f\u7528\u591a\u7ebf\u7a0b\u6765\u8ba1\u7b97\u4e00\u4e2a\u795e\u7ecf\u7f51\u7edc":106,"\u6307\u793a\u662f\u5426\u5f00\u542f\u53c2\u6570\u670d\u52a1\u5668":106,"\u6307\u793a\u662f\u5426\u663e\u793a\u53c2\u6570\u670d\u52a1\u5668\u4e0a\u7684\u7a00\u758f\u53c2\u6570\u5206\u5e03\u7684\u65e5\u5fd7\u7ec6\u8282":106,"\u6307\u793a\u662f\u5426\u68c0\u67e5\u6240\u6709\u53c2\u6570\u670d\u52a1\u5668\u4e0a\u7684\u7a00\u758f\u53c2\u6570\u7684\u5206\u5e03\u662f\u5747\u5300\u7684":106,"\u6309\u542f\u53d1\u5f0f\u635f\u5931\u7684\u5927\u5c0f\u9012\u589e\u6392\u5e8f":106,"\u6309\u7167\u4e0b\u9762\u6b65\u9aa4\u5373\u53ef":111,"\u6309\u7167\u5176\u5185\u5bb9\u521b\u5efa\u4e00\u4e2a\u540d\u4e3a":94,"\u6309\u7167\u5177\u4f53\u5b9e\u73b0\u65b9\u5f0f\u53ef\u4ee5\u5f52\u7eb3\u4e3a2\u7c7b":81,"\u6309\u94ae":95,"\u633a":90,"\u633a\u597d":90,"\u6355\u83b7":123,"\u6362":90,"\u6392\u6210\u4e00\u5217\u7684\u591a\u4e2a\u5143\u7d20":89,"\u63a5\u4e0a\u4e00\u4e2a\u5168\u8fde\u63a5\u5c42":87,"\u63a5\u4e0a\u5e73\u65b9\u8bef\u5dee\u5c42":87,"\u63a5\u4e0b\u6765":[97,123],"\u63a5\u4e0b\u6765\u53ef\u4ee5\u8003\u8651\u4e0b\u65f6\u95f4\u7ebf\u7684\u5206\u6790":103,"\u63a5\u4e0b\u6765\u5c31\u53ef\u4ee5\u4f7f\u7528":103,"\u63a5\u4e0b\u6765\u6211\u4eec\u521b\u5efa\u4e00\u4e2a\u539f\u59cb":95,"\u63a5\u4e0b\u6765\u6211\u4eec\u53d6\u6d88\u5bf9":95,"\u63a5\u4e0b\u6765\u6211\u4eec\u5c06\u5c55\u793a\u5982\u4f55\u7528paddlepaddle\u8bad\u7ec3\u4e00\u4e2a\u6587\u672c\u5206\u7c7b\u6a21\u578b":123,"\u63a5\u4e0b\u6765\u7b49\u5f85":95,"\u63a5\u53d7\u4e00\u4e2a\u8f93\u5165\u53c2\u6570":97,"\u63a5\u53e3":[54,55,97,98],"\u63a5\u53e3\u4f1a\u88ab\u8c03\u7528":98,"\u63a5\u53e3\u5c42\u505a\u8fc7\u591a\u5c01\u88c5":55,"\u63a5\u53e3\u63d0\u53d6\u7684\u7ed3\u679c\u662f\u4e00\u81f4\u7684":122,"\u63a5\u53e3\u662f":35,"\u63a5\u53e3\u6700\u7ec8\u4f1a\u8c03\u7528\u5bf9\u5e94":98,"\u63a5\u53e3\u6709\u4e00\u4e2a":80,"\u63a5\u53e3\u6765\u52a0\u8f7d\u6570\u636e":123,"\u63a5\u53e3\u6765\u52a0\u8f7d\u8be5\u6587\u4ef6":122,"\u63a5\u53e3\u6765\u6253\u5f00\u8be5\u6587\u4ef6":122,"\u63a5\u53e3\u7684":80,"\u63a5\u6536\u5904\u7406pfsclient\u7aef\u7684\u6587\u4ef6\u7ba1\u7406\u8bf7\u6c42":44,"\u63a7\u5236":106,"\u63a7\u5236\u7528\u6237\u6743\u9650":35,"\u63a8\u5bfc\u8be5\u5c42\u524d\u5411\u548c\u540e\u5411\u4f20\u9012\u7684\u65b9\u7a0b":96,"\u63a8\u8350":90,"\u63a8\u8350\u4f7f\u7528":2,"\u63a8\u8350\u4f7f\u7528centos\u7684devtools2":83,"\u63a8\u8350\u6e05\u7406\u6574\u4e2a\u7f16\u8bd1\u76ee\u5f55":83,"\u63a8\u8350\u76f4\u63a5\u5b58\u653e\u5230\u8bad\u7ec3\u76ee\u5f55":1,"\u63a8\u9001\u5230\u8fdc\u7a0b\u4ed3\u5e93":95,"\u63cf\u8ff0\u7684\u9ed8\u8ba4\u5165\u53e3\u7a0b\u5e8f":94,"\u63cf\u8ff0\u7f51\u7edc\u7ed3\u6784\u548c\u4f18\u5316\u7b97\u6cd5":123,"\u63cf\u8ff0\u8be5op\u7684\u8f93\u5165":97,"\u63cf\u8ff0\u95ee\u9898":95,"\u63cf\u8ff0kubernetes\u4e0a\u8fd0\u884c\u7684\u4f5c\u4e1a":109,"\u63d0\u4ea4\u65b9\u5f0f\u53c2\u89c1":99,"\u63d0\u4ea4pull":95,"\u63d0\u4f9b":104,"\u63d0\u4f9b\u4e03\u5c42\u534f\u8bae\u7684\u53cd\u5411\u4ee3\u7406":44,"\u63d0\u4f9b\u4e86\u4e00\u4e2a\u542f\u52a8\u811a\u672c":111,"\u63d0\u4f9b\u4e86\u547d\u4ee4\u6837\u4f8b\u6765\u8fd0\u884c":104,"\u63d0\u4f9b\u4e86\u65b9\u4fbf\u7684\u548c":102,"\u63d0\u4f9b\u4e86\u81ea\u52a8\u5316\u811a\u672c\u6765\u542f\u52a8\u4e0d\u540c\u8282\u70b9\u4e2d\u7684\u6240\u6709":104,"\u63d0\u4f9b\u51e0\u4e4e\u6240\u6709\u8bad\u7ec3\u7684\u5185\u90e8\u8f93\u51fa\u65e5\u5fd7":104,"\u63d0\u4f9b\u5e38\u7528\u7684\u547d\u4ee4\u884c\u7ba1\u7406\u547d\u4ee4\u7ba1\u7406\u6587\u4ef6\u548c\u76ee\u5f55":44,"\u63d0\u4f9b\u6269\u5c55\u7684\u957f\u5ea6\u4fe1\u606f":89,"\u63d0\u4f9b\u7528\u6237\u7ba1\u7406\u6587\u4ef6\u7684\u547d\u4ee4":44,"\u63d0\u4f9b\u7ed9paddle\u4f5c\u4e3a\u8bad\u7ec3\u6570\u636e":35,"\u63d0\u4f9b\u8bad\u7ec3\u8fc7\u7a0b\u7684":104,"\u63d0\u51fa\u7684\u4ee3\u7801\u9700\u6c42":121,"\u63d0\u793a":77,"\u641c\u7d22\u4ee3\u7801\u5e93":99,"\u642d\u5efa\u795e\u7ecf\u7f51\u7edc\u5c31\u50cf\u4f7f\u7528\u79ef\u6728\u642d\u5efa\u5b9d\u5854\u4e00\u6837":87,"\u64cd\u4f5c":90,"\u64cd\u4f5c\u7cfb\u7edf":[86,94],"\u652f\u6301\u4ea4\u53c9\u7f16\u8bd1":117,"\u652f\u6301\u53cc\u5c42\u5e8f\u5217\u4f5c\u4e3a\u8f93\u5165\u7684layer":[91,92],"\u652f\u6301\u5927\u6587\u4ef6\u7684\u65ad\u70b9\u4e0a\u4f20":44,"\u652f\u6301\u5927\u89c4\u6a21\u96c6\u7fa4\u751f\u4ea7\u73af\u5883\u7684\u5b8c\u6574\u96c6\u7fa4\u65b9\u6848":104,"\u652f\u6301\u7684\u6700\u5c0f\u7684android":115,"\u652f\u6301\u7684\u6700\u5c0fandroid":115,"\u652f\u6301\u7f16\u8bd1\u5668":115,"\u652f\u6301rbd":109,"\u653e\u5728\u8fd9\u4e2a\u76ee\u5f55\u91cc\u7684\u6587\u4ef6\u5176\u5b9e\u662f\u4fdd\u5b58\u5230\u4e86mfs\u4e0a":111,"\u653e\u5fc3":90,"\u6545\u800c\u662f\u4e00\u4e2a\u5355\u5c42\u65f6\u95f4\u5e8f\u5217":90,"\u6548\u679c\u5982\u4e0b":102,"\u6548\u679c\u603b\u7ed3":123,"\u6559\u7a0b":84,"\u6570":92,"\u6570\u5fc5\u987b\u4e25\u683c\u76f8\u7b49":92,"\u6570\u636e":44,"\u6570\u636e\u4e2d0":82,"\u6570\u636e\u5206\u7247":104,"\u6570\u636e\u5217\u8868":122,"\u6570\u636e\u5c06\u4fdd\u5b58\u5728":121,"\u6570\u636e\u63d0\u4f9b\u5668":105,"\u6570\u636e\u7c7b\u578b":4,"\u6570\u636e\u7f13\u5b58\u7684\u7b56\u7565":2,"\u6570\u636e\u8bbf\u95ee":0,"\u6570\u636e\u8bfb\u53d6\u5747\u4ea4\u7531\u5176\u4ed6\u8bed\u8a00\u5b8c\u6210":54,"\u6570\u636e\u8f93\u5165":92,"\u6570\u636e\u8f93\u5165\u683c\u5f0f":2,"\u6570\u636e\u957f\u5ea6\u53ca\u6821\u9a8c\u503c\u7ec4\u6210":44,"\u6570\u636e\u96c6\u9700\u8981\u9884\u5148\u88ab\u8f6c\u6362\u6210paddlepaddle\u5206\u5e03\u5f0f\u8bad\u7ec3\u4f7f\u7528\u7684\u5b58\u50a8\u683c":35,"\u6570\u636e\u9884\u5904\u7406\u4efb\u52a1":35,"\u6570\u636e\u9884\u5904\u7406\u5b8c\u6210\u4e4b\u540e":123,"\u6570\u76ee":108,"\u6574\u4f53":90,"\u6574\u4f53\u6570\u636e\u548c\u539f\u59cb\u6570\u636e\u5b8c\u5168\u4e00\u6837":90,"\u6574\u4f53\u7684\u7ed3\u6784\u56fe\u5982\u4e0b":111,"\u6574\u6570":96,"\u6574\u6570\u6807\u7b7e":[2,87],"\u6574\u6d01":90,"\u6587\u4ef6":[54,94,95,97,110],"\u6587\u4ef6\u4e2d":[97,111,122],"\u6587\u4ef6\u4e2d\u6307\u5b9a\u6a21\u578b\u8def\u5f84\u548c\u8f93\u51fa\u7684\u76ee\u5f55":122,"\u6587\u4ef6\u4e2d\u6307\u5b9a\u8981\u63d0\u53d6\u7279\u5f81\u7684\u7f51\u7edc\u5c42\u7684\u540d\u5b57":122,"\u6587\u4ef6\u4e2d\u6ce8\u518c\u524d\u5411":97,"\u6587\u4ef6\u4e2d\u6ce8\u518c\u8be5op\u548ckernel":97,"\u6587\u4ef6\u4e2d\u6ce8\u518cgpu":97,"\u6587\u4ef6\u4e2d\u7684":122,"\u6587\u4ef6\u4e3a":80,"\u6587\u4ef6\u4e4b\u5916":95,"\u6587\u4ef6\u4e5f\u53ef\u4ee5\u7528\u4e8e\u5bf9\u6837\u672c\u8fdb\u884c\u9884\u6d4b":122,"\u6587\u4ef6\u4f20\u8f93\u7684\u7684\u5173\u952e\u5728\u4e8e\u9700\u8981pfsclient\u7aef\u5bf9\u6bd4source\u548cdestination\u7684\u6587\u4ef6chunks\u7684checksum\u662f\u5426\u4fdd\u6301\u4e00\u81f4":44,"\u6587\u4ef6\u5185\u5bb9\u4e3a":54,"\u6587\u4ef6\u540d":102,"\u6587\u4ef6\u540d\u4e3a\u4efb\u610f\u6587\u4ef6\u540d":104,"\u6587\u4ef6\u540d\u4e3a\u6b64uuid":34,"\u6587\u4ef6\u547d\u540d\u4ee5":97,"\u6587\u4ef6\u59390":111,"\u6587\u4ef6\u5bf9\u5e94\u7684data":35,"\u6587\u4ef6\u5de5\u5177\u662f\u4f7f\u7528docker":99,"\u6587\u4ef6\u7684\u4e0a\u4f20\u548c\u4e0b\u8f7d\u90fd\u662f\u901a\u8fc7\u5bf9chunk\u7684\u64cd\u4f5c\u6765\u5b9e\u73b0\u7684":44,"\u6587\u4ef6\u7684\u6539\u53d8":95,"\u6587\u4ef6\u7ed9\u51fa\u4e86\u5b8c\u6574\u4f8b\u5b50":123,"\u6587\u4ef6model":108,"\u6587\u5b57\u7684\u4ea4\u4e92\u5f0f\u6587\u6863":84,"\u6587\u672c\u4e2d\u7684\u5355\u8bcd\u7528\u7a7a\u683c\u5206\u9694":123,"\u6587\u672c\u4fe1\u606f\u5c31\u662f\u4e00\u4e2a\u5e8f\u5217\u6570\u636e":2,"\u6587\u672c\u5206\u7c7b\u95ee\u9898":123,"\u6587\u672c\u5377\u79ef\u5206\u53ef\u4e3a\u4e09\u4e2a\u6b65\u9aa4":123,"\u6587\u6863":77,"\u6587\u68631":98,"\u6587\u68632":98,"\u6587\u6863\u8f83\u5c11":98,"\u6587\u6863\u90fd\u662f\u901a\u8fc7":99,"\u6587\u7ae0":111,"\u65b0":90,"\u65b0\u5efa\u4e00\u4e2a\u6743\u91cd":96,"\u65b0\u624b\u5165\u95e8":114,"\u65b0\u624b\u5165\u95e8\u7ae0\u8282":71,"\u65b9\u4fbf":90,"\u65b9\u4fbf\u5feb\u901f\u5b89\u88c5":85,"\u65b9\u4fbf\u6d4b\u8bd5\u4eba\u5458\u6d4b\u8bd5paddlepaddle\u7684\u884c\u4e3a":71,"\u65b9\u4fbf\u7528\u6237\u4e0a\u4f20\u81ea\u5df1\u7684\u8bad\u7ec3\u6570\u636e\u4ee5\u8fdb\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3":44,"\u65b9\u5f0f1":80,"\u65b9\u5f0f2":80,"\u65b9\u6cd5\u4e00":108,"\u65b9\u6cd5\u4e09":108,"\u65b9\u6cd5\u4e8c":108,"\u65c1\u8fb9":90,"\u65e0":90,"\u65e0\u5ef6\u8fdf":106,"\u65e0\u6cd5\u505a\u5230\u5bf9\u4e8e\u5404\u79cd\u8bed\u8a00\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u7684\u9002\u914d":54,"\u65e0\u8bba\u5728\u672c\u5730\u8fd8\u662f\u5728\u4e91\u7aef":35,"\u65e0\u8bba\u662f\u4ece":35,"\u65e0\u8bba\u662f\u5728\u672c\u5730\u6216\u662f\u4e91\u7aef\u8f6c\u6362":35,"\u65e0\u9ed8\u8ba4\u503c":[115,117],"\u65e5\u5fd7\u62a5\u9519\u4e3a\u7f51\u7edc\u901a\u4fe1\u7c7b\u9519\u8bef":78,"\u65e9\u9910":90,"\u65f6":[34,80,82,89,93,96,106,111,115],"\u65f6\u5019":90,"\u65f6\u5e8f\u6a21\u578b\u5747\u4f7f\u7528\u8be5\u811a\u672c":123,"\u65f6\u5e8f\u6a21\u578b\u662f\u6307\u6570\u636e\u7684\u67d0\u4e00\u7ef4\u5ea6\u662f\u4e00\u4e2a\u5e8f\u5217\u5f62\u5f0f":2,"\u65f6\u6709\u6548":116,"\u65f6\u88ab\u8bad\u7ec3\u7684":96,"\u65f6\u8bbe\u5907id\u53f7\u7684\u5206\u914d":108,"\u65f6\u95f4":90,"\u65f6\u95f4\u6b65\u7684\u6982\u5ff5":90,"\u65f6\u987b\u4ece\u7b2c17\u5b57\u8282\u5f00\u59cb":82,"\u6620\u5c04\u4e3a":94,"\u6620\u5c04\u5230\u4e00\u4e2a\u7ef4\u5ea6\u4e3a":96,"\u662f":[44,77,86,90],"\u662f\u4e00\u4e2a\u51681\u7684\u5411\u91cf":96,"\u662f\u4e00\u4e2a\u5185\u7f6e\u7684\u5b9a\u65f6\u5668\u5c01\u88c5":103,"\u662f\u4e00\u4e2a\u52a8\u6001\u7a0b\u5e8f\u5206\u6790\u7684\u672f\u8bed":103,"\u662f\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":89,"\u662f\u4e00\u4e2a\u53cc\u5c42\u7684\u5e8f\u5217":89,"\u662f\u4e00\u4e2a\u591a\u8bed\u8a00\u63a5\u53e3\u7684\u4ee3\u7801\u751f\u6210\u5668":54,"\u662f\u4e00\u4e2a\u5c01\u88c5\u5bf9\u8c61":103,"\u662f\u4e00\u4e2a\u5f88\u6709\u7528\u7684\u53c2\u6570":108,"\u662f\u4e00\u4e2a\u7c7b\u578b\u7684\u6807\u5fd7":55,"\u662f\u4e00\u4e2a\u975e\u7ebf\u6027\u7684":96,"\u662f\u4e00\u4e2apython\u7684":2,"\u662f\u4e00\u4e2apython\u7684\u7b2c\u4e09\u65b9\u5e93":102,"\u662f\u4e00\u4e2aunbound":92,"\u662f\u4e00\u6761\u65f6\u95f4\u5e8f\u5217":[2,87],"\u662f\u4e00\u79cd\u4efb\u610f\u590d\u6742\u7684rnn\u5355\u5143":92,"\u662f\u4e00\u7ec4":109,"\u662f\u4e0d\u5305\u62ec\u6e90\u7801\u7684":110,"\u662f\u4e0d\u5e38\u89c1\u7684\u505a\u6cd5":54,"\u662f\u4e0d\u662f\u5f88\u7b80\u5355\u5462":2,"\u662f\u4e0d\u662f\u8981\u5bf9\u6570\u636e\u505ashuffl":2,"\u662f\u4ec0\u4e48\u4e5f\u6ca1\u5173\u7cfb":2,"\u662f\u4f7f\u5f97\u8981\u5171\u4eab\u7684\u53c2\u6570\u4f7f\u7528\u540c\u6837\u7684":82,"\u662f\u504f\u5dee":93,"\u662f\u51e0\u4e4e\u4e0d\u5360\u5185\u5b58\u7684":2,"\u662f\u5404\u4e2a\u5b9e\u73b0\u4e2d\u5171\u4eab\u7684\u5934\u6587\u4ef6":55,"\u662f\u5411\u91cf":96,"\u662f\u5426\u4ec5\u7f16\u8bd1capi":83,"\u662f\u5426\u4ee5\u9006\u5e8f\u5904\u7406\u8f93\u5165\u5e8f\u5217":92,"\u662f\u5426\u4f7f\u7528":116,"\u662f\u5426\u4f7f\u7528\u53cc\u7cbe\u5ea6\u6d6e\u70b9\u6570":83,"\u662f\u5426\u4f7f\u7528\u65e7\u7684remoteparameterupdat":106,"\u662f\u5426\u4f7f\u7528\u6743\u91cd":96,"\u662f\u5426\u4f7f\u7528arm\u6a21\u5f0f":115,"\u662f\u5426\u4f7f\u7528eigen\u5e93\u8fdb\u884c\u77e9\u9635\u8ba1\u7b97":[115,116],"\u662f\u5426\u4f7f\u7528mkl\u6570\u5b66\u5e93":83,"\u662f\u5426\u4f7f\u7528neon\u6307\u4ee4":[115,117],"\u662f\u5426\u4f7f\u80fd":116,"\u662f\u5426\u5141\u8bb8\u6682\u5b58\u7565\u5fae\u591a\u4f59pool_size\u7684\u6570\u636e":2,"\u662f\u5426\u5185\u5d4cpython\u89e3\u91ca\u5668":83,"\u662f\u5426\u5219\u5171\u4eab\u540c\u4e00\u4e2a":97,"\u662f\u5426\u542f\u7528gpu\u8bad\u7ec3":104,"\u662f\u5426\u5c06\u5168\u5c40\u79cd\u5b50\u5e94\u7528\u4e8e\u672c\u5730\u7ebf\u7a0b\u7684\u968f\u673a\u6570":106,"\u662f\u5426\u5f00\u542f\u5355\u5143\u6d4b\u8bd5":83,"\u662f\u5426\u5fc5\u9009":104,"\u662f\u5426\u6253\u5370\u7248\u672c\u4fe1\u606f":106,"\u662f\u5426\u652f\u6301gpu":83,"\u662f\u5426\u663e\u793a":106,"\u662f\u5426\u7a00\u758f":96,"\u662f\u5426\u7f16\u8bd1\u4e2d\u82f1\u6587\u6587\u6863":83,"\u662f\u5426\u7f16\u8bd1\u542b\u6709avx\u6307\u4ee4\u96c6\u7684paddlepaddle\u4e8c\u8fdb\u5236\u6587\u4ef6":83,"\u662f\u5426\u7f16\u8bd1\u65f6\u8fdb\u884c\u4ee3\u7801\u98ce\u683c\u68c0\u67e5":83,"\u662f\u5426\u7f16\u8bd1c":116,"\u662f\u5426\u7f16\u8bd1go\u8bed\u8a00\u7684\u53ef\u5bb9\u9519paramet":83,"\u662f\u5426\u7f16\u8bd1python\u7684swig\u63a5\u53e3":83,"\u662f\u5426\u8fd0\u884c\u65f6\u52a8\u6001\u52a0\u8f7dcuda\u52a8\u6001\u5e93":83,"\u662f\u5426\u9700\u8981\u7b49\u5f85\u8be5\u8f6e\u6a21\u578b\u53c2\u6570":106,"\u662f\u56e0\u4e3a\u8fd9\u4e2a\u6d41\u7a0b\u6bd4\u5176\u4ed6\u65b9\u6cd5\u90fd\u66f4\u7b80\u4fbf":94,"\u662f\u56e0\u4e3ac99\u652f\u6301":54,"\u662f\u5728paddlepaddle\u4e2d\u6784\u9020\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u65f6\u6700\u91cd\u8981\u7684\u6982\u5ff5":93,"\u662f\u5b58\u6709\u4e00\u7cfb\u5217\u53d8\u6362\u77e9\u9635\u7684\u6743\u91cd":96,"\u662f\u5b58\u6709\u504f\u7f6e\u5411\u91cf\u7684\u6743\u91cd":96,"\u662f\u5bf9\u7528\u6237\u6587\u4ef6\u5b58\u50a8\u7a7a\u95f4\u7684\u62bd\u8c61":44,"\u662f\u5bfb\u627e\u74f6\u9888\u7684\u5173\u952e\u6307\u6807":102,"\u662f\u5f00\u542favx\u7f16\u8bd1\u7684":84,"\u662f\u5f85\u6269\u5c55\u7684\u6570\u636e":89,"\u662f\u6211\u4eec":95,"\u662f\u6211\u4eec\u8981\u5206\u6790\u7684\u7a0b\u5e8f":102,"\u662f\u6307":55,"\u662f\u6307\u4e00\u7cfb\u5217\u7684\u7279\u5f81\u6570\u636e":90,"\u662f\u6307recurrent_group\u7684\u591a\u4e2a\u8f93\u5165\u5e8f\u5217":90,"\u662f\u6570\u636e\u8f93\u5165":93,"\u662f\u6709\u610f\u4e49\u7684":90,"\u662f\u6784\u5efa\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u6700\u91cd\u8981\u7684\u5de5\u5177":93,"\u662f\u6ca1\u6709\u540d\u5b57\u7684":94,"\u662f\u7528\u6237\u4f7f\u7528c":55,"\u662f\u7684":94,"\u662f\u77e9\u9635":96,"\u662f\u7f51\u7edc\u5c42\u5b9e\u4f8b\u7684\u540d\u5b57\u6807\u8bc6\u7b26":96,"\u662f\u7f51\u7edc\u5c42\u7684\u6807\u8bc6\u7b26":96,"\u662f\u7f51\u7edc\u5c42\u7684\u7c7b\u578b":96,"\u662f\u7f51\u7edc\u5c42\u8f93\u51fa\u7684\u5927\u5c0f":96,"\u662f\u8be5\u5c42\u7684\u6807\u8bc6\u7b26":96,"\u662f\u8be5\u5c42\u7684\u7c7b\u540d":96,"\u662f\u8be5\u7f51\u7edc\u5c42\u7684":96,"\u662f\u8f93\u5165":93,"\u662f\u8fd9\u4e00\u7c7b\u7684":81,"\u662f\u901a\u7528\u7269\u4f53\u5206\u7c7b\u9886\u57df\u4e00\u4e2a\u4f17\u6240\u5468\u77e5\u7684\u6570\u636e\u5e93":122,"\u662f\u9700\u8981\u4e86\u89e3\u54ea\u4e9b\u6b65\u9aa4\u62d6\u6162\u4e86\u6574\u4f53":103,"\u662fc":55,"\u662fdecoder\u7684\u6570\u636e\u8f93\u5165":92,"\u662fgoogle\u5f00\u6e90\u7684\u5bb9\u5668\u96c6\u7fa4\u7ba1\u7406\u7cfb\u7edf":109,"\u662fnvidia\u6027\u80fd\u5206\u6790\u5de5\u5177":103,"\u662fpaddlepaddle\u652f\u6301\u7684\u4e00\u79cd\u4efb\u610f\u590d\u6742\u7684rnn\u5355\u5143":92,"\u662fpaddlepaddle\u8d1f\u8d23\u63d0\u4f9b\u6570\u636e\u7684\u6a21\u5757":123,"\u662fpod\u5185\u7684\u5bb9\u5668\u90fd\u53ef\u4ee5\u8bbf\u95ee\u7684\u5171\u4eab\u76ee\u5f55":109,"\u662fpython\u5c01\u88c5\u7684\u7c7b\u540d":96,"\u662frnn\u72b6\u6001":93,"\u663e":123,"\u663e\u5f0f\u6307\u5b9a\u8fd4\u56de\u7684\u662f\u4e00\u4e2a28":2,"\u663e\u7136":102,"\u665a":90,"\u6682\u4e0d\u8003\u8651\u5728\u5185":80,"\u6682\u65e0":86,"\u6682\u65f6\u4e0d\u652f\u6301python3":86,"\u6682\u65f6\u4e0d\u8003\u8651\u591a\u4e2aparamet":34,"\u66b4\u9732\u8fd9\u4e2a\u6982\u5ff5\u5fc5\u8981\u51fd\u6570":55,"\u66f4\u522b\u63d0\u7b80\u5316\u95ee\u9898\u590d\u73b0\u5e26\u6765\u7684\u597d\u5904\u4e86":94,"\u66f4\u591a\u5173\u4e8edocker\u7684\u5b89\u88c5\u4e0e\u4f7f\u7528":77,"\u66f4\u591a\u7684\u8f6c\u6362\u65b9\u6cd5\u8bf7\u53c2\u8003eigen":98,"\u66f4\u597d\u5730\u5b8c\u6210\u4e00\u4e9b\u590d\u6742\u7684\u8bed\u8a00\u7406\u89e3\u4efb\u52a1":92,"\u66f4\u5feb":93,"\u66f4\u65b0":77,"\u66f4\u65b0\u53ef\u80fd\u5bfc\u81f4\u9700\u8981\u65b0\u7684\u5f00\u53d1\u5de5\u5177":94,"\u66f4\u65b0\u6a21\u5f0f":80,"\u66f4\u65b0\u7684\u6587\u6863\u4ee5pr\u7684\u5f62\u5f0f\u63d0\u4ea4\u5230github\u4e2d":99,"\u66f4\u65b0\u7f51\u7edc\u53c2\u6570\u65f6\u5e94\u7528":80,"\u66f4\u65b9\u4fbf\u7684\u8bbe\u7f6e\u65b9\u5f0f":82,"\u66f4\u8be6\u7ec6\u6570\u636e\u683c\u5f0f\u548c\u7528\u4f8b\u8bf7\u53c2\u8003":123,"\u66f4\u8be6\u7ec6\u7684\u5b89\u88c5\u548c\u7f16\u8bd1\u65b9\u6cd5\u53c2\u8003":88,"\u66f4\u8be6\u7ec6\u7684\u7f51\u7edc\u914d\u7f6e\u8fde\u63a5\u8bf7\u53c2\u8003":123,"\u66f4\u8be6\u7ec6\u7684\u8bf4\u660e":123,"\u66f4\u8fdb\u4e00\u6b65":92,"\u66f4\u9ad8":93,"\u66ff\u6211\u4eec\u5b8c\u6210\u4e86\u539f\u59cb\u8f93\u5165\u6570\u636e\u7684\u62c6\u5206":92,"\u6700":90,"\u6700\u4e3b\u8981\u7684\u5de5\u4f5c\u5c31\u662f\u89e3\u6790\u51fa":111,"\u6700\u4f73\u63a8\u8350":2,"\u6700\u540e":[2,84,95,96,104,123],"\u6700\u540e\u4e00\u4e2a":89,"\u6700\u540e\u4e00\u5c42cost\u4e2d\u8bb0\u5f55\u4e86\u795e\u7ecf\u7f51\u7edc\u7684\u6240\u6709\u62d3\u6251\u7ed3\u6784":87,"\u6700\u540e\u518d\u8c03\u7528mutabl":98,"\u6700\u540e\u5220\u9664":71,"\u6700\u540e\u6211\u4eec\u4f7f\u7528\u94fe\u5f0f\u6cd5\u5219\u8ba1\u7b97":96,"\u6700\u5c0f\u7684ios\u90e8\u7f72\u7248\u672c":116,"\u6700\u5c11\u663e\u793a\u591a\u5c11\u4e2a\u8282\u70b9":106,"\u6700\u5e38\u89c1\u7684\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u662fexcept":54,"\u6700\u65b0\u7684\u4ee3\u7801":95,"\u6700\u65b0\u7684paddlepaddl":[77,84],"\u6700\u7ec8":96,"\u6700\u7ec8\u5b9e\u73b0\u4e00\u4e2a\u5c42\u6b21\u5316\u7684\u590d\u6742rnn":92,"\u6700\u7ec8\u6211\u4eec\u53ef\u4ee5\u8c03\u7528trainer\u7684train\u65b9\u6cd5\u542f\u52a8\u8bad\u7ec3":87,"\u6700\u7ec8\u7684\u8f93\u51fa\u7ed3\u679c":92,"\u6708\u6e56":90,"\u6709":90,"\u6709\u4e00\u4e2a\u57fa\u672c\u7684\u8ba4\u8bc6":109,"\u6709\u4e00\u4e9b\u5fc5\u987b\u914d\u7f6e\u7684\u53c2\u6570":[115,116,117],"\u6709\u4e9b\u5c42\u53ef\u80fd\u9700\u8981\u9ad8\u7cbe\u5ea6\u6765\u4fdd\u8bc1\u68af\u5ea6\u68c0\u67e5\u5355\u6d4b\u6b63\u786e\u6267\u884c":96,"\u6709\u4e9b\u5c42\u6216\u8005\u6fc0\u6d3b\u9700\u8981\u505a\u5f52\u4e00\u5316\u4ee5\u4fdd\u8bc1\u5b83\u4eec\u7684\u8f93\u51fa\u7684\u548c\u662f\u4e00\u4e2a\u5e38\u6570":96,"\u6709\u4e9b\u7279\u5f81\u7684\u53d6\u503c\u8fbe\u5230\u6570\u767e\u4e07":80,"\u6709\u4eba\u7528\u865a\u62df\u673a\u6765\u7c7b\u6bd4":94,"\u6709\u4ee5\u4e0b\u5efa\u8bae":[115,116],"\u6709\u5173":90,"\u6709\u5173\u53c2\u6570\u914d\u7f6e\u7684\u8be6\u7ec6\u8bf4\u660e\u89c1":115,"\u6709\u5173\u7ebf\u6027\u56de\u5f52\u7684\u5b9e\u9645\u5e94\u7528":87,"\u6709\u5173kubernetes\u76f8\u5173\u6982\u5ff5\u4ee5\u53ca\u5982\u4f55\u642d\u5efa\u548c\u914d\u7f6ekubernetes\u96c6\u7fa4":111,"\u6709\u52a9\u4e8e\u8bca\u65ad\u5206\u5e03\u5f0f\u9519\u8bef":104,"\u6709\u591a\u96be":94,"\u6709\u65f6\u5019\u6211\u4eec\u4f1a\u5e0c\u671b\u6e05\u7406\u6389\u5df2\u7ecf\u4e0b\u8f7d\u7684\u7b2c\u4e09\u65b9\u4f9d\u8d56\u4ee5\u53ca\u5df2\u7ecf\u7f16\u8bd1\u7684\u4e8c\u8fdb\u5236\u6587\u4ef6":94,"\u6709\u65f6\u5019\u6211\u4eec\u53ea\u60f3\u8fd0\u884c\u4e00\u4e2a\u7279\u5b9a\u7684\u5355\u5143\u6d4b\u8bd5":94,"\u6709\u6807\u51c6\u7684":54,"\u6709\u7684\u65f6\u5019":54,"\u6709\u7684\u65f6\u5019\u7b80\u7b80\u5355\u5355\u7684\u6539\u53d8\u5c31\u80fd\u5728\u6027\u80fd\u4e0a\u4ea7\u751f\u660e\u663e\u7684\u4f18\u5316\u6548\u679c":103,"\u6709\u7684\u8bdd\u9700\u8981\u5148\u5378\u8f7d":77,"\u6709\u975e\u5e38\u5927\u7684\u5dee\u522b":102,"\u670d\u52a1":90,"\u670d\u52a1\u5458":90,"\u670d\u52a1\u5668\u4e4b\u95f4\u53ef\u4ee5\u901a\u8fc7\u5c40\u57df\u7f51":104,"\u672a\u6307\u5b9a\u6309\u7167double\u7cbe\u5ea6\u7f16\u8bd1":82,"\u672a\u77e5\u8bcd":121,"\u672a\u8bbe\u7f6e":116,"\u672c\u4f8b\u4e2d\u4e3a0":121,"\u672c\u4f8b\u4e2d\u4e3a32":121,"\u672c\u4f8b\u4e2d\u4e3a4":121,"\u672c\u4f8b\u4e2d\u4f7f\u7528for\u5faa\u73af\u8fdb\u884c\u591a\u6b21\u8c03\u7528":2,"\u672c\u4f8b\u4e2d\u7684\u539f\u59cb\u6570\u636e\u4e00\u5171\u670910\u4e2a\u6837\u672c":90,"\u672c\u4f8b\u4e2d\u7684\u8f93\u5165\u7279\u5f81\u662f\u8bcdid\u7684\u5e8f\u5217":2,"\u672c\u4f8b\u6839\u636e\u7f51\u7edc\u914d\u7f6e\u4e2d":2,"\u672c\u4f8b\u6bcf\u884c\u4fdd\u5b58\u4e00\u6761\u6837\u672c":123,"\u672c\u4f8b\u7531\u6613\u5230\u96be\u5c55\u793a4\u79cd\u4e0d\u540c\u7684\u6587\u672c\u5206\u7c7b\u7f51\u7edc\u914d\u7f6e":123,"\u672c\u4f8b\u7684":2,"\u672c\u4f8b\u7684\u6240\u6709\u5b57\u7b26\u90fd\u5c06\u8f6c\u6362\u4e3a\u8fde\u7eed\u6574\u6570\u8868\u793a\u7684id\u4f20\u7ed9\u6a21\u578b":123,"\u672c\u4f8b\u91c7\u7528\u82f1\u6587\u60c5\u611f\u5206\u7c7b\u7684\u6570\u636e":2,"\u672c\u4f8b\u91c7\u7528adam\u4f18\u5316\u65b9\u6cd5":123,"\u672c\u5217\u8868\u8bf4\u660epaddlepaddle\u53d1\u7248\u4e4b\u524d\u9700\u8981\u6d4b\u8bd5\u7684\u529f\u80fd\u70b9":71,"\u672c\u5730":[77,86],"\u672c\u5730\u6d4b\u8bd5":105,"\u672c\u5730\u8bad\u7ec3":105,"\u672c\u5730\u8bad\u7ec3\u4e0e\u9884\u6d4b":79,"\u672c\u5730\u8bad\u7ec3\u7684\u5b9e\u9a8c":108,"\u672c\u5b9e\u4f8b\u4e2d":121,"\u672c\u5c0f\u8282\u6211\u4eec\u5c06\u4ecb\u7ecd\u6a21\u578b\u7f51\u7edc\u7ed3\u6784":123,"\u672c\u5c42\u5c3a\u5bf8":122,"\u672c\u5c42\u6709\u56db\u4e2a\u53c2\u6570":122,"\u672c\u6559\u7a0b\u4e2d\u6211\u4eec\u7ed9\u51fa\u4e86\u4e09\u4e2aresnet\u6a21\u578b":122,"\u672c\u6559\u7a0b\u4e3b\u8981\u4ecb\u7ecd\u5e26kernel\u7684op\u5982\u4f55\u5199":97,"\u672c\u6559\u7a0b\u5c06\u6307\u5bfc\u4f60\u5982\u4f55\u5728":93,"\u672c\u6559\u7a0b\u63d0\u4f9b\u4e86\u4e00\u4e2a\u7528\u4e8eimagenet\u4e0a\u7684\u5377\u79ef\u5206\u7c7b\u7f51\u7edc\u6a21\u578b":122,"\u672c\u6587\u4e2d\u6240\u6709\u7684\u4f8b\u5b50":90,"\u672c\u6587\u4e2d\u7684\u4f8b\u5b50\u91cc":94,"\u672c\u6587\u4e2d\u793a\u4f8b\u6240\u4f7f\u7528\u7684\u5355\u5143\u6d4b\u8bd5\u6587\u4ef6\u662f":90,"\u672c\u6587\u4ee5paddlepaddle\u7684\u53cc\u5c42rnn\u5355\u5143\u6d4b\u8bd5\u4e3a\u793a\u4f8b":90,"\u672c\u6587\u53ea\u4f7f\u7528\u4e86\u9ed8\u8ba4\u547d\u540d\u7a7a\u95f4":109,"\u672c\u6587\u5c06\u4ecb\u7ecd\u5728kubernetes\u5bb9\u5668\u7ba1\u7406\u5e73\u53f0\u4e0a\u5feb\u901f\u6784\u5efapaddlepaddle\u5bb9\u5668\u96c6\u7fa4":111,"\u672c\u6587\u5c06\u4ecb\u7ecd\u5982\u4f55\u4f7f\u7528paddlepaddle\u5728\u4e0d\u540c\u7684\u96c6\u7fa4\u6846\u67b6\u4e0b\u5b8c\u6210\u5206\u5e03\u5f0f\u8bad\u7ec3":104,"\u672c\u6587\u6863\u4ecb\u7ecd\u5982\u4f55\u5728paddlepaddle\u5e73\u53f0\u4e0a":121,"\u672c\u6587\u6863\u5185\u4e0d\u91cd\u590d\u4ecb\u7ecd":109,"\u672c\u6587\u6863\u5c06\u4ee5linux":115,"\u672c\u6587\u6863\u63cf\u8ff0paddl":55,"\u672c\u6587\u7684\u5c06\u4ecb\u7ecd\u5728macos\u4e0a":116,"\u672c\u6765":90,"\u672c\u6b21\u8bad\u7ec3\u6587\u4ef6\u6240\u5728\u76ee\u5f55":111,"\u672c\u6b21\u8bad\u7ec3\u7684yaml\u6587\u4ef6\u53ef\u4ee5\u5199\u6210":111,"\u672c\u6b21\u8bad\u7ec3\u8981\u6c42\u67093\u4e2apaddlepaddle\u8282\u70b9":111,"\u672c\u6b21\u8bd5\u9a8c":123,"\u672c\u793a\u4f8b\u4e2d\u4f7f\u7528\u7684\u539f\u59cb\u6570\u636e\u5982\u4e0b":90,"\u672c\u793a\u4f8b\u610f\u56fe\u4f7f\u7528\u5355\u5c42rnn\u548c\u53cc\u5c42rnn\u5b9e\u73b0\u4e24\u4e2a\u5b8c\u5168\u7b49\u4ef7\u7684\u5168\u8fde\u63a5rnn":90,"\u673a\u5668\u4e0a\u4ee5\u53ca":117,"\u673a\u5668\u7684\u8bbe\u5907":108,"\u673a\u5668\u7ffb\u8bd1":71,"\u6743\u91cd\u66f4\u65b0\u7684\u68af\u5ea6":106,"\u6761\u4ef6\u4e0b":109,"\u6765":90,"\u6765\u4ee3\u66ff":95,"\u6765\u4f7f\u7528dropout":81,"\u6765\u4f7f\u7528dropout\u7684":81,"\u6765\u4fdd\u8bc1\u8bad\u7ec3\u8fc7\u7a0b\u53ef\u4ee5\u4ece\u4e2d\u95f4\u72b6\u6001\u91cd\u65b0\u542f\u52a8":34,"\u6765\u505a\u68af\u5ea6\u68c0\u67e5":96,"\u6765\u5206\u6790\u6267\u884c\u6587\u4ef6":103,"\u6765\u521d\u59cb\u5316\u53c2\u6570":82,"\u6765\u542f\u52a8\u548c":94,"\u6765\u5b8c\u6210\u7f51\u7edc\u7684\u8bad\u7ec3":87,"\u6765\u5b9a\u4e49\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":93,"\u6765\u5bf9\u6bd4\u5206\u6790\u4e24\u8005\u8bed\u4e49\u76f8\u540c\u7684\u539f\u56e0":90,"\u6765\u5e2e\u52a9\u4f60\u7406\u89e3paddlepaddle\u7684\u5185\u90e8\u8fd0\u884c\u673a\u5236":123,"\u6765\u5f71\u54cdpaddlepaddle\u7684\u7f16\u8bd1\u8fc7\u7a0b":[115,116],"\u6765\u5f97\u5230\u67d0\u4e2a\u7279\u5b9a\u53c2\u6570\u7684\u68af\u5ea6\u77e9\u9635":96,"\u6765\u6267\u884c":94,"\u6765\u6307\u5b9a\u7f51\u7edc\u5c42\u7684\u6570\u76ee":122,"\u6765\u63a5\u53d7\u4e0d\u4f7f\u7528\u7684\u51fd\u6570\u4ee5\u4fdd\u8bc1\u517c\u5bb9\u6027":2,"\u6765\u63cf\u8ff0\u7684":98,"\u6765\u63cf\u8ff0\u8be5op\u7684\u8f93\u5165":97,"\u6765\u642d\u5efa\u795e\u7ecf\u7f51\u7edc":87,"\u6765\u663e\u793a\u6027\u80fd\u5206\u6790\u7ed3\u679c":102,"\u6765\u67e5\u770b\u6027\u80fd\u5206\u6790\u7ed3\u679c":102,"\u6765\u6ce8\u518c\u8be5\u5c42":96,"\u6765\u6df7\u5408\u4f7f\u7528gpu\u548ccpu\u8ba1\u7b97\u7f51\u7edc\u5c42\u7684\u53c2\u6570":108,"\u6765\u6e05\u7406\u8fd9\u4e9b\u5185\u5bb9":94,"\u6765\u786e\u4fdd\u628a":54,"\u6765\u786e\u5b9a\u5bf9\u5e94\u5173\u7cfb":2,"\u6765\u7f16\u8bd1":94,"\u6765\u81ea\u5b9a\u4e49\u4f20\u6570\u636e\u7684\u8fc7\u7a0b":1,"\u6765\u83b7\u5f97\u8f93\u51fa\u7684\u68af\u5ea6":96,"\u6765\u8868\u793a":93,"\u6765\u8868\u793a\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":93,"\u6765\u8868\u793apaddle\u5185\u90e8\u7c7b":54,"\u6765\u89e3\u51b3\u4e0a\u9762\u7684\u95ee\u9898":80,"\u6765\u8ba1\u7b97\u68af\u5ea6":96,"\u6765\u8bb2\u89e3\u5982\u4f55\u4f7f\u7528\u53cc\u5c42rnn":90,"\u6765\u8bbe\u7f6e":82,"\u6765\u8bbf\u95ee\u7528\u6237\u81ea\u5df1\u7684\u6570\u636e":35,"\u6765\u8bf4\u660epydataprovider2\u7684\u7b80\u5355\u4f7f\u7528\u573a\u666f":2,"\u6765\u8c03\u6574":95,"\u6765\u8c03\u7528":94,"\u6765\u8fd0\u884c\u5305\u62ec":94,"\u6765\u8fd0\u884c\u5355\u5143\u6d4b\u8bd5\u4e86":94,"\u6765\u8fd0\u884c\u6027\u80fd\u5206\u6790\u548c\u8c03\u4f18":103,"\u6765\u8fd0\u884c\u955c\u50cf":84,"\u6765\u8fdb\u884c\u8ba8\u8bba":55,"\u6765\u9884\u6d4b\u8fd9\u4e2a\u4e2d\u95f4\u7684\u8bcd":80,"\u676f\u5b50":90,"\u6784\u5efa":115,"\u6784\u5efa\u597d\u5f00\u53d1\u955c\u50cf\u540e":115,"\u6784\u5efa\u76ee\u6807\u4e3a":116,"\u6784\u6210\u4e86\u8f93\u51fa\u53cc\u5c42\u5e8f\u5217\u7684\u7b2ci\u4e2a":89,"\u6784\u9020":111,"\u6784\u9020\u51fd\u6570\u542b\u67092\u4e2a\u53c2\u6570":97,"\u6784\u9020\u51fd\u6570\u91cc\u901a\u8fc7":97,"\u6784\u9020paddl":4,"\u67b6\u6784\u7684\u6a21\u62df\u5668\u5e73\u53f0":116,"\u67b6\u6784\u7684iphone\u6216\u8005ipad\u7b49\u7269\u7406\u8bbe\u5907":116,"\u67d0\u4e00\u4e2a\u795e\u7ecf\u5143\u7684\u4e00\u4e2a\u8f93\u5165\u4e3a\u4e0a\u4e00\u4e2a\u65f6\u95f4\u6b65\u7f51\u7edc\u4e2d\u67d0\u4e00\u4e2a\u795e\u7ecf\u5143\u7684\u8f93\u51fa":90,"\u67d0\u4e9b\u53c2\u6570\u53ea\u53ef\u7528\u4e8e\u7279\u5b9a\u7684\u5c42\u4e2d":105,"\u67e5\u770b":[95,123],"\u67e5\u770b\u5f53\u524d\u72b6\u6001":95,"\u67e5\u770b\u5f53\u524d\u8fdc\u7a0b\u4ed3\u5e93\u7684\u540d\u5b57":95,"\u67e5\u770b\u6587\u4ef6\u5177\u4f53\u88ab\u4fee\u6539\u7684\u5185\u5bb9":95,"\u67e5\u770b\u662f\u5426\u662f\u5176\u4ed6\u9519\u8bef\u5f15\u53d1\u7684\u62a5\u9519":78,"\u67e5\u770bjob\u7684\u8be6\u7ec6\u60c5\u51b5":110,"\u6807\u51c6":86,"\u6807\u51c6\u5dee\u4e3a":82,"\u6807\u51c6\u8868\u793apaddlepaddle\u7248\u672c\u53f7":71,"\u6807\u8bc6\u4e86\u4e00\u4e2a\u8f93\u51fa\u7684\u6587\u4ef6\u540d":102,"\u6807\u8bc6\u6027\u80fd\u5206\u6790\u7684\u7ed3\u679c\u6587\u4ef6":102,"\u6807\u8bc6\u662f\u5426\u4e3a\u8fde\u7eed\u7684batch\u8ba1\u7b97":106,"\u6807\u8bc6\u88ab\u6027\u80fd\u5206\u6790\u7684\u6e90\u6587\u4ef6":102,"\u6807\u8bc6http\u670d\u52a1\u7684\u7aef\u53e3":102,"\u6807\u8bc6http\u670d\u52a1\u7ed1\u5b9a\u7684ip":102,"\u6838\u4e00\u6837\u591a\u7684\u8fdb\u7a0b\u6765\u5e76\u884c\u7f16\u8bd1":94,"\u6839\u636e\u4e2a\u4eba\u7684\u9700\u6c42\u4fee\u6539\u5b9a\u5236docker\u5bb9\u5668\u6240\u6267\u884c\u7684\u811a\u672c":115,"\u6839\u636e\u4f60\u7684\u4efb\u52a1":108,"\u6839\u636e\u524d\u6587\u7684\u63cf\u8ff0":111,"\u6839\u636e\u7528\u6237\u6307\u5b9a\u7684\u5b57\u5178":121,"\u6839\u636e\u7f51\u7edc\u914d\u7f6e\u4e2d\u7684":106,"\u6839\u636e\u8f93\u5165tensor\u7684\u5927\u5c0f\u6765\u8bbe\u7f6e\u8f93\u51fatensor\u7684\u5927\u5c0f":98,"\u6839\u636e\u8fd9\u4e9b\u53c2\u6570\u7684\u4f7f\u7528\u573a\u5408":105,"\u6839\u636e\u9ed8\u8ba4\u503c\u9012\u589e":106,"\u6839\u636e\u9ed8\u8ba4\u7aef\u53e3\u53f7\u9012\u589e":106,"\u6839\u636ejob\u5bf9\u5e94\u7684pod\u4fe1\u606f":110,"\u6839\u636eport":104,"\u683c\u5f0f":106,"\u683c\u5f0f\u5982\u4e0b":123,"\u683c\u5f0f\u7684\u6587\u4ef6\u6765\u5b58\u653e":97,"\u683c\u5f0f\u8bf4\u660e":121,"\u6846\u67b6\u63d0\u4f9b\u7684blas\u51fd\u6570\u8fdb\u884c\u77e9\u9635\u8ba1\u7b97":116,"\u6846\u67b6\u8fdb\u884cblas\u77e9\u9635\u8ba1\u7b97":116,"\u68af\u5ea6\u4f1a\u5c31\u5730":96,"\u68af\u5ea6\u4f1a\u6709\u566a\u58f0":104,"\u68af\u5ea6\u53c2\u6570\u7684\u5206\u5757\u6570\u76ee":106,"\u68af\u5ea6\u5c31\u53ef\u4ee5\u901a\u8fc7\u8fd9\u4e2a\u65b9\u7a0b\u8ba1\u7b97\u5f97\u5230":96,"\u68af\u5ea6\u670d\u52a1\u5668\u7684\u6570\u91cf":106,"\u68af\u5ea6\u68c0\u67e5\u5355\u5143\u6d4b\u8bd5\u901a\u8fc7\u6709\u9650\u5dee\u5206\u6cd5\u6765\u9a8c\u8bc1\u4e00\u4e2a\u5c42\u7684\u68af\u5ea6":96,"\u68af\u5ea6\u68c0\u67e5\u7684\u8f93\u5165\u6570\u636e\u7684\u6279\u6b21\u5927\u5c0f":96,"\u68c0\u67e5\u70b9\u4fdd\u5b58\u7a0b\u5e8f\u6d41\u7a0b":34,"\u68c0\u67e5\u8f93\u5165\u6570\u636e\u7ef4\u5ea6":97,"\u68d2":123,"\u697c\u5c42":90,"\u6a21\u5757\u4e0b\u7684\u76f8\u5173":98,"\u6a21\u5757\u4e2d\u7684":2,"\u6a21\u578b\u4e00\u76f4\u4e0d\u6536\u655b":80,"\u6a21\u578b\u5171\u5305\u542b1":121,"\u6a21\u578b\u53c2\u6570\u68c0\u67e5\u70b9\u901a\u8fc7\u5b9a\u671f\u5411\u78c1\u76d8\u4e0a\u4fdd\u5b58\u4e00\u4efd\u5b58\u50a8\u5728paramet":34,"\u6a21\u578b\u5b58\u50a8\u8def\u5f84":123,"\u6a21\u578b\u6570\u636e\u68c0\u67e5\u70b9\u7684\u5b9e\u73b0":34,"\u6a21\u578b\u6587\u4ef6\u5c06\u88ab\u5199\u5165\u8282\u70b9":104,"\u6a21\u578b\u6765\u6307\u5bfc\u4f60\u5b8c\u6210\u8fd9\u4e9b\u6b65\u9aa4":93,"\u6a21\u578b\u6f14\u793a\u5982\u4f55\u914d\u7f6e\u590d\u6742\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u6a21\u578b":93,"\u6a21\u578b\u7684\u4ee3\u7801\u53ef\u4ee5\u5728":93,"\u6a21\u578b\u7684\u7ed3\u6784\u548c\u8bad\u7ec3\u8fc7\u7a0b":121,"\u6a21\u578b\u7684\u7f16\u7801\u5668\u90e8\u5206\u5982\u4e0b\u6240\u793a":93,"\u6a21\u578b\u8bad\u7ec3\u4f1a\u770b\u5230\u7c7b\u4f3c\u4e0a\u9762\u8fd9\u6837\u7684\u65e5\u5fd7\u4fe1\u606f":123,"\u6a21\u578b\u8bad\u7ec3\u7b49\u4efb\u52a1":87,"\u6a21\u578b\u8def\u5f84":122,"\u6a21\u578b\u914d\u7f6e":[0,79],"\u6a21\u578b\u914d\u7f6e\u89e3\u6790":54,"\u6a21\u578b\u91c7\u7528":121,"\u6a21\u578b\u9884\u6d4b":4,"\u6a21\u5f0f\u4e0b\u7684\u6027\u80fd\u6d4b\u8bd5\u662f\u6ca1\u6709\u610f\u4e49\u7684":102,"\u6a2a\u5411\u62fc\u63a5":80,"\u6b21":90,"\u6b22\u8fce\u901a\u8fc7":95,"\u6b63\u6837\u672c":123,"\u6b63\u786e\u7684\u89e3\u51b3\u65b9\u6cd5\u662f":77,"\u6b63\u8d1f\u5bf9\u9a8c\u8bc1":105,"\u6b64\u547d\u4ee4\u5c06\u5728":115,"\u6b64\u5904":121,"\u6b64\u5904\u90fd\u4e3a2":90,"\u6b64\u5916":[81,94,95,115],"\u6b64\u6559\u7a0b\u4f1a\u4ecb\u7ecd\u5982\u4f55\u4f7f\u7528python\u7684cprofile\u5305":102,"\u6b64\u6559\u7a0b\u5c06\u5411\u60a8\u5206\u6b65\u4ecb\u7ecd\u5982\u4f55\u4f7f\u7528\u5185\u7f6e\u7684\u5b9a\u65f6\u5de5\u5177":103,"\u6b64\u65f6\u53ea\u9700\u8981":94,"\u6b64\u65f6\u53ef\u4ee5\u5728\u8c03\u7528infer\u63a5\u53e3\u65f6\u901a\u8fc7\u8bbe\u7f6e":80,"\u6b64\u65f6\u53ef\u4ee5\u8df3\u8fc7paddlepaddle\u6a21\u578b\u53c2\u6570\u6587\u4ef6\u7684\u5934\u4fe1\u606f":82,"\u6b64\u65f6master\u5c06\u8d1f\u8d23\u542f\u52a8\u4e00\u4e2a\u65b0\u7684train":34,"\u6b64\u7c7b\u62a5\u9519\u901a\u5e38\u662f\u7531\u4e8e\u67d0\u4e00\u4e2a\u8282\u70b9\u7684\u9519\u8bef\u5bfc\u81f4\u8fd9\u4e2a\u8282\u70b9\u7684\u8bad\u7ec3\u8fdb\u7a0b\u9000\u51fa":78,"\u6b64\u90e8\u5206\u7684\u4f7f\u7528\u65b9\u6cd5\u53ef\u4ee5\u53c2\u8003":104,"\u6b65\u9aa4":80,"\u6bb5\u843d\u53ef\u4ee5\u770b\u4f5c\u662f\u4e00\u4e2a\u5d4c\u5957\u7684\u53cc\u5c42\u7684\u5e8f\u5217":92,"\u6bcf\u4e00\u4e2a":71,"\u6bcf\u4e00\u4e2a\u4efb\u52a1\u6d41\u7a0b\u90fd\u53ef\u4ee5\u88ab\u5212\u5206\u4e3a\u5982\u4e0b\u4e94\u4e2a\u6b65\u9aa4":123,"\u6bcf\u4e00\u4e2a\u6587\u4ef6\u662f\u6570\u636e\u96c6\u7684\u4e00\u4e2ashard":35,"\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65":90,"\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65\u4e4b\u95f4\u7684\u795e\u7ecf\u7f51\u7edc\u5177\u6709\u4e00\u5b9a\u7684\u76f8\u5173\u6027":90,"\u6bcf\u4e00\u4e2a\u8282\u70b9\u90fd\u6709\u76f8\u540c\u7684\u65e5\u5fd7\u7ed3\u6784":104,"\u6bcf\u4e00\u4e2alayer\u8f93\u51fa\u77e9\u9635\u7684\u9ad8\u5ea6":80,"\u6bcf\u4e00\u5217\u7684\u542b\u4e49\u662f":102,"\u6bcf\u4e00\u7ec4\u5185\u7684\u6240\u6709\u53e5\u5b50\u548clabel":90,"\u6bcf\u4e2a\u503c\u7684\u7c7b\u578b\u53ef\u4ee5\u662f\u6574\u5f62":35,"\u6bcf\u4e2a\u5143\u7d20\u662f\u4e00\u4e2a0\u5c42\u5e8f\u5217":89,"\u6bcf\u4e2a\u5143\u7d20\u662f\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":89,"\u6bcf\u4e2a\u5355\u5c42rnn":92,"\u6bcf\u4e2a\u53c2\u6570\u670d\u52a1\u5668\u53ea\u4fdd\u5b58\u6574\u4e2a\u795e\u7ecf\u7f51\u7edc\u6240\u6709\u53c2\u6570\u7684\u4e00\u90e8\u5206":104,"\u6bcf\u4e2a\u53e5\u5b50\u53c8\u662f\u5355\u8bcd\u7684\u6570\u7ec4":90,"\u6bcf\u4e2a\u53e5\u5b50\u90fd\u4ee5\u5f00\u59cb\u6807\u8bb0\u5f00\u5934":93,"\u6bcf\u4e2a\u53e5\u5b50\u90fd\u4ee5\u7ed3\u675f\u6807\u8bb0\u7ed3\u5c3e":93,"\u6bcf\u4e2a\u5b50\u5e8f\u5217\u957f\u5ea6\u53ef\u4ee5\u4e0d\u4e00\u81f4":90,"\u6bcf\u4e2a\u5c42\u5728\u5176":96,"\u6bcf\u4e2a\u5c42\u90fd\u6709\u4e00\u4e2a\u6216\u591a\u4e2ainput":123,"\u6bcf\u4e2a\u6279\u6b21\u6570\u636e":106,"\u6bcf\u4e2a\u65f6\u95f4\u6b65\u4e4b\u5185\u7684\u8fd0\u7b97\u662f\u72ec\u7acb\u7684":92,"\u6bcf\u4e2a\u65f6\u95f4\u6b65\u90fd\u7528\u4e86\u4e0a\u4e00\u4e2a\u65f6\u95f4\u6b65\u7684\u8f93\u51fa\u7ed3\u679c":90,"\u6bcf\u4e2a\u6743\u91cd\u5bf9\u5e94\u4e00\u4e2a\u8f93\u5165":96,"\u6bcf\u4e2a\u6837\u672c\u7531\u4e24\u90e8\u5206\u7ec4\u6210":90,"\u6bcf\u4e2a\u6837\u672c\u95f4\u7528\u7a7a\u884c\u5206\u5f00":90,"\u6bcf\u4e2a\u72b6\u6001":92,"\u6bcf\u4e2a\u7ebf\u7a0b":106,"\u6bcf\u4e2a\u7ebf\u7a0b\u5206\u914d\u5230128\u4e2a\u6837\u672c\u7528\u4e8e\u8bad\u7ec3":106,"\u6bcf\u4e2a\u8bad\u7ec3\u8282\u70b9\u5fc5\u987b\u6307\u5b9a\u4e00\u4e2a\u552f\u4e00\u7684id\u53f7":106,"\u6bcf\u4e2a\u8f93\u5165\u90fd\u662f\u4e00\u4e2a":96,"\u6bcf\u4e2a\u8f93\u51fa\u8282\u70b9\u90fd\u8fde\u63a5\u5230\u6240\u6709\u7684\u8f93\u5165\u8282\u70b9\u4e0a":96,"\u6bcf\u4e2a\u90e8\u5206\u5206\u522b\u7ed9\u6bcf\u4e2atrainer\u4f7f\u7528":104,"\u6bcf\u4e2acommit\u53ea\u505a\u4e86\u5c11\u91cf\u7684\u4fee\u6539":95,"\u6bcf\u4e2adata":35,"\u6bcf\u4e2aparamet":34,"\u6bcf\u4e2apass\u7684\u7b2c0\u4e2abatch\u5230\u5f53\u524dbatch\u6240\u6709\u6837\u672c\u7684\u5e73\u5747\u5206\u7c7b\u9519\u8bef\u7387":123,"\u6bcf\u4e2apass\u7684\u7b2c0\u4e2abatch\u5230\u5f53\u524dbatch\u6240\u6709\u6837\u672c\u7684\u5e73\u5747cost":123,"\u6bcf\u4e2apod\u5305\u542b\u4e00\u4e2apaddlepaddle\u5bb9\u5668":111,"\u6bcf\u4e2ashard\u5206\u522b\u5b58\u50a8\u5728\u5176\u4e2d\u4e00\u53f0paramet":34,"\u6bcf\u4e2atrainer\u542f\u52a8\u540e\u8bfb\u53d6\u5207\u5206\u597d\u7684\u4e00\u90e8\u5206\u6570\u636e":104,"\u6bcf\u4e2atrainer\u7684\u552f\u4e00id":104,"\u6bcf\u4e2atrainer\u8fdb\u7a0b\u9700\u8981\u80fd\u591f\u8bfb\u53d6\u5c5e\u4e8e\u81ea\u5df1\u7684\u4e00\u4efd\u6570\u636e":104,"\u6bcf\u53f0\u670d\u52a1\u5668\u5177\u6709\u96c6\u7fa4\u4e2d\u552f\u4e00\u7684ip\u5730\u5740":104,"\u6bcf\u5c42\u4e0a\u53ea\u80fd\u4fdd\u5b58\u56fa\u5b9a\u6570\u76ee\u4e2a\u6700\u597d\u7684\u72b6\u6001":106,"\u6bcf\u5c42\u4f7f\u7528\u7684gpu\u53f7\u4f9d\u8d56\u4e8e\u53c2\u6570train":108,"\u6bcf\u6279\u6b21":106,"\u6bcf\u6b21\u63d0\u4ea4\u4ee3\u7801":95,"\u6bcf\u6b21\u63d0\u4ea4\u65f6":95,"\u6bcf\u6b21\u8bfb\u53d6\u4e00\u6761\u6570\u636e\u540e":123,"\u6bcf\u6b21\u8c03\u7528\u7684\u8017\u65f6\u4e5f\u5f88\u957f":102,"\u6bcf\u6b21\u8f93\u51fa\u4e00\u4e2adata":35,"\u6bcf\u6b21\u90fd\u4f1a\u4ecepython\u7aef\u8bfb\u53d6\u6570\u636e":2,"\u6bcf\u884c\u5b58\u50a8\u4e00\u4e2a\u8bcd":121,"\u6bcf\u884c\u5b58\u50a8\u7684\u662f\u4e00\u4e2a\u6837\u672c\u7684\u7279\u5f81":122,"\u6bcf\u884c\u6253\u537032\u4e2a\u53c2\u6570\u4ee5":121,"\u6bcf\u884c\u8868\u793a\u4e00\u4e2a\u6279\u6b21\u4e2d\u7684\u5355\u4e2a\u8f93\u5165":96,"\u6bcf\u8f6e\u4f1a\u5c06\u6570\u636e\u96c6\u4e2d\u7684\u6240\u6709\u8bad\u7ec3\u6837\u672c\u4f7f\u7528\u4e00\u6b21":106,"\u6bcf\u8f6e\u7ed3\u675f\u65f6\u5bf9\u6240\u6709\u6d4b\u8bd5\u6570\u636e\u8fdb\u884c\u6d4b\u8bd5":106,"\u6bcf\u8f6e\u90fd\u4f1a\u4fdd\u5b58\u9884\u6d4b\u7ed3\u679c":106,"\u6bcf\u8fd0\u884c\u591a\u5c11\u4e2a\u6279\u6b21\u6267\u884c\u4e00\u6b21\u7a00\u758f\u53c2\u6570\u5206\u5e03\u7684\u68c0\u67e5":106,"\u6bcf\u969410\u5206\u949f":34,"\u6bcf\u9694\u591a\u5c11batch\u6253\u5370\u4e00\u6b21\u65e5\u5fd7":123,"\u6bcfdot":106,"\u6bcflog":106,"\u6bcfsave":106,"\u6bcftest":106,"\u6bd4\u5982":[35,78,80,84,94,95,123],"\u6bd4\u5982\u4e00\u53e5\u8bdd\u4e2d\u7684\u6bcf\u4e00\u4e2a\u5355\u8bcd":90,"\u6bd4\u5982\u5728":84,"\u6bd4\u5982\u5982\u679c\u8981build\u4e00\u4e2a\u4e0d\u4f9d\u8d56gpu":95,"\u6bd4\u5982\u5c06":71,"\u6bd4\u5982\u5e0c\u671b\u6700\u5c0f\u5316\u751f\u6210\u5e93\u7684\u5927\u5c0f":116,"\u6bd4\u5982\u5e0c\u671b\u6700\u5c0f\u5316\u751f\u6210\u7684\u5e93\u7684\u5927\u5c0f":[115,117],"\u6bd4\u5982\u6bcf\u969410\u5206\u949f\u6700\u65b0\u7684\u5feb\u7167":34,"\u6bd4\u5982\u6d41\u5f0f\u6570\u636e\u5904\u7406":35,"\u6bd4\u5982\u8282\u70b9\u7684id":104,"\u6bd4\u5982\u8bbe\u7f6e\u4e00\u4e2a\u5168\u8fde\u63a5\u5c42\u7684\u53c2\u6570\u521d\u59cb\u5316\u65b9\u5f0f\u548cbias\u521d\u59cb\u5316\u65b9\u5f0f":82,"\u6bd4\u5982\u901a\u8fc78080\u7aef\u53e3":109,"\u6bd4\u5982cento":86,"\u6bd4\u5982fpe":78,"\u6bd4\u5982ide\u914d\u7f6e\u91cc":95,"\u6bd4\u5982imagenet\u8fd9\u4e2a\u6570\u636e\u96c6\u53ef\u80fd\u88ab\u5206\u62101000\u4e2ashard":35,"\u6bd4\u5982pil\u5e93\u7b49":104,"\u6bd5\u7adf\u5355\u7ebf\u7a0b\u8c03\u8bd5\u66f4\u5bb9\u6613":102,"\u6c34\u6e29":90,"\u6c49\u5ead":90,"\u6ca1":90,"\u6ca1\u6709\u4f5c\u7528":2,"\u6ca1\u6709\u5b9e\u9645\u610f\u4e49":121,"\u6ca1\u6709\u627e\u5230\u548c\u5f53\u524d\u7cfb\u7edf\u5339\u914d\u7684paddlepaddle\u5b89\u88c5\u5305":[77,86],"\u6ca1\u6709\u6d4b\u8bd5\u6570\u636e":2,"\u6ca1\u6709\u8bbe\u7f6e":[115,117],"\u6ce8":[34,84],"\u6ce8\u518c":97,"\u6ce8\u518ccpu":97,"\u6ce8\u518cop":97,"\u6ce8\u518cop\u65f6\u7684\u7c7b\u578b\u540d":97,"\u6ce8\u610f":[2,83,87,93,96,99,102,104,111,115,116,117],"\u6ce8\u610f\u4e0a\u8ff0\u547d\u4ee4\u4e2d":111,"\u6ce8\u610f\u5230\u6211\u4eec\u5df2\u7ecf\u5047\u8bbe\u673a\u5668\u4e0a\u67094\u4e2agpu":108,"\u6ce8\u610f\u9884\u6d4b\u6570\u636e\u901a\u5e38\u4e0d\u5305\u542blabel":4,"\u6ce8\u610fnode":111,"\u6ce8\u91ca":97,"\u6cf3\u6c60":90,"\u6d41":90,"\u6d41\u7a0b\u6765\u63d0\u4ea4\u4ee3\u7801":95,"\u6d44":90,"\u6d4b\u8bd5":95,"\u6d4b\u8bd5\u524d\u5411oper":97,"\u6d4b\u8bd5\u65f6\u6307\u5b9a\u7684\u5b58\u50a8\u6a21\u578b\u5217\u8868\u7684\u6587\u4ef6":106,"\u6d4b\u8bd5\u65f6\u9ed8\u8ba4\u4e0dshuffl":2,"\u6d4b\u8bd5\u662f":95,"\u6d4b\u8bd5\u7684\u6a21\u578b\u5305\u62ec\u4ece\u7b2cm\u8f6e\u5230\u7b2cn":108,"\u6d4b\u8bd5docker\u955c\u50cf":71,"\u6d4b\u8bd5model_list":105,"\u6d4b\u8bd5save_dir":105,"\u6d6e\u70b9\u578b\u6570\u636e":35,"\u6d6e\u70b9\u5f02\u5e38\u901a\u5e38\u7684\u539f\u56e0\u662f\u6d6e\u70b9\u6570\u6ea2\u51fa":80,"\u6d6e\u70b9\u6570\u5360\u7528\u7684\u5b57\u8282\u6570":121,"\u6d6e\u70b9\u7a00\u758f\u6570\u636e":96,"\u6df7\u5408\u4ee3\u7801\u7684\u6027\u80fd\u5206\u6790\u6765\u8fdb\u884c\u8c03\u4f18":102,"\u6df7\u5408\u4ee3\u7801\u7684\u6027\u80fd\u74f6\u9888\u4e5f\u662f\u8981\u770b":102,"\u6df7\u5408\u5f53\u524d\u8bcd\u5411\u91cf\u548cattention\u52a0\u6743\u7f16\u7801\u5411\u91cf":93,"\u6dfb\u52a0\u4e86\u4e00\u4e2a\u8f93\u51fa":97,"\u6dfb\u52a0\u542f\u52a8\u811a\u672c":111,"\u6dfb\u52a0\u8f93\u5165\u53c2\u6570":97,"\u6dfb\u52a0\u8f93\u51fa\u53c2\u6570":97,"\u6dfb\u52a0op\u7684\u6ce8\u91ca":97,"\u6e05\u7406":94,"\u6e05\u7406\u6389\u8001\u65e7\u7684paddlepaddle\u5b89\u88c5\u5305":77,"\u6e29\u99a8":90,"\u6e90\u4ee3\u7801":123,"\u6e90\u4ee3\u7801\u683c\u5f0f":95,"\u6e90\u5e8f\u5217":93,"\u6e90\u7801\u4e0edemo":110,"\u6e90\u7801\u6811\u6839\u76ee\u5f55":94,"\u6e90\u8bed\u8a00\u548c\u76ee\u6807\u8bed\u8a00\u5171\u4eab\u76f8\u540c\u7684\u7f16\u7801\u5b57\u5178":121,"\u6e90\u8bed\u8a00\u548c\u76ee\u6807\u8bed\u8a00\u90fd\u662f\u76f8\u540c\u7684\u8bed\u8a00":121,"\u6e90\u8bed\u8a00\u77ed\u8bed\u548c\u76ee\u6807\u8bed\u8a00\u77ed\u8bed\u7684\u5b57\u5178\u5c06\u88ab\u5408\u5e76":121,"\u6ee4\u6ce2\u5668\u6838\u5728\u5782\u76f4\u65b9\u5411\u4e0a\u7684\u5c3a\u5bf8":122,"\u6ee4\u6ce2\u5668\u6838\u5728\u6c34\u5e73\u65b9\u5411\u4e0a\u7684\u5c3a\u5bf8":122,"\u6fc0\u6d3b":96,"\u6fc0\u6d3b\u51fd\u6570\u7c7b\u578b":123,"\u6fc0\u6d3b\u65b9\u7a0b":96,"\u6fc0\u6d3b\u7684\u7c7b\u578b":96,"\u70b9\u51fb":86,"\u70b9\u51fb\u8fd9\u91cc":99,"\u70ed\u60c5":90,"\u7136\u540e":[103,104,121],"\u7136\u540e\u4e0b\u8f7d\u4f18\u5316\u66f4\u65b0\u540e\u7684\u795e\u7ecf\u7f51\u7edc\u53c2\u6570":104,"\u7136\u540e\u4ea4\u7ed9step\u51fd\u6570":92,"\u7136\u540e\u4f7f\u7528":116,"\u7136\u540e\u4f7f\u7528resize\u63a5\u53e3\u8bbe\u7f6etensor\u7684\u5927\u5c0f":98,"\u7136\u540e\u5355\u51fb":95,"\u7136\u540e\u53ef\u4ee5\u4ecehead\u8282\u70b9ssh\u65e0\u5bc6\u7801\u767b\u5f55\u5230openmpi\u7684\u6bcf\u4e2a\u8282\u70b9\u4e0a":104,"\u7136\u540e\u53ef\u4ee5\u4f7f\u7528\u547d\u4ee4\u884c\u5de5\u5177\u521b\u5efajob":111,"\u7136\u540e\u53ef\u4ee5\u8f6c\u6362\u4e3a\u56fe\u7247":122,"\u7136\u540e\u5728\u4e0b\u4e00\u4e2a\u65f6\u95f4\u6b65\u8f93\u5165\u7ed9\u53e6\u4e00\u4e2a\u795e\u7ecf\u5143":90,"\u7136\u540e\u5728\u6d4f\u89c8\u5668\u4e2d\u8f93\u5165\u4ee5\u4e0b\u7f51\u5740":84,"\u7136\u540e\u5728dataprovider\u91cc\u9762\u6839\u636e\u8be5\u5730\u5740\u52a0\u8f7d\u5b57\u5178":82,"\u7136\u540e\u5728etcd\u7684":34,"\u7136\u540e\u5b89\u88c5paddle\u7684python\u73af\u5883":77,"\u7136\u540e\u5b9a\u4e49":93,"\u7136\u540e\u5c06\u6784\u5efa\u6210\u529f\u7684\u955c\u50cf\u4e0a\u4f20\u5230\u955c\u50cf\u4ed3\u5e93":111,"\u7136\u540e\u5c06\u8fd9\u4e9blayer\u7684\u53c2\u6570":81,"\u7136\u540e\u5c31\u53ef\u4ee5\u5e76\u53d1\u5199\u5165\u591a\u4e2achunk":44,"\u7136\u540e\u6240\u6709\u7528":95,"\u7136\u540e\u624d\u80fd\u4f7f\u7528pfsclient":44,"\u7136\u540e\u6267\u884c\u4e0b\u9762\u7684\u547d\u4ee4":122,"\u7136\u540e\u628a\u8fd9\u4e2a\u5305\u542b\u4e86\u8bad\u7ec3\u6570\u636e\u7684container\u4fdd\u5b58\u4e3a\u4e00\u4e2a\u65b0\u7684\u955c\u50cf":110,"\u7136\u540e\u63d0\u4ea4\u65b0\u6dfb\u52a0\u7684":95,"\u7136\u540e\u70b9\u51fb":95,"\u7136\u540e\u7533\u660e\u4e00\u4e2a\u5b58\u50a8\u5377":111,"\u7136\u540e\u89c2\u5bdf\u5230\u8f93\u51fa\u7684\u53d8\u5316\u4e3a":96,"\u7136\u540e\u8fd4\u56de\u7ed9paddlepaddle\u8fdb\u7a0b":2,"\u7136\u540e\u901a\u8fc7\u51fd\u6570":111,"\u7136\u540e\u901a\u8fc7\u81ea\u8eab\u7684ip\u5730\u5740\u5728":111,"\u7136\u540e\u91cd\u65b0cmake\u5373\u53ef":77,"\u7136\u800c":[93,106],"\u7248\u672c":[83,86,94,116],"\u7248\u672c\u5206\u652f":71,"\u7248\u672c\u53f7":71,"\u7248\u672c\u53f7rc":71,"\u7248\u672c\u5728":95,"\u7248\u672c\u8bf4\u660e":86,"\u7248\u672cfork\u51fa\u81ea\u5df1\u7684\u529f\u80fd\u5206\u652f":71,"\u7279\u522b\u662f\u5728lstm\u7b49rnn\u4e2d":80,"\u7279\u5f81\u56fe\u5747\u503c":122,"\u7279\u5f81\u56fe\u65b9\u5dee":122,"\u7279\u5f81\u5c06\u4f1a\u5b58\u5230":122,"\u72ec\u7acb\u5b9a\u5236\u7684\u4e8c\u8fdb\u5236\u65f6\u624d\u9700\u8981\u7f16\u8bd1":85,"\u72ec\u7acb\u5de5\u5177\u94fe":115,"\u72ec\u7acb\u5de5\u5177\u94fe\u6240\u5728\u7684\u7edd\u5bf9\u8def\u5f84":115,"\u73af\u5883\u53d8\u91cf":[104,111],"\u73af\u5883\u53d8\u91cf\u6765\u6307\u5b9a\u7279\u5b9a\u7684gpu":80,"\u73b0\u9636\u6bb5paddle\u6709\u4e00\u4e2a\u95ee\u9898\u662f":54,"\u7406\u89e3":94,"\u751a\u81f3\u80fd\u89e3\u91ca\u4e3a\u4ec0\u4e48\u67d0\u4e2a\u64cd\u4f5c\u82b1\u4e86\u5f88\u957f\u65f6\u95f4":103,"\u751f\u4ea7\u73af\u5883\u4e2d\u7684\u8bad\u7ec3\u6570\u636e\u96c6\u901a\u5e38\u4f53\u79ef\u5f88\u5927":35,"\u751f\u4ea7\u73af\u5883\u7684\u65e5\u5fd7\u6570\u636e\u4f1a\u901a\u8fc7\u5b9e\u65f6\u6d41\u7684\u65b9\u5f0f":35,"\u751f\u4ea7\u955c\u50cf":95,"\u751f\u6210":111,"\u751f\u6210\u5404\u79cd\u8bed\u8a00\u7684\u7ed1\u5b9a\u4ee3\u7801":54,"\u751f\u6210\u540e\u7684\u6587\u6863\u5206\u522b\u5b58\u50a8\u5728\u7f16\u8bd1\u76ee\u5f55\u7684":99,"\u751f\u6210\u5e8f\u5217\u7684\u6700\u5927\u957f\u5ea6":93,"\u751f\u6210\u6587\u6863":54,"\u751f\u6210\u7684":35,"\u751f\u6210\u7684\u6027\u80fd\u5206\u6790\u6587\u4ef6\u4e3a":102,"\u751f\u6210\u7684\u6570\u636e\u5c06\u4f1a\u5b58\u50a8\u5728\u8fd9\u4e2avolume\u4e0b":111,"\u751f\u6210\u7684\u6570\u636e\u7f13\u5b58\u5728\u5185\u5b58\u91cc":80,"\u751f\u6210\u7ed9\u5b9a":35,"\u751f\u6210\u7f51\u7edc\u5c42\u914d\u7f6e":96,"\u751f\u6210\u81ea\u5df1\u76ee\u5f55\u4e0b\u7684\u4ed3\u5e93":95,"\u751f\u6210\u8c03\u8bd5\u4fe1\u606f":102,"\u751f\u6210\u968f\u673a\u7684\u8f93\u5165\u6570\u636e":97,"\u751f\u6210api\u6587\u6863":54,"\u751f\u6210pfsclient\u548cpfsserver\u7684\u6846\u67b6\u90e8\u5206":44,"\u751f\u6210python\u6027\u80fd\u5206\u6790\u7684\u547d\u4ee4\u5982\u4e0b":102,"\u7528":44,"\u75280\u548c1\u8868\u793a":2,"\u7528\u4e86\u4e24\u4e2a\u6708\u4e4b\u540e\u8fd9\u4e2a\u663e\u793a\u5668\u5c4f\u5e55\u788e\u4e86":123,"\u7528\u4e8e\u521d\u59cb\u5316\u53c2\u6570\u548c\u8bbe\u7f6e":96,"\u7528\u4e8e\u5c06\u4e0b\u4e00\u884c\u7684\u6570\u636e\u8f93\u5165\u51fd\u6570\u6807\u8bb0\u6210\u4e00\u4e2apydataprovider2":2,"\u7528\u4e8e\u5c06\u53c2\u6570\u4f20\u9012\u7ed9\u7f51\u7edc\u914d\u7f6e":108,"\u7528\u4e8e\u6307\u5b9a\u5176\u8981\u5173\u8054\u7684layer":81,"\u7528\u4e8e\u6307\u5b9a\u7f51\u7edc\u914d\u7f6e\u6587\u4ef6":106,"\u7528\u4e8e\u6784\u6210\u65b0\u7684\u8bcd\u8868":121,"\u7528\u4e8e\u6ce8\u518c\u6ca1\u6709\u53cd\u5411\u7684op":97,"\u7528\u4e8e\u7a00\u758f\u7c7b\u578b\u53c2\u6570\u901a\u4fe1\u7684\u7aef\u53e3\u4e2a\u6570":104,"\u7528\u4e8e\u7a00\u758f\u8bad\u7ec3\u4e2d":106,"\u7528\u4e8e\u81ea\u5b9a\u4e49\u6bcf\u6761\u6570\u636e\u7684batch":2,"\u7528\u4e8e\u83b7\u53d6\u7279\u5b9alayer\u4e0a\u4e00\u65f6\u95f4\u6b65\u7684\u8f93\u51fa":81,"\u7528\u4e8e\u8ba1\u7b97\u7f16\u7801\u5411\u91cf\u7684\u52a0\u6743\u548c":93,"\u7528\u4e8e\u8bad\u7ec3\u795e\u7ecf\u7f51\u7edc\u7684\u6570\u636e":104,"\u7528\u53cc\u5411\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7f16\u7801":93,"\u7528\u591a\u5bf9\u6548\u679c\u5b8c\u5168\u76f8\u540c\u7684":90,"\u7528\u6237\u4e00\u822c\u901a\u8fc7\u8c03\u7528":102,"\u7528\u6237\u4e0a\u4f20\u6570\u636e\u540e":35,"\u7528\u6237\u4e5f\u53ef\u4ee5\u4e0a\u4f20label":35,"\u7528\u6237\u4e5f\u53ef\u4ee5\u5728c":1,"\u7528\u6237\u4eceapp":116,"\u7528\u6237\u53ea\u9700\u5b9a\u4e49rnn\u5728\u4e00\u4e2a\u65f6\u95f4\u6b65\u5185\u5b8c\u6210\u7684\u8ba1\u7b97":92,"\u7528\u6237\u53ef\u4ee5\u5206\u522b\u67e5\u770b\u6700\u65b0\u7684":99,"\u7528\u6237\u53ef\u4ee5\u53c2\u8003\u4e0b\u6587":115,"\u7528\u6237\u53ef\u4ee5\u53c2\u8003sphinx\u6559\u7a0b\u8fdb\u884c\u4e66\u5199":99,"\u7528\u6237\u53ef\u4ee5\u5728\u8f93\u51fa\u7684\u6587\u672c\u6a21\u578b\u4e2d\u770b\u5230":121,"\u7528\u6237\u53ef\u4ee5\u5b89\u5168\u7684\u91ca\u653e\u67d0\u4e2ac":55,"\u7528\u6237\u53ef\u4ee5\u628a\u81ea\u5df1\u7684\u6570\u636e\u5206\u4eab\u7ed9\u522b\u4eba":35,"\u7528\u6237\u53ef\u4ee5\u6839\u636e\u8bad\u7ec3\u65e5\u5fd7":123,"\u7528\u6237\u53ef\u4ee5\u76f4\u63a5\u4f7f\u7528\u8fd9\u4e2a\u52a8\u6001\u5e93\u6765\u5f15\u5165paddl":55,"\u7528\u6237\u53ef\u4ee5\u81ea\u5b9a\u4e49beam":106,"\u7528\u6237\u53ef\u4ee5\u8bbe\u7f6e":108,"\u7528\u6237\u53ef\u4ee5\u901a\u8fc7\u7b80\u5355\u4f7f\u7528python\u63a5\u53e3":1,"\u7528\u6237\u53ef\u5728\u81ea\u5df1\u719f\u6089\u7684\u5f00\u53d1\u5e73\u53f0\u4e0a\u7f16\u8bd1android\u5e73\u53f0\u4e0a\u9002\u7528\u7684paddlepaddle\u5e93":115,"\u7528\u6237\u53ef\u5728\u8c03\u7528cmake\u7684\u65f6\u5019\u8bbe\u7f6e\u5b83\u4eec":83,"\u7528\u6237\u53ef\u5c06":115,"\u7528\u6237\u53ef\u5c06\u5408\u6210\u7684fat\u5e93\u7528\u4e8e\u6df1\u5ea6\u5b66\u4e60\u76f8\u5173\u7684io":116,"\u7528\u6237\u53ef\u6839\u636e\u81ea\u5df1\u7684\u7f16\u8bd1\u76ee\u6807\u67b6\u6784":115,"\u7528\u6237\u53ef\u81ea\u884c\u524d\u5f80\u4e0b\u8f7d\u9884\u7f16\u8bd1\u597d\u7684\u7248\u672c":115,"\u7528\u6237\u53ef\u901a\u8fc7\u5982\u4e0b\u4e24\u79cd\u65b9\u5f0f":115,"\u7528\u6237\u5728\u4f7f\u7528\u8fd9\u4e00\u7c7brecurr":81,"\u7528\u6237\u5728\u4f7f\u7528paddlepaddl":77,"\u7528\u6237\u5728\u672c\u5730\u8f6c\u6362\u597d\u518d\u4e0a\u4f20":35,"\u7528\u6237\u5b9a\u4e49\u7684\u53c2\u6570":2,"\u7528\u6237\u5c06\u53c2\u6570\u8f7d\u5165":82,"\u7528\u6237\u5c06\u914d\u7f6e\u4e0e\u8bad\u7ec3\u6570\u636e\u5207\u5206\u597d\u653e\u5728\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf\u9884\u5148\u5206\u914d\u597d\u7684\u76ee\u5f55\u4e2d":111,"\u7528\u6237\u5f3a\u5236\u6307\u5b9a\u7279\u5b9a\u7684python\u7248\u672c":77,"\u7528\u6237\u6307\u5b9a\u65b0\u7684\u5b57\u5178\u7684\u8def\u5f84":121,"\u7528\u6237\u6587\u4ef6\u53ef\u80fd\u662f\u6bd4\u8f83\u5927\u7684":44,"\u7528\u6237\u8fd8\u53ef\u6839\u636e\u81ea\u5df1\u7684\u9700\u6c42\u8bbe\u7f6e\u5176\u4ed6\u7f16\u8bd1\u53c2\u6570":[115,116,117],"\u7528\u6237\u901a\u8fc7\u53c2\u6570":[81,82],"\u7528\u6237\u901a\u8fc7c":55,"\u7528\u6237\u9700\u8981\u5728\u7f51\u7edc\u914d\u7f6e\u4e2d\u6307\u5b9a":108,"\u7528\u6237\u9700\u8981\u5728cmake\u65f6\u624b\u52a8\u8bbe\u7f6e\u8fd9\u4e9b\u503c":[115,117],"\u7528\u6237\u9700\u8981\u6307\u5b9a\u672c\u673a\u4e0apython\u7684\u8def\u5f84":77,"\u7528\u6237\u9700\u8981\u63d0\u524d\u51c6\u5907\u597d\u4ea4\u53c9\u7f16\u8bd1\u73af\u5883":115,"\u7528\u672c\u673a\u7684\u7b2c\u4e00\u4e2a":94,"\u7528\u6765\u4ece\u53c2\u6570\u670d\u52a1\u5668\u9884\u53d6\u53c2\u6570\u77e9\u9635\u76f8\u5e94\u7684\u884c":96,"\u7528\u6765\u5b58\u50a8\u672c\u6b21\u6027\u80fd\u5206\u6790\u7684\u7ed3\u679c":102,"\u7528\u8fd9\u4e2a\u955c\u50cf\u521b\u5efa\u7684\u5bb9\u5668\u9700\u8981\u6709\u4ee5\u4e0b\u4e24\u4e2a\u529f\u80fd":111,"\u7528docker\u7f16\u8bd1\u548c\u6d4b\u8bd5paddlepaddl":85,"\u7528web\u6d4f\u89c8\u5668\u8bbf\u95ee\u5bf9\u5e94\u7f51\u5740":102,"\u7531":[81,92],"\u7531\u4e8e\u5b83\u5185\u90e8\u5305\u542b\u4e86\u6bcf\u7ec4\u6570\u636e\u4e2d\u7684\u6240\u6709\u53e5\u5b50":90,"\u7531\u4e8e\u5bb9\u5668\u4e4b\u95f4\u5171\u4eabnet":109,"\u7531\u4e8e\u5bf9parameters\u7684\u66f4\u65b0\u9700\u8981\u83b7\u53d6parameters\u5185\u5b58\u7684":34,"\u7531\u4e8e\u6211\u4eec\u60f3\u8981\u7684\u53d8\u6362\u662f\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217":90,"\u7531\u4e8e\u6211\u4eec\u652f\u6301\u8bad\u7ec3\u6570\u636e\u6709\u4e0d\u540c\u7684\u6279\u6b21\u5927\u5c0f":96,"\u7531\u4e8e\u6bcf\u4e2a\u5377\u79ef\u5c42\u540e\u9762\u8fde\u63a5\u7684\u662fbatch":122,"\u7531\u4e8e\u8fd9\u4e2a\u5730\u5740\u4f1a\u88abdataprovider\u4f7f\u7528":1,"\u7531\u4e8e\u8fd9\u6837\u505a\u53ef\u4ee5\u907f\u514d\u5f88\u591a\u6b7b\u9501\u95ee\u9898":2,"\u7531\u4e8e\u96c6\u7fa4\u4e2d\u540c\u65f6\u5b58\u5728\u4e24\u53f0\u673a\u5668\u6545\u969c\u7684\u6982\u7387\u6781\u4f4e":34,"\u7531\u4e8e\u987a\u5e8f\u8c03\u7528\u8fd9\u4e9bgenerator\u4e0d\u4f1a\u51fa\u73b0\u4e0a\u8ff0\u95ee\u9898":2,"\u7531\u4e8earm64\u67b6\u6784\u8981\u6c42android":115,"\u7531\u4e8ec":54,"\u7531\u4e8echunk\u6bd4\u8f83\u5c0f":44,"\u7531\u4e8eeigen":98,"\u7531\u4e8epypi":71,"\u7531\u4e8estep":92,"\u7531\u4e8etensor\u7684rank\u662f\u6a21\u677f\u53c2\u6570":98,"\u7531\u4e8etest_data\u5305\u542b\u4e24\u6761\u9884\u6d4b\u6570\u636e":4,"\u7531\u8bcd\u8bed\u6784\u6210\u7684\u53e5\u5b50":89,"\u7533\u8bf7\u7528\u6237\u7a7a\u95f4":44,"\u7535\u8111":90,"\u767b\u5f55\u5230head\u8282\u70b9":104,"\u7684":[86,90,94,95,98,104,110,111,116,123],"\u768410\u7ef4\u6574\u6570\u6807\u7b7e":2,"\u7684\u4e00\u4e2a\u7b80\u5355\u8c03\u7528\u5982\u4e0b":92,"\u7684\u4e3a0":106,"\u7684\u4efb\u4e00\u4e00\u79cd":80,"\u7684\u4f5c\u7528\u662f\u5ef6\u8fdf\u5206\u914d\u5185\u5b58":98,"\u7684\u4f7f\u7528\u793a\u4f8b\u5982\u4e0b":89,"\u7684\u503c":[115,116,117],"\u7684\u503c\u81ea\u52a8\u63a8\u5bfc\u5f97\u5230":115,"\u7684\u504f\u7f6e\u5411\u91cf":96,"\u7684\u5177\u4f53\u8ba1\u7b97\u903b\u8f91":97,"\u7684\u5185\u5b58":80,"\u7684\u5185\u5bb9\u6765\u5b9a\u5236imag":111,"\u7684\u5185\u6838block\u4f7f\u7528\u60c5\u51b5":103,"\u7684\u5206\u7c7b\u4efb\u52a1\u4e2d\u8d62\u5f97\u4e86\u7b2c\u4e00\u540d":122,"\u7684\u522b\u540d":[5,6,8],"\u7684\u5355\u5143\u6d4b\u8bd5":97,"\u7684\u53cd\u5411\u4f20\u64ad\u5c06\u4f1a\u6253\u5370\u65e5\u5fd7\u4fe1\u606f":106,"\u7684\u53d8\u6362\u77e9\u9635":96,"\u7684\u540d\u5b57":2,"\u7684\u540d\u79f0\u76f8\u540c":93,"\u7684\u5411\u91cf":96,"\u7684\u542f\u52a8\u53c2\u6570":111,"\u7684\u542f\u52a8\u53c2\u6570\u5e76\u6267\u884c\u8fdb\u7a0b":111,"\u7684\u547d\u4ee4\u548c\u4e00\u822c\u7684":102,"\u7684\u547d\u540d\u98ce\u683c\u5e76\u4e0d\u80fd\u9002\u5e94\u5176\u4ed6\u7b2c\u4e09\u65b9\u8bed\u8a00":54,"\u7684\u5730\u5740":109,"\u7684\u5730\u65b9":95,"\u7684\u5747\u5300\u5206\u5e03":82,"\u7684\u591a\u79cd\u5b89\u88c5\u65b9\u5f0f":104,"\u7684\u5934\u6587\u4ef6":54,"\u7684\u5b9e\u73b0":97,"\u7684\u5e73\u5747\u503c":89,"\u7684\u5e8f\u5217\u5f62\u72b6\u4e00\u81f4":90,"\u7684\u5f00\u53d1\u5de5\u4f5c\u90fd\u5e94\u8be5\u5728\u4e00\u4e2a\u65b0\u7684\u5206\u652f\u4e0a\u5b8c\u6210":95,"\u7684\u5f00\u53d1\u6d41\u7a0b":94,"\u7684\u5f00\u59cb\u8bf7\u52a0\u4e0a\u5b8f\u5b9a\u4e49":97,"\u7684\u6027\u80fd\u5206\u6790\u4e0e\u8c03\u4f18\u5206\u4e3a\u4e24\u4e2a\u90e8\u5206":102,"\u7684\u6027\u80fd\u5206\u6790\u5de5\u5177\u975e\u5e38\u591a":102,"\u7684\u6027\u80fd\u6709\u95ee\u9898":102,"\u7684\u63a5\u53e3\u6837\u5f0f":54,"\u7684\u63cf\u8ff0\u8bf4\u660e\u4e2d":95,"\u7684\u64cd\u4f5c":98,"\u7684\u6570\u636e\u6d41\u56fe":35,"\u7684\u6570\u636e\u8bfb\u53d6\u811a\u672c\u548c\u7c7b\u4f3c\u4e8e":123,"\u7684\u6570\u76ee\u4e00\u81f4":89,"\u7684\u6587\u4ef6\u4e5f\u5e26\u5230\u65b0\u5206\u652f\u4e0a":95,"\u7684\u65b9\u7a0b":96,"\u7684\u65f6\u95f4\u6b65\u4fe1\u606f\u6210\u6b63\u6bd4":80,"\u7684\u66f4\u8be6\u7ec6\u51c6\u786e\u7684\u5b9a\u4e49":90,"\u7684\u6700\u5c0f\u503c":106,"\u7684\u6700\u65b0\u4ee3\u7801\u5e76\u66f4\u65b0\u5f53\u524d\u5206\u652f":95,"\u7684\u6784\u9020\u51fd\u6570":97,"\u7684\u67b6\u6784\u7684\u793a\u4f8b":93,"\u7684\u6837\u5f0f":95,"\u7684\u6838\u5fc3\u662f\u8bbe\u8ba1step\u51fd\u6570\u7684\u8ba1\u7b97\u903b\u8f91":92,"\u7684\u6839\u76ee\u5f55":116,"\u7684\u6bb5\u843d\u5b9a\u4e49\u4e3a\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":92,"\u7684\u6e90\u7801\u4ee5\u53ca\u751f\u6210\u6587\u6863\u9700\u8981\u591a\u79cd\u5f00\u53d1\u5de5\u5177":95,"\u7684\u6e90\u7801\u91cc\u4f7f\u7528\u4e86":54,"\u7684\u7248\u672c":[94,117],"\u7684\u7248\u672c\u53f7":121,"\u7684\u7279\u5f81":122,"\u7684\u72b6\u6001":92,"\u7684\u72ec\u7acb\u5de5\u5177\u94fe":115,"\u7684\u77e9\u9635":[80,96],"\u7684\u7a20\u5bc6\u5411\u91cf\u4f5c\u4e3a\u8f93\u5165":96,"\u7684\u7b2ci\u4e2a\u503c":96,"\u7684\u7b2cj\u4e2a\u503c":96,"\u7684\u7cfb\u7edf":94,"\u7684\u7ef4\u5ea6":121,"\u7684\u7f16\u5199":104,"\u7684\u7f16\u8bd1\u5de5\u5177\u94fe":115,"\u7684\u7f29\u5199":44,"\u7684\u89c4\u8303":54,"\u7684\u89d2\u5ea6":35,"\u7684\u8ba1\u7b97\u4ee3\u7801":98,"\u7684\u8ba1\u7b97\u8fc7\u7a0b\u4e66\u5199\u66f4\u52a0\u7b80\u5355":97,"\u7684\u8bad\u7ec3\u6a21\u578b\u811a\u672c":123,"\u7684\u8bdd":80,"\u7684\u8be6\u7ec6\u4fe1\u606f":102,"\u7684\u8f93\u5165":92,"\u7684\u8f93\u51fa":103,"\u7684\u8f93\u51fa\u4fe1\u606f\u5165\u624b\u662f\u4e2a\u4e0d\u9519\u7684\u9009\u62e9":103,"\u7684\u8f93\u51fa\u51fd\u6570\u8fd4\u56de\u7684\u662f\u4e0b\u4e00\u4e2a\u65f6\u523b\u8f93\u51fa\u8bcd\u7684":93,"\u7684\u8f93\u51fa\u683c\u5f0f":90,"\u7684\u8f93\u51fa\u88ab\u7528\u4f5c":93,"\u7684\u8f93\u51fab\u662f\u4e00\u4e2a":80,"\u7684\u8fd0\u884c\u73af\u5883":94,"\u7684\u8fdc\u7a0b\u4ed3\u5e93\u7684\u540d\u5b57":95,"\u7684\u914d\u7f6e":121,"\u7684\u914d\u7f6e\u5199\u5230\u914d\u7f6e\u6587\u4ef6\u4e2d":35,"\u7684\u96c6\u88c5\u7bb1\u6280\u672f":94,"\u7684\u9875\u9762\u5220\u9664\u8fdc\u7a0b\u4ed3\u5e93\u7684\u5206\u652f":95,"\u7684cpu":97,"\u7684linux\u670d\u52a1\u5668\u7ec4\u6210":104,"\u7684paddlepaddle\u5e93":115,"\u76d1\u542c\u7684\u7aef\u53e3\u4e2a\u6570":104,"\u76ee\u524d":92,"\u76ee\u524d\u4f7f\u7528":95,"\u76ee\u524d\u53ea\u8003\u8651\u52a8\u6001\u6269\u5bb9trainer\u6570\u91cf":34,"\u76ee\u524d\u5d4c\u5165python\u89e3\u91ca\u5668":54,"\u76ee\u524d\u5fc5\u987b\u8bbe\u7f6e\u6210":117,"\u76ee\u524d\u6211\u4eec\u7528cephfs\u6765\u642d\u5efa":44,"\u76ee\u524d\u652f\u6301":115,"\u76ee\u524d\u652f\u6301\u4e24\u79cd":89,"\u76ee\u524d\u652f\u6301cento":88,"\u76ee\u524d\u652f\u6301fail":106,"\u76ee\u524d\u8be5\u53c2\u6570\u4ec5\u7528\u4e8eaucvalidationlayer\u548cpnpairvalidationlayer\u5c42":106,"\u76ee\u524d\u8fd8\u672a\u652f\u6301":92,"\u76ee\u524dpaddle\u7684\u8fdb\u7a0b\u6a21\u578b\u662fc":54,"\u76ee\u524dpaddlepaddle\u7684develop\u5206\u652f\u7684\u6587\u6863\u662f\u81ea\u52a8\u89e6\u53d1\u66f4\u65b0\u7684":99,"\u76ee\u5f55":[84,94,104,110,111,115,116,117],"\u76ee\u5f55\u4e0b":[55,96,104,123],"\u76ee\u5f55\u4e0b\u627e\u5230":123,"\u76ee\u5f55\u4e0b\u65b0\u589e\u7684":97,"\u76ee\u5f55\u4e0b\u6700\u65b0\u7684":116,"\u76ee\u5f55\u4e0b\u7684\u751f\u6210\u6587\u4ef6\u7528\u4e8e\u6df1\u5ea6\u5b66\u4e60\u76f8\u5173android":115,"\u76ee\u5f55\u4e0b\u7684demo\u8bad\u7ec3\u51fa\u6765":4,"\u76ee\u5f55\u4e0b\u7684python\u5305":77,"\u76ee\u5f55\u4e2d":104,"\u76ee\u5f55\u4e2d\u4f1a\u5305\u542b":[115,117],"\u76ee\u5f55\u4e2d\u4f1a\u5305\u542b\u4ee5\u4e0b\u5185\u5bb9":116,"\u76ee\u5f55\u4e2d\u7684":103,"\u76ee\u5f55\u4e2dpaddl":111,"\u76ee\u5f55\u548c":[115,116,117],"\u76ee\u5f55\u5c31\u6210\u4e3a\u4e86\u5171\u4eab\u5b58\u50a8":111,"\u76ee\u5f55\u751f\u6210\u4e00\u5957\u72ec\u7acb\u7f16\u8bd1\u5de5\u5177\u94fe":115,"\u76ee\u5f55\u91cc\u627e\u5230\u4ea4\u53c9\u7f16\u8bd1\u5668":117,"\u76ee\u5f55\u91cc\u63d0\u4f9b\u4e86\u8be5\u6570\u636e\u7684\u4e0b\u8f7d\u811a\u672c\u548c\u9884\u5904\u7406\u811a\u672c":123,"\u76ee\u6807\u5411\u91cf":93,"\u76ee\u6807\u5de5\u5177\u94fe":115,"\u76ee\u6807\u673a\u7248protobuf\u5e93":[115,117],"\u76ee\u6807\u67b6\u6784":116,"\u76ee\u6807\u67b6\u6784abi":115,"\u76f4\u5230\u8bad\u7ec3\u6536\u655b\u4e3a\u6b62":82,"\u76f4\u63a5\u4f7f\u7528\u4e0a\u8ff0\u5b89\u88c5\u6d41\u7a0b":85,"\u76f4\u63a5\u4f7f\u7528c\u8bed\u8a00\u7684":54,"\u76f4\u63a5\u5220\u9664\u8fd9\u4e2a\u53c2\u6570\u5373\u53ef":55,"\u76f4\u63a5\u5bfc\u51fa\u5230c\u7684\u63a5\u53e3\u6bd4\u8f83\u56f0\u96be":54,"\u76f4\u63a5\u8fd0\u884c":84,"\u76f4\u63a5\u8fd4\u56de\u8ba1\u7b97\u7ed3\u679c":4,"\u76f4\u63a5\u8fdb\u5165\u8bad\u7ec3\u6a21\u578b\u7ae0\u8282":123,"\u76f8\u5173\u6982\u5ff5\u662f":2,"\u76f8\u540c\u540d\u5b57\u7684\u53c2\u6570":82,"\u76f8\u5bf9":90,"\u76f8\u5bf9\u4e8epaddlepaddle\u7a0b\u5e8f\u8fd0\u884c\u65f6\u7684\u8def\u5f84":1,"\u76f8\u5bf9mnist\u800c\u8a00":2,"\u76f8\u5e94\u7684\u6570\u636e\u8bfb\u53d6\u811a\u672c\u548c\u8bad\u7ec3\u6a21\u578b\u811a\u672c":123,"\u76f8\u5f53":90,"\u76f8\u6bd4":97,"\u770b\u5f53\u524dmpi\u96c6\u7fa4\u662f\u5426\u652f\u6301resourc":78,"\u77a7":88,"\u77e9\u9635":105,"\u77e9\u9635\u4e58\u6cd5\u7684\u516c\u5f0f":97,"\u786e\u4fdd\u7f16\u8bd1\u5668\u9009\u9879":95,"\u78c1\u76d8\u4e0d\u591f":94,"\u78c1\u76d8\u7a7a\u95f4\u4e0d\u8db3\u7b49":78,"\u793a":123,"\u793a\u4f8b":[80,82,122],"\u793a\u4f8b3\u5bf9\u4e8e\u5355\u5c42rnn\u548c\u53cc\u5c42rnn\u6570\u636e\u5b8c\u5168\u76f8\u540c":90,"\u793a\u4f8b3\u7684\u914d\u7f6e\u4f7f\u7528\u4e86\u5355\u5c42rnn\u548c\u53cc\u5c42rnn":90,"\u793a\u4f8b3\u7684\u914d\u7f6e\u5206\u522b\u4e3a":90,"\u793a\u4f8b\u4ee3\u7801\u5982\u4e0b":80,"\u793a\u4f8b\u5982\u4e0b":82,"\u793a\u4f8b\u7a0b\u5e8f":104,"\u793e\u533a\u53c2\u4e0e\u56f0\u96be":54,"\u793e\u533a\u8d21\u732e\u4ee3\u7801\u5b66\u4e60\u6210\u672c\u9ad8":54,"\u795e\u7ecf\u7f51\u7edc\u4e2d\u7684\u53c2\u6570":34,"\u795e\u7ecf\u7f51\u7edc\u4e5f\u9700\u8981\u4e00\u4e9b\u7279\u5b9a\u7684layer\u4f5c\u4e3a\u8f93\u5165\u63a5\u53e3":87,"\u795e\u7ecf\u7f51\u7edc\u53c2\u6570\u4ee5\u53ca\u8fed\u4ee3\u65b9\u7a0b":87,"\u795e\u7ecf\u7f51\u7edc\u5728\u8bad\u7ec3\u7684\u65f6\u5019":80,"\u795e\u7ecf\u7f51\u7edc\u672c\u8d28\u4e0a\u662f\u4e00\u4e2a\u8ba1\u7b97\u56fe":98,"\u795e\u7ecf\u7f51\u7edc\u7684\u7f51\u7edc\u7ed3\u6784\u4e2d\u5177\u6709\u6709\u5411\u73af\u7ed3\u6784":90,"\u795e\u7ecf\u7f51\u7edc\u7684\u8bad\u7ec3\u672c\u8eab\u662f\u4e00\u4e2a\u975e\u5e38\u6d88\u8017\u5185\u5b58\u548c\u663e\u5b58\u7684\u5de5\u4f5c":80,"\u79bb":90,"\u79bb\u7ebf\u6279\u5904\u7406":35,"\u79f0\u4e3a":[93,95],"\u79f0\u4e3a\u5f00\u53d1\u955c\u50cf":115,"\u79f0\u4e4b\u4e3a\u53cc\u5c42\u5e8f\u5217\u7684\u4e00\u4e2a\u5b50\u5e8f\u5217":89,"\u79f0\u4e4b\u4e3a\u96c6\u675f\u5927\u5c0f":106,"\u79f0\u4f5c\u6709kernel":97,"\u79f0\u4f5ckernel":97,"\u7a00\u758f\u6570\u636e\u7684\u683c\u5f0f":96,"\u7a00\u758f\u66f4\u65b0\u7684\u7aef\u53e3\u6570\u91cf":111,"\u7a00\u758f\u768401\u5411\u91cf":[2,87],"\u7a00\u758f\u7684\u5411\u91cf":[2,87],"\u7a00\u758f\u77e9\u9635\u7684\u4e58\u79ef\u5e94\u7528\u4e8e\u524d\u5411\u4f20\u64ad\u8fc7\u7a0b":108,"\u7a0b\u5e8f\u4ece\u6b64\u76ee\u5f55\u62f7\u8d1d\u6587\u4ef6\u5230\u5bb9\u5668\u5185\u8fdb\u884c\u8bad\u7ec3":111,"\u7a0b\u5e8f\u4f9d\u8d56":104,"\u7a0b\u5e8f\u505c\u6b62":106,"\u7a0b\u5e8f\u662f\u4e00\u6837\u7684":102,"\u7a0b\u5e8f\u76f4\u63a5\u9000\u51fa":106,"\u7a20\u5bc6\u5411\u91cf":96,"\u7a20\u5bc6\u66f4\u65b0\u7684\u7aef\u53e3\u6570\u91cf":111,"\u7a20\u5bc6\u7684\u6d6e\u70b9\u6570\u5411\u91cf":[2,87],"\u7a97\u6237":90,"\u7acb\u523b\u9000\u51fa":80,"\u7acb\u5373\u6267\u884c\u5355\u5143\u6d4b\u8bd5":83,"\u7ae0\u8282":115,"\u7aef\u53e3":78,"\u7aef\u7684":102,"\u7aef\u81ea\u5b9a\u4e49\u4e00\u4e2a":1,"\u7aef\u8bfb\u53d6\u6570\u636e":80,"\u7b2c":90,"\u7b2c\u4e00\u4e2a":95,"\u7b2c\u4e00\u4e2a\u53c2\u6570":97,"\u7b2c\u4e00\u4e2a\u53c2\u6570\u662fsettings\u5bf9\u8c61":2,"\u7b2c\u4e00\u4e2a\u6837\u672c\u540c\u65f6encode\u4e24\u6761\u6570\u636e\u6210\u4e24\u4e2a\u5411\u91cf":90,"\u7b2c\u4e00\u4e2apass\u4f1a\u4ecepython\u7aef\u8bfb\u53d6\u6570\u636e":2,"\u7b2c\u4e00\u4e2atag\u4e3a":71,"\u7b2c\u4e00\u5929":90,"\u7b2c\u4e00\u7ae0\u8282":87,"\u7b2c\u4e00\u884c\u5b58\u7684\u662f\u56fe\u50cf":122,"\u7b2c\u4e00\u884c\u662f":121,"\u7b2c\u4e00\u90e8\u5206\u662f\u56fe\u7247\u7684\u6807\u7b7e":2,"\u7b2c\u4e09\u4e2a\u53c2\u6570":97,"\u7b2c\u4e09\u6b65":122,"\u7b2c\u4e09\u6b65\u5b8c\u6210\u540e":71,"\u7b2c\u4e8c\u4e2a":80,"\u7b2c\u4e8c\u4e2a\u4e3a":71,"\u7b2c\u4e8c\u4e2a\u53c2\u6570":97,"\u7b2c\u4e8c\u6b65":[121,122],"\u7b2c\u4e8c\u7c7b":81,"\u7b2c\u4e8c\u884c\u5b58\u7684\u662f\u56fe\u50cf":122,"\u7b2c\u4e8c\u90e8\u5206\u662f28":2,"\u7b2ci\u884c\u7b2cj\u5217\u7684\u6570\u503c":96,"\u7b49":[55,78,97],"\u7b49\u4e8e\u6837\u672c\u6570":80,"\u7b49\u5168\u90e8\u9759\u6001\u5e93\u4e2d\u7684\u76ee\u6807\u6587\u4ef6\u5168\u90e8\u6253\u5305\u540e\u4ea7\u751f\u7684\u6587\u4ef6":55,"\u7b49\u53c2\u6570":111,"\u7b49\u591a\u79cd\u516c\u6709\u4e91\u73af\u5883":109,"\u7b49\u5f85\u8fd9\u4e2a\u7a0b\u5e8f\u6267\u884c\u6210\u529f\u5e76\u8fd4\u56de0\u5219\u6210\u529f\u9000\u51fa":109,"\u7b49\u6587\u4ef6":55,"\u7b49\u7b49":123,"\u7b49\u90fd\u5c5e\u4e8e\u4e00\u4e2a\u547d\u540d\u7a7a\u95f4":109,"\u7b80\u4ecb":100,"\u7b80\u5199":97,"\u7b80\u5355\u4ecb\u7ecd\u9700\u8981\u7528\u5230\u57fa\u7c7b":97,"\u7b80\u5355\u603b\u7ed3op\u9700\u8981\u5305\u542b\u7684\u5185\u5bb9\u5982\u4e0b":97,"\u7b80\u5355\u6765\u8bf4":103,"\u7b80\u5355\u7684\u5168\u8fde\u63a5\u7f51\u7edc":82,"\u7b80\u5355\u7684\u6027\u80fd\u5206\u6790":103,"\u7b80\u5355\u7684pydataprovider2\u6837\u4f8b\u5c31\u8bf4\u660e\u5b8c\u6bd5\u4e86":2,"\u7b80\u5355\u7684yaml\u6587\u4ef6\u5982\u4e0b":110,"\u7b80\u76f4":90,"\u7b97\u6cd5":[80,93],"\u7b97\u6cd5\u4e2d\u7684beam\u5927\u5c0f":93,"\u7c7b\u4f3c":[55,89],"\u7c7b\u4f5c\u4e3a\u53c2\u6570\u7684\u62bd\u8c61":96,"\u7c7b\u522b\u4e2d\u7684\u53c2\u6570\u53ef\u7528\u4e8e\u6240\u6709\u573a\u5408":105,"\u7c7b\u522bid":123,"\u7c7b\u522bid\u548c\u6587\u672c\u4fe1\u606f\u7528":123,"\u7c7b\u540d\u548cc":54,"\u7c7b\u578b":[54,97,106],"\u7c7b\u578b\u4e3a":97,"\u7c7b\u578b\u4ecd\u7136\u4e3aeigenvector":98,"\u7c7b\u578b\u53ef\u4ee5\u662fpaddlepaddle\u652f\u6301\u7684\u4efb\u610f\u8f93\u5165\u6570\u636e\u7c7b\u578b":89,"\u7c7b\u578b\u540d\u4e3a":97,"\u7c7b\u578b\u662fnumpy\u7684ndarrai":80,"\u7c7b\u578b\u662fsparse_binary_vector":[2,87],"\u7c7b\u578b\u662fsparse_float_vector":[2,87],"\u7c7b\u578b\u6765\u8bbe\u7f6e":2,"\u7c7b\u578b\u7684":90,"\u7c7b\u578b\u7b49\u662f\u5426\u5408\u6cd5":97,"\u7c7b\u7684\u5b9a\u4e49\u5199\u5728":97,"\u7c7b\u7684\u6784\u9020\u51fd\u6570\u548c\u6790\u6784\u51fd\u6570":96,"\u7c7b\u91cd\u5199":97,"\u7c7b\u9700\u8981\u5b9e\u73b0\u521d\u59cb\u5316":96,"\u7cfb\u6570":97,"\u7cfb\u7edf\u4e2d\u7684\u74f6\u9888\u53ef\u80fd\u548c\u7a0b\u5e8f\u5458\u5f00\u53d1\u8fc7\u7a0b\u4e2d\u60f3\u8c61\u7684\u74f6\u9888\u76f8\u53bb\u751a\u8fdc":102,"\u7cfb\u7edf\u4f1a\u5bf9\u65b0\u589e\u7684op\u81ea\u52a8\u7ed1\u5b9apython":97,"\u7cfb\u7edf\u4f1a\u63d0\u4f9b\u4e00\u4e2a\u5206\u5e03\u5f0f\u5b58\u50a8\u670d\u52a1":104,"\u7cfb\u7edf\u4f1a\u6839\u636e\u6587\u4ef6\u540d\u81ea\u52a8\u6784\u5efaop\u548c\u5176\u5bf9\u5e94\u7684python\u6269\u5c55":97,"\u7ea2\u697c\u68a6":121,"\u7ebf\u7a0bid\u53f7":108,"\u7ec6\u8282\u63cf\u8ff0":107,"\u7ecf\u5e38\u4f1a\u6d88\u8017\u657010gb\u7684\u5185\u5b58\u548c\u6570gb\u7684\u663e\u5b58":80,"\u7ed3\u5408":109,"\u7ed3\u5c3e":97,"\u7ed3\u675f\u6807\u8bb0":93,"\u7ed3\u6784\u5982\u4e0b\u56fe":121,"\u7ed3\u679c\u5982\u4e0b\u56fe\u6240\u793a":102,"\u7ed3\u679c\u8f93\u51fa\u5230":94,"\u7ed3\u8bba":54,"\u7ed9":90,"\u7ed9\u4e2a\u7b80\u5355\u7684":95,"\u7ed9\u5b9aencoder\u8f93\u51fa\u548c\u5f53\u524d\u8bcd":92,"\u7edf\u4e00\u7528":35,"\u7ee7\u627f\u81ea":97,"\u7ee7\u627f\u81eaoperatorbas":97,"\u7ee7\u7eed\u8bad\u7ec3\u6216\u9884\u6d4b":2,"\u7ef4\u57fa\u767e\u79d1\u4e2d\u6587\u9875\u9762":90,"\u7ef4\u57fa\u767e\u79d1\u9875\u9762":90,"\u7ef4\u5ea6\u4e3aword_dim":123,"\u7ef4\u5ea6\u662f\u7c7b\u522b\u4e2a\u6570":123,"\u7ef4\u5ea6\u662f\u8bcd\u5178\u5927\u5c0f":123,"\u7ef4\u62a4":109,"\u7ef4\u7a7a\u95f4":93,"\u7ef4\u7a7a\u95f4\u5b8c\u6210":93,"\u7f13\u5b58\u6c60\u7684\u51cf\u5c0f":80,"\u7f13\u5b58\u8bad\u7ec3\u6570\u636e\u5230\u5185\u5b58":2,"\u7f16\u5199":84,"\u7f16\u5199\u4e86\u4e00\u4e2apaddlepaddle\u7684\u7a0b\u5e8f":84,"\u7f16\u5199\u5b8cyaml\u6587\u4ef6\u540e":111,"\u7f16\u5199\u672c\u6b21\u8bad\u7ec3\u7684yaml\u6587\u4ef6":111,"\u7f16\u5199\u6df1\u5ea6\u5b66\u4e60\u7a0b\u5e8f":102,"\u7f16\u5199\u7684\u90e8\u5206":86,"\u7f16\u5199\u96c6\u7fa4\u4efb\u52a1\u63d0\u4ea4\u548c\u7ba1\u7406\u811a\u672c":104,"\u7f16\u53f7\u4ece0\u5f00\u59cb":80,"\u7f16\u7801\u5411\u91cf":93,"\u7f16\u7801\u5668\u8f93\u51fa":93,"\u7f16\u7801\u6e90\u5e8f\u5217":93,"\u7f16\u8bd1":[84,94,95,115],"\u7f16\u8bd1\u540e\u7684\u6587\u4ef6\u5c06\u88ab\u5b58\u50a8\u5728\u5de5\u4f5c\u76ee\u5f55":99,"\u7f16\u8bd1\u548c\u5b89\u88c5paddlepaddl":117,"\u7f16\u8bd1\u548c\u5b89\u88c5paddlepaddle\u9884\u6d4b\u5e93":[115,116],"\u7f16\u8bd1\u548c\u6267\u884c":97,"\u7f16\u8bd1\u5668":[115,116,117],"\u7f16\u8bd1\u5668\u6ca1\u6709":54,"\u7f16\u8bd1\u5668\u8981\u6c42\u7cfb\u7edf\u652f\u6301":115,"\u7f16\u8bd1\u578b\u8bed\u8a00":54,"\u7f16\u8bd1\u5b89\u88c5\u4e0e\u5355\u5143\u6d4b\u8bd5":79,"\u7f16\u8bd1\u5b89\u88c5\u7ed3\u675f\u4e4b\u540e":115,"\u7f16\u8bd1\u5b8c\u6210\u4e4b\u540e":99,"\u7f16\u8bd1\u5b8c\u6210\u540e\u4f1a\u5728build":83,"\u7f16\u8bd1\u5de5\u5177\u94fe":115,"\u7f16\u8bd1\u5de5\u5177\u94fe\u6240\u5728\u7684\u7edd\u5bf9\u8def\u5f84":117,"\u7f16\u8bd1\u6027\u80fd\u4f1a\u548c":102,"\u7f16\u8bd1\u6210\u529f\u540e":97,"\u7f16\u8bd1\u6210\u52a8\u6001\u5e93":106,"\u7f16\u8bd1\u65f6\u4e00\u5b9a\u8981\u5f00\u542f\u4f18\u5316":102,"\u7f16\u8bd1\u65f6\u53ef\u80fd\u4f1a\u53bb\u6389\u8c03\u8bd5\u4fe1\u606f":102,"\u7f16\u8bd1\u65f6\u6307\u5b9a":102,"\u7f16\u8bd1\u751f\u6210":99,"\u7f16\u8bd1\u8fd9\u4e2a\u7248\u672c\u7684docker\u53d1\u884c\u955c\u50cf":71,"\u7f16\u8bd1\u8fd9\u4e2a\u7248\u672c\u7684python":71,"\u7f16\u8bd1\u8fd9\u4e2a\u7248\u672c\u7684ubuntu":71,"\u7f16\u8bd1c":55,"\u7f16\u8bd1master\u5206\u652f\u7684docker\u53d1\u884c\u955c\u50cf":71,"\u7f16\u8bd1paddlepaddl":83,"\u7f16\u8bd1ubuntu\u7684deb\u5305":71,"\u7f16\u8f91":109,"\u7f29\u653e\u53c2\u6570":122,"\u7f51\u7edc\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf":109,"\u7f51\u7edc\u540d\u79f0":123,"\u7f51\u7edc\u5c42\u53ef\u4ee5\u6709\u591a\u4e2a\u8f93\u5165":96,"\u7f51\u7edc\u5c42\u7684\u6807\u8bc6\u7b26\u4e3a":96,"\u7f51\u7edc\u5c42\u7684\u7c7b\u578b":96,"\u7f51\u7edc\u5c42\u7684\u7ec6\u8282\u53ef\u4ee5\u901a\u8fc7\u4e0b\u9762\u8fd9\u4e9b\u4ee3\u7801\u7247\u6bb5\u6765\u6307\u5b9a":96,"\u7f51\u7edc\u5c42\u7684\u8f93\u51fa\u662f\u7ecf\u8fc7\u6fc0\u6d3b\u51fd\u6570\u4e4b\u540e\u7684\u503c":106,"\u7f51\u7edc\u5c42\u914d\u7f6e\u5305\u542b\u4ee5\u4e0b\u51e0\u9879":96,"\u7f51\u7edc\u6a21\u5757":122,"\u7f51\u7edc\u901a\u4fe1":96,"\u7f51\u7edc\u914d\u7f6e":123,"\u7f51\u7edc\u914d\u7f6e\u6587\u4ef6":122,"\u800c":[81,93,97,102,110],"\u800c\u4e0d\u4f1a\u6539\u53d8\u539f\u6709tensor\u7684shape\u4fe1\u606f":98,"\u800c\u4e0d\u5fc5\u5728\u610fpaddl":55,"\u800c\u4e0d\u652f\u6301pypy\u89e3\u91ca\u5668":54,"\u800c\u4e0d\u662f\u5728layer\u91cc\u5b9e\u73b0":81,"\u800c\u4e0d\u662f\u6e90\u7801\u76ee\u5f55\u91cc":77,"\u800c\u4e0d\u662f\u7279\u5f81\u7684\u96c6\u5408":90,"\u800c\u4e0d\u66b4\u9732\u6982\u5ff5\u7684\u5b9e\u73b0":55,"\u800c\u4e0d\u7528\u5173\u5fc3\u6570\u636e\u5982\u4f55\u4f20\u8f93":2,"\u800c\u4e14\u4e2a\u6570\u5e76\u4e0d\u786e\u5b9a":104,"\u800c\u4e14\u5305\u542b\u4e86c":86,"\u800c\u4e14\u5728\u4f20\u8f93\u7684\u8fc7\u7a0b\u4e2d\u4e5f\u53ef\u80fd\u51fa\u73b0\u7f51\u7edc\u4e0d\u7a33\u5b9a\u7684\u60c5\u51b5":44,"\u800c\u4e14cento":86,"\u800c\u4e4b\u524d\u7684\u53c2\u6570\u5c06\u4f1a\u88ab\u5220\u9664":106,"\u800c\u4ece\u5e94\u7528\u7684\u89d2\u5ea6":103,"\u800c\u4f18\u5316\u6027\u80fd\u7684\u9996\u8981\u4efb\u52a1":103,"\u800c\u5176\u4ed6\u5c42\u4f7f\u7528cpu\u8ba1\u7b97":108,"\u800c\u51fa\u73b0\u9636\u6bb5\u6027\u7684\u8fd0\u884c\u505c\u6ede":34,"\u800c\u53cc\u5c42rnn\u662f\u53ef\u4ee5\u5904\u7406\u8fd9\u79cd\u8f93\u5165\u6570\u636e\u7684\u7f51\u7edc\u7ed3\u6784":90,"\u800c\u53ea\u9700\u8981\u83b7\u5f97recurr":81,"\u800c\u53f3\u56fe\u7684\u74f6\u9888\u8fde\u63a5\u6a21\u5757\u7528\u4e8e50\u5c42":122,"\u800c\u5728\u8ba1\u7b97\u7ed3\u675f\u4e4b\u540e":98,"\u800c\u5728cpp\u91cc\u9762\u5b9e\u73b0\u8fd9\u4e2ac\u7684\u63a5\u53e3":54,"\u800c\u591a\u8bed\u8a00\u63a5\u53e3\u9700\u8981\u76f4\u63a5\u8bfb\u53d6\u751f\u6210\u7684\u4e8c\u8fdb\u5236":54,"\u800c\u5b89\u88c5\u5305":[77,86],"\u800c\u5b89\u88c5\u5305\u662f":[77,86],"\u800c\u5bf9\u4e8e\u53cc\u5c42\u5e8f\u5217":90,"\u800c\u5bf9\u4e8e\u6bcf\u4e00\u4e2a\u5185\u5c42\u7279\u5f81\u6570\u636e\u800c\u8a00":90,"\u800c\u5bf9\u4e8egolang":54,"\u800c\u5bf9\u4e8egolang\u9519\u8bef\u5904\u7406\u5e94\u8be5\u4f7f\u7528\u8fd4\u56de\u503c":54,"\u800c\u5c06\u8fd9\u4e2a\u6bb5\u843d\u7684\u6bcf\u4e00\u53e5\u8bdd\u7528lstm\u7f16\u7801\u6210\u4e00\u4e2a\u5411\u91cf":90,"\u800c\u5f53\u524d\u5df2\u7ecf\u67095":103,"\u800c\u662f\u76f4\u63a5\u4ece\u5185\u5b58\u7684\u7f13\u5b58\u91cc\u8bfb\u53d6\u6570\u636e":80,"\u800c\u662f\u76f4\u63a5\u4fee\u6539paddl":55,"\u800c\u662f\u76f4\u63a5\u7528api\u7684\u63a5\u53e3\u8fdc\u7a0b\u8bbf\u95ee":35,"\u800c\u66f4\u6df1\u5165\u7684\u5206\u6790":103,"\u800c\u6709\u4e9b\u53c2\u6570\u9700\u8981\u5728\u96c6\u7fa4\u591a\u673a\u8bad\u7ec3\u4e2d\u4f7f\u7528\u7b49":105,"\u800c\u6e90\u5e8f\u5217\u7684\u7f16\u7801\u5411\u91cf\u53ef\u4ee5\u88ab\u65e0\u8fb9\u754c\u7684memory\u8bbf\u95ee":93,"\u800c\u795e\u7ecf\u7f51\u7edc\u662f\u6211\u4eec\u8981\u642d\u5efa\u7684\u5b9d\u5854":87,"\u800c\u7a00\u758f\u66f4\u65b0\u5728\u53cd\u5411\u4f20\u64ad\u4e4b\u540e\u7684\u6743\u91cd\u66f4\u65b0\u65f6\u8fdb\u884c":108,"\u800c\u8ba1\u7b97\u8fc7\u7a0b\u662f\u7531":98,"\u800c\u8fd9\u4e00\u53e5\u8bdd\u5c31\u53ef\u4ee5\u8868\u793a\u6210\u8fd9\u4e9b\u4f4d\u7f6e\u7684\u6570\u7ec4":90,"\u800c\u8fd9\u4e2acontext\u53ef\u80fd\u4f1a\u975e\u5e38\u5927":2,"\u800c\u8fd9\u6bcf\u4e00\u4e2a\u6570\u7ec4\u5143\u7d20":90,"\u800c\u975e\u76f4\u63a5\u56de\u590d\u7684\u65b9\u5f0f":95,"\u800c\u975e\u9759\u6001\u52a0\u8f7dcuda\u52a8\u6001\u5e93":83,"\u800ceigenvector":98,"\u800cpaddlepaddle\u5219\u4f1a\u5e2e\u7528\u6237\u505a\u4ee5\u4e0b\u5de5\u4f5c":2,"\u800crnn\u662f\u6700\u6d41\u884c\u7684\u9009\u62e9":92,"\u800cswig\u53ea\u80fd\u7b80\u5355\u7684\u66b4\u9732c":54,"\u800ctrainer\u9700\u8981\u8bfb\u53d6\u8bad\u7ec3\u6570\u636e\u8fdb\u884c\u8bad\u7ec3":87,"\u800cy_predict\u662f\u63a5\u6536x\u4f5c\u4e3a\u8f93\u5165":87,"\u8054\u901a":104,"\u80fd\u591f\u5904\u7406\u53cc\u5c42\u5e8f\u5217":92,"\u80fd\u591f\u5bf9\u53cc\u5411\u5e8f\u5217\u8fdb\u884c\u5904\u7406\u7684\u6709":92,"\u80fd\u591f\u627e\u5230\u8fd9\u91cc\u4f7f\u7528\u7684\u6240\u6709\u6570\u636e":123,"\u80fd\u591f\u8bb0\u5f55\u4e0a\u4e00\u4e2asubseq":92,"\u80fd\u591f\u9488\u5bf9cpu\u548cgpu\u7684\u8ba1\u7b97\u505a\u66f4\u591a\u4f18\u5316":81,"\u80fd\u83b7\u53d6":104,"\u811a\u672c":[94,115],"\u811a\u672c\u5f00\u59cb\u65f6":111,"\u81ea\u52a8\u5173\u95ed\u5bf9\u5e94\u7684":95,"\u81ea\u52a8\u5730\u5c06\u8fd9\u4e9b\u9009\u9879\u5e94\u7528\u5230":104,"\u81ea\u52a8\u5b8c\u6210\u8fd9\u4e00\u8fc7\u7a0b":92,"\u81ea\u52a8\u6302\u8f7d\u5206\u5e03\u5f0f\u5b58\u50a8\u76ee\u5f55":34,"\u81ea\u52a8\u6784\u5efa\u72ec\u7acb\u5de5\u5177\u94fe":115,"\u81ea\u52a8\u751f\u6210":99,"\u81ea\u52a8\u83b7\u53d6\u4e0a\u4e00\u4e2a\u751f\u6210\u7684\u8bcd":93,"\u81ea\u52a8\u9009\u62e9":116,"\u81ea\u5e95\u5411\u4e0a\u6cd5":123,"\u81ea\u6b64":[115,116],"\u81ea\u7136\u4e5f\u5c31\u6709\u7ba1\u7406\u5458\u6743\u9650":94,"\u81ea\u7136\u8bed\u8a00\u5904\u7406\u7b49":108,"\u81f3\u4e8e\u4e3a\u4ec0\u4e48\u9700\u8981c":55,"\u81f3\u5c11\u5305\u542bgcc_3":86,"\u81f3\u5c11\u5305\u542bglibcxx_3":86,"\u81f3\u6b64":[2,90,95],"\u8212\u9002":90,"\u826f\u597d\u7684\u6587\u6863":54,"\u8282\u70b9":104,"\u82e5":96,"\u82e5\u5728paddlepaddle\u7f16\u8bd1\u65f6":82,"\u82e5\u5e0c\u671b\u5f97\u5230\u6700\u5feb\u7684\u6267\u884c\u901f\u5ea6":116,"\u82e5\u5e0c\u671b\u6700\u5feb\u7684\u6267\u884c\u901f\u5ea6":[115,117],"\u82e5\u5e72\u4e2a\u53e5\u5b50\u6784\u6210\u4e00\u4e2a\u6bb5\u843d":89,"\u82e5\u6709\u4e0d\u4e00\u81f4\u4e4b\u5904":103,"\u82e5\u6709\u5fc5\u8981":96,"\u82e5\u672a\u663e\u5f0f\u6307\u5b9a":116,"\u82e5\u6ca1\u6709\u663e\u5f0f\u8bbe\u7f6e":115,"\u82e5\u73af\u5883\u53d8\u91cf":[115,116,117],"\u82e5\u8981\u5bf9\u8fd9\u51e0\u4e2alayer\u4f7f\u7528dropout":81,"\u82e5\u8f93\u51fa\u662f\u5355\u5c42\u5e8f\u5217":89,"\u82e5\u8f93\u51fa\u662f\u53cc\u5c42\u5e8f\u5217":89,"\u82f1\u6587\u6587\u6863":99,"\u82f1\u6587\u6587\u6863\u76ee\u5f55":99,"\u8303\u56f4":108,"\u83b7\u53d6":95,"\u83b7\u53d6\u5229\u7528":123,"\u83b7\u53d6\u53ef\u9009\u7684tag":84,"\u83b7\u53d6\u5f53\u524d\u7cfb\u7edf\u652f\u6301\u7684\u5b89\u88c5\u5305\u683c\u5f0f":86,"\u83b7\u53d6\u5f53\u524d\u7cfb\u7edf\u652f\u6301\u7684python\u5305\u7684\u540e\u7f00":77,"\u83b7\u53d6\u6700\u65b0\u7684\u68c0\u67e5\u70b9\u7684\u6587\u4ef6uuid":34,"\u83b7\u53d6\u6e90\u7801":94,"\u83b7\u53d6\u8be5\u6761\u6837\u672c\u7c7b\u522bid":123,"\u83b7\u53d6trainer":111,"\u83b7\u5f97\u53c2\u6570\u5c3a\u5bf8":96,"\u83b7\u5f97\u57fa\u672c\u7684docker\u5b89\u88c5\u548c\u4f7f\u7528\u65b9\u6cd5":84,"\u83b7\u5f97\u8fd9\u4e9b\u8282\u70b9\u7684ip\u5730\u5740":104,"\u83b7\u5f97head\u548cnode\u8282\u70b9\u7684ip\u5730\u5740":104,"\u865a\u62df\u673a\u4e0a":94,"\u867d\u7136\u4e0d\u9f13\u52b1\u8fd9\u6837":55,"\u867d\u7136\u5f02\u6b65sgd\u65b9\u5f0f\u4f1a\u63d0\u9ad8\u53c2\u6570\u66f4\u65b0\u5e76\u884c\u5ea6":104,"\u867d\u7136\u6bcf\u4e2agenerator\u5728\u6ca1\u6709\u8c03\u7528\u7684\u65f6\u5019":2,"\u867d\u7136paddle\u770b\u8d77\u6765\u5305\u542b\u4e86\u4f17\u591a\u53c2\u6570":105,"\u884c":121,"\u884c\u4f18\u5148\u6b21\u5e8f\u5b58\u50a8":122,"\u884c\u5185\u4f7f\u7528":2,"\u884c\u53f7":102,"\u8865\u5145\u4e0a\u6b21\u7684commit":95,"\u8868\u660e\u4e86\u8fd9\u4e9b\u884c\u7684\u6807\u53f7":96,"\u8868\u660e\u8fd9\u4e2a\u5c42\u7684\u4e00\u4e2a\u5b9e\u4f8b\u662f\u5426\u9700\u8981\u504f\u7f6e":96,"\u8868\u793a":97,"\u8868\u793a\u4e00\u4e2akubernetes\u96c6\u7fa4\u4e2d\u7684\u4e00\u4e2a\u5de5\u4f5c\u8282\u70b9":109,"\u8868\u793a\u4e3adeviceid":108,"\u8868\u793a\u5c06\u5916\u5c42\u7684outer_mem\u4f5c\u4e3a\u5185\u5c42memory\u7684\u521d\u59cb\u72b6\u6001":90,"\u8868\u793a\u5f53\u524d\u96c6\u7fa4\u4f5c\u4e1a\u7684\u8282\u70b9":104,"\u8868\u793a\u6570\u636e\u7c7b\u578b":97,"\u8868\u793a\u7528\u4e8e\u8bad\u7ec3\u6216\u9884\u6d4b":2,"\u8868\u793a\u7684\u6bcf\u4e2a\u5355\u8bcd":123,"\u8868\u793a\u8bbe\u5907\u7c7b\u578b":97,"\u8868\u793a\u8bfb\u8005\u6240\u4f7f\u7528\u7684docker\u955c\u50cf\u4ed3\u5e93\u5730\u5740":111,"\u8868\u793a\u8fc7\u4e8620\u4e2abatch":123,"\u8868\u793a\u8fc7\u4e862560\u4e2a\u6837\u672c":123,"\u8868\u793a\u8fd9\u4e2ajob\u7684\u540d\u5b57":111,"\u8868\u793a\u9700\u8981\u6784\u5efa\u63a8\u7406\u5e93":117,"\u88ab":95,"\u88ab\u5207\u5206\u6210\u591a\u4e2a\u90e8\u5206":104,"\u88ab\u6269\u5c55\u4e3a\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":89,"\u88ab\u653e\u5728":96,"\u88ab\u79f0\u4e3a":93,"\u8981\u4f7f\u7528\u547d\u4ee4\u884c\u5206\u6790\u5de5\u5177":103,"\u8981\u5728\u5df2\u6709\u7684kubernetes\u96c6\u7fa4\u4e0a\u8fdb\u884cpaddlepaddle\u7684\u5206\u5e03\u5f0f\u8bad\u7ec3":111,"\u8981\u6c42\u5355\u5c42\u5e8f\u5217\u542b\u6709\u5143\u7d20\u7684\u6570\u76ee":89,"\u8981\u751f\u6210\u7684\u76ee\u6807\u5e8f\u5217":92,"\u8981\u8c03\u7528":96,"\u89e3\u51b3\u529e\u6cd5\u662f":77,"\u89e3\u51b3\u65b9\u6848\u662f":82,"\u89e3\u6790\u6a21\u578b\u914d\u7f6e\u6587\u4ef6":4,"\u89e3\u6790\u73af\u5883\u53d8\u91cf\u5f97\u5230":111,"\u89e3\u6790\u8bad\u7ec3\u6a21\u578b\u65f6\u7528\u7684\u914d\u7f6e\u6587\u4ef6":4,"\u89e3\u7801\u5668\u4f7f\u7528":93,"\u89e3\u7801\u5668\u57fa\u4e8e\u7f16\u7801\u6e90\u5e8f\u5217\u548c\u6700\u540e\u751f\u6210\u7684\u76ee\u6807\u8bcd\u9884\u6d4b\u4e0b\u4e00\u76ee\u6807\u8bcd":93,"\u89e3\u7801\u5668\u662f\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":93,"\u89e3\u91ca":123,"\u89e3\u91ca\u578b\u8bed\u8a00\u53ea\u80fd\u8c03\u7528\u52a8\u6001\u5e93":54,"\u89e3\u91ca\u6027\u8bed\u8a00\u5b9e\u9645\u8fd0\u884c\u7684\u4e8c\u8fdb\u5236\u662f\u89e3\u91ca\u5668\u672c\u8eab":54,"\u8ba1\u7b97":[93,104],"\u8ba1\u7b97\u504f\u7f6e\u7684\u68af\u5ea6":96,"\u8ba1\u7b97\u53cd\u5411rnn\u7684\u7b2c\u4e00\u4e2a\u5b9e\u4f8b":93,"\u8ba1\u7b97\u53d8\u6362\u77e9\u9635\u7684\u5927\u5c0f\u548c\u683c\u5f0f":96,"\u8ba1\u7b97\u5f53\u524d\u5c42\u6743\u91cd\u7684\u68af\u5ea6":96,"\u8ba1\u7b97\u6548\u7387\u66f4\u9ad8":81,"\u8ba1\u7b97\u6bcf\u4e2a\u8bcd\u7684\u8bcd\u5411\u91cf":93,"\u8ba1\u7b97\u6fc0\u6d3b\u51fd\u6570\u7684\u68af\u5ea6":96,"\u8ba1\u7b97\u7684\u7ec6\u8282\u5c06\u5728\u4e0b\u9762\u7684\u5c0f\u8282\u7ed9\u51fa":96,"\u8ba1\u7b97\u8282\u70b9":104,"\u8ba1\u7b97\u8282\u70b9\u4e4b\u95f4\u4e5f\u4e0d\u4f1a\u76f8\u4e92\u4f9d\u8d56":104,"\u8ba1\u7b97\u8f6c\u6362\u77e9\u9635\u548c\u8f93\u5165\u7684\u68af\u5ea6":96,"\u8ba1\u7b97\u8f93\u5165\u548c\u53c2\u6570\u7684\u68af\u5ea6":96,"\u8ba1\u7b97\u8f93\u5165\u5c42\u7684\u504f\u5dee":96,"\u8ba1\u7b97\u8f93\u51fa":96,"\u8ba1\u7b97\u8fd9\u4e2a\u6587\u4ef6\u7684md5":34,"\u8ba1\u7b97\u96c6\u7fa4\u901a\u5e38\u7531\u4e00\u7ec4":104,"\u8ba1\u7b97\u9700\u8981\u7684\u6570\u636e\u5b58\u653e\u5728":98,"\u8ba9\u6a21\u578b\u80fd\u591f\u5f97\u5230\u8bad\u7ec3\u66f4\u65b0":123,"\u8ba9\u795e\u7ecf\u7f51\u7edc\u53ef\u4ee5\u8fdb\u884c\u8bad\u7ec3\u6216\u9884\u6d4b":1,"\u8ba9paddle\u6838\u5fc3\u4e2d":55,"\u8bad\u7ec3":105,"\u8bad\u7ec3\u4e0e\u5e94\u7528":0,"\u8bad\u7ec3\u4efb\u52a1\u7684\u8fd0\u884c\u53ef\u80fd\u4f1a\u5360\u6ee1trainer\u548cparamet":34,"\u8bad\u7ec3\u548c\u7eaf\u4f7f\u7528":71,"\u8bad\u7ec3\u5931\u8d25\u65f6\u53ef\u4ee5\u68c0\u67e5\u9519\u8bef\u65e5\u5fd7":104,"\u8bad\u7ec3\u597d\u4e00\u4e2a\u6df1\u5c42\u795e\u7ecf\u7f51\u7edc\u901a\u5e38\u8981\u8017\u8d39\u975e\u5e38\u957f\u7684\u65f6\u95f4":103,"\u8bad\u7ec3\u6570\u636e\u662f":2,"\u8bad\u7ec3\u6570\u636e\u6709\u95ee\u9898":80,"\u8bad\u7ec3\u6570\u636e\u683c\u5f0f\u548c\u8bad\u7ec3\u7a0b\u5e8f\u7684":104,"\u8bad\u7ec3\u65f6":111,"\u8bad\u7ec3\u65f6\u6240\u9700\u8bbe\u7f6e\u7684\u4e3b\u8981\u53c2\u6570\u5982\u4e0b":123,"\u8bad\u7ec3\u65f6\u9ed8\u8ba4shuffl":2,"\u8bad\u7ec3\u6a21\u578b\u540e":93,"\u8bad\u7ec3\u6a21\u578b\u6b63\u786e\u6027":71,"\u8bad\u7ec3\u7a0b\u5e8f":104,"\u8bad\u7ec3\u7ed3\u675f\u540e\u67e5\u770b\u8f93\u51fa\u7ed3\u679c":111,"\u8bad\u7ec3\u811a\u672c":123,"\u8bad\u7ec3\u811a\u672c\u7b49\u7b49":123,"\u8bad\u7ec3\u8282\u70b9\u6570\u91cf":111,"\u8bad\u7ec3\u8bed\u8a00\u6a21\u578b\u8ddd\u79bb":80,"\u8bad\u7ec3\u8f6e\u6b21":123,"\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u53c2\u6570\u6216\u8005\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u7684\u68af\u5ea6\u5c3a\u5ea6\u8fc7\u5927":80,"\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u6d4b\u8bd5test_period":105,"\u8bad\u7ec3\u8fc7\u7a0b\u662f\u5426\u4e3a\u672c\u5730\u6a21\u5f0f":106,"\u8bad\u7ec3\u8fc7\u7a0b\u662f\u5426\u4f7f\u7528gpu":106,"\u8bad\u7ec3\u914d\u7f6e\u4e2d\u7684\u8bbe\u5907\u5c5e\u6027\u5c06\u4f1a\u65e0\u6548":106,"\u8bad\u7ec3dot_period":105,"\u8bb0\u5f55\u4e0b\u6240\u6709\u5931\u8d25\u7684\u4f8b\u5b50":71,"\u8bb0\u5fc6\u6a21\u5757":93,"\u8bba\u6587":122,"\u8bbe\u4e3a\u5df2\u90e8\u7f72\u7684\u5de5\u4f5c\u7a7a\u95f4\u76ee\u5f55":104,"\u8bbe\u4e3a\u672c\u5730":104,"\u8bbe\u5b9a":81,"\u8bbe\u7f6e":[55,80,81,83,115,116],"\u8bbe\u7f6e\u4e3a":96,"\u8bbe\u7f6e\u4e3a\u4e0d\u540c\u7684\u503c":81,"\u8bbe\u7f6e\u4e3atrue\u4f7f\u7528\u672c\u5730\u8bad\u7ec3\u6216\u8005\u4f7f\u7528\u96c6\u7fa4\u4e0a\u7684\u4e00\u4e2a\u8282\u70b9":106,"\u8bbe\u7f6e\u4e3atrue\u4f7f\u7528gpu\u6a21\u5f0f":106,"\u8bbe\u7f6e\u4e86\u76f8\u540c\u7684\u53d6\u503c":81,"\u8bbe\u7f6e\u5176\u53c2\u6570\u5c5e\u6027":82,"\u8bbe\u7f6e\u5185\u5b58\u4e2d\u6682\u5b58\u7684\u6570\u636e\u6761\u6570":2,"\u8bbe\u7f6e\u5185\u5b58\u4e2d\u6700\u5c0f\u6682\u5b58\u7684\u6570\u636e\u6761\u6570":2,"\u8bbe\u7f6e\u53c2\u6570\u7684\u540d\u5b57":82,"\u8bbe\u7f6e\u547d\u4ee4\u884c\u53c2\u6570":[80,100],"\u8bbe\u7f6e\u5b66\u4e60\u7387\u8870\u51cf\u56e0\u5b50\u5206\u6bb5\u51fd\u6570":82,"\u8bbe\u7f6e\u6210":82,"\u8bbe\u7f6e\u6210\u4e00\u4e2a\u5c0f\u4e00\u4e9b\u7684\u503c":80,"\u8bbe\u7f6e\u8f93\u51fa\u7684\u5c3a\u5bf8":96,"\u8bbe\u7f6e\u8f93\u51fatensor\u7684\u5f62\u72b6":97,"\u8bbe\u7f6e\u8fd9\u4e2apydataprovider2\u8fd4\u56de\u4ec0\u4e48\u6837\u7684\u6570\u636e":2,"\u8bbe\u7f6e\u9ed8\u8ba4\u8bbe\u5907\u53f7\u4e3a0":108,"\u8bbe\u7f6egpu":106,"\u8bbf\u95ee\u5bf9\u5e94\u7684\u7f51\u5740":102,"\u8bbf\u95eekubernetes\u7684\u63a5\u53e3\u6765\u67e5\u8be2\u6b64job\u5bf9\u5e94\u7684\u6240\u6709pod\u4fe1\u606f":111,"\u8bc4\u4f30\u8be5\u4ea7\u54c1\u7684\u8d28\u91cf":123,"\u8bc4\u5ba1\u4eba\u4e00\u822c\u4e0d\u505a\u8bc4\u5ba1":95,"\u8bc4\u5ba1\u4eba\u7684\u6bcf\u4e2a\u610f\u89c1\u90fd\u5fc5\u987b\u56de\u590d":95,"\u8bc4\u5ba1\u4eba\u9700\u8981\u9010\u4e00\u67e5\u770b\u6bcf\u4e2acommit\u624d\u80fd\u77e5\u9053\u505a\u4e86\u54ea\u4e9b\u4fee\u6539":95,"\u8bc4\u8bba\u6846\u4e2d\u52a0\u4e0a":95,"\u8bc6\u522b\u6570\u5b57":71,"\u8bcd\u5411\u91cf":[71,121],"\u8bcd\u5411\u91cf\u6a21\u578b\u540d\u79f0":121,"\u8bcd\u672c\u8eab\u548c\u8bcd\u9891":121,"\u8bd5\u7740\u8ba9\u8f93\u51fa\u7684\u5206\u6790\u6570\u636e\u548c\u7406\u8bba\u503c\u5bf9\u5e94":103,"\u8be5\u51fd\u6570\u5177\u6709\u4e24\u4e2a\u53c2\u6570":2,"\u8be5\u51fd\u6570\u5728\u521d\u59cb\u5316\u7684\u65f6\u5019\u4f1a\u88ab\u8c03\u7528":2,"\u8be5\u51fd\u6570\u7684\u529f\u80fd\u662f":2,"\u8be5\u53c2\u6570\u5728\u7f51\u7edc\u914d\u7f6e\u7684output":106,"\u8be5\u53c2\u6570\u5728\u96c6\u7fa4\u63d0\u4ea4\u73af\u5883\u4e2d\u81ea\u52a8\u8bbe\u7f6e":106,"\u8be5\u53c2\u6570\u5df2\u7ecf\u5728\u96c6\u7fa4\u63d0\u4ea4\u73af\u5883\u4e2d\u5b8c\u6210\u8bbe\u7f6e":106,"\u8be5\u53c2\u6570\u5fc5\u987b\u80fd\u88abflag":106,"\u8be5\u53c2\u6570\u6307\u793a\u662f\u5426\u6253\u5370\u65e5\u5fd7\u622a\u65ad\u4fe1\u606f":106,"\u8be5\u53c2\u6570\u6307\u793a\u662f\u5426\u6253\u5370\u9519\u8bef\u622a\u65ad\u65e5\u5fd7":106,"\u8be5\u53c2\u6570\u7528\u4e8e\u6307\u5b9a\u52a8\u6001\u5e93\u8def\u5f84":106,"\u8be5\u53c2\u6570\u7684\u610f\u601d\u662f\u8bad\u7ec3num":106,"\u8be5\u53c2\u6570\u9ed8\u8ba4\u4e3anull":106,"\u8be5\u5bf9\u8c61\u5177\u6709\u4ee5\u4e0b\u4e24\u4e2a\u5c5e\u6027":2,"\u8be5\u5c42\u4ec5\u9700\u8981\u8fd9\u4e9b\u975e\u96f6\u6837\u672c\u4f4d\u7f6e\u6240\u5bf9\u5e94\u7684\u53d8\u6362\u77e9\u9635\u7684\u90a3\u4e9b\u884c":96,"\u8be5\u5c42\u795e\u7ecf\u5143\u4e2a\u6570":123,"\u8be5\u622a\u65ad\u4f1a\u5f71\u54cd":106,"\u8be5\u6279\u6b21\u7684\u8f93\u5165\u4e2d\u4ec5\u6709\u4e00\u4e2a\u5b50\u96c6\u662f\u975e\u96f6\u7684":96,"\u8be5\u63a5\u53e3\u4f7f\u7528\u591a\u7ebf\u7a0b\u8bfb\u53d6\u6570\u636e":2,"\u8be5\u63a5\u53e3\u53ef\u7528\u4e8e\u9884\u6d4b\u548c\u5b9a\u5236\u5316\u8bad\u7ec3":83,"\u8be5\u6570\u636e\u96c6":121,"\u8be5\u6570\u76ee\u662f\u63d0\u524d\u5b9a\u4e49\u597d\u7684":106,"\u8be5\u6587\u4ef6\u662f\u7531cpickle\u4ea7\u751f\u7684":122,"\u8be5\u65f6\u95f4\u53bb\u9664\u6389\u672c\u51fd\u6570\u8c03\u7528\u5176\u4ed6\u51fd\u6570\u7684\u65f6\u95f4":102,"\u8be5\u6a21\u578b\u4f9d\u7136\u4f7f\u7528\u903b\u8f91\u56de\u5f52\u5206\u7c7b\u7f51\u7edc\u7684\u6846\u67b6":123,"\u8be5\u6a21\u578b\u7684\u8bf4\u660e\u5982\u4e0b\u56fe\u6240\u793a":93,"\u8be5\u7c7b\u7684":97,"\u8be5\u7c7b\u7684\u5b9e\u73b0\u7ec6\u8282\u5728":96,"\u8be5\u811a\u672c\u4e2d\u8bb0\u5f55\u4e86\u4ea4\u53c9\u7f16\u8bd1android\u7248paddlepaddle\u5e93\u5e38\u7528\u7684cmake\u914d\u7f6e":115,"\u8be5\u811a\u672c\u4f1a\u751f\u6210\u4e00\u4e2adot\u6587\u4ef6":122,"\u8be5\u8bed\u53e5\u4f1a\u4e3a\u6bcf\u4e2a\u5c42\u521d\u59cb\u5316\u5176\u6240\u9700\u8981\u7684\u53d8\u91cf\u548c\u8fde\u63a5":96,"\u8be5layer\u662f\u901a\u8fc7\u53c2\u6570":81,"\u8be6\u7ec6\u4ecb\u7ecd\u53ef\u4ee5\u53c2\u8003":90,"\u8be6\u7ec6\u4ecb\u7ecd\u8bf7\u53c2\u8003\u8bbe\u8ba1\u6587\u6863":97,"\u8be6\u7ec6\u4fe1\u606f\u8bf7\u68c0\u67e5":104,"\u8be6\u7ec6\u5185\u5bb9\u8bf7\u53c2\u89c1":123,"\u8be6\u7ec6\u53c2\u8003":83,"\u8be6\u7ec6\u53ef\u53c2\u8003":95,"\u8be6\u7ec6\u5730\u5c55\u793a\u4e86\u6574\u4e2a\u7279\u5f81\u63d0\u53d6\u7684\u8fc7\u7a0b":122,"\u8be6\u7ec6\u6587\u6863\u53c2\u8003":80,"\u8be6\u7ec6\u7684\u53c2\u6570\u89e3\u91ca":123,"\u8be6\u7ec6\u7684cmake\u4f7f\u7528\u65b9\u6cd5\u53ef\u4ee5\u53c2\u8003":83,"\u8be6\u7ec6\u89c1":89,"\u8be6\u7ec6\u8bbe\u8ba1":44,"\u8bed\u610f\u89d2\u8272\u6807\u6ce8":71,"\u8bed\u8a00\u6a21\u578b":121,"\u8bed\u8a00\u91cd\u6784\u540e\u7684":102,"\u8bf4\u660e":[34,83,86,104,111],"\u8bf4\u660e\u63d0\u4ea4\u7684\u4ee3\u7801\u5b58\u5728\u95ee\u9898":95,"\u8bf4\u660e\u8fd9\u4e2a\u5c42\u7684\u8f93\u5165":96,"\u8bf7\u4e0d\u8981\u521b\u5efa\u7a7a\u7684":97,"\u8bf7\u4e0d\u8981\u5fd8\u8bb0\u63d0\u524d\u5728\u7269\u7406\u673a\u4e0a\u5b89\u88c5gpu\u6700\u65b0\u9a71\u52a8":84,"\u8bf7\u4fdd\u8bc1travi":95,"\u8bf7\u5148\u4f7f\u7528":[115,116,117],"\u8bf7\u53c2\u7167\u7f51\u7edc\u914d\u7f6e\u7684\u6587\u6863\u4e86\u89e3\u66f4\u8be6\u7ec6\u7684\u4fe1\u606f":108,"\u8bf7\u53c2\u8003":[2,55,77,80,87,90,96,97,123],"\u8bf7\u53c2\u8003\u5982\u4e0b\u8868\u683c":123,"\u8bf7\u53c2\u89c1":95,"\u8bf7\u53c2\u9605":93,"\u8bf7\u5728\u8be5pull":95,"\u8bf7\u60a8\u6bcf\u6b21\u63d0\u4ea4\u4ee3\u7801\u65f6":95,"\u8bf7\u60a8\u9075\u5b88\u4ee5\u4e0b\u7ea6\u5b9a":95,"\u8bf7\u6307\u5b9a\u7684paddlepaddle\u5de5\u4f5c\u76ee\u5f55\u7ed9\u73af\u5883\u53d8\u91cf":99,"\u8bf7\u6307\u5b9a\u8be5\u76ee\u5f55":106,"\u8bf7\u663e\u793a\u5730\u8c03\u7528":97,"\u8bf7\u67e5\u770b":121,"\u8bf7\u68c0\u67e5python\u7248\u672c\u662f\u5426\u4e3a2":86,"\u8bf7\u6ce8\u610f":[93,97,110,121],"\u8bf7\u6ce8\u610f\u6bcf\u4e2acommit\u7684\u540d\u79f0":95,"\u8bf7\u6ce8\u610f\u8fd9\u4e2a\u547d\u4ee4\u7ed3\u5c3e\u5904\u7684":94,"\u8bf7\u6ce8\u610fcommit\u7684\u6570\u91cf":95,"\u8bf7\u76f4\u63a5\u586b\u51450":82,"\u8bf7\u770b\u4e0b\u9762\u7684\u4f8b\u5b50":108,"\u8bf7\u786e\u4fdd":95,"\u8bf7\u7ed9\u51fa\u603b\u4f53\u7684\u4fee\u6539\u60c5\u51b5":95,"\u8bf7\u7ed9\u51fa\u60a8\u81ea\u5df1\u7684\u53cd\u9a73\u7406\u7531":95,"\u8bf7\u9009\u62e9\u5408\u9002\u7684\u8bcd\u6c47":95,"\u8bf7\u9009\u62e9\u6b63\u786e\u7684\u7248\u672c":77,"\u8bf7\u9075\u5b88":95,"\u8bf7\u91c7\u7528":95,"\u8bf8\u5982\u56fe\u50cf\u5206\u7c7b":108,"\u8bfb\u53d6\u6570\u636e":2,"\u8bfb\u53d6\u6bcf\u4e00\u884c":2,"\u8bfb\u53d6volume\u4e2d\u7684\u6570\u636e\u8fdb\u884c\u8fd9\u6b21\u5206\u5e03\u5f0f\u8bad\u7ec3":111,"\u8bfb\u8005\u53ef\u4ee5\u67e5\u770b":111,"\u8bfb\u8005\u9700\u8981\u66ff\u6362\u6210\u81ea\u5df1\u4f7f\u7528\u7684\u4ed3\u5e93\u5730\u5740":111,"\u8c03\u7528":[96,97,116],"\u8c03\u7528\u4e00\u6b21":2,"\u8c03\u7528\u5bf9\u5e94":98,"\u8c03\u7528\u65b9\u6cd5\u89c1c":[115,116],"\u8c03\u7528\u7528":102,"\u8c03\u7528\u7684\u4e00\u4e9b\u7528\u6237\u5b9a\u4e49\u7684\u5e93\u51fd\u6570":104,"\u8c03\u7528\u7684\u51fd\u6570\u662f\u5426\u652f\u6301\u4e0d\u540c\u8bbe\u5907":97,"\u8c03\u7528\u7684pydataprovider2\u662f":2,"\u8c03\u7528\u7b2c\u4e8c\u6b21\u7684\u65f6\u5019":2,"\u8c03\u7528\u8be5\u51fd\u6570\u540e":96,"\u8c03\u7528\u8fd9\u4e2apydataprovider2\u7684\u65b9\u6cd5":2,"\u8d1f\u6837\u672c":123,"\u8d21\u732e\u6587\u6863":99,"\u8d44\u6e90\u5bf9\u8c61\u7684\u540d\u5b57\u662f\u552f\u4e00\u7684":109,"\u8d77":90,"\u8def\u5f84\u4e0b":122,"\u8df3\u8f6c\u5230":95,"\u8df3\u8fc7":80,"\u8f6c\u6362\u751f\u6210\u7684\u6587\u4ef6\u540d\u4f1a\u662f\u4ee5\u4e0b\u683c\u5f0f":35,"\u8f6c\u6362\u8fc7\u6765\u7684":122,"\u8f83":90,"\u8f93\u5165":[89,93],"\u8f93\u5165\u4e86\u6027\u80fd\u5206\u6790\u7ed3\u679c":102,"\u8f93\u5165\u548c\u8f93\u51fa\u90fd\u662f\u5355\u5c42\u5e8f\u5217":92,"\u8f93\u5165\u548c\u8f93\u51fa\u90fd\u662f\u53cc\u5c42\u5e8f\u5217":92,"\u8f93\u5165\u5c42\u5c3a\u5bf8":122,"\u8f93\u5165\u5e8f\u5217\u4e2d\u5143\u7d20\u7684\u603b\u6570":80,"\u8f93\u5165\u6570\u636e\u4e3a\u4e00\u4e2a\u5b8c\u6574\u7684\u65f6\u95f4\u5e8f\u5217":90,"\u8f93\u5165\u6570\u636e\u4e3a\u5728\u5355\u5c42rnn\u6570\u636e\u91cc\u9762":90,"\u8f93\u5165\u6570\u636e\u6574\u4f53\u4e0a\u662f\u4e00\u4e2a\u65f6\u95f4\u5e8f\u5217":90,"\u8f93\u5165\u6570\u636e\u7684\u5b57\u5178\u7ef4\u6570\u662f1\u767e\u4e07":108,"\u8f93\u5165\u6587\u672c":121,"\u8f93\u5165\u6587\u672c\u4e2d\u6ca1\u6709\u5934\u90e8":121,"\u8f93\u5165\u662f\u5426\u662f\u8f6c\u7f6e\u7684":96,"\u8f93\u5165\u662f\u7531\u4e00\u4e2alist\u4e2d\u7684\u7f51\u7edc\u5c42\u5b9e\u4f8b\u7684\u540d\u5b57\u7ec4\u6210\u7684":96,"\u8f93\u5165\u7279\u5f81\u56fe\u7684\u901a\u9053\u6570\u76ee":122,"\u8f93\u5165\u7684":121,"\u8f93\u5165\u7684\u540d\u5b57":96,"\u8f93\u5165\u7684\u5927\u5c0f":96,"\u8f93\u5165\u7684\u6587\u672c\u683c\u5f0f\u5982\u4e0b":121,"\u8f93\u5165\u7684\u6587\u672c\u8bcd\u5411\u91cf\u6a21\u578b\u540d\u79f0":121,"\u8f93\u5165\u7684\u7c7b\u578b":96,"\u8f93\u5165n\u4e2a\u5355\u8bcd":123,"\u8f93\u51fa":[89,93,97],"\u8f93\u51fa\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":92,"\u8f93\u51fa\u4e00\u4e2a\u53cc\u5c42\u5e8f\u5217":92,"\u8f93\u51fa\u4e3an\u4e2aword_dim\u7ef4\u5ea6\u5411\u91cf":123,"\u8f93\u51fa\u51fd\u6570":93,"\u8f93\u51fa\u5e8f\u5217\u7684\u7c7b\u578b":89,"\u8f93\u51fa\u5e8f\u5217\u7684\u8bcd\u8bed\u6570\u548c\u8f93\u5165\u5e8f\u5217\u4e00\u81f4":92,"\u8f93\u51fa\u6587\u4ef6\u7684\u683c\u5f0f\u8bf4\u660e":121,"\u8f93\u51fa\u67092\u5217":121,"\u8f93\u51fa\u7279\u5f81\u56fe\u7684\u901a\u9053\u6570\u76ee":122,"\u8f93\u51fa\u7684\u4e8c\u8fdb\u5236\u8bcd\u5411\u91cf\u6a21\u578b\u540d\u79f0":121,"\u8f93\u51fa\u7684\u6587\u672c\u6a21\u578b\u540d\u79f0":121,"\u8f93\u51fa\u7684\u68af\u5ea6":106,"\u8f93\u51fa\u76ee\u5f55":122,"\u8f93\u51fa\u7ed3\u679c\u53ef\u80fd\u4f1a\u968f\u7740\u5bb9\u5668\u7684\u6d88\u8017\u800c\u88ab\u5220\u9664":110,"\u8fbe\u5230\u5bb9\u707e\u7684\u76ee\u7684":34,"\u8fc7\u4e86\u4e00\u4e2a\u5f88\u7b80\u5355\u7684recurrent_group":90,"\u8fc7\u5b8c\u6240\u6709\u8bad\u7ec3\u6570\u636e\u5373\u4e3a\u4e00\u4e2apass":80,"\u8fd0\u884c\u4e00\u4e2a":94,"\u8fd0\u884c\u4e0b\u9762\u547d\u4ee4\u53ef\u4ee5\u8fdb\u884c\u7f16\u8bd1":97,"\u8fd0\u884c\u4ee5\u4e0b\u7684\u547d\u4ee4\u4e0b\u8f7d\u548c\u83b7\u53d6\u6211\u4eec\u7684\u5b57\u5178\u548c\u9884\u8bad\u7ec3\u6a21\u578b":121,"\u8fd0\u884c\u4ee5\u4e0b\u7684\u547d\u4ee4\u4e0b\u8f7d\u6570\u636e\u96c6":121,"\u8fd0\u884c\u5355\u5143\u6d4b\u8bd5":94,"\u8fd0\u884c\u5355\u5143\u6d4b\u8bd5\u6d4b\u65f6\u9700\u8981\u7f16\u8bd1\u6574\u4e2a\u5de5\u7a0b":97,"\u8fd0\u884c\u5931\u8d25":108,"\u8fd0\u884c\u5b8c\u4ee5\u4e0a\u547d\u4ee4":121,"\u8fd0\u884c\u5b8c\u6210\u540e":104,"\u8fd0\u884c\u5b8c\u6bd5\u540e\u8f93\u51fa":102,"\u8fd0\u884c\u6027\u80fd\u5206\u6790\u7684\u65f6\u5019":102,"\u8fd0\u884c\u6210\u529f\u4ee5\u540e":121,"\u8fd0\u884c\u65e5\u5fd7":104,"\u8fd0\u884c\u65f6\u4e5f\u53ef\u80fd\u56e0\u4e3a\u591a\u7ebf\u7a0b\u4ea7\u751f\u6df7\u4e71\u4e0d\u53ef\u8bfb\u7684\u6027\u80fd\u5206\u6790\u7ed3\u679c":102,"\u8fd0\u884c\u65f6\u4f1a\u81ea\u52a8\u627e\u5230\u7cfb\u7edf\u4e2d\u5b89\u88c5\u7684cuda\u548ccudnn\u5e93\u8fdb\u884c\u7f16\u8bd1\u548c\u6267\u884c":83,"\u8fd0\u884c\u7684\u4e00\u4e9b\u53c2\u6570\u901a\u8fc7\u8fd9\u79cd\u65b9\u5f0f\u4f20\u9012\u5230\u5bb9\u5668\u5185":111,"\u8fd0\u884c\u8be5\u7f16\u8bd1\u5de5\u5177\u94fe\u9700\u8981\u4e00\u53f0":117,"\u8fd1":90,"\u8fd4\u56de":[7,8,9,13,14,15,21,23,28],"\u8fd4\u56de0":2,"\u8fd4\u56de\u4e00\u6761\u5b8c\u6574\u7684\u6837\u672c":2,"\u8fd4\u56de\u65f6":2,"\u8fd4\u56de\u7684\u662f":[2,87],"\u8fd4\u56de\u7684\u987a\u5e8f\u9700\u8981\u548cinput_types\u4e2d\u5b9a\u4e49\u7684\u987a\u5e8f\u4e00\u81f4":2,"\u8fd4\u56de\u7b2c\u4e8c\u6b65":71,"\u8fd4\u56de\u7b2ci\u4e2a\u8f93\u5165\u77e9\u9635":96,"\u8fd4\u56de\u7c7b\u578b":[7,8,9,13,14,15,21,23,28],"\u8fd4\u56depython\u7aef\u7684\u8ba1\u7b97\u7ed3\u679c":97,"\u8fd8\u4f1a":90,"\u8fd8\u4f1a\u4e0b\u8f7dmkl":83,"\u8fd8\u4f1a\u5f3a\u5236\u8bbe\u7f6e\u4e00\u4e9bpaddlepaddle\u53c2\u6570\u7684\u503c":115,"\u8fd8\u4f1a\u8f93\u51fa\u4e00\u4e2a":95,"\u8fd8\u53ef\u4ee5\u901a\u8fc7\u51cf\u5c0f\u5b66\u4e60\u7387\u6216\u8005\u5bf9\u6570\u636e\u8fdb\u884c\u5f52\u4e00\u5316\u5904\u7406\u6765\u89e3\u51b3\u8fd9\u7c7b\u95ee\u9898":80,"\u8fd8\u662f":90,"\u8fd8\u662f\u4ece":35,"\u8fd8\u662f\u865a\u62df\u673a":94,"\u8fd8\u6709":90,"\u8fd8\u9700\u8981\u5728\u8282\u70b9\u4e0a\u5b89\u88c5\u5bf9\u5e94\u7684gpu\u9a71\u52a8\u4ee5\u53cacuda":104,"\u8fd8\u9700\u8981\u91cd\u5199":97,"\u8fd9":[80,90,123],"\u8fd98\u79cdlearning_rate_schedule\u53ca\u5176\u5bf9\u5e94\u5b66\u4e60\u7387\u8ba1\u7b97\u65b9\u5f0f\u5982\u4e0b":82,"\u8fd9\u4e00\u5757\u7684\u8017\u65f6\u6bd4\u4f8b\u771f\u7684\u592a\u9ad8":103,"\u8fd9\u4e00\u5c42\u8fdb\u884c\u5c01\u88c5":55,"\u8fd9\u4e00\u6982\u5ff5\u4e0d\u518d\u7410\u788e":55,"\u8fd9\u4e00\u8ba1\u7b97\u5355\u5143":81,"\u8fd9\u4e00\u8fc7\u7a0b\u5bf9\u7528\u6237\u662f\u5b8c\u5168\u900f\u660e\u7684":92,"\u8fd9\u4e09\u4e2a\u5206\u652f":71,"\u8fd9\u4e09\u4e2a\u6b65\u9aa4\u53ef\u914d\u7f6e\u4e3a":123,"\u8fd9\u4e24\u4e2a\u6307\u6807\u4ee3\u8868\u4e86\u67d0\u4e00\u4e2a\u51fd\u6570\u771f\u5b9e\u7684\u8fd0\u884c\u65f6\u95f4":102,"\u8fd9\u4e2a":[86,90,94,109],"\u8fd9\u4e2a\u4efb\u52a1\u7684\u914d\u7f6e\u4e3a":80,"\u8fd9\u4e2a\u4efb\u52a1\u7684dataprovider\u4e3a":80,"\u8fd9\u4e2a\u4f8b\u5b50\u6709\u4e24\u5904\u4e0d\u540c":97,"\u8fd9\u4e2a\u51fd\u6570\u7684":93,"\u8fd9\u4e2a\u51fd\u6570\u8fdb\u884c\u53d8\u6362":90,"\u8fd9\u4e2a\u51fd\u6570\u9700\u8981\u8bbe\u7f6e":93,"\u8fd9\u4e2a\u52a8\u6001\u5e93\u7684\u8fde\u63a5\u53c2\u6570\u4e0epaddle\u7684\u5176\u4ed6\u4e8c\u8fdb\u5236":55,"\u8fd9\u4e2a\u53c2\u6570\u4e5f\u4e0d\u4f1a\u4e00\u5e76\u5220\u9664":55,"\u8fd9\u4e2a\u5730\u5740\u5219\u4e3a\u5b83\u7684\u7edd\u5bf9\u8def\u5f84\u6216\u76f8\u5bf9\u8def\u5f84":1,"\u8fd9\u4e2a\u5730\u5740\u6765\u8868\u793a\u6b64\u6b65\u9aa4\u6240\u6784\u5efa\u51fa\u7684\u955c\u50cf":111,"\u8fd9\u4e2a\u57fa\u7c7b":96,"\u8fd9\u4e2a\u5934\u6587\u4ef6\u4e0d\u5047\u8bbe\u5176\u4ed6\u6587\u4ef6\u7684\u5f15\u7528\u987a\u5e8f":55,"\u8fd9\u4e2a\u5e8f\u5217\u7684\u6bcf\u4e2a\u5143\u7d20\u53c8\u662f\u4e00\u4e2a\u5e8f\u5217":92,"\u8fd9\u4e2a\u60c5\u51b5\u4e0b\u6240\u6709\u7684\u6587\u4ef6\u4f1a\u5b58\u5728\u6574\u7406\u8fc7\u7684\u7684\u6587\u4ef6\u76ee\u5f55":99,"\u8fd9\u4e2a\u63a5\u53e3\u9700\u8981\u505a\u5230":54,"\u8fd9\u4e2a\u6570\u636e\u4e5f\u88ab\u5355\u5c42rnn\u7f51\u7edc\u76f4\u63a5\u4f7f\u7528":90,"\u8fd9\u4e2a\u6587\u4ef6\u5177\u6709\u72ec\u7279\u7684\u8bed\u6cd5":54,"\u8fd9\u4e2a\u662f\u76ee\u524d\u63a8\u8350\u7684\u4f7f\u7528\u65b9\u6cd5":99,"\u8fd9\u4e2a\u663e\u793a\u5668\u5f88\u68d2":123,"\u8fd9\u4e2a\u73af\u5883\u53d8\u91cf\u5173\u95edopenmp\u4f18\u5316":102,"\u8fd9\u4e2a\u76ee\u5f55\u4e2d\u9664\u4e86":55,"\u8fd9\u4e2a\u795e\u7ecf\u7f51\u7edc\u5355\u5143\u5c31\u53ebmemori":90,"\u8fd9\u4e2a\u7c7b\u7684\u53c2\u6570\u5305\u62ec":96,"\u8fd9\u4e2a\u7c7b\u9700\u8981\u7ee7\u627f":96,"\u8fd9\u4e2a\u7ed3\u6784\u4f53\u4e2d\u7684\u53e6\u4e00\u4e2a\u9879\u76ee\u662f":55,"\u8fd9\u4e2a\u7ed3\u6784\u4f53\u5305\u542b\u4e24\u4e2a\u9879\u76ee":55,"\u8fd9\u4e2a\u811a\u672c\u8c03\u7528":94,"\u8fd9\u4e2a\u8282\u70b9\u53ef\u4ee5\u662f\u7269\u7406\u673a\u6216\u8005\u865a\u62df\u673a":109,"\u8fd9\u4e2a\u8868\u683c":109,"\u8fd9\u4e2a\u8f93\u5165\u4e0d\u53c2\u4e0e":97,"\u8fd9\u4e2a\u8fc7\u7a0b\u5bf9\u7528\u6237\u4e5f\u662f\u900f\u660e\u7684":92,"\u8fd9\u4e2a\u8fc7\u7a0b\u9664\u4e86\u7f16\u8bd1paddlepaddle\u4e3a":95,"\u8fd9\u4e2a\u914d\u7f6e\u4e0e":121,"\u8fd9\u4e2a\u914d\u7f6e\u6587\u4ef6":109,"\u8fd9\u4e2a\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u751f\u6210\u4e00\u7cfb\u5217\u6743\u91cd":93,"\u8fd9\u4e2a\u95ee\u9898\u662fpydataprovider\u8bfb\u6570\u636e\u65f6\u5019\u7684\u903b\u8f91\u95ee\u9898":2,"\u8fd9\u4e2a\u9759\u6001\u5e93\u5305\u542b\u4e86paddle\u7684\u5168\u90e8\u7b26\u53f7":55,"\u8fd9\u4e2adataprovider\u8f83\u590d\u6742":2,"\u8fd9\u4e2ainstance\u53ef\u4ee5\u662f\u5355\u4e2a\u503c":35,"\u8fd9\u4e2aissu":94,"\u8fd9\u4e2ajob\u624d\u7b97\u6210\u529f\u7ed3\u675f":111,"\u8fd9\u4e2alayer\u7684\u8f93\u51fa\u4f1a\u4f5c\u4e3a\u6574\u4e2a":92,"\u8fd9\u4e5f\u4f1a\u6781\u5927\u51cf\u5c11\u6570\u636e\u8bfb\u5165\u7684\u8017\u65f6":80,"\u8fd9\u4e9b\u4f8b\u5b50\u90fd\u53ef\u4ee5\u5728":104,"\u8fd9\u4e9b\u51fd\u6570\u4f1a\u5c06\u5bf9\u5e94\u5185\u5bb9\u6dfb\u52a0\u5230":97,"\u8fd9\u4e9b\u5206\u5e03\u5f0f\u5b58\u50a8\u670d\u52a1\u901a\u5e38\u4f1a\u628a\u6570\u636e\u5207\u5272\u6210\u591a\u4e2a\u5206\u7247\u5206\u5e03\u5f0f\u7684\u5b58\u50a8\u5728\u591a\u4e2a\u8282\u70b9\u4e4b\u4e0a":35,"\u8fd9\u4e9b\u53c2\u6570\u53ef\u4ee5\u901a\u8fc7\u73af\u5883\u53d8\u91cf":104,"\u8fd9\u4e9b\u53c2\u6570\u7684\u5177\u4f53\u63cf\u8ff0":111,"\u8fd9\u4e9b\u540d\u5b57\u5fc5\u987b\u8981\u5199\u5bf9":96,"\u8fd9\u4e9b\u6570\u636e\u4f1a\u88ab\u7528\u6765\u66f4\u65b0\u53c2\u6570":80,"\u8fd9\u4e9b\u6570\u636e\u4f7f\u7528\u7684\u5185\u5b58\u4e3b\u8981\u548c\u4e24\u4e2a\u53c2\u6570\u6709\u5173\u7cfb":80,"\u8fd9\u4e9b\u6587\u4ef6\u5c06\u4f1a\u88ab\u4fdd\u5b58\u5728":122,"\u8fd9\u4e9b\u6a21\u578b\u90fd\u662f\u7531\u539f\u4f5c\u8005\u63d0\u4f9b\u7684\u6a21\u578b":122,"\u8fd9\u4e9b\u7279\u5f81\u503c\u4e0e\u4e0a\u8ff0\u4f7f\u7528c":122,"\u8fd9\u4e9b\u7279\u5f81\u6570\u636e\u4e4b\u95f4\u7684\u987a\u5e8f\u662f\u6709\u610f\u4e49\u7684":90,"\u8fd9\u4efd\u6559\u7a0b\u5c55\u793a\u4e86\u5982\u4f55\u5728paddlepaddle\u4e2d\u5b9e\u73b0\u4e00\u4e2a\u81ea\u5b9a\u4e49\u7684\u7f51\u7edc\u5c42":96,"\u8fd9\u4f1a\u63d0\u793a\u5f53\u524d\u76ee\u5f55\u7684\u4e00\u4e9b\u53d8\u5316":95,"\u8fd9\u4f1a\u7ed9\u8bc4\u5ba1\u4eba\u5e26\u6765\u5f88\u5927\u56f0\u6270":95,"\u8fd9\u4f1a\u81ea\u52a8\u8fdb\u884c\u7f51\u7edc\u914d\u7f6e\u4e2d\u58f0\u660e\u7684\u6fc0\u6d3b\u64cd\u4f5c":96,"\u8fd9\u4fbf\u662f\u4e00\u79cd\u53cc\u5c42rnn\u7684\u8f93\u5165\u6570\u636e":90,"\u8fd9\u51e0\u4e2a\u7f16\u8bd1\u9009\u9879\u7684\u8bbe\u7f6e":83,"\u8fd9\u53e5\u8868\u793a\u4f7f\u7528\u57fa\u7c7b":97,"\u8fd9\u53ef\u4ee5\u5e2e\u60a8\u7701\u6389\u82b1\u4e00\u5c0f\u65f6\u5b89\u88c5\u548c\u914d\u7f6e\u5404\u79cd\u5f00\u53d1\u5de5\u5177":94,"\u8fd9\u53ef\u4ee5\u8ba9\u5176\u4ed6\u4eba\u77e5\u9053\u8fd9\u6b21\u63d0\u4ea4\u505a\u4e86\u54ea\u4e9b\u6539\u53d8":95,"\u8fd9\u53ef\u4ee5\u901a\u8fc7":95,"\u8fd9\u548c\u5355\u5c42rnn\u7684\u914d\u7f6e\u662f\u7b49\u4ef7\u7684":90,"\u8fd9\u56db\u6761\u6570\u636e\u540c\u65f6\u5904\u7406\u7684\u53e5\u5b50\u6570\u91cf\u4e3a":90,"\u8fd9\u5728\u6784\u9020\u975e\u5e38\u590d\u6742\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u65f6\u662f\u6709\u7528\u7684":93,"\u8fd9\u5bf9\u4e8e\u901a\u5e38\u7684java\u7684\u5f00\u53d1\u8005\u6765\u8bf4":54,"\u8fd9\u5c06\u4f1a\u5bfc\u81f4\u5355\u5143\u6d4b\u8bd5\u51fa\u9519":97,"\u8fd9\u5c06\u4f1a\u5bfc\u81f4\u7f16\u8bd1\u51fa\u9519":97,"\u8fd9\u610f\u5473\u7740":93,"\u8fd9\u610f\u5473\u7740\u9664\u4e86\u6307\u5b9adevic":108,"\u8fd9\u65f6":80,"\u8fd9\u65f6\u5728\u4f7f\u7528":82,"\u8fd9\u65f6\u8fdb\u884c\u77e9\u9635\u4e58\u6cd5\u8fd0\u7b97\u5c31\u53ef\u80fd\u5bfc\u81f4\u6d6e\u70b9\u6570\u6ea2\u51fa":80,"\u8fd9\u662f\u4e00\u79cd\u6309\u5df2\u8bad\u7ec3\u6837\u672c\u6570\u5206\u6bb5\u53d6\u503c\u7684\u5b66\u4e60\u7387\u9000\u706b\u65b9\u6cd5":82,"\u8fd9\u662f\u4e00\u79cd\u6309\u5df2\u8bad\u7ec3pass\u6570\u5206\u6bb5\u53d6\u503c\u7684\u5b66\u4e60\u7387\u9000\u706b\u65b9\u6cd5":82,"\u8fd9\u662f\u4e00\u79cd\u975e\u5e38\u7075\u6d3b\u7684\u6570\u636e\u7ec4\u7ec7\u65b9\u5f0f":89,"\u8fd9\u662f\u56e0\u4e3a":54,"\u8fd9\u662f\u5f00\u6e90\u793e\u533a\u7684\u57fa\u672c\u793c\u8c8c":95,"\u8fd9\u662f\u666e\u901a\u7684\u5355\u5c42\u65f6\u95f4\u5e8f\u5217\u7684dataprovider\u4ee3\u7801":90,"\u8fd9\u662f\u76ee\u524dcmake\u5bfb\u627epython\u7684\u903b\u8f91\u5b58\u5728\u7f3a\u9677":77,"\u8fd9\u6837":[55,104],"\u8fd9\u6837\u4fdd\u5b58\u5728\u5206\u5e03\u5f0f\u5b58\u50a8\u4e2d\u7684\u6570\u636e\u53ef\u4ee5\u88ab\u96c6\u7fa4\u4e2d\u7684\u6bcf\u4e2a\u8282\u70b9\u8bfb\u53d6\u5230":104,"\u8fd9\u6837\u4fdd\u8bc1":71,"\u8fd9\u6837\u4fdd\u8bc1\u8fd0\u884c\u7ed3\u675f\u4e4b\u540e\u7684":94,"\u8fd9\u6837\u505a\u53ef\u4ee5\u6781\u5927\u7684\u51cf\u5c11\u5185\u5b58\u5360\u7528":80,"\u8fd9\u6837\u53ef\u4ee5\u514d\u53bb\u5355\u72ec\u5b89\u88c5\u7f16\u8bd1\u4f9d\u8d56\u7684\u6b65\u9aa4":83,"\u8fd9\u6837\u53ef\u4ee5\u51cf\u5c0fgpu\u5185\u5b58":108,"\u8fd9\u6837\u5982\u679c\u9047\u5230\u95ee\u9898":94,"\u8fd9\u6837\u5bb9\u5668\u7684":111,"\u8fd9\u6837\u5c31\u53ef\u4ee5\u5728\u4e91\u7aef\u6267\u884c\u591a\u79cd\u6570\u636e\u7c7b\u8ba1\u7b97\u4efb\u52a1":35,"\u8fd9\u6837\u5df2\u7ecf\u4f20\u8f93\u6210\u529f\u7684\u90e8\u5206\u5c31\u4e0d\u7528\u91cd\u65b0\u4f20\u8f93\u4e86":44,"\u8fd9\u6837\u5f53\u8be5pull":95,"\u8fd9\u6837\u6781\u5927\u5730\u63d0\u9ad8\u4e86\u8ba1\u7b97\u7684\u5e76\u884c\u6027":104,"\u8fd9\u6837\u7684\u88c5\u9970\u5668":96,"\u8fd9\u6837\u7684\u8bdd":110,"\u8fd9\u6837\u8bad\u7ec3\u6587\u4ef6\u7684\u4e2a\u6570\u4f1a\u6bd4\u8f83\u591a":104,"\u8fd9\u6b63\u662f\u5b83\u4eec\u901f\u5ea6\u5feb\u7684\u539f\u56e0":103,"\u8fd9\u7528\u4e8e\u5728\u591a\u7ebf\u7a0b\u548c\u591a\u673a\u4e0a\u66f4\u65b0\u53c2\u6570":96,"\u8fd9\u79cd\u521d\u59cb\u5316\u65b9\u5f0f\u5728\u4e00\u822c\u60c5\u51b5\u4e0b\u4e0d\u4f1a\u4ea7\u751f\u5f88\u5dee\u7684\u7ed3\u679c":82,"\u8fd9\u79cd\u60c5\u51b5\u4e0b\u4e0d\u9700\u8981\u91cd\u5199\u8be5\u51fd\u6570":96,"\u8fd9\u79cd\u60c5\u51b5\u5e38\u5e38\u53d1\u751f\u5728":80,"\u8fd9\u79cd\u65b9\u5f0f\u5bf9\u5185\u5b58\u6d88\u8017\u8f83\u5927":81,"\u8fd9\u79cd\u65b9\u5f0f\u5fc5\u987b\u4f7f\u7528paddle\u5b58\u50a8\u7684\u6a21\u578b\u8def\u5f84\u683c\u5f0f":108,"\u8fd9\u79cd\u751f\u6210\u6280\u672f\u53ea\u7528\u4e8e\u7c7b\u4f3c\u89e3\u7801\u5668\u7684\u751f\u6210\u8fc7\u7a0b":93,"\u8fd9\u79cd\u7c7b\u578b\u7684\u8f93\u5165\u5fc5\u987b\u901a\u8fc7":92,"\u8fd9\u79cd\u96c6\u7fa4\u8282\u70b9\u7ba1\u7406\u65b9\u5f0f\u4f1a\u5728\u5c06\u6765\u4f7f\u7528":111,"\u8fd9\u7bc7":84,"\u8fd9\u7bc7\u6587\u6863":95,"\u8fd9\u7bc7\u6587\u6863\u4ecb\u7ecd\u5728":117,"\u8fd9\u7bc7\u6587\u6863\u4ecb\u7ecd\u57fa\u4e8e":94,"\u8fd9\u7bc7\u6587\u7ae0":94,"\u8fd9\u7ec4\u8bed\u4e49\u76f8\u540c\u7684\u793a\u4f8b\u914d\u7f6e\u5982\u4e0b":90,"\u8fd9\u884c\u547d\u4ee4\u4e2d":102,"\u8fd9\u901a\u8fc7\u83b7\u5f97\u53cd\u5411\u5faa\u73af\u7f51\u7edc\u7684\u7b2c\u4e00\u4e2a\u5b9e\u4f8b":93,"\u8fd9\u90fd\u9700\u8981\u8fd9\u4e2a\u63a5\u53e3\u6309\u7167\u7ea6\u5b9a\u4fd7\u6210\u7684\u89c4\u5219\u6765\u6ce8\u91ca\u5b8c\u5907":54,"\u8fd9\u91cc":[82,83,93,95,109,111,122],"\u8fd9\u91cc\u4e0d\u518d\u8d58\u8ff0":97,"\u8fd9\u91cc\u4ee5":123,"\u8fd9\u91cc\u4f7f\u7528\u4e86\u7528":102,"\u8fd9\u91cc\u4f7f\u7528\u4e86paddlepaddle\u9884\u5b9a\u4e49\u597d\u7684rnn\u5904\u7406\u51fd\u6570":90,"\u8fd9\u91cc\u4f7f\u7528\u7b80\u5355\u7684":80,"\u8fd9\u91cc\u53ea\u7b80\u5355\u4ecb\u7ecd\u4e86\u5355\u673a\u8bad\u7ec3":123,"\u8fd9\u91cc\u5c06\u4ecb\u7ecdpaddlepaddle\u7684\u57fa\u672c\u4f7f\u7528\u6982\u5ff5":87,"\u8fd9\u91cc\u6211\u4eec\u5c55\u793a\u4e00\u4efd\u7b80\u5316\u8fc7\u7684\u4ee3\u7801":96,"\u8fd9\u91cc\u6211\u4eec\u901a\u8fc7\u5728kubernetes\u96c6\u7fa4\u4e0a\u542f\u52a8\u4e00\u4e2ajob\u6765\u4e0b\u8f7d\u5e76\u5207\u5272\u6570\u636e":111,"\u8fd9\u91cc\u6307\u5b9a\u8bcd\u5178":123,"\u8fd9\u91cc\u6709\u4e24\u79cd\u6709\u6548\u7684\u89e3\u51b3\u65b9\u6cd5":80,"\u8fd9\u91cc\u68c0\u9a8c\u8fd0\u884c\u65f6\u95f4\u6a21\u578b\u7684\u6536\u655b":104,"\u8fd9\u91cc\u7684eigentensor\u4e4b\u95f4\u7684\u8fd0\u7b97\u53ea\u662f\u6539\u53d8\u4e86\u539f\u6709tensor\u4e2d\u7684\u6570\u636e":98,"\u8fd9\u91cc\u76f4\u63a5\u901a\u8fc7\u9884\u6d4b\u811a\u672c":123,"\u8fd9\u91cc\u7ed9\u51fa\u96c6\u4e2d\u5e38\u89c1\u7684\u90e8\u7f72\u65b9\u6cd5":109,"\u8fd9\u91cc\u91c7\u7528adam\u4f18\u5316\u65b9\u6cd5":123,"\u8fd9\u91cc\u9700\u8981\u7528\u6237\u989d\u5916\u6ce8\u610f":34,"\u8fd9\u9700\u8981\u8054\u5408\u6211\u4eec\u7b2c\u4e8c\u8282":102,"\u8fdb\u4e3b\u4ed3\u5e93\u540e":95,"\u8fdb\u5165\u5bb9\u5668":110,"\u8fdb\u7a0b\u542f\u52a8\u7684\u5fc5\u8981\u53c2\u6570":111,"\u8fdb\u7a0b\u7684":104,"\u8fdb\u7a0b\u7684\u542f\u52a8\u53c2\u6570":111,"\u8fdb\u7a0b\u7684\u8fd0\u884c\u73af\u5883":111,"\u8fdb\u7a0b\u9700\u8981\u7684":111,"\u8fdb\u800c\u591a\u673a":102,"\u8fdb\u800c\u6211\u4eec\u53ef\u4ee5\u4f7f\u7528\u5982\u4e0b\u547d\u4ee4\u5f00\u542f\u4e00\u4e2ahttp\u670d\u52a1":102,"\u8fdb\u800c\u6307\u5b9a\u4e86python\u53ef\u6267\u884c\u6587\u4ef6\u7684\u8def\u5f84":102,"\u8fdb\u800c\u8fdb\u884c\u4ee3\u7801\u8bc4\u5ba1":71,"\u8fdb\u884c\u4e86":90,"\u8fdb\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3\u7684\u65b9\u6848":111,"\u8fdb\u884c\u5206\u5e03\u5f0f\u8bad\u7ec3\u7684\u65b9\u6cd5":111,"\u8fdb\u884c\u56de\u590d":95,"\u8fdb\u884c\u5f00\u53d1":95,"\u8fdb\u884c\u62c6\u89e3":90,"\u8fdb\u884c\u6fc0\u6d3b\u64cd\u4f5c":96,"\u8fdb\u884c\u7f16\u8bd1\u548c\u5b89\u88c5":115,"\u8fdb\u884c\u8bbe\u7f6e":97,"\u8fdb\u884c\u9884\u6d4b":123,"\u8fdb\u884cpython\u4e0ec":102,"\u8fdb\u9636\u6307\u5357":[87,114],"\u8fde\u63a5":92,"\u8fde\u63a5\u5230pserver\u7684\u7aef\u53e3":104,"\u8fde\u63a5\u5230pserver\u7684\u7aef\u53e3\u4e2a\u6570":104,"\u9000\u51fa\u5bb9\u5668":110,"\u9002\u4e2d":90,"\u9009":90,"\u9009\u62e9":90,"\u9009\u62e9\u4e0b\u8f7d\u4f7f\u7528\u4e0d\u540c\u7684blas\u5e93\u7684docker\u955c\u50cf":84,"\u9009\u62e9\u6d4b\u8bd5\u7ed3\u679c\u6700\u597d\u7684\u6a21\u578b\u6765\u9884\u6d4b":123,"\u9009\u62e9\u76ee\u6807\u5206\u652f":95,"\u9009\u62e9\u8def\u5f84\u6765\u52a8\u6001\u52a0\u8f7dnvidia":106,"\u9009\u62e9\u9002\u5408\u60a8\u7684\u573a\u666f\u7684\u5408\u9002\u65b9\u6848":109,"\u9009\u9879":[83,94,121],"\u900f\u4f20\u7528\u6237\u8eab\u4efd\u7684\u529e\u6cd5":44,"\u9012\u5f52\u795e\u7ecf\u7f51\u7edc":105,"\u901a\u5e38":[55,97,102],"\u901a\u5e38\u4f1a\u4f7f\u7528\u73af\u5883\u53d8\u91cf\u914d\u7f6ejob\u7684\u914d\u7f6e\u4fe1\u606f":111,"\u901a\u5e38\u4f1a\u4f7f\u7528mapreduce\u4efb\u52a1\u7684\u8f93\u51fa\u7ed3\u679c\u4f5c\u4e3a\u8bad\u7ec3\u7ed3\u679c":104,"\u901a\u5e38\u4f7f\u7528\u7a00\u758f\u8bad\u7ec3\u6765\u52a0\u901f\u8ba1\u7b97\u8fc7\u7a0b":108,"\u901a\u5e38\u4f7f\u7528cento":86,"\u901a\u5e38\u505a\u6cd5\u662f\u4ece\u4e00\u4e2a\u6bd4\u8f83\u5927\u7684learning_rate\u5f00\u59cb\u8bd5":82,"\u901a\u5e38\u540d\u5b57\u662f":95,"\u901a\u5e38\u60c5\u51b5\u4e0b":103,"\u901a\u5e38\u6211\u4eec\u4f1a\u5b89\u88c5ceph\u7b49\u5206\u5e03\u5f0f\u6587\u4ef6\u7cfb\u7edf\u6765\u5b58\u50a8\u8bad\u7ec3\u6570\u636e":110,"\u901a\u5e38\u6307\u5c06\u4e00\u4e2a\u6574\u4f53\u62c6\u5206\u6210\u591a\u4efd\u7684\u5176\u4e2d\u7684\u4e00\u4efd":34,"\u901a\u5e38\u6709\u4e24\u4e2a\u65b9\u6cd5\u6765\u6784\u5efa\u57fa\u4e8e":117,"\u901a\u5e38\u6bcf\u4e2ajob\u5305\u62ec\u4e00\u4e2a\u6216\u8005\u591a\u4e2apod":109,"\u901a\u5e38\u7684\u505a\u6cd5\u662f\u4f7f\u7528":93,"\u901a\u5e38\u7684\u505a\u6cd5\u662f\u5c06\u914d\u7f6e\u5b58\u4e8e":96,"\u901a\u5e38\u8981\u6c42\u65f6\u95f4\u6b65\u4e4b\u95f4\u5177\u6709\u4e00\u4e9b\u4f9d\u8d56\u6027":90,"\u901a\u5e38\u89c2\u5bdf\u70ed\u70b9\u51fd\u6570\u95f4\u7684\u8c03\u7528\u5173\u7cfb":102,"\u901a\u5e38\u90fd\u4f1a\u4f7f\u7528\u4e0b\u9762\u8fd9\u4e9b\u547d\u4ee4\u884c\u53c2\u6570":108,"\u901a\u7528":105,"\u901a\u77e5":90,"\u901a\u8fc7":[80,90,95,96,97,123],"\u901a\u8fc7\u4e24\u4e2a\u5d4c\u5957\u7684":92,"\u901a\u8fc7\u4f7f\u7528":83,"\u901a\u8fc7\u51fd\u6570":111,"\u901a\u8fc7\u547d\u4ee4\u884c\u53c2\u6570":80,"\u901a\u8fc7\u5f15\u7528memory\u5f97\u5230\u8fd9\u4e2alayer\u4e0a\u4e00\u4e2a\u65f6\u523b\u7684\u8f93\u51fa":92,"\u901a\u8fc7\u5f15\u7528memory\u5f97\u5230\u8fd9\u4e2alayer\u4e0a\u4e00\u4e2a\u65f6\u523b\u8f93\u51fa":92,"\u901a\u8fc7\u6240\u6709\u5355\u5143\u6d4b\u8bd5":95,"\u901a\u8fc7\u67e5\u770b\u4e70\u5bb6\u5bf9\u67d0\u4e2a\u4ea7\u54c1\u7684\u8bc4\u4ef7\u53cd\u9988":123,"\u901a\u8fc7\u6a21\u578b\u63a8\u65adapi\u7684\u5b9e\u73b0\u4f5c\u4e3a\u4e00\u4e2a\u6837\u4f8b":55,"\u901a\u8fc7\u7ec4\u5408\u4e0d\u540c\u7684layer":87,"\u901a\u8fc7\u7f16\u8bd1\u4f1a\u751f\u6210py_paddle\u8f6f\u4ef6\u5305":4,"\u901a\u8fc7\u7f51\u7edc\u5c42\u7684\u6807\u8bc6\u7b26\u6765\u6307\u5b9a":96,"\u901a\u8fc7\u8ba1\u7b97\u8282\u70b9\u548c\u53c2\u6570\u670d\u52a1\u5668\u7684\u5206\u5e03\u5f0f\u534f\u4f5c":104,"\u901a\u8fc7\u8be5\u53c2\u6570\u53ef\u83b7\u53d6\u5230\u8f93\u5165\u8f93\u51fa\u4ee5\u53ca\u5c5e\u6027":97,"\u901a\u8fc7\u8c03\u7528":4,"\u901a\u8fc7\u914d\u7f6e\u7c7b\u4f3c\u4e8e":123,"\u901a\u8fc7data":92,"\u901a\u8fc7ssh\u7b49\u65b9\u5f0f\u767b\u5f55\u5230raspberri":117,"\u901a\u8fc7volum":109,"\u903b\u8f91\u5212\u4e0a\u6587\u4ef6\u5206\u5757\u7684\u5355\u4f4d":44,"\u903b\u8f91\u56de\u5f52":123,"\u9047\u5230\u8be5\u9519\u8bef\u65f6":81,"\u9053\u6b49":90,"\u9069":90,"\u9075\u5b88\u4ee5\u4e0b\u7ea6\u5b9a":95,"\u9075\u5faa\u4ee5\u4e0b\u6d41\u7a0b":71,"\u9075\u5faa\u6587\u7ae0":121,"\u90a3\u4e48":[55,92,96],"\u90a3\u4e480\u5c42\u5e8f\u5217\u5373\u4e3a\u4e00\u4e2a\u8bcd\u8bed":92,"\u90a3\u4e48\u53ef\u4ee5\u8ba4\u4e3a\u8bad\u7ec3\u4e0d\u6536\u655b":82,"\u90a3\u4e48\u5728":97,"\u90a3\u4e48\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u4e0d\u4f1a\u6267\u884c\u6d4b\u8bd5\u64cd\u4f5c":1,"\u90a3\u4e48\u5982\u4f55\u5224\u65ad\u8bad\u7ec3\u4e0d\u6536\u655b\u5462":82,"\u90a3\u4e48\u5e38\u6570\u8f93\u51fa\u6240\u80fd\u8fbe\u5230\u7684\u6700\u5c0fcost\u662f":82,"\u90a3\u4e48\u5f53check\u51fa\u6570\u636e\u4e0d\u5408\u6cd5\u65f6":2,"\u90a3\u4e48\u6211\u4eec\u4e5f\u5c31\u4e0d\u9700\u8981":94,"\u90a3\u4e48\u6211\u4eec\u53ef\u4ee5\u5224\u65ad\u4e3a\u8bad\u7ec3\u4e0d\u6536\u655b":82,"\u90a3\u4e48\u63a8\u8350\u4f7f\u7528":93,"\u90a3\u4e48\u63a8\u8350\u4f7f\u7528\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u65b9\u6cd5":93,"\u90a3\u4e48\u6536\u655b\u53ef\u80fd\u5f88\u6162":82,"\u90a3\u4e48\u6700\u597d\u5c06\u6570\u636e\u6587\u4ef6\u5728\u6bcf\u6b21\u8bfb\u53d6\u4e4b\u524d\u505a\u4e00\u6b21shuffl":80,"\u90a3\u4e48\u7528\u6237\u9700\u8981\u62c9\u53d6\u6240\u6709\u7684\u8fdc\u7a0b\u5206\u652f\u5230\u672c\u673a":77,"\u90a3\u4e48\u7f16\u8bd1\u8fc7\u7a0b\u53ea\u4f1a\u4ea7\u751f":94,"\u90a3\u4e48\u8bad\u7ec3\u6709\u53ef\u80fd\u4e0d\u6536\u655b":82,"\u90a3\u4e48\u8be5\u4f18\u5316\u7b97\u6cd5\u81f3\u5c11\u9700\u8981":80,"\u90a3\u4e48fc1\u548cfc2\u5c42\u5c06\u4f1a\u4f7f\u7528\u7b2c1\u4e2agpu\u6765\u8ba1\u7b97":108,"\u90a3\u4e48paddlepaddle\u4f1a\u6839\u636elayer\u7684\u58f0\u660e\u987a\u5e8f":2,"\u90a3\u4e5f\u5c31\u4e0d\u9700\u8981\u6025\u7740\u4f18\u5316\u6027\u80fd\u5566":103,"\u90a3\u4f30\u8ba1\u8fd9\u91cc\u7684\u6f5c\u529b\u5c31\u6ca1\u5565\u597d\u6316\u7684\u4e86":103,"\u90a3\u51cf\u5c11\u5b66\u4e60\u738710\u500d\u7ee7\u7eed\u8bd5\u9a8c":82,"\u90a3\u6211\u4f1a\u671f\u671b\u5206\u6790\u5de5\u5177\u7edf\u8ba1\u5230\u901f\u5ea6\u662f100gb":103,"\u90a3\u7a0b\u5e8f\u5206\u6790\u5de5\u5177\u662f\u5fc5\u4e0d\u53ef\u5c11\u7684\u5229\u5668":103,"\u90e8\u7f72\u548c\u914d\u7f6e\u6bd4\u8f83\u7b80\u5355":109,"\u90fd":90,"\u90fd\u4e0d\u9700\u8981":94,"\u90fd\u4f1a\u4ea7\u751f\u5f53\u524d\u5c42\u72b6\u6001\u7684\u6240\u6709\u7ee7\u627f\u7ed3\u679c":106,"\u90fd\u53ea\u662f\u4ecb\u7ecd\u53cc\u5c42rnn\u7684api\u63a5\u53e3":90,"\u90fd\u53ef\u4ee5\u8fd0\u884c":94,"\u90fd\u662f\u4e94\u4f4d\u7684\u6570\u5b57":35,"\u90fd\u662f\u5bf9layer1\u5143\u7d20\u7684\u62f7\u8d1d":89,"\u90fd\u662f\u5c06\u6bcf\u4e00\u53e5\u5206\u597d\u8bcd\u540e\u7684\u53e5\u5b50":90,"\u90fd\u662fabi\u8c03\u7528\u6807\u51c6\u7684":54,"\u90fd\u7528":95,"\u90fd\u9700\u8981\u5199\u63d0\u4ea4\u8bf4\u660e":95,"\u90fd\u9700\u8981\u8c03\u7528\u4e00\u6b21":96,"\u914d\u7f6e\u5982\u4e0b":121,"\u914d\u7f6e\u6253\u5f00":103,"\u914d\u7f6e\u6587\u4ef6":123,"\u914d\u7f6e\u6587\u4ef6\u63a5\u53e3\u662ffc_layer":96,"\u914d\u7f6e\u6587\u4ef6\u91cc\u52a0\u4e24\u884c":94,"\u914d\u7f6e\u6a21\u578b\u6587\u4ef6":121,"\u914d\u7f6e\u7684\u65b9\u6cd5\u53c2\u8003":44,"\u914d\u7f6e\u7b80\u5355\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u7684\u4f8b\u5b50":93,"\u914d\u7f6e\u7f51\u7edc\u5c42\u7684\u8f93\u5165":96,"\u914d\u7f6eapi":89,"\u9152\u5e97":90,"\u91c7\u7528\u5747\u5300\u5206\u5e03\u6216\u8005\u9ad8\u65af\u5206\u5e03\u521d\u59cb\u5316":106,"\u91c7\u7528multi":82,"\u91ca\u653e\u5bf9paramters\u5185\u5b58\u7684\u9501\u5b9a":34,"\u91cc":94,"\u91cc\u4ecb\u7ecd\u4e86\u7528paddle\u6e90\u7801\u4e2d\u7684\u811a\u672c\u4e0b\u8f7d\u8bad\u7ec3\u6570\u636e\u7684\u8fc7\u7a0b":110,"\u91cc\u53ef\u4ee5\u6807\u51c6\u5316\u7f16\u8bd1\u73af\u5883":94,"\u91cc\u5b8c\u6210":97,"\u91cc\u6240\u6709\u7684\u7b26\u53f7\u90fd\u5199\u5165\u81ea\u5df1\u7684\u7a0b\u5e8f\u7684\u4e8c\u8fdb\u5236\u6587\u4ef6\u91cc":54,"\u91cc\u6307\u5b9a\u56fe\u50cf\u6570\u636e\u5217\u8868":122,"\u91cc\u7684":94,"\u91cc\u7684\u65e5\u5fd7":104,"\u91cc\u7684\u6e90\u7801":94,"\u91cc\u8fd0\u884c\u7684\u7f16\u8bd1\u5de5\u5177\u5b9e\u9645\u4e0a\u90fd\u662f\u5728\u672c\u673a\u7684":94,"\u91cc\u9762":97,"\u91cc\u9762\u6db5\u76d6\u4e86\u4ea4\u53c9\u7f16\u8bd1android\u7248paddlepaddle\u5e93\u9700\u8981\u7684\u6240\u6709\u7f16\u8bd1\u5de5\u5177":115,"\u91cd\u547d\u540d\u6210":54,"\u91cd\u65b0\u7f16\u8bd1paddlepaddl":103,"\u9488\u5bf9\u4e0d\u540c\u7684":116,"\u9488\u5bf9\u4efb\u52a1\u8fd0\u884c\u5b8c\u6210\u540e\u5bb9\u5668\u81ea\u52a8\u9000\u51fa\u7684\u573a\u666f":110,"\u9488\u5bf9\u5185\u5b58\u548c\u663e\u5b58":80,"\u94fe\u63a5\u4e2d\u627e\u5230":86,"\u94fe\u63a5\u4f55\u79cdblas\u5e93\u7b49":83,"\u94fe\u63a5\u5230\u81ea\u5df1\u7684\u7a0b\u5e8f\u91cc":54,"\u94fe\u63a5\u5f85\u8865\u5145":123,"\u9519\u8bef\u5904\u7406":54,"\u9519\u8bef\u5904\u7406\u65b9\u5f0f\u662f\u8fd4\u56de\u503c":54,"\u9519\u8bef\u5904\u7406\u7684\u65b9\u5f0f\u4e5f\u4e0d\u5c3d\u76f8\u540c":54,"\u9519\u8bef\u7387":123,"\u9519\u8bef\u7684define_py_data_sources2\u7c7b\u4f3c":82,"\u952e\u6765\u542f\u52a8\u7f16\u8bd1\u4e86":94,"\u955c\u50cf\u91cc\u6709":110,"\u957f\u5ea6":80,"\u95e8\u63a7\u5faa\u73af\u5355\u5143\u5355\u6b65\u51fd\u6570\u548c\u8f93\u51fa\u51fd\u6570":93,"\u95e8\u63a7\u5faa\u73af\u5355\u5143\u7684\u8f93\u51fa\u88ab\u7528\u4f5c\u8f93\u51famemori":93,"\u95f4\u9694":123,"\u9650\u5236\u5957\u63a5\u5b57\u53d1\u9001\u7f13\u51b2\u533a\u7684\u5927\u5c0f":106,"\u9650\u5236\u5957\u63a5\u5b57\u63a5\u6536\u7f13\u51b2\u533a\u7684\u5927\u5c0f":106,"\u9664\u4e86":2,"\u9664\u4e86\u53ef\u4ee5\u81ea\u52a8\u7f16\u8bd1\u6587\u6863":99,"\u9664\u4e86boot_lay":90,"\u9664\u53bbdata\u5c42":123,"\u9664\u6784\u9020\u67d0\u79cd\u7c7b\u578b\u7684\u51fd\u6570":55,"\u9664\u6b64\u4e4b\u5916":80,"\u9664\u8bcd\u5411\u91cf\u6a21\u578b\u5916\u7684\u53c2\u6570\u5c06\u4f7f\u7528\u6b63\u6001\u5206\u5e03\u968f\u673a\u521d\u59cb\u5316":121,"\u9664\u96f6\u7b49\u95ee\u9898":80,"\u968f\u540e\u53ef\u4ee5\u7528\u8fd9\u4e2a\u5f00\u53d1\u955c\u50cf\u5f00\u59cbbuild":95,"\u968f\u673a\u6570\u7684\u79cd\u5b50":106,"\u968f\u673a\u6570seed":105,"\u9694\u5f00":[104,122],"\u96c6\u675f\u641c\u7d22\u4f7f\u7528\u5e7f\u5ea6\u4f18\u5148\u641c\u7d22\u7684\u65b9\u5f0f\u6784\u5efa\u67e5\u627e\u6811":106,"\u96c6\u7fa4\u4e0a\u542f\u52a8\u4e00\u4e2a\u5355\u673a\u4f7f\u7528cpu\u7684paddle\u8bad\u7ec3\u4f5c\u4e1a":110,"\u96c6\u7fa4\u4e2d\u7684\u6bcf\u53f0\u8ba1\u7b97\u673a\u901a\u5e38\u88ab\u6210\u4e3a\u4e00\u4e2a":104,"\u96c6\u7fa4\u4efb\u52a1":104,"\u96c6\u7fa4\u4f5c\u4e1a\u5c06\u4f1a\u5728\u51e0\u79d2\u540e\u542f\u52a8":104,"\u96c6\u7fa4\u6d4b\u8bd5":105,"\u96c6\u7fa4\u7ba1\u7406\u5de5\u5177":104,"\u96c6\u7fa4\u8bad\u7ec3":105,"\u96c6\u7fa4\u8bad\u7ec3\u4e0e\u9884\u6d4b":79,"\u96c6\u7fa4\u8fdb\u7a0b":104,"\u9700\u52a0\u8be5\u6a21\u677f\u53c2\u6570":97,"\u9700\u5728nvvp\u754c\u9762\u4e2d\u9009\u4e0a\u624d\u80fd\u5f00\u542f":103,"\u9700\u8981":[35,94,97],"\u9700\u8981\u4e3a":97,"\u9700\u8981\u4f7f\u7528\u5176\u5236\u5b9a\u7684\u65b9\u5f0f\u6302\u8f7d\u540e\u5e76\u5bfc\u5165\u6570\u636e":111,"\u9700\u8981\u4f7f\u7528\u6700\u65b0\u7684pip":86,"\u9700\u8981\u4fdd\u6301\u5f53\u524d\u5206\u652f\u76ee\u5f55":95,"\u9700\u8981\u4fee\u6539build":71,"\u9700\u8981\u5148\u6302\u8f7d\u5230\u670d\u52a1\u5668node\u4e0a\u518d\u901a\u8fc7kubernet":109,"\u9700\u8981\u5347\u7ea7pip\u7248\u672c\u5230\u6700\u65b0":[77,86],"\u9700\u8981\u5355\u72ec":84,"\u9700\u8981\u53ef\u4ee5\u8de8\u5e73\u53f0\u6267\u884c":44,"\u9700\u8981\u540c\u6b65\u539f\u4ed3\u5e93":95,"\u9700\u8981\u542f\u52a8\u7684\u8282\u70b9\u4e2a\u6570\u4ee5\u53ca":111,"\u9700\u8981\u548c\u8be5op\u7684\u540d\u5b57\u4e00\u6837":97,"\u9700\u8981\u5728":97,"\u9700\u8981\u5728\u521b\u5efa\u5bb9\u5668\u524d\u6302\u8f7d\u5377\u4ee5\u4fbf\u6211\u4eec\u4fdd\u5b58\u8bad\u7ec3\u7ed3\u679c":110,"\u9700\u8981\u5728\u7cfb\u7edf\u91cc\u5148\u5b89\u88c5\u597ddocker\u5de5\u5177\u5305":99,"\u9700\u8981\u5728cmake\u7684\u65f6\u5019":55,"\u9700\u8981\u5728macos\u7cfb\u7edf\u4e0a\u8fdb\u884c":116,"\u9700\u8981\u5b89\u88c5graphviz\u6765\u8f6c\u6362dot\u6587\u4ef6\u4e3a\u56fe\u7247":122,"\u9700\u8981\u5bf9":109,"\u9700\u8981\u5c06\u5176parameter\u8bbe\u7f6e\u6210":80,"\u9700\u8981\u5c06bugfix\u7684\u5206\u652f\u540c\u65f6merge\u5230":71,"\u9700\u8981\u5f15\u7528":55,"\u9700\u8981\u5f3a\u8c03\u7684\u662f":94,"\u9700\u8981\u6267\u884c":[83,86,88],"\u9700\u8981\u6307\u5b9a\u4e0e\u67d0\u4e00\u4e2a\u8f93\u5165\u7684\u5e8f\u5217\u4fe1\u606f\u662f\u4e00\u81f4\u7684":90,"\u9700\u8981\u6307\u5b9alayer\u7684\u8f93\u5165\u6765\u6e90":87,"\u9700\u8981\u660e\u786e\u6307\u5b9a":106,"\u9700\u8981\u6709\u4e00\u4e2a\u5916\u90e8\u7684\u5b58\u50a8\u670d\u52a1\u6765\u4fdd\u5b58\u8bad\u7ec3\u6240\u9700\u6570\u636e\u548c\u8bad\u7ec3\u8f93\u51fa":109,"\u9700\u8981\u6709\u7a33\u5b9a\u7684\u5bfc\u51fa\u7b26\u53f7":54,"\u9700\u8981\u6839\u636e\u4e0d\u540c\u7684\u5206\u5e03\u5f0f\u5b58\u50a8\u6765\u7ed1\u5b9a\u4e00\u4e2a":111,"\u9700\u8981\u6ce8\u610f":97,"\u9700\u8981\u6ce8\u610f\u7684\u662f":[71,80,106],"\u9700\u8981\u6ce8\u610f\u7684\u662f\u68af\u5ea6\u68c0\u67e5\u4ec5\u4ec5\u9a8c\u8bc1\u4e86\u68af\u5ea6\u7684\u8ba1\u7b97":96,"\u9700\u8981\u6ce8\u610f\u7684\u662fpaddlepaddle\u76ee\u524d\u53ea\u652f\u6301\u5b50\u5e8f\u5217\u6570\u76ee\u4e00\u6837\u7684\u591a\u8f93\u5165\u53cc\u5c42rnn":90,"\u9700\u8981\u7528\u5230\u7684\u7f16\u8bd1\u5de5\u5177\u548c\u7cfb\u7edf\u5e93":115,"\u9700\u8981\u7528\u6237\u663e\u5f0f\u8bbe\u5b9a":81,"\u9700\u8981\u88ab\u66b4\u9732\u5230\u5176\u4ed6\u8bed\u8a00":55,"\u9700\u8981\u8bf7\u7ba1\u7406\u5458\u5b89\u88c5\u548c\u914d\u7f6e\u597d":94,"\u9700\u8981\u9075\u5faa\u4ee5\u4e0b\u7ea6\u5b9a":92,"\u9700\u8981\u91cd\u547d\u540dwheel\u5305\u4e2dplatform\u76f8\u5173\u7684\u540e\u7f00":71,"\u9700\u8981\u989d\u5916\u6ce8\u610f\u7684\u662f":98,"\u975e\u5e38\u6570":96,"\u975e\u96f6\u6570\u5b57\u7684\u4e2a\u6570":96,"\u9762\u5411\u67b6\u6784\u4e3a32\u4f4darm\u67b6\u6784":115,"\u9762\u5411\u67b6\u6784\u4e3a64\u4f4darm64\u67b6\u6784":115,"\u9879\u76ee\u5728\u52aa\u529b\u5f00\u59cb\u652f\u6301\u5176\u4ed6\u4e0d\u9700\u8981":94,"\u987a\u5e8f":90,"\u9884\u63d0\u4ea4\u94a9\u5b50":95,"\u9884\u6d4b\u6982\u7387\u53d6\u5e73\u5747":122,"\u9884\u6d4b\u7ed3\u679c\u4ee5\u6587\u672c\u7684\u5f62\u5f0f\u4fdd\u5b58\u5728":123,"\u9884\u6d4bid":123,"\u9884\u8bad\u7ec3\u6a21\u578b\u4f7f\u7528\u7684\u5b57\u5178\u7684\u8def\u5f84":121,"\u9884\u8bad\u7ec3\u8bcd\u5411\u91cf\u5b57\u5178\u6a21\u578b\u7684\u8def\u5f84":121,"\u989c\u8272\u901a\u9053\u987a\u5e8f\u4e3a":122,"\u989d\u5916\u7684\u53c2\u6570":123,"\u9996\u5148":[2,90,93,96,121,122,123],"\u9996\u5148\u5728\u7cfb\u7edf\u8def\u5f84":83,"\u9996\u5148\u5b89\u88c5\u5e76\u5728\u5f53\u524d\u76ee\u5f55\u8fd0\u884c\u5b83":95,"\u9996\u5148\u5b9a\u4e49":97,"\u9996\u5148\u5bf9\u8f93\u5165\u505a\u4e00\u4e2a\u5c0f\u7684\u6270\u52a8":96,"\u9996\u5148\u6211\u4eec\u9700\u8981\u63a8\u5bfc\u8be5\u7f51\u7edc\u5c42\u7684":96,"\u9996\u5148\u6784\u9020\u5934\u4fe1\u606f":82,"\u9996\u5148\u901a\u8fc7":95,"\u9996\u5148\u9700\u8981\u52a0\u8f7d\u76f8\u5e94\u7684python\u5e93":87,"\u9a71\u52a8":99,"\u9ad8\u4eae\u90e8\u5206":90,"\u9ad8\u53ef\u7528":109,"\u9ad8\u5ea6\u652f\u6301\u7075\u6d3b\u548c\u9ad8\u6548\u7684\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u914d\u7f6e":93,"\u9ad8\u65af\u5206\u5e03":82,"\u9ed8\u8ba4":[2,104,106],"\u9ed8\u8ba4256k":44,"\u9ed8\u8ba4\u4e00\u4e2apass\u4fdd\u5b58\u4e00\u6b21\u6a21\u578b":123,"\u9ed8\u8ba4\u4e0d\u663e\u793a":106,"\u9ed8\u8ba4\u4e0d\u8bbe\u7f6e":92,"\u9ed8\u8ba4\u4e3a0":[106,108],"\u9ed8\u8ba4\u4e3a1":[2,108],"\u9ed8\u8ba4\u4e3a100":108,"\u9ed8\u8ba4\u4e3a4096mb":106,"\u9ed8\u8ba4\u4e3a\u7b2c\u4e00\u4e2a\u8f93\u5165":92,"\u9ed8\u8ba4\u4e3anull":106,"\u9ed8\u8ba4\u4f1a\u5c06a\u548cb":80,"\u9ed8\u8ba4\u4f7f\u7528concurrentremoteparameterupdat":106,"\u9ed8\u8ba4\u4f7f\u7528mkl":83,"\u9ed8\u8ba4\u503c":[83,89,104,108],"\u9ed8\u8ba4\u503c\u4e3a":[115,116,117],"\u9ed8\u8ba4\u503c\u4e3a\u73af\u5883\u53d8\u91cf":116,"\u9ed8\u8ba4\u521d\u59cb\u72b6\u4e3a0":92,"\u9ed8\u8ba4\u60c5\u51b5\u4e0b":[82,104],"\u9ed8\u8ba4\u60c5\u51b5\u4e0b\u6309\u7167float\u7cbe\u5ea6\u8ba1\u7b97":82,"\u9ed8\u8ba4\u6307\u5b9a\u7b2c\u4e00\u4e2a\u8f93\u5165":90,"\u9ed8\u8ba4\u662f\u4f7f\u7528mkl\u7684\u955c\u50cf":84,"\u9ed8\u8ba4\u6ca1\u6709\u5b89\u88c5vim":84,"\u9ed8\u8ba4\u7684":110,"\u9ed8\u8ba4\u7f16\u8bd1\u6240\u6709\u67b6\u6784":116,"\u9ed8\u8ba4\u8bbe\u7f6e\u4e3a\u771f":108,"\u9ed8\u8ba4\u8bbe\u7f6e\u6210\u73af\u5883\u53d8\u91cf":[115,117],"\u9ed8\u8ba4\u8c03\u7528":94,"abi\u7684paddlepaddle\u5e93":115,"abstract":[59,70,72],"adamax\u7b49":123,"amazon\u7535\u5b50\u4ea7\u54c1\u8bc4\u8bba\u6570\u636e":123,"android\u5b98\u65b9\u63d0\u4f9b\u7684":115,"android\u5e73\u53f0\u4e0a\u4f7f\u7528\u7684c":115,"android\u5e73\u53f0\u53ef\u9009\u914d\u7f6e\u53c2\u6570":115,"android\u5e73\u53f0\u7f16\u8bd1\u6307\u5357":118,"android\u7684docker\u5f00\u53d1\u955c\u50cf\u5411\u7528\u6237\u63d0\u4f9b\u4e24\u4e2a\u53ef\u914d\u7f6e\u7684\u53c2\u6570":115,"api\u4e0d\u5c0f\u4e8e21":115,"api\u4e2d\u4f7f\u7528":54,"api\u5b8c\u6210\u5206\u5e03\u5f0f\u8bad\u7ec3":104,"api\u5bf9\u6bd4\u4ecb\u7ecd":91,"api\u5bfc\u51fa\u7684\u52a8\u6001\u5e93":55,"api\u5bfc\u51fa\u7684\u9759\u6001\u5e93":55,"api\u5e93\u5c06\u88ab\u5b89\u88c5\u5230":115,"api\u5f00\u53d1\u5305\u5e76\u5b89\u88c5":86,"api\u63a5\u53d7\u7684\u7c7b\u578b\u5168\u662f":55,"api\u63a5\u53e3":[44,109],"api\u63a5\u53e3\u751f\u6210":97,"api\u63a5\u53e3\u7684\u53c2\u6570\u8f6c\u53d1\u7ed9":55,"api\u63a5\u53e3\u7684\u751f\u6210":97,"api\u6587\u6863":[115,116],"api\u65f6":55,"api\u65f6\u6240\u552f\u4e00\u9700\u8981\u5f15\u5165\u7684\u5934\u6587\u4ef6":55,"api\u662f\u591a\u8bed\u8a00api\u7684\u57fa\u7840\u90e8\u5206":55,"api\u66b4\u9732\u7684\u7c7b\u578b":55,"api\u6765\u9884\u6d4b":[115,116],"api\u751f\u6210\u7684\u4e8c\u8fdb\u5236\u6587\u4ef6\u4f1a\u88ab\u5b89\u88c5\u5230":55,"api\u7684\u5934\u6587\u4ef6":[115,116,117],"api\u7684\u5b9e\u4f8b":55,"api\u7684\u5b9e\u73b0\u7ec6\u8282":55,"api\u7684\u63a5\u53e3":55,"api\u7684\u65f6\u5019\u63a8\u8350paddle\u4e0d\u5d4c\u5165python\u89e3\u91ca\u5668":55,"api\u7684\u7f16\u8bd1\u9009\u9879\u9ed8\u8ba4\u5173\u95ed":55,"api\u76ee\u5f55\u7ed3\u6784\u5982\u4e0a\u56fe\u8868\u6240\u793a":55,"api\u7ea7\u522b":115,"api\u7ea7\u522b\u4e0d\u5c0f\u4e8e21":115,"api\u7ea7\u522b\u4e3a21":115,"api\u83b7\u5f97\u4e86\u795e\u7ecf\u7f51\u7edc\u7684\u53c2\u6570\u5b9e\u4f8b":55,"api\u9759\u6001\u5e93":116,"api\u9884\u6d4b\u5e93":116,"app\u4e2d":[115,116],"apple\u5b98\u65b9\u4e3aios\u5f00\u53d1\u63d0\u4f9b\u4e86\u5b8c\u6574\u7684\u4ea4\u53c9\u7f16\u8bd1\u5de5\u5177\u548c\u96c6\u6210\u5f00\u53d1\u73af\u5883":116,"async_sgd\u8fdb\u884c\u8bad\u7ec3\u65f6":82,"avx\u662f\u4e00\u79cdcpu\u6307\u4ee4\u96c6":84,"avx\u7248\u672c":84,"avx\u7684\u955c\u50cf":84,"awselasticblockstore\u7b49":109,"batch\u4e2d\u5305\u542b":80,"batches\u4e2a\u6279\u6b21\u4fdd\u5b58\u4e00\u6b21\u53c2\u6570":106,"batches\u6b21":106,"block\u6784\u6210\u4e00\u4e2amodel":34,"book\u4e00\u5b9a\u662f\u60a8\u6700\u597d\u7684\u9009\u62e9":84,"book\u4e2d\u6240\u6709\u7ae0\u8282\u529f\u80fd\u7684\u6b63\u786e\u6027":71,"book\u662f\u4e3a\u7528\u6237\u548c\u5f00\u53d1\u8005\u5236\u4f5c\u7684\u4e00\u4e2a\u4ea4\u4e92\u5f0f\u7684jupyt":84,"book\u7684":87,"book\u7684docker\u955c\u50cf":84,"bool\u578b\u53c2\u6570":2,"boolean":[21,45,51,54],"break":[13,32,75],"bugfix\u5206\u652f\u4e5f\u662f\u5728\u5f00\u53d1\u8005\u81ea\u5df1\u7684fork\u7248\u672c\u5e93\u7ef4\u62a4":71,"bugfix\u5206\u652f\u9700\u8981\u5206\u522b\u7ed9\u4e3b\u7248\u672c\u5e93\u7684":71,"byte":[15,44,53,82],"c99\u662f\u76ee\u524dc\u6700\u5e7f\u6cdb\u7684\u4f7f\u7528\u6807\u51c6":54,"c\u6709\u6807\u51c6\u7684abi":54,"c\u8bed\u8a00\u662f\u6709\u5bfc\u51fa\u7b26\u53f7\u7684\u6807\u51c6\u7684":54,"case":[8,20,21,36,55,59,62,64,65,68,103,119],"cc\u4e2d\u7684":98,"cells\u7b49":81,"char":38,"class":[5,6,7,8,9,10,11,13,14,18,20,23,26,28,29,31,42,46,47,49,50,52,54,57,58,61,65,68,69,70,72,73,74,75,82,96,97,98],"cmake\u4e2d\u5c06":103,"cmake\u5b98\u65b9\u5bf9android\u5e73\u53f0\u7684\u4ea4\u53c9\u7f16\u8bd1\u63d0\u4f9b\u4e86\u901a\u7528\u7684\u652f\u6301":115,"cmake\u627e\u5230\u7684python\u5e93\u548cpython\u89e3\u91ca\u5668\u7248\u672c\u53ef\u80fd\u6709\u4e0d\u4e00\u81f4\u73b0\u8c61":77,"cmake\u7cfb\u7edf\u5bf9\u4ea4\u53c9\u7f16\u8bd1\u63d0\u4f9b\u4e86\u652f\u6301":115,"cmake\u7f16\u8bd1\u65f6":83,"cmake\u7f16\u8bd1\u7684\u76ee\u6807\u5e73\u53f0":[115,116,117],"cmake\u914d\u7f6e\u4e2d\u5c06":103,"cmake\u914d\u7f6e\u5b8c\u6210\u540e":[115,116,117],"const":[31,36,38,46,52,60,62,69,72,74,75,96,97,98],"container\u4e2d":110,"core\u4e2d\u7684\u6a21\u578b\u8fd8\u5728\u4f7f\u7528\u8fd9\u4e2a\u53c2\u6570":55,"core\u4e2d\u8fd9\u4e00\u7c7b\u578b\u63a5\u53e3\u7684\u667a\u80fd\u6307\u9488":55,"core\u662f\u5426\u8fd8\u5728\u4f7f\u7528\u8fd9\u4e2a\u5b9e\u4f8b":55,"core\u6982\u5ff5":55,"cost\u63a5\u6536y_predict\u4e0ey\u4f5c\u4e3a\u8f93\u5165":87,"cost\u8fd8\u5927\u4e8e\u8fd9\u4e2a\u6570":82,"count\u4e2agpu\u4e0a\u4f7f\u7528\u6570\u636e\u5e76\u884c\u6765\u8ba1\u7b97\u67d0\u4e00\u5c42":108,"count\u548cgpu":108,"cuda\u5e93":106,"cudnn\u5e93":[83,106],"cumtime\u7684\u6bcf\u6b21\u8c03\u7528\u5e73\u5747\u65f6\u95f4":102,"data\u5230\u5206\u5e03\u5f0f\u5b58\u50a8\u8865\u5145\u8bad\u7ec3\u6570\u636e":35,"data\u63a5\u53e3\u5206\u914d\u5b9e\u9645\u7684\u5185\u5b58":98,"data\u76ee\u5f55\u4e2d\u5b58\u653e\u5207\u5206\u597d\u7684\u6570\u636e":111,"dataprovider\u5171\u8fd4\u56de\u4e24\u4e2a\u6570\u636e":90,"dataprovider\u5171\u8fd4\u56de\u4e24\u7ec4\u6570\u636e":90,"dataprovider\u662fpaddlepaddle\u8d1f\u8d23\u63d0\u4f9b\u6570\u636e\u7684\u6a21\u5757":1,"dataprovider\u7684\u4ecb\u7ecd":[3,123],"dataprovider\u7f13\u51b2\u6c60\u5185\u5b58":80,"deb\u5305":71,"deb\u5305\u7f16\u8bd1\u95ee\u9898":71,"decoder\u5faa\u73af\u5c55\u5f00\u7684\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65\u4f1a\u5f15\u7528\u5168\u90e8\u7ed3\u679c":92,"decoder\u63a5\u53d7\u4e24\u4e2a\u8f93\u5165":92,"decoder\u6bcf\u6b21\u9884\u6d4b\u4ea7\u751f\u4e0b\u4e00\u4e2a\u6700\u53ef\u80fd\u7684\u8bcd\u8bed":92,"decoer\u67b6\u6784":92,"default":[6,7,8,9,10,11,13,14,15,18,21,25,28,29,31,32,48,52,53,56,62,63,72,73,74,76,101,108,110,111,115,119],"demo\u9884\u6d4b\u8f93\u51fa\u5982\u4e0b":4,"dictionary\u662f\u4ece\u7f51\u7edc\u914d\u7f6e\u4e2d\u4f20\u5165\u7684dict\u5bf9\u8c61":2,"dist\u76ee\u5f55\u4e0b\u751f\u6210\u8f93\u51fa\u7684whl\u5305":83,"dnn\u6570\u5b66\u5e93":83,"docker\u5b89\u88c5\u65b9\u5f0f\u53ef\u4ee5\u8fdb\u5165docker\u5bb9\u5668\u6267\u884c":104,"docker\u5b89\u88c5\u8bf7\u53c2\u8003":99,"docker\u5b89\u88c5\u8bf7\u53c2\u8003docker\u7684\u5b98\u7f51":99,"docker\u5b98\u7f51":84,"docker\u5bb9\u5668\u4e2d\u5c06\u9ed8\u8ba4\u4f7f\u7528":115,"docker\u7684\u5b98\u7f51":99,"docker\u7f16\u8bd1\u73af\u5883\u955c\u50cf\u5b8c\u6210\u7f16\u8bd1":83,"docker\u80fd\u5728\u6240\u6709\u4e3b\u8981\u64cd\u4f5c\u7cfb\u7edf":115,"docker\u955c\u50cf":84,"docker\u955c\u50cf\u4e3a\u4e86\u51cf\u5c0f\u4f53\u79ef":84,"docker\u955c\u50cf\u9ed8\u8ba4":84,"dockerhub\u7f51\u7ad9":84,"double\u7c7b\u578b\u65f6\u4e3a8":82,"dropout\u7684\u6bd4\u4f8b":96,"eigenscalar\u7684\u8f6c\u6362":98,"elec\u6d4b\u8bd5\u96c6":123,"embedding\u6a21\u578b\u9700\u8981\u7a0d\u5fae\u6539\u53d8\u63d0\u4f9b\u6570\u636e\u7684python\u811a\u672c":123,"encode\u6210\u7684\u6700\u540e\u4e00\u4e2a\u5411\u91cf":90,"encoder\u548cdecoder\u53ef\u4ee5\u662f\u80fd\u591f\u5904\u7406\u5e8f\u5217\u7684\u4efb\u610f\u795e\u7ecf\u7f51\u7edc\u5355\u5143":92,"encoder\u8f93\u51fa":92,"entropy\u4f5c\u4e3acost":82,"enum":[36,38,61,73,74,76],"export":[50,77,84,99,104],"f\u4ee3\u8868\u4e00\u4e2a\u6d6e\u70b9\u6570":[2,87],"false\u7684\u60c5\u51b5":2,"fc1\u548cfc2\u5c42\u5728gpu\u4e0a\u8ba1\u7b97":108,"fc3\u5c42\u4f7f\u7528cpu\u8ba1\u7b97":108,"final":[8,9,30,50,56,57,75],"flatten\u65b9\u6cd5\u662f\u628apaddle\u4e2d\u7684\u4e00\u4e2atensor\u8fdb\u884creshape\u64cd\u4f5c":98,"float":[2,6,7,8,10,13,21,46,52,74,97,98,103,122],"float\u7b49":108,"from\u65b9\u6cd5\u662f\u628apaddle\u4e2d\u7684\u4e00\u7ef4tensor\u8f6c\u4e3aeigen\u7684\u4e00\u7ef4tensor":98,"from\u662feigentensor\u6a21\u677f\u63d0\u4f9b\u7684\u4e00\u4e2a\u63a5\u53e3":98,"full\u53c2\u6570\u63d0\u4ea4":78,"function":[8,9,13,21,26,29,31,33,37,38,39,41,42,46,49,52,56,57,59,60,61,62,64,65,66,68,69,70,72,74,75,93,101,102,119],"function\u4f7f\u7528":81,"generator\u4fbf\u4f1a\u5b58\u4e0b\u5f53\u524d\u7684\u4e0a\u4e0b\u6587":2,"generator\u81f3\u5c11\u9700\u8981\u8c03\u7528\u4e24\u6b21\u624d\u4f1a\u77e5\u9053\u662f\u5426\u505c\u6b62":2,"git\u6d41\u5206\u652f\u6a21\u578b":95,"github\u9996\u9875":95,"glibc\u81f3\u5c11\u5305\u542bglibc_2":86,"golang\u53ef\u4ee5\u4f7f\u7528":54,"golang\u7684":54,"google\u5f00\u6e90\u7684\u5bb9\u5668\u96c6\u7fa4\u7684\u8c03\u5ea6\u6846\u67b6":104,"gpu\u4e8c\u8fdb\u5236\u6587\u4ef6":83,"gpu\u5171\u4eabkernel\u5b9e\u73b0\u5728":97,"gpu\u5219\u8fd8\u9700\u8981\u9ad8\u5e76\u884c\u6027":103,"gpu\u53cc\u7f13\u5b58":2,"gpu\u5b9e\u73b0\u5171\u4eab\u540c\u4e00\u4e2a":97,"gpu\u5b9e\u73b0\u5728":97,"gpu\u6027\u80fd\u5206\u6790\u4e0e\u8c03\u4f18":100,"gpu\u6267\u884c":98,"gpu\u6838\u5728\u8bad\u7ec3\u914d\u7f6e\u4e2d\u6307\u5b9a":106,"gpu\u7684\u4ee3\u7801\u53ef\u4ee5\u590d\u7528":97,"gpu\u7684docker\u955c\u50cf\u7684\u65f6\u5019":77,"group\u6559\u7a0b":91,"group\u7684\u5b9e\u73b0\u65b9\u5f0f":81,"gru\u6216lstm":93,"gru\u6a21\u578b":123,"gru\u6a21\u578b\u914d\u7f6e":123,"h\u5e76\u4e0d\u56f0\u96be":54,"html\u5373\u53ef\u8bbf\u95ee\u672c\u5730\u6587\u6863":99,"i\u4ee3\u8868\u4e00\u4e2a\u6574\u6570":[2,87],"id\u4e3a0\u7684\u6982\u7387":123,"id\u4e3a1\u7684\u6982\u7387":123,"id\u6307\u5b9a\u4f7f\u7528\u54ea\u4e2agpu\u6838":106,"id\u6307\u5b9a\u7684gpu":108,"id\u65e0\u6548":106,"image\u91cc":110,"images\u6570\u636e\u96c6\u4e0a\u4f20\u5230\u4e91\u7aef\u7684":35,"imikolov\u6570\u636e\u96c6":104,"import":[2,4,7,8,29,31,32,48,50,51,56,57,61,66,72,86,87,88,97,104,121,122],"infer\u63a5\u53e3\u7684\u8fd4\u56de\u503c\u662f\u4e00\u4e2apython":80,"ingress\u9700\u8981\u628apfsclient\u7684\u8eab\u4efd\u4fe1\u606f\u4f20\u7ed9pfsserv":44,"init_hook\u53ef\u4ee5\u4f20\u5165\u4e00\u4e2a\u51fd\u6570":2,"instance\u4e0e\u751f\u6210\u6570\u636e\u96c6\u65f6":35,"instance\u5305\u6db5\u4e24\u4e2a\u503c":35,"instance\u662f\u4e00\u6a21\u4e00\u6837\u7684":35,"int":[2,6,7,8,9,13,14,15,21,31,36,37,38,41,51,52,54,55,61,63,64,73,74,75,76,90,96,98,104,108],"interface\u6587\u4ef6\u7684\u5199\u6cd5\u975e\u5e38":54,"ios\u5e73\u53f0\u53ef\u9009\u914d\u7f6e\u53c2\u6570":116,"ios\u5e73\u53f0\u7f16\u8bd1\u6307\u5357":118,"ip\u548c\u4efb\u52a1\u8282\u70b9\u4e2a\u6570\u7b49":104,"issue\u7f16\u53f7":95,"job\u542f\u52a8\u540e\u4f1a\u521b\u5efa\u8fd9\u4e9bpod\u5e76\u5f00\u59cb\u6267\u884c\u4e00\u4e2a\u7a0b\u5e8f":109,"job\u6216\u8005\u5e94\u7528\u7a0b\u5e8f\u5728\u5bb9\u5668\u4e2d\u8fd0\u884c\u65f6\u751f\u6210\u7684\u6570\u636e\u4f1a\u5728\u5bb9\u5668\u9500\u6bc1\u65f6\u6d88\u5931":109,"job\u662f\u672c\u6b21\u8bad\u7ec3\u5bf9\u5e94\u7684job":111,"job\u7684\u540d\u5b57":111,"kernel\u5b9e\u73b0":97,"kernel\u6ce8\u518ccpu\u5b9e\u73b0\u5728":97,"kernel\u7684\u5b9e\u73b0\u57fa\u4e8eeigen":97,"kubernetes\u4e3a\u8fd9\u6b21\u8bad\u7ec3\u521b\u5efa\u4e863\u4e2apod\u5e76\u4e14\u8c03\u5ea6\u5230\u4e863\u4e2anode\u4e0a\u8fd0\u884c":111,"kubernetes\u5206\u5e03\u5f0f\u8bad\u7ec3":100,"kubernetes\u5355\u673a\u8bad\u7ec3":100,"kubernetes\u53ef\u4ee5\u5728\u7269\u7406\u673a\u6216\u865a\u62df\u673a\u4e0a\u8fd0\u884c":109,"kubernetes\u53ef\u4ee5\u901a\u8fc7yaml\u6587\u4ef6\u6765\u521b\u5efa\u76f8\u5173\u5bf9\u8c61":111,"kubernetes\u5c31\u4f1a\u521b\u5efa3\u4e2apod\u4f5c\u4e3apaddlepaddle\u8282\u70b9\u7136\u540e\u62c9\u53d6\u955c\u50cf":111,"kubernetes\u63d0\u4f9b\u4e86\u591a\u79cd\u96c6\u7fa4\u90e8\u7f72\u7684\u65b9\u6848":109,"kubernetes\u652f\u6301\u591a\u79cdvolum":109,"kubernetes\u6709job\u7c7b\u578b\u7684\u8d44\u6e90\u6765\u652f\u6301":110,"kubernetes\u96c6\u7fa4\u5c31\u662f\u7531node\u8282\u70b9\u4e0emaster\u8282\u70b9\u7ec4\u6210\u7684":109,"label\u662f\u539f\u59cb\u6570\u636e\u4e2d\u5bf9\u4e8e\u6bcf\u4e00\u53e5\u8bdd\u7684\u5206\u7c7b\u6807\u7b7e":90,"labels\u662f\u6bcf\u7ec4\u5185\u6bcf\u4e2a\u53e5\u5b50\u7684\u6807\u7b7e":90,"layer1\u5fc5\u987b\u662f\u4e00\u4e2a0\u5c42\u5e8f\u5217":89,"layer1\u5fc5\u987b\u662f\u4e00\u4e2a\u5355\u5c42\u5e8f\u5217":89,"layer\u4f5c\u4e3a\u4e00\u4e2a\u6574\u4f53\u6765\u5b9e\u73b0":81,"layer\u62ff\u5230\u7684\u7528\u6237\u8f93\u5165":92,"layer\u65f6":81,"layer\u662f\u6211\u4eec\u7684\u79ef\u6728":87,"layer\u7c7b\u53ef\u4ee5\u81ea\u52a8\u8ba1\u7b97\u4e0a\u9762\u7684\u5bfc\u6570":96,"layer\u8ba1\u7b97\u7684\u8f93\u51fa":81,"linux\u4e2d":84,"list\u4e2d\u7684\u6bcf\u4e00\u884c\u90fd\u4f20\u9012\u7ed9process\u51fd\u6570":2,"list\u4f5c\u4e3a\u68c0\u67e5\u5217\u8868":71,"list\u5199\u5165\u90a3\u4e2a\u6587\u672c\u6587\u4ef6\u7684\u5730\u5740":2,"list\u548ctest":1,"list\u5982\u4e0b\u6240\u793a":108,"list\u5b58\u653e\u5728\u672c\u5730":1,"list\u6307\u5b9a\u6d4b\u8bd5\u7684\u6a21\u578b\u5217\u8868":108,"long":[8,9,13,21],"lstm\u6a21\u578b":123,"lstm\u6a21\u578b\u914d\u7f6e":123,"memory\u4e0d\u80fd\u72ec\u7acb\u5b58\u5728":92,"memory\u4e5f\u53ef\u4ee5\u5177\u6709":93,"memory\u4e5f\u53ef\u4ee5\u662f\u5e8f\u5217":93,"memory\u53ea\u80fd\u5728":92,"memory\u53ef\u4ee5\u7f13\u5b58\u4e0a\u4e00\u4e2a\u65f6\u523b\u67d0\u4e00\u4e2a\u795e\u7ecf\u5143\u7684\u8f93\u51fa":90,"memory\u6307\u5411\u4e00\u4e2alay":92,"memory\u662f\u5728\u5355\u6b65\u51fd\u6570\u4e2d\u5faa\u73af\u4f7f\u7528\u7684\u72b6\u6001":93,"memory\u662fpaddlepaddle\u5b9e\u73b0rnn\u65f6\u5019\u4f7f\u7528\u7684\u4e00\u4e2a\u6982\u5ff5":90,"memory\u7684":93,"memory\u7684\u521d\u59cb\u72b6\u6001":92,"memory\u7684\u65f6\u95f4\u5e8f\u5217\u957f\u5ea6\u4e00\u81f4\u7684\u60c5\u51b5":90,"memory\u7684\u66f4\u591a\u8ba8\u8bba\u8bf7\u53c2\u8003\u8bba\u6587":92,"memory\u7684\u8f93\u51fa\u5b9a\u4e49\u5728":93,"memory\u7684i":92,"memory\u9ed8\u8ba4\u521d\u59cb\u5316\u4e3a0":92,"mnist\u662f\u4e00\u4e2a\u5305\u542b\u670970":2,"model\u505a\u5206\u652f\u7ba1\u7406":71,"model\u53ef\u4ee5\u901a\u8fc7":4,"model\u6765\u5b9e\u73b0\u624b\u5199\u8bc6\u522b\u7684\u9884\u6d4b\u4ee3\u7801":4,"name\u7ec4\u5408\u53ef\u4ee5\u627e\u5230\u672c\u6b21\u8bad\u7ec3\u9700\u8981\u7684\u6587\u4ef6\u8def\u5f84":111,"ndarray\u7c7b\u578b\u7684\u503c\u548c\u6574\u578b\u7684\u503c":35,"ndk\u4e2d\u5305\u542b\u4e86\u6240\u6709android":115,"new":[8,13,30,31,32,33,36,37,38,39,40,46,57,59,63,64,65,67,68,70,74,75,95,96,119],"nfs\u7684\u90e8\u7f72\u65b9\u6cd5\u53ef\u4ee5\u53c2\u8003":109,"normalization\u5c42":122,"normalization\u5c42\u7684\u53c2\u6570":122,"note\u7684\u4e66\u5199":71,"null":[50,96,106],"num\u51b3\u5b9a":104,"num_gradient_servers\u53c2\u6570":111,"num_samples_processed\u4e3a\u5df2\u8bad\u7ec3\u6837\u672c\u6570":82,"only\u7684\u4e8c\u8fdb\u5236":83,"op\u4e0d\u9700\u8981\u5b9a\u4e49opprotomak":97,"op\u5b9a\u4e49":97,"op\u6709\u8ba1\u7b97\u51fd\u6570":97,"op\u6ce8\u518c\u5b9e\u73b0\u5728":97,"op\u8ba1\u7b97\u51fd\u6570\u7684\u57fa\u7c7b":97,"opprotomake\u5b9a\u4e49":97,"org\u5de5\u5177\u7684\u8be6\u7ec6\u4fe1\u606f":99,"org\u76ee\u524d\u9075\u5faa":71,"outer_mem\u662f\u4e00\u4e2a\u5b50\u53e5\u7684\u6700\u540e\u4e00\u4e2a\u5411\u91cf":90,"output\u53ef\u4ee5\u662f\u4efb\u610f\u7ef4\u5ea6\u7684tensor":98,"output\u6587\u4ef6\u5939\u5b58\u653e\u8bad\u7ec3\u7ed3\u679c\u4e0e\u65e5\u5fd7":111,"output\u7684\u539f\u6709shape\u4fe1\u606f\u4e0d\u53d8":98,"packages\u91cc\u9762":77,"packages\u91cc\u9762\u7684python\u5305":77,"paddepaddle\u901a\u8fc7\u7f16\u8bd1\u65f6\u6307\u5b9a\u8def\u5f84\u6765\u5b9e\u73b0\u5f15\u7528\u5404\u79cdbla":83,"paddle\u4e00\u4e2a\u52a8\u6001\u5e93\u53ef\u4ee5\u5728\u4efb\u4f55linux\u7cfb\u7edf\u4e0a\u8fd0\u884c":54,"paddle\u4e2d\u7ecf\u5e38\u4f1a\u5c06\u65f6\u95f4\u5e8f\u5217\u6210\u4e3a":90,"paddle\u4e8c\u8fdb\u5236\u5728\u8fd0\u884c\u65f6\u6355\u83b7\u4e86\u6d6e\u70b9\u6570\u5f02\u5e38":80,"paddle\u5185\u5d4c\u7684python\u89e3\u91ca\u5668\u548c\u5916\u90e8\u4f7f\u7528\u7684python\u5982\u679c\u7248\u672c\u4e0d\u540c":54,"paddle\u5185\u90e8\u7684\u7c7b\u4e3ac":54,"paddle\u7684\u5404\u7248\u672c\u955c\u50cf\u53ef\u4ee5\u53c2\u8003":110,"paddle\u7684\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0\u5305\u62ec\u4e00\u4e0b\u51e0\u4e2a\u65b9\u9762":54,"paddle\u7684\u7c7b\u578b\u5168\u90e8\u9000\u5316\u6210":55,"paddle\u7684\u94fe\u63a5\u65b9\u5f0f\u6bd4\u8f83\u590d\u6742":54,"paddle\u7684c":55,"paddle\u7684dock":110,"paddle\u8bad\u7ec3\u4efb\u52a1":35,"paddle\u8def\u5f84\u4e0b":55,"paddle\u955c\u50cf":110,"paddle\u9700\u8981\u4e00\u4e2a\u591a\u8bed\u8a00\u63a5\u53e3":54,"paddle\u9700\u8981\u66b4\u9732\u7684api\u5f88\u591a":55,"paddle\u9759\u6001\u5e93\u94fe\u63a5\u590d\u6742":54,"paddle_\u7c7b\u578b\u540d":55,"paddle_\u7c7b\u578b\u540d_\u51fd\u6570\u540d":55,"paddlepaddle\u4e2d":[89,92],"paddlepaddle\u4e2d\u7684\u8bb8\u591alayer\u5e76\u4e0d\u5728\u610f\u8f93\u5165\u662f\u5426\u662f\u65f6\u95f4\u5e8f\u5217":90,"paddlepaddle\u4e2d\u8fd8\u5305\u542b":81,"paddlepaddle\u4e2d\u901a\u8fc7reader\u6765\u52a0\u8f7d\u6570\u636e":87,"paddlepaddle\u4e3a\u4ea4\u53c9\u7f16\u8bd1\u63d0\u4f9b\u4e86\u5de5\u5177\u94fe\u914d\u7f6e\u6587\u6863":[115,116],"paddlepaddle\u4e3a\u6df1\u5ea6\u5b66\u4e60\u7814\u7a76\u4eba\u5458\u63d0\u4f9b\u4e86\u4e30\u5bcc\u7684api":87,"paddlepaddle\u4e3ano":84,"paddlepaddle\u4e3b\u8981\u4f7f\u7528":83,"paddlepaddle\u4f1a\u5728\u8c03\u7528\u8bfb\u53d6\u6570\u636e\u7684python\u811a\u672c\u4e4b\u524d":123,"paddlepaddle\u4f1a\u81ea\u52a8\u8bbe\u5b9a":81,"paddlepaddle\u4f7f\u7528\u540c\u6b65\u5c4f\u969c":104,"paddlepaddle\u4f7f\u7528\u5747\u503c0":82,"paddlepaddle\u4f7f\u7528avx":77,"paddlepaddle\u4f7f\u7528git":71,"paddlepaddle\u4f7f\u7528swig\u5bf9\u5e38\u7528\u7684\u9884\u6d4b\u63a5\u53e3\u8fdb\u884c\u4e86\u5c01\u88c5":4,"paddlepaddle\u4fdd\u5b58\u7684\u6a21\u578b\u53c2\u6570\u6587\u4ef6\u5185\u5bb9\u753116\u5b57\u8282\u5934\u4fe1\u606f\u548c\u7f51\u7edc\u53c2\u6570\u4e24\u90e8\u5206\u7ec4\u6210":82,"paddlepaddle\u4fdd\u5b58\u7684\u6a21\u578b\u53c2\u6570\u6587\u4ef6\u524d16\u5b57\u8282\u4e3a\u5934\u4fe1\u606f":82,"paddlepaddle\u4fdd\u7559\u6dfb\u52a0\u53c2\u6570\u7684\u6743\u529b":2,"paddlepaddle\u5206\u5e03\u5f0f\u8bad\u7ec3":100,"paddlepaddle\u53d1\u5e03\u7684\u5b89\u88c5\u5305\u4f1a\u5c3d\u91cf\u5bf9\u9f50":86,"paddlepaddle\u53ef\u4ee5\u4f7f\u7528\u591a\u79cd\u5206\u5e03\u5f0f\u8ba1\u7b97\u5e73\u53f0\u6784\u5efa\u5206\u5e03\u5f0f\u8ba1\u7b97\u4efb\u52a1":104,"paddlepaddle\u53ef\u4ee5\u4f7f\u7528\u5e38\u7528\u7684python\u5305\u7ba1\u7406\u5de5\u5177":86,"paddlepaddle\u53ef\u4ee5\u4f7f\u7528cudnn":83,"paddlepaddle\u53ef\u4ee5\u540c\u65f6\u652f\u6301\u540c\u6b65\u968f\u673a\u68af\u5ea6\u4e0b\u964d":104,"paddlepaddle\u53ef\u4ee5\u6267\u884c\u7528\u6237\u7684python\u811a\u672c\u7a0b\u5e8f\u6765\u8bfb\u53d6\u5404\u79cd\u683c\u5f0f\u7684\u6570\u636e\u6587\u4ef6":123,"paddlepaddle\u53ef\u4ee5\u6bd4\u8f83\u7b80\u5355\u7684\u5224\u65ad\u54ea\u4e9b\u8f93\u51fa\u662f\u5e94\u8be5\u8de8\u8d8a\u65f6\u95f4\u6b65\u7684":90,"paddlepaddle\u53ef\u4ee5\u901a\u8fc7\u8be5\u673a\u5236\u5224\u65ad\u662f\u5426\u5df2\u7ecf\u6536\u96c6\u9f50\u6240\u6709\u7684\u68af\u5ea6":96,"paddlepaddle\u5728\u5b9e\u73b0rnn\u7684\u65f6\u5019":90,"paddlepaddle\u5728\u6fc0\u6d3b\u51fd\u6570\u91cc\u5b9e\u73b0dropout":81,"paddlepaddle\u5728\u7f16\u8bd1\u65f6":83,"paddlepaddle\u5b58\u7684\u662f\u6709\u503c\u4f4d\u7f6e\u7684\u7d22\u5f15":[2,87],"paddlepaddle\u5b89\u88c5\u5305\u7531\u4e8e\u4e0d\u4ec5\u4ec5\u5305\u542b":86,"paddlepaddle\u5b9a\u4e49\u7684\u53c2\u6570":2,"paddlepaddle\u5c06\u4ee5\u8bbe\u7f6e\u53c2\u6570\u7684\u65b9\u5f0f\u6765\u8bbe\u7f6e":123,"paddlepaddle\u5c06\u4f1a\u6839\u636e":116,"paddlepaddle\u5c06\u4f1a\u81ea\u52a8\u9009\u62e9":116,"paddlepaddle\u5c06\u6839\u636e":115,"paddlepaddle\u5c06\u81ea\u52a8\u4e0b\u8f7d\u548c\u7f16\u8bd1\u6240\u6709\u7b2c\u4e09\u65b9\u4f9d\u8d56\u5e93":[115,116,117],"paddlepaddle\u5c06train":2,"paddlepaddle\u5e93\u5df2\u7ecf\u5b89\u88c5\u5b8c\u6210":116,"paddlepaddle\u5f00\u53d1\u8fc7\u7a0b\u4f7f\u7528":71,"paddlepaddle\u63d0\u4f9b":85,"paddlepaddle\u63d0\u4f9b\u4e13\u7528\u7684":35,"paddlepaddle\u652f\u6301":83,"paddlepaddle\u652f\u6301\u4e0d\u540c\u7c7b\u578b\u7684\u8f93\u5165\u6570\u636e":87,"paddlepaddle\u652f\u6301\u4f7f\u7528pip\u5feb\u901f\u5b89\u88c5":88,"paddlepaddle\u652f\u6301\u975e\u5e38\u591a\u7684\u4f18\u5316\u7b97\u6cd5":80,"paddlepaddle\u652f\u6301sparse\u7684\u8bad\u7ec3":80,"paddlepaddle\u6587\u6863\u4f7f\u7528":99,"paddlepaddle\u662f\u6e90\u4e8e\u767e\u5ea6\u7684\u4e00\u4e2a\u6df1\u5ea6\u5b66\u4e60\u5e73\u53f0":87,"paddlepaddle\u6bcf\u6b21\u53d1\u65b0\u7684\u7248\u672c":71,"paddlepaddle\u6bcf\u6b21\u53d1\u7248\u672c\u9996\u5148\u8981\u4fdd\u8bc1paddlepaddl":71,"paddlepaddle\u7684\u4e3b\u7248\u672c\u5e93\u9075\u5faa":71,"paddlepaddle\u7684\u5185\u5b58\u5360\u7528\u4e3b\u8981\u5206\u4e3a\u5982\u4e0b\u51e0\u4e2a\u65b9\u9762":80,"paddlepaddle\u7684\u53c2\u6570\u4f7f\u7528\u540d\u5b57":82,"paddlepaddle\u7684\u5b89\u88c5\u53ef\u4ee5\u53c2\u8003":104,"paddlepaddle\u7684\u5df2\u7ecf\u5b89\u88c5\u5b8c\u6210":115,"paddlepaddle\u7684\u6240\u6709layer\u90fd\u6709\u552f\u4e00\u7684nam":81,"paddlepaddle\u7684\u6570\u636e\u5305\u62ec\u56db\u79cd\u4e3b\u8981\u7c7b\u578b":2,"paddlepaddle\u7684\u6587\u6863\u5305\u62ec\u82f1\u6587\u6587\u6863":99,"paddlepaddle\u7684\u6587\u6863\u6784\u5efa\u6709\u4e09\u79cd\u65b9\u5f0f":99,"paddlepaddle\u7684\u6e90\u7801":95,"paddlepaddle\u7684\u7f16\u8bd1\u9009\u9879":83,"paddlepaddle\u7684bas":96,"paddlepaddle\u7684c":115,"paddlepaddle\u7684cmake\u7cfb\u7edf\u4f1a\u81ea\u52a8\u7f16\u8bd1\u6240\u6709\u7684\u7b2c\u4e09\u65b9\u4f9d\u8d56\u5e93":116,"paddlepaddle\u7684cmake\u7cfb\u7edf\u5c06\u6839\u636e\u8be5\u503c\u81ea\u52a8\u63a8\u5bfc\u548c\u8bbe\u7f6e\u9700\u8981\u4f7f\u7528\u7684\u4ea4\u53c9\u7f16\u8bd1\u5668":115,"paddlepaddle\u7684cmake\u7cfb\u7edf\u5c06\u6839\u636e\u8be5\u503c\u81ea\u52a8\u8bbe\u7f6e\u9700\u8981\u4f7f\u7528\u7684\u4ea4\u53c9\u7f16\u8bd1\u5668":117,"paddlepaddle\u7684cmake\u7cfb\u7edf\u624d\u8ba4\u4e3a\u5728\u662f\u5728\u4ea4\u53c9\u7f16\u8bd1raspberri":117,"paddlepaddle\u7684cmake\u7cfb\u7edf\u624d\u8ba4\u4e3a\u662f\u5728\u4ea4\u53c9\u7f16\u8bd1android\u7cfb\u7edf\u7684\u7248\u672c":115,"paddlepaddle\u76ee\u524d\u53ea\u652f\u6301\u5728\u6bcf\u4e2a\u65f6\u95f4\u6b65\u4e2d":90,"paddlepaddle\u76ee\u524d\u63d0\u4f9b\u4e24\u79cd\u53c2\u6570\u521d\u59cb\u5316\u7684\u65b9\u5f0f":82,"paddlepaddle\u76ee\u524d\u652f\u63018\u79cdlearning_rate_schedul":82,"paddlepaddle\u7f16\u8bd1\u9700\u8981\u4f7f\u7528\u5230\u4e0b\u9762\u7684\u4f9d\u8d56":83,"paddlepaddle\u82e5\u68c0\u6d4b\u5230\u7528\u6237\u4f7f\u7528\u7684cmake\u7248\u672c\u4e0d\u4f4e\u4e8e3":115,"paddlepaddle\u8981\u6c42\u4f7f\u7528\u7684\u7f16\u8bd1\u5de5\u5177\u94fe\u6240\u652f\u6301\u7684android":115,"paddlepaddle\u8c03\u7528process\u51fd\u6570\u6765\u8bfb\u53d6\u6570\u636e":123,"paddlepaddle\u8d1f\u8d23\u5b8c\u6210\u4fe1\u606f\u548c\u68af\u5ea6\u5728\u65f6\u95f4\u5e8f\u5217\u4e0a\u7684\u4f20\u64ad":92,"paddlepaddle\u8d1f\u8d23\u5b8c\u6210\u4fe1\u606f\u548c\u8bef\u5dee\u5728\u65f6\u95f4\u5e8f\u5217\u4e0a\u7684\u4f20\u64ad":92,"paddlepaddle\u955c\u50cf\u9700\u8981\u63d0\u4f9b":111,"paddlepaddle\u9700\u8981\u7528\u6237\u5728\u7f51\u7edc\u914d\u7f6e":1,"pass\u4e2a\u6a21\u578b\u5230\u7b2c":106,"pass\u5c06\u4e0d\u8d77\u4f5c\u7528":106,"pass\u8f6e\u5f00\u59cb\u8bad\u7ec3":106,"pass\u8f6e\u7684\u6a21\u578b\u7528\u4e8e\u6d4b\u8bd5":106,"passes\u8f6e":106,"patch\u53f7":71,"patch\u53f7\u52a0\u4e00":71,"path\u6307\u5b9a\u6d4b\u8bd5\u7684\u6a21\u578b":108,"perftools\u6765\u8fdb\u884c\u6027\u80fd\u5206\u6790":102,"period\u4e2a\u6279\u6b21\u5bf9\u6240\u6709\u6d4b\u8bd5\u6570\u636e\u8fdb\u884c\u6d4b\u8bd5":106,"period\u4e2a\u6279\u6b21\u6253\u5370\u65e5\u5fd7\u8fdb\u5ea6":106,"period\u4e2a\u6279\u6b21\u8f93\u51fa\u53c2\u6570\u7edf\u8ba1":106,"period\u4e2a\u6279\u6b21\u8f93\u51fa\u7b26\u53f7":106,"period\u6574\u9664":106,"period\u8f6e\u4fdd\u5b58\u8bad\u7ec3\u53c2\u6570":106,"pfsclient\u9700\u8981\u548cingress\u4e4b\u95f4\u505a\u53cc\u5411\u9a8c\u8bc1":44,"pfsclient\u9700\u8981\u5728\u4f20\u8f93\u5b8c\u6bd5\u6700\u540e\u4e00\u4e2achunk\u7684\u65f6\u5019\u68c0\u67e5destination\u6587\u4ef6\u7684md5\u503c\u662f\u5426\u548csource\u6587\u4ef6\u4e00\u81f4":44,"pfsserver\u63d0\u4f9brest":44,"pi\u5e73\u53f0\u4e0a\u9002\u7528\u7684paddlepaddle\u7684\u65b9\u6cd5\u548c\u6b65\u9aa4":117,"pi\u5e73\u53f0\u7f16\u8bd1\u6307\u5357":118,"pi\u7248\u672c\u7684\u5e93":117,"pi\u7248\u672cpaddlepaddle\u5e93\u65f6":117,"pi\u7684\u914d\u7f6e\u4fe1\u606f\u5728":117,"pi\u7cfb\u7edf\u4e0a\u6765\u6784\u5efa":117,"pi\u7cfb\u7edf\u7684\u7248\u672c":117,"pip\u548cdocker\u7684\u5b89\u88c5\u65b9\u5f0f":85,"pod\u4e2d\u7684\u5bb9\u5668\u5171\u4eabnet":109,"pod\u662fkubernetes\u7684\u6700\u5c0f\u8c03\u5ea6\u5355\u5143":109,"process\u51fd\u6570\u4f1a\u7528yield\u8bed\u53e5\u8f93\u51fa\u8fd9\u6761\u6570\u636e":123,"pserver\u5730\u5740\u7b49\u53c2\u6570\u4f7ftrainer\u53ef\u4ee5\u6b63\u786e\u8fde\u63a5\u5230pserv":104,"pserver\u76d1\u542c\u7684\u8d77\u59cb\u7aef\u53e3":104,"public":[14,18,31,46,47,52,69,72,74,75,96,97,98,110],"pwd\u53d8\u91cf\u4f1a\u5c55\u5f00\u4e3a\u5f53\u524d\u8def\u5f84\u7684\u7edd\u5bf9\u8def\u5f84":84,"py\u4e2d":71,"py\u7a0b\u5e8f":86,"py_paddle\u91cc\u9762\u63d0\u4f9b\u4e86\u4e00\u4e2a\u5de5\u5177\u7c7b":4,"pydataprovider2\u4f1a\u5c3d\u53ef\u80fd\u591a\u7684\u4f7f\u7528\u5185\u5b58":2,"pydataprovider2\u63d0\u4f9b\u4e86\u4e24\u79cd\u7b80\u5355\u7684cache\u7b56\u7565":2,"pydataprovider2\u662fpaddlepaddle\u4f7f\u7528python\u63d0\u4f9b\u6570\u636e\u7684\u63a8\u8350\u63a5\u53e3":2,"pydataprovider2\u7684\u4f7f\u7528":[1,3,80,123],"pydataprovider\u4f7f\u7528\u7684\u662f\u5f02\u6b65\u52a0\u8f7d":80,"pypi\u4e0a\u7684package\u540d\u79f0\u4e3apaddlepaddle\u548cpaddlepaddl":71,"pypi\u5b89\u88c5\u5305\u53ef\u4ee5\u5728":86,"python\u53ef\u4ee5\u89e3\u9664\u6389\u5185\u90e8\u53d8\u91cf\u7684\u5f15\u7528":2,"python\u5b89\u88c5\u5305\u652f\u6301linux":77,"python\u5c01\u88c5\u7684\u5b9e\u73b0\u4f7f\u5f97\u6211\u4eec\u53ef\u4ee5\u5728\u914d\u7f6e\u6587\u4ef6\u4e2d\u4f7f\u7528\u65b0\u5b9e\u73b0\u7684\u7f51\u7edc\u5c42":96,"python\u5e93yep":102,"python\u6807\u51c6\u5e93\u4e2d\u63d0\u4f9b\u4e86\u6027\u80fd\u5206\u6790\u7684\u5de5\u5177\u5305":102,"reader\u7684\u4f7f\u7528\u65b9\u5f0f\u90fd\u662f\u4e00\u81f4\u7684":35,"reader\u8f93\u51fa\u7684data":35,"recommendation\u6587\u4ef6\u5939\u5185\u5b58\u653e\u8bad\u7ec3\u6587\u4ef6":111,"release\u9875\u9762":71,"request\u524d":95,"request\u7684":95,"request\u88ab\u5408\u5e76\u540e":95,"return":[2,6,8,9,11,13,14,15,18,21,28,29,30,31,35,36,38,41,42,46,47,48,50,52,56,57,58,61,62,63,65,67,69,72,74,75,82,87,90,93,96,98,111,122],"rnn\u5373\u65f6\u95f4\u9012\u5f52\u795e\u7ecf\u7f51\u7edc":90,"rnn\u5bf9\u4e8e\u6bcf\u4e00\u4e2a\u65f6\u95f4\u6b65\u901a\u8fc7\u4e86\u4e00\u4e2alstm\u7f51\u7edc":90,"rnn\u603b\u662f\u5f15\u7528\u4e0a\u4e00\u65f6\u523b\u9884\u6d4b\u51fa\u7684\u8bcd\u7684\u8bcd\u5411\u91cf":92,"rnn\u6a21\u578b":123,"rnn\u76f8\u5173\u6a21\u578b":100,"rnn\u914d\u7f6e":91,"s3\u4e4b\u7c7b\u7684\u5206\u5e03\u5f0f\u5b58\u50a8\u4e4b\u4e0a":35,"search\u7684\u65b9\u6cd5":106,"sentences\u662f\u53cc\u5c42\u65f6\u95f4\u5e8f\u5217\u7684\u6570\u636e":90,"seq\u53c2\u6570\u5fc5\u987b\u4e3afals":92,"server\u4e2a\u6279\u6b21\u6253\u5370\u65e5\u5fd7\u8fdb\u5ea6":106,"server\u4e4b\u4e0a":34,"server\u4e4b\u95f4\u7684\u7f51\u7edc\u5e26\u5bbd":34,"server\u4f1a\u6682\u505c\u53c2\u6570\u66f4\u65b0\u5e76\u7b49\u5f85":34,"server\u4f1a\u83b7\u53d6parameters\u5185\u5b58\u7684":34,"server\u5185\u5b58\u4e2d\u7684\u6a21\u578b\u6570\u636e\u7684\u5b8c\u6574\u955c\u50cf":34,"server\u540c\u6b65\u7684\u4fdd\u5b58\u4e00\u4e2a\u7279\u5b9a\u65f6\u95f4\u70b9\u7684\u5168\u5c40\u68c0\u67e5\u70b9":34,"server\u5728\u96c6\u7fa4\u4e2d\u542f\u52a8\u540e":34,"server\u6545\u969c\u540e\u88abkubernetes\u91cd\u65b0\u542f\u52a8":34,"server\u6b64\u65f6\u8fd8\u9700\u8981\u901a\u8fc7\u7f51\u7edc\u8bbf\u95ee\u5206\u5e03\u5f0f\u5b58\u50a8\u4ee5\u4fdd\u5b58\u5feb\u7167":34,"server\u751f\u6210\u4e00\u4e2auuid":34,"server\u7684\u5355\u70b9\u6216\u591a\u70b9\u540c\u65f6\u6545\u969c":34,"server\u7684\u6570\u636e\u5feb\u7167":34,"server\u7684\u68c0\u67e5\u70b9\u5404\u81ea\u72ec\u7acb\u4fdd\u5b58":34,"server\u7b2c\u4e00\u6b21\u542f\u52a8\u6216\u4efb\u610f\u65f6\u95f4paramet":34,"sh\u8c03\u7528\u4e86":122,"short":[8,9,21,46,48,63,72,75],"simd\u6307\u4ee4\u63d0\u9ad8cpu\u6267\u884c\u6548\u7387":77,"size\u4e3a512":106,"size\u53ef\u80fd\u4f1a\u5bf9\u8bad\u7ec3\u7ed3\u679c\u4ea7\u751f\u5f71\u54cd":80,"size\u672c\u8eab\u662f\u795e\u7ecf\u7f51\u7edc\u7684\u8d85\u53c2\u6570":80,"size\u7684\u503c":2,"softmax\u5c42":121,"softmax\u6fc0\u6d3b\u7684\u8f93\u51fa\u7684\u548c\u603b\u662f1":96,"sparse\u8bad\u7ec3\u9700\u8981\u8bad\u7ec3\u7279\u5f81\u662f":80,"static":[28,38,55,72,74,119],"step\u51fd\u6570\u4e2d\u7684memori":92,"step\u51fd\u6570\u5185\u90e8\u53ef\u4ee5\u81ea\u7531\u7ec4\u5408paddlepaddle\u652f\u6301\u7684\u5404\u79cdlay":92,"store\u4e0b\u8f7d\u5b89\u88c5xcode\u5373\u53ef":116,"subseq\u7684\u6bcf\u4e2a\u5143\u7d20\u662f\u4e00\u4e2a0\u5c42\u5e8f\u5217":89,"super":[63,96],"swig\u652f\u6301\u7684\u8bed\u8a00\u6216\u8005\u89e3\u91ca\u5668\u6709\u5c40\u9650":54,"swig\u66b4\u9732\u7684\u63a5\u53e3\u4fdd\u7559\u4e86c":54,"swig\u751f\u6210\u7684\u4ee3\u7801\u4e0d\u80fd\u4fdd\u8bc1\u591a\u8bed\u8a00\u4ee3\u7801\u98ce\u683c\u7684\u4e00\u81f4\u6027":54,"swig\u76f4\u63a5\u8bfb\u53d6c":54,"swig\u9700\u8981\u5199\u4e00\u4e2ainterface\u6587\u4ef6":54,"swig_paddle\u4e2d\u7684\u9884\u6d4b\u63a5\u53e3\u7684\u53c2\u6570\u662f\u81ea\u5b9a\u4e49\u7684c":4,"switch":[31,55,67],"tag\u4e3a":71,"tensor\u5230\u5bf9eigentensor\u7684\u8f6c\u6362":98,"tensor\u5230eigentensor":98,"tensor\u5b9a\u4e49\u5728framework\u76ee\u5f55\u4e0b":98,"tensor\u662f\u4e00\u4e2a\u6b63\u5728\u5f00\u53d1\u4e2d\u7684\u6a21\u5757":98,"tensor\u6a21\u5757\u5bf9el":98,"tensor\u6a21\u5757\u6765\u5b9e\u73b0":97,"tensor\u6a21\u5757\u7684\u6587\u6863\u8f83\u5c11":98,"tensor\u6a21\u5757\u7684\u8be6\u7ec6\u4ecb\u7ecd\u8bf7\u53c2\u8003":98,"tests\u7684paddlepaddl":95,"tflops\u4e86":103,"tottime\u7684\u6bcf\u6b21\u8c03\u7528\u5e73\u5747\u65f6\u95f4":102,"trainer\u542f\u52a8\u9700\u8981\u4f20\u5165\u7aef\u53e3":104,"trainer\u63a5\u6536\u4e09\u4e2a\u53c2\u6570":87,"trainer\u8282\u70b9\u4e2a\u6570":104,"trainer\u9700\u8981\u548cpserver\u4fdd\u6301\u7f51\u7edc\u8054\u901a\u4ee5\u5b8c\u6210\u8bad\u7ec3":104,"true":[6,7,8,9,10,11,13,15,20,21,28,29,31,36,51,58,61,62,64,71,74,75,80,82,90,93,96,108,111,122],"true\u8868\u793a\u53cd\u5411\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":93,"try":[32,33,36,37,38,50,64,72,77],"type\u5b57\u6bb5\u5747\u4e0d\u5c3d\u76f8\u540c":55,"type\u6307\u5b9a\u4e3a":102,"ubuntu\u4e0b\u5b89\u88c5\u547d\u4ee4\u4e3a":102,"ubuntu\u5b89\u88c5\u5305\u7684\u529f\u80fd\u6b63\u786e\u6027":71,"unit\u5728\u4e00\u4e2a\u65f6\u95f4\u6b65\u5185\u8ba1\u7b97\u5f97\u5230\u7684\u4e2d\u95f4\u503c":81,"unsupported\u6a21\u5757":97,"update\u53c2\u6570\u65f6\u624d\u6709\u6548":106,"utf8\u7f16\u7801":121,"uts\u7b49linux":109,"v1\u7248\u672c":77,"var":[31,47,49,51,58,61,62,63,68,72,75,99],"vector\u662frank\u4e3a1\u7684tensor":98,"void":[31,36,38,46,47,52,53,54,55,61,62,73,74,96,97,98],"volume\u6302\u8f7d\u5230\u5bb9\u5668\u4e2d":109,"w0\u548c":122,"wbias\u662f\u9700\u8981\u5b66\u4e60\u7684\u53c2\u6570":122,"wheel\u5305":71,"while":[6,13,21,31,40,50,57,59,60,64,70,72,111],"wise\u8ba1\u7b97\u63d0\u4f9b\u4e86\u5f3a\u5927\u7684\u652f\u6301":98,"wmt14\u6570\u636e\u7684\u63d0\u4f9b\u6587\u4ef6\u5728":93,"words\u5373\u4e3a\u8fd9\u4e2a\u6570\u636e\u4e2d\u7684\u5355\u5c42\u65f6\u95f4\u5e8f\u5217":90,"words\u662f\u539f\u59cb\u6570\u636e\u4e2d\u7684\u6bcf\u4e00\u53e5\u8bdd":90,"x86_64\u548cmaco":77,"x\u4e0ey\u4e3a\u4e4b\u524d\u63cf\u8ff0\u7684\u8f93\u5165\u5c42":87,"x\u548cwindow":115,"y\u8868\u793a\u8f93\u5165\u6570\u636e\u662f\u4e00\u4e2a\u7ef4\u5ea6\u4e3a1\u7684\u7a20\u5bc6\u5411\u91cf":87,"yaml\u6587\u4ef6\u4e2d\u5404\u4e2a\u5b57\u6bb5\u7684\u5177\u4f53\u542b\u4e49":111,"yaml\u6587\u4ef6\u63cf\u8ff0\u4e86\u8fd9\u6b21\u8bad\u7ec3\u4f7f\u7528\u7684docker\u955c\u50cf":111,"zero\u4e09\u79cd\u64cd\u4f5c":106,AGE:110,AWS:[13,35,109,112,113],And:[7,8,10,13,14,15,28,30,36,40,41,43,48,50,64,72],But:[8,9,13,21,30,47,72,77,119],EOS:8,For:[4,7,8,9,10,13,21,28,29,31,37,38,39,41,42,47,49,52,53,56,57,59,60,61,62,63,64,65,67,68,69,70,73,74,76,101,103,119],IDE:94,IDs:[14,21,40,57],IRs:65,Its:[7,73],K8s:119,NFS:109,NMS:8,NOT:63,Not:[29,33,119],OPs:[65,66],One:[7,9,28,30,40,53,67,72],Ops:[68,70,74],PFS:44,QoS:110,Such:[52,63,75],TLS:[29,44],That:[8,13],The:[2,5,6,7,8,9,10,13,14,15,18,21,25,28,29,30,32,33,37,39,40,41,42,43,45,46,50,53,55,56,57,59,61,62,63,64,65,66,67,70,72,73,74,75,76,96,97,98,101,111,120,123],Their:[8,33],Then:[8,9,47,52,62,101],There:[7,8,14,21,28,29,31,32,33,38,40,41,45,46,50,56,57,59,60,63,65,70,72,73],These:[7,15,31,46,49,58,70,73,74],Use:[7,13,29,45,64,68,101],Used:[9,18,22,68],Using:[33,59,64,68,70,72],Will:[13,28],With:[8,9,61,75],YES:41,Yes:84,___embedding_0__:111,___embedding_1__:111,__align__:46,__cuda_align__:46,__device__:46,__doc__:74,__file__:41,__forceinline__:46,__fp16:46,__global__:46,__gradient_machines__:28,__hadd:46,__half:46,__half_raw:46,__impl__:74,__init__:[42,48,58,63,75,96,101,102],__main__:[4,48,122],__metaclass__:97,__name__:[4,48,122],__param_conf__:28,__rnn_step__:93,__square_error_cost_0__:111,__tmp_params__:28,__va_args__:69,__x:46,_binari:32,_create_global_var:63,_dtype:50,_librari:32,_link:9,_loss:48,_op:[50,97],_proj:8,_recurrent_group:93,_res2_1_branch1_bn:122,_source_language_embed:[93,121],_target_language_embed:[93,121],_test:32,_update_op:42,_value_index:50,a75:46,a_op:97,aaaaa:35,abbrevi:15,abc:8,abil:48,abl:[8,29,52,58,63,65,67,76,119],about:[8,9,15,31,32,41,45,56,64,65,72,74,101],abov:[2,7,8,21,29,31,32,33,37,46,47,49,56,57,58,59,61,63,65,67,74,75,101,103,119],abs:[8,9,20,30,48],abs_numerical_grad:30,absp:8,acceler:[8,34,59],accept:[6,8,13,29,68],access:[8,9,21,29,32,37,40,41,63,65],accessor:63,accord:[7,8,15,21,30,38,49,57,65,66,68,75],accordingli:[7,8],account:[68,119],accrodingli:36,accumul:[33,38,42,59],accur:[30,40],accuraci:[7,18,42],achiev:59,across:[8,13,56],act1:50,act2:50,act:[8,9,21,22,31,50,57,63,65,67,75,80,87,88,90,93],act_output:74,act_typ:[50,123],action:65,activ:[9,21,27,32,50,57,63,65,70,74,80,87,88,93,123],activi:9,actual:[8,36,48,50,59,74],adadelta:[80,123],adagrad:[23,59,73,123],adagradoptim:58,adam:[23,29,38,48,82,123],adamax:23,adamoptim:[121,123],adapt:[7,10,21,28],add:[8,9,13,20,21,23,26,28,30,31,32,36,40,42,46,47,51,58,59,62,63,65,66,68,70,72,77,95,98],add_activ:63,add_bia:63,add_depend:32,add_execut:32,add_input:[56,96],add_memori:56,add_output:56,add_scalar:[31,57,61],add_sum:63,add_test:[32,96],add_to:81,add_two:[31,56],add_unittest_without_exec:96,addattr:[74,97],addbia:96,addcom:[74,97],added:[7,8,18,26,28,31,46,59,66,70,95],adding:70,addinput:[74,97],addit:[8,9,68,70,75],addition:56,addop:47,addoutput:97,addr:33,address:[33,38,65,67,119],addrow:96,addtolay:8,addtyp:74,admin:119,administr:[40,119],adopt:[46,48],advanc:30,advantag:[30,46,59,64],adversari:[48,64],affect:[8,31],affili:57,afford:37,afi:2,aforement:32,after:[8,9,14,15,31,32,37,38,40,45,46,60,63,65,66,95,101],again:[29,33,59],age:[14,111],agg_level:[8,89,90],aggreg:42,aggregatelevel:[89,90],ago:32,alexnet_pass1:108,alexnet_pass2:108,algo_hrnn_demo:90,algorithm:[8,10,37,57,59,70,121],alia:21,align:[8,9,13],all:[2,6,7,8,18,20,21,28,29,31,32,33,36,38,40,41,43,45,48,49,50,53,55,56,57,58,59,61,63,65,67,68,74,80,92,111,119],alloc:[6,38,41,98],allow:[29,38,59,65,70],allow_only_one_model_on_one_gpu:[105,106,108],along:[15,21],alpha:[32,70],alreadi:[32,33,63,65,66,72,77],also:[8,9,14,21,29,31,32,36,39,46,47,48,49,50,56,57,59,60,61,62,63,64,70,72,74,75,103,119,123],altern:101,altogeth:119,alwai:[8,9,28,32,53,67,73,111],amazon:110,ambigu:64,amd64:109,amend:95,amnih:8,amount:21,analysi:101,ancestor:[61,63],android:115,android_abi:115,android_api:115,android_arm_neon:115,android_native_api_level:115,android_standalone_toolchain:115,android_toolchain:115,androideabi:115,angl:8,ani:[8,9,13,21,23,29,32,33,38,40,41,46,52,53,57,59,63,64,65,66,67,69,70],announc:46,anoth:[8,13,28,29,31,41,57,63,72,74],anroid_arm_mod:115,anyth:[13,57,64],anytim:48,anywai:101,api:[14,18,28,29,32,38,39,41,42,44,47,48,50,56,60,68,71,75,76,86,101,102,103,104,111,114,115,119,120],api_shar:32,api_test:32,api_trainer_config_helpers_lay:[93,123],apiserv:109,apivers:[109,110,111],appear:49,append:[2,13,21,28,42,57,63,64,90,93,104,111],append_backward:[101,102],append_backward_op:[23,58],append_batch_s:21,append_gradient_machin:28,append_op:63,append_oper:63,appleyard:103,appli:[8,21,48,49,72],applic:[25,46,49,63,68,101,103,110,119],applyl1:36,approach:[8,59,60,65,66,70,119,123],appropri:65,approxim:[20,59],apt:[84,101,102],arang:21,arbitrari:[8,53,65],arch:115,architectur:46,archiv:[14,54,55],area:48,arg:[2,7,9,25,50,58,74,82,97,111,122,123],arg_nam:8,argpars:111,args_ext:111,argu:62,argument:[2,8,13,15,25,31,36,37,58,60,62,63,67,111,115],argumentpars:111,argv:122,arithmet:46,arm64:[115,116],arm64_standalone_toolchain:115,arm:[46,115,116,117],arm_soft_fp_abi:115,arm_standalone_toolchain:115,armeabi:115,armv7:[46,116],armv8:46,around:[2,8,40,63,119],arrai:[4,6,8,13,15,21,28,38,49,57,61,63,64,68,75,80,82,87,97,122],arrang:75,arrow:48,articl:[49,95],artifici:20,arxiv:[8,9,20,48],as_row_vector:8,as_step_input:31,asgd:59,ask:[33,40],assert:4,assign:[7,8,37,46,119],associ:[60,69],assum:[7,8,31,65],assumpt:65,astyp:[64,97],asyc:33,async:[33,105],async_count:[105,106],async_lagged_grad_discard_ratio:106,async_lagged_ratio_default:[105,106],async_lagged_ratio_min:[105,106],asynchron:33,att_seq:9,attach:9,attend:9,attended_sequ:9,attenion:9,attent:[8,9],attr1:8,attr2:8,attr:[6,8,9,21,31,50,61,62,63,74,80,81,82,93,97],attr_map:74,attrdesc:61,attribut:[8,9,21,27,31,61,66,68,72,74,75],attributemap:97,attrproto:74,attrtyp:[61,74,97],attrvalu:74,auc:[42,105],author:[44,109],auto:[31,36,54,62,68,72,75,83,96,97,98,103],automat:[8,29,38,47,58,65,66,68,74,101],avail:[33,38,46,66,67,119],averag:[7,8,11,28,37],average_test_period:[105,106],avg:[89,103,123],avgcost:123,avgpool:[8,123],avoid:[30,31,33,59,60,65,103],avx:84,await:110,awar:[29,42,56,63,101],awk:104,aws:44,axes:21,axi:[8,21,80],axis:8,aync:66,azur:109,b2t:121,b363:110,b8561f5c79193550d64fa47418a9e67ebdd71546186e840f88de5026b8097465:110,back:[8,21,28,33,46,48,59,65],background:[7,8,70],background_id:[7,8],backpropag:30,backward:[5,8,9,26,30,31,36,38,48,58,59,60,62,69,70,96,97],backward_first:93,backward_op:30,backwardactiv:96,baidu:110,bake:65,balanc:66,bandwidth:46,bare:[109,110,119],barrier:104,barrierstatset:103,basci:50,base:[7,8,11,13,14,18,20,21,23,26,29,37,42,46,52,58,59,65,68,69,70,75,94,101],baseactiv:9,baseev:28,basematrix:96,basenam:7,basepoolingtyp:[8,9],basestr:[6,7,8,9,11,28],bash:[83,84,94,95,104,110,111],basic:[8,28,50,61,65,68,69,75],batch:[8,9,13,15,18,21,28,29,31,33,35,36,42,48,51,56,57,59,75,87,110,111,122,123],batch_0:122,batch_id:[28,48,87],batch_im:48,batch_images_from_tar:15,batch_label:48,batch_norm:48,batch_norm_lay:9,batch_norm_typ:8,batch_read:[35,64],batch_siz:[13,48,57,80,87,121,123],batch_szi:48,batch_z:48,batchnorm:[21,48],batchsiz:[8,96],bazel:32,bbbbb:35,bbox:[7,8],bcd:8,bcebo:14,bcm2708:117,bdist_wheel:71,beacus:50,beam:[8,93],beam_gen:[8,93],beam_search:[28,57,92,93],beam_siz:[8,57,93,105,106,108],becaus:[7,8,14,29,31,32,33,38,46,57,60,63,64,70,72,73,75,76,90,101],becom:[66,72],been:[8,9,32,37],befor:[8,9,33,40,45,49,59,60,64,70,77,80,97,101,119],begin:[7,8,18,36,38,42,45,49,57],beginiter:[28,29],beginn:93,beginpass:[28,29],begintrain:29,behind:75,being:[21,40,62,64,101],belong:[7,8,65,72],below:[8,31,33,38,46,53,60,64,65,66,70,75,76],benchmark:53,benefit:[9,40,41,57],bengio:20,besid:[8,14,65],best:32,besteffort:110,beta1:[10,23],beta2:[10,23],beta:[48,122],better:[9,32,57,119],between:[7,8,15,21,28,32,33,38,46,55,60,65,66,67,69,72],bgr:[15,122],bi_gru:9,bi_lstm:9,bia:[8,9,21,57,63,96,122],bias:8,bias_attr:[8,9,21,63,80,82,90,93],bias_initi:21,bias_param_attr:9,biases_:96,biasparameter_:96,biassiz:96,bidi:110,bidirect:[8,9],bidirectional_lstm:81,big:[66,119],bigger:33,bilinear:8,bilinear_interpol:8,bilinearfwdbwd:103,bin:[84,104,109,110,111],binari:[7,8,13,32,41,46,48,53,65,101],bind:[46,47,72],bit:46,bitcod:116,black:48,blank:8,block:[8,34,36,38,42,43,52,56,58,65,66,98],block_i:8,block_id:43,block_x:8,blockdesc:[31,49,63,68],blockdescbind:52,blueprint:57,bn_bias_attr:9,bn_layer_attr:9,bn_param_attr:9,book:[14,68,84,93,99,120],bool:[6,7,8,9,10,11,13,15,28,31,46,51,62,63,73,74,75,76,96,106,108],boot:[8,92,93,119],boot_bia:8,boot_bias_active_typ:8,boot_lay:[90,93],boot_stat:75,boot_with_const_id:8,bootstrapp:119,borrow:[48,75],bos_id:[8,93],both:[5,6,8,9,15,21,29,31,32,33,40,46,48,52,57,62,65,66,73],bottom:[28,123],bound:8,boundari:65,boundri:7,box:[8,48],brace:[31,49],brain:40,branch:[8,29,31,32,51,61,65,71,95],break_if:75,brief:[32,38,46,98],broadcast:[21,33,68,119],broken:95,browser:101,bsd:94,buf:36,buf_siz:13,buffer:[13,36,53,59,64,72],buffer_s:13,buffered_read:64,bufsiz:13,bug:95,build:[8,14,32,41,49,50,59,65,70,71,74,83,94,95,99,101,102,104,111,112,113,115,116,117],build_dict:14,build_doc:99,build_model:48,built:[32,46,65,74,75,101,102,103,119],bunch:53,c11:54,c703c041:95,c99:55,c_pre_init:21,cach:[46,65,80],cache_pass_in_mem:[2,80],cachetyp:[2,80],cacul:[9,42],caff:31,caffe2:31,caffe_poli:82,calc_batch_s:2,calcul:[7,8,9,18,21,30,33,38,42,46],call:[7,8,9,13,21,28,29,30,31,36,37,38,39,41,48,49,56,57,58,63,65,68,69,72,74,75,101,102,103,111,123],callabl:[6,8,13,14],callback:96,caller:[30,101],calrnn:90,can:[6,7,8,9,13,14,15,21,25,28,29,30,31,32,33,36,37,40,41,46,47,48,49,50,52,56,57,58,59,61,62,63,64,65,66,67,68,69,70,74,75,101,103,119],can_over_batch_s:2,cancel:40,candid:[8,57],candidate_activ:21,cannot:[67,68,72,75,77],cantain:50,capabl:[46,60,68],capac:70,capi:54,capi_prvi:55,caption:57,captur:8,care:[9,41,64,119],caret:28,carpedm20:48,carri:21,cast:46,cast_to_op_attr:74,cat:[13,15,84,104,111,122],categori:[8,14,33],categorig:14,categoryfil:110,caus:[33,45],cburg:8,cc_:32,cc_binari:32,cc_test:32,cclient:39,cde:8,cdn:14,ceil:8,ceil_mod:8,cell:[8,9,21],cell_activ:21,center:15,center_crop:15,cento:[86,119],central:70,ceph:[13,35,109],cephf:[35,41,44],certain:[58,72],certif:[29,44,109],cffi:54,cfg:110,cgo:54,chain:[13,49],challeng:[8,33,51],chanc:[29,46],chang:[8,14,32,37,41,60,61,64,69,72,95],channel:[8,9,15,21,103],channel_shar:8,chapter:[56,57],chapter_data:56,chapter_out:56,check:[2,13,31,32,62,68,82,95,96,106],check_align:13,check_attr:74,check_eq:96,check_fail_continu:2,check_grad:[30,97],check_l:96,check_sparse_distribution_batch:[105,106],check_sparse_distribution_in_pserv:[105,106],check_sparse_distribution_ratio:[105,106],check_sparse_distribution_unbalance_degre:[105,106],checker:68,checkgrad:106,checkgrad_ep:106,checkmark:119,checkout:95,checkpoint:[62,66],checksum:44,child:31,choic:32,choos:43,chosen:48,chunk:[37,44],chunk_schem:7,chunktyp:7,chw:15,circl:49,claimnam:111,clang:[46,54,95,115],clarifi:7,clariti:57,classic:8,classif:[8,20,49,123],classifi:[8,48,122],classification_cost:[65,80,90,123],classification_error_evalu:[7,123],classification_evalu:7,clean:[31,32,60,68,77,95],clear:[7,32,57,60,72],clearer:[60,63],clearli:72,click:101,client:[36,39,68,109],clip:[6,9,106,123],clock:8,clone:[8,83,94,99,101,102,115,117],close:[2,64,67,95],cloud:[32,33,41,44,45,67,68,119],cloud_read:13,cluster:[13,28,29,31,33,38,65,67,104,109,111],cluster_test_fil:104,cluster_train:[80,104,123],cluster_train_fil:104,cluster_train_v2:104,cmake:[55,77,83,94,95,97,99,101,103,115,116,117],cmake_build_typ:[101,115,116,117],cmake_c:[115,116],cmake_system_nam:[115,116,117],cmakelist:[32,96],cmatrix:[54,55],cmd:13,cnn:[8,110,123],coars:47,code:[2,4,8,13,29,32,40,43,46,47,48,49,53,58,59,60,62,64,65,66,68,69,70,74,75,96,110],codebas:68,coded_stream:82,codedinputstream:82,coeff:8,collabor:33,collect:[8,14,28],collectbia:96,color:[15,122],colour:14,column:[7,8,21,49,64,67,101],column_evalu:7,com:[8,9,14,32,48,83,94,95,99,101,102,109,110,115,117,119,122],combin:[7,8,9,13,23,28,58,68,72],come:[21,42,61,65,75],comma:[25,28,38],command:[13,25,32,36,41,45,94,96,101,108,110,111,112,113],commandlin:[103,111],comment:[32,50,74,90,111],commit:[32,110],common:[15,20,23,26,35,70],commonli:[45,70,101],commun:[33,38,39,65,66],compar:[30,32,68],comparison:32,compat:[46,47],compil:[8,32,50,52,65,69,73,74,76,94,104,115,116,117],complaint:32,complet:[8,9,14,23,28,31,33,37,38,44,49,53,68,101,110,111,119],complex:[9,40,57,68],complic:[8,47,64,65,75],compon:[50,65,75],compos:[13,29,47,50,56,63,68],composenotalign:13,composit:47,compress:37,comput:[8,9,21,25,29,30,33,46,50,53,58,59,65,66,67,69,72,76,97,98,101],computation:8,computationgraph:50,concat:[48,93],concaten:[8,9,48,56,75,80],concentr:68,concept:[7,29,47,48,50,56,57,59,60,61,67,72,75,76],conceptu:[48,50],concern:[29,42],concis:[48,75],conckerneltrac:25,concret:68,concurr:[33,40,66],cond:[21,31,51,61],condit:[8,37,51,65,110],condtion:48,conf:[4,8,82,90,104,111,121,122],conf_paddle_gradient_num:111,conf_paddle_n:111,conf_paddle_port:111,conf_paddle_ports_num:111,conf_paddle_ports_num_spars:111,confer:20,confid:8,confidence_threshold:8,config:[6,8,25,35,45,57,87,96,105,106,108,109,110,111,119,123],config_:[36,106],config_arg:[105,106,108,122,123],config_bas:[7,8,28],config_lay:96,config_len:38,config_pars:[4,96],config_proto:38,configprotostr:82,configur:[8,21,28,36,38,40,41,50,63,66,88,96,119,121,122],confirm:45,conflict:[72,95],confus:[15,48],conll:14,connect:[9,21,41,65,66,78,110,119,123],connectionist:8,consequ:[8,9],consid:[7,8,20,62,119],consider:[8,9],consist:[7,8,14,15,37,53,61,64,68,69,74],consolid:31,const_cast:97,constant:[8,20,21,50,52,67,82],constantiniti:21,constraint:[65,72],construct:[7,29,50,56,63,68,72,74,76],constructbackwardgraph:49,constructoptimizationgraph:49,constructor:[21,46,63,68,72,74],consum:[33,101],contact:40,contain:[2,7,8,9,11,13,14,15,21,28,29,31,37,43,48,50,53,60,63,67,68,69,72,73,74,75,76,94,110,111],content:[38,45,53,57,99],content_dir:99,content_len:38,context:[2,8,9,14,72,73,80,93,97,98,109],context_attr:9,context_len:[8,9,123],context_proj_layer_nam:9,context_proj_param_attr:9,context_project:9,context_start:[8,9,123],continu:[7,33,53],contrast:8,contrib:70,contribut:70,contributor:68,control:[6,31,67,110,119],conv2d:48,conv:[9,21,48],conv_act:[9,22],conv_batchnorm_drop_r:[9,22],conv_bias_attr:9,conv_filter_s:[9,22],conv_layer_attr:9,conv_num_filt:[9,22],conv_op:8,conv_pad:[9,22],conv_param_attr:9,conv_strid:9,conv_with_batchnorm:[9,22],conveni:[29,50,58,74],convent:38,convers:[46,65],convert:[2,4,14,21,35,46,64,69],convlay:8,convolut:[8,9,13,21,22,48,63],convoper:8,convproject:8,convtranslay:8,convtransproject:8,cool:95,coordin:[33,38],copi:[28,29,37,40,45,49,56,57,59,75],core:[6,18,50,55,59,60,75],coreo:119,corner:68,corpu:14,correct:[8,21,30,46],correctli:[7,13,46,48],corresond:46,correspoind:29,correspond:[8,21,26,29,31,32,46,50,51,56,57,63,67,68,69,70,74,82,101],corss_entropi:29,cortex:46,cos:[8,74],cosin:[8,21,74],cosineop:74,cosineopproto:74,cosineopprotomak:74,cost:[21,28,29,49,58,61,62,65,67,87],cost_id:8,cost_np:62,cost_val:65,could:[8,13,28,29,30,37,46,56,58,59,60,61,63,64,65,66,69,101],count:[7,33,41,42,62,64,103,104,106,108,110],counter:[25,33,37,49],cours:[7,41],covari:8,cp27:86,cp27m:86,cp27mu:86,cpp:[30,36,47,54,55,60,66,68,76,82,90,96,103,111,123],cprofil:[101,102],cprofilev:[101,102],cpu:[2,6,8,30,41,46,59,60,65,67,68,70,71,94,97,98,101,103,108,110,111],cpu_avx_mkl:86,cpu_avx_openbla:86,cpu_per_p:67,cpu_per_train:67,cpuinfo:84,cpuplac:[97,98],cpusparsematrix:55,crash:[33,103],creat:[6,8,13,18,21,28,29,30,31,33,38,42,44,45,46,47,48,49,56,58,59,60,63,69,70,82,87,95,96,99,104,110,111,119],create_backward_pass:58,create_bias_paramet:96,create_block:63,create_cloud_job:67,create_doc_str:74,create_input_paramet:96,create_op:97,create_oper:47,create_optimization_pass:[23,58],create_paramet:63,create_python_ops_creatation_funct:74,create_rnn:31,create_rnn_op:56,create_st:18,create_tmp_var:63,create_tmp_vari:63,create_var:63,create_whileloop:75,createfromconfigproto:4,creategradientoper:69,createop:74,createoper:31,createvari:31,creation:47,creator:[13,14,35,68,69],creator_:69,credenti:45,critic:[48,101],crlf:95,crop:[15,122],crop_siz:[15,122],crope:15,cross:[8,63,82,115,116,117,123],cross_entropi:[8,29,48,67],cross_entropy_with_selfnorm:8,crt:[44,109],csc:96,csr:96,csv:[25,82],ctc:7,ctc_evalu:7,ctest:[83,94,95,97],ctor:63,ctrl:[94,104],ctx:[97,98],cuda7:86,cuda8:[83,86],cuda:[25,32,68,94,103,106],cuda_dir:[105,106],cuda_fp16:46,cuda_profil:25,cuda_so:[77,84],cuda_visible_devic:80,cudaconfigurecal:103,cudadevicegetattribut:103,cudaeventcr:103,cudaeventcreatewithflag:103,cudafre:103,cudagetdevic:103,cudagetdevicecount:103,cudagetdeviceproperti:103,cudagetlasterror:103,cudahostalloc:103,cudalaunch:103,cudamalloc:103,cudamemcpi:103,cudaprofilerstart:103,cudaprofilerstop:103,cudaprofilestop:103,cudaruntimegetvers:103,cudasetdevic:103,cudasetupargu:103,cudastreamcr:103,cudastreamcreatewithflag:103,cudastreamsynchron:103,cudeviceget:103,cudevicegetattribut:103,cudevicegetcount:103,cudevicegetnam:103,cudevicetotalmem:103,cudnn:[8,11,32],cudnn_batch_norm:8,cudnn_conv:8,cudnn_conv_workspace_limit_in_mb:[105,106],cudnn_convt:8,cudnn_dir:[105,106],cudnnavginclpadpool:8,cudnnavgpool:8,cudnnv5:83,cudrivergetvers:103,cuinit:103,cumtim:[101,102],cumul:8,cur_mem:57,curl:[13,109],curli:[31,49],current:[2,8,31,32,33,36,38,42,56,57,59,60,63,66,67,72,75,99,109,120],current_block:[61,63],current_oper:61,current_word:[80,93],currentcost:123,currentev:123,curv:29,custom:[23,29,41,46,48,57,59,68],custom_batch_read:64,cut:[66,75],cut_lin:13,cutoff:14,cv2:15,cxx:116,cxx_compil:[115,116,117],cxx_flag:116,cxx_flags_minsizerel:115,cxxabi_1:86,cycl:33,cyclic:8,cython:54,d_b0:48,d_b1:48,d_b2:48,d_block:48,d_f:48,d_g:48,d_h0:48,d_h0_bn:48,d_h0_relu:48,d_h1:48,d_h1_bn:48,d_h1_relu:48,d_h2:48,d_loss:48,d_loss_fak:48,d_loss_real:48,d_optim:48,d_step:48,d_t:48,d_w0:48,d_w1:48,d_w2:48,dalla:2,dandroid_abi:115,dandroid_arm_mod:115,dandroid_arm_neon:115,dandroid_standalone_toolchain:115,darwin:109,dash:48,dat:35,data:[2,7,14,15,18,28,29,30,31,35,36,37,42,44,46,48,49,50,52,53,56,57,58,59,60,61,63,66,67,68,70,72,73,74,75,76,80,87,88,90,93,98,104,105,110,111,112,121,122,123],data_batch:80,data_config:4,data_dir:121,data_fil:15,data_i:48,data_lay:[2,36,80,90,123],data_layout:21,data_read:[13,64],data_reader_creator_random_imag:64,data_shar:75,data_typ:[13,14,53,73,76,87,88,93],data_x:48,databas:14,datacent:[35,45],datacenter1:35,datacenter2:35,datacenter_1:35,datacenter_2:35,datacenter_nam:35,datadim:8,datafeed:16,dataflow:50,dataprovid:[1,80,82,111,123],dataprovider_:123,dataprovider_bow:123,dataprovider_emb:123,dataproviderconvert:4,datasci:8,dataset:[35,41,59,64,87,88,93,101,102,104,122],dataset_nam:15,datatyp:[14,18,73,76],dcgan:48,dcmake_build_typ:99,dcmake_install_prefix:[115,116,117],dcmake_system_nam:[115,116,117],dcuda_arch_nam:83,dcudnn_root:83,ddim:98,dead:33,deal:119,deb:95,debug:[30,45,65,99,101,102],debug_str:50,decai:[10,23,26],decar:13,decayr:36,decent:37,decid:[29,40,48,53,59,69,70,73,120],declar:[8,21,31,48,56],declear:21,decod:[8,9,92,93],decoder_boot:93,decoder_dim:57,decoder_group_nam:93,decoder_input:[57,80,93],decoder_mem:[57,93],decoder_prev:9,decoder_s:[80,93],decoder_st:[9,93],deconv:[8,48],deconvolut:[8,21],decor:[2,13],deduc:68,deep:[8,20,40,48,49,68,70,103,122],def:[2,4,8,13,29,30,35,41,42,47,48,50,56,57,58,63,64,74,75,80,82,87,90,93,96,97,111,122,123],def_block:48,default_block:48,default_decor:111,default_devic:108,default_main_program:18,default_param_attr:63,default_st:75,default_startup_program:18,default_valu:108,defaultinfervartyp:52,defect:60,defer:40,defin:[2,8,9,13,20,23,26,28,29,31,32,33,40,46,47,48,50,56,61,63,64,66,68,72,74,75,80,87,97,101,123],define_py_data_sources2:[2,82,122,123],definit:[31,33,37,43,61,65,69,74,75,101,121],definiton:47,degre:8,delai:59,delet:[41,44,95,120],delimit:[7,82],deliv:119,delta:[8,30],delv:[8,20],demand:33,demo:[4,8,14,68,110,112,121,122,123],dens:[8,13,38,39,73],dense_arrai:13,dense_vector:[2,4,13,87,88],dense_vector_sequ:13,dep:32,depend:[31,32,33,41,43,50,62,65,66,73,119],dependent_var:62,deploi:[8,119],deploy:[50,53,68,119],deprec:8,depth:31,dequeu:66,deriv:[5,29,51,58],desc:[31,53,63,74,75],desc_:31,descend:75,descent:[8,33,59],descproto:53,describ:[29,31,32,37,53,56,57,60,61,63,68,73,74,76,110],descript:[7,31,32,52,53,69,73,76,111],deseri:[28,53,60],deserializ:68,desgin:49,design:[8,13,20,36,54,59,70,119],desir:[13,33,59,110],destin:[38,45],destroi:31,destruct:72,det_output:7,detail:[6,7,8,9,10,21,30,37,41,45,48,50,53,56,63,65,67,70,72,75,76,101,119],detect:[52,95],detection_evalu:7,detection_output:7,determin:[8,13,31,68],dev:[68,77,84,94,95,101,102,115,119],dev_ctx:31,develop:[32,52,60,63,69,71,95,101,116],deviat:[6,20],devic:[6,42,43,46,50,60,65,67,68,77,84,98,108],device_context:97,device_context_:97,devicecontext:[31,97],deviceid:108,devid:8,dhcp:119,diagram:56,diamond:48,dic:15,dict:[2,7,14,28,63,67,82,90,111],dict_dim:[80,90],dict_fil:[7,90],dict_siz:[14,36,57],dictionari:[2,7,8,14,28,29,30,63,67,80,123],dictrionari:123,did:60,diff:95,diff_mat:30,differ:[7,8,28,31,32,33,38,40,42,43,46,48,50,51,57,59,62,65,66,67,69,72,75,101],differenti:47,difficult:[7,30],difficulti:20,digit:8,digraph:50,dilat:8,dilation_i:8,dim0:97,dim1:97,dim:[8,13,36,53,56,68,73,76,96,97,98,121],dim_:98,dimens:[5,8,9,11,13,21,48,68,73,75,80,98],dimension:[8,21],dimes:8,dios_arch:116,dios_enable_bitcod:116,dios_platform:116,dios_use_veclib_for_bla:116,dir:[111,115,122],direct:[8,9,15,59,101],directli:[9,20,23,26,32,39,41,46,60,65,74,75],directori:[8,32,35,40,44,45,95,99,103,110,120],disabl:82,disadvantag:[59,63],discard:[13,33,37,57,95,106],discexp:82,discount:8,discov:33,discret:8,discrim:48,discuss:[29,31,37,38,39,65],disk:53,dispatch:[60,65],displai:[41,45],dist:[71,77,83],dist_train:[29,41],distanc:[7,8],distinguish:32,distribut:[8,20,31,37,38,39,40,42,66,67,68,76,106,112,113,119],distribute_test:[105,106],disucss:29,div:21,divid:[8,10,42,43,74,76,101],diy_beam_search_prob_so:[105,106],do_forward_backward:64,doc:[4,13,50,56,75,97,99,104,111],doc_cn:99,docker:[71,77,83,84,95,99,104,110,111,112,113,115,119],docker_build:29,docker_clust:104,docker_push:29,dockerfil:[94,95,111,115,117],document:[8,9,30,44,49,56,57,65,68],documentari:2,doe:[9,33,37,38,40,41,46,50,56,60,63,65,66,68,69,70],doesn:[6,8,13,29,31,64,67,101],dog:122,doing:[36,40,49,65],don:[9,29,32,47,49,64],done:[7,8,9,32,33,37,38,52,53,59,65,69,70,95,101,103,111],dot:[8,9,97],dot_period:[106,108,111],dotmuloper:8,dotmulproject:8,doubl:[46,49,65,97,106],down:103,download:[14,32,33,36,40,44,110,119],doxygen:95,dozen:32,dpython_execut:77,dpython_include_dir:77,dpython_librari:77,draw:57,drive:72,drop:[8,9,21,57],drop_fc:81,drop_rat:[6,81],dropout:[6,9,81],dropout_prob:21,dropout_r:[8,81],drpi_arm_neon:117,drpi_toolchain:117,drwxr:110,dst:38,dtoh:103,dtype:[4,18,21,50,63,82,122],due:[37,40,48,57,63,101,102],dummi:[28,37],dump:53,duplic:[21,66],durat:37,dure:[2,8,9,21,26,31,33,37,40,41,42,59,68,119],duse_eigen_for_bla:115,dwith_c_api:[55,115,116,117],dwith_doc:99,dwith_gpu:[83,99,117],dwith_mkl:99,dwith_profil:103,dwith_python:[55,117],dwith_swig_pi:[55,115,116,117],dwith_test:[83,97,116],dwith_tim:103,dynam:[2,38,55,56,64],dynamic_cast:96,dynamic_recurrent_op:75,e2e:119,each:[2,7,8,9,11,13,14,18,21,28,30,32,33,36,37,38,40,41,42,43,49,52,56,57,60,62,63,64,65,66,68,69,72,73,74,75,101,119],each_feature_vector:5,each_pixel_str:2,each_time_step_output:5,each_word:2,earli:46,eas:[13,52],easi:[30,57,59,64,68,70],easier:[29,46,64,66,75],easili:[29,48,64,69,72],ec2:109,echo:[77,84],edg:[15,67],edit:7,editor:63,edu:[8,14,110],effect:[8,28],effici:[8,53,64,65],effort:65,efg:8,eigen:[46,59,68,70,97],eigen_use_gpu:97,eigenmatrix:98,eigentensor:98,eigenvector:98,either:[8,9,13,28,29,48,51,52,56,59,65,70],electron:110,elem_dim:8,element:[7,8,9,13,15,21,28,30,37,50,57,66,67,68,97],element_typ:38,elementari:68,elementwis:21,elif:[29,74],els:[29,36,41,48,51,52,65,66,72,74,84,90,96,122],emac:94,emailweixu:32,emb1:[36,90],emb2:[36,90],emb:[80,82,90,110,123],emb_group:90,emb_para:82,emb_param_fil:82,emb_sum:80,embed:[29,31,36,52,57,66,73,75,82,93,121],embedding_lay:[36,80,90,123],embedding_nam:[8,93],embedding_s:[8,93],empir:8,emplace_back:96,emploi:74,empti:[7,13,33,57],emul:46,enabl:[6,8,31,32,37,50,66,94,103],enable_grad_shar:[105,106],enable_parallel_vector:106,enableonstart:25,enc_proj:[9,93],enc_seq:9,enc_vec:93,encapsul:[38,67],encod:[9,37,57,90],encoded_proj:[9,93],encoded_sequ:[9,93],encoded_vector:93,encoder1:90,encoder1_expand:90,encoder1_rep:90,encoder2:90,encoder2_rep:90,encoder_ctx:57,encoder_ctx_expand:57,encoder_ctx_proj:57,encoder_dim:57,encoder_last:8,encoder_out_seq:57,encoder_s:93,encount:36,encourag:65,end2end:119,end:[7,8,28,31,50,57,60,64,72,93,95],end_pass:29,end_po:8,endforwardbackward:28,endian:53,enditer:[28,29,87],endpass:[28,29],endpoint:[13,35],endtrain:29,engin:41,english:8,enlarg:97,enough:31,enqueu:66,ensembl:9,ensur:[33,72],enter:31,enterpris:68,entir:[8,9,38,40],entiti:[7,31,72],entranc:43,entri:[13,37,41,52],entropi:[8,63,123],entry_point:41,enueu:66,enumer:[5,82],env:[80,99,101,102,111],environ:[29,67,77,101,103,110],environmenterror:104,eol:95,eos_id:[8,93],epoch:48,epsilon:[8,10,21,23],equal:[8,9,21,33,75,90,97],equat:[7,8,9,10,21,97],equival:[29,31,51,74,119],error:[6,7,8,9,21,29,30,37,45,46,72,77,82,97,106,123],error_clipping_threshold:[6,80],especi:[8,9],essenc:29,essenti:[8,29,43,46],estim:[8,29,59,66],eta:110,etc:[7,13,21,31,42,59,64,65,72,119],etcd:[13,28,33,37,38,40],etcd_addr:38,etcd_endpoint:13,eth0:111,euclidean:8,eval:[7,18,31,42,48,67,68,123],eval_program:[18,42],eval_result:42,evalu:[8,16,27,28,40,43,50,62,65,67,103],evaluate_difficult:7,even:[29,46,63,64],evenli:38,event:[87,110],event_handl:[28,29,87],eventu:[65,75],everi:[7,8,9,13,18,29,33,37,38,40,42,49,50,52,56,63,66,72,74],everyth:[48,65,66],evid:60,exactli:[8,9,80],exampl:[7,8,9,13,14,15,21,28,31,41,42,45,47,48,49,50,52,56,57,60,61,63,64,66,68,69,70,73,75,101,122],exc_path:77,exceed:8,except:[8,14,40,49,75],exchang:60,exclud:8,exclude_mod:8,excluded_chunk_typ:7,exconv:8,exconvt:8,excut:13,exdb:14,exe:109,execut:[32,33,37,41,42,43,48,50,67,69,101],executioncontext:[97,98],executor:[16,18,42,46,48,58,61,101,102],exist:[29,31,33,45,57,63,64,67,69,74,75,98],exit:[38,45,110],exp:82,expand:[57,90],expand_a:[8,89,90],expand_lay:90,expand_level:[8,89],expandconvlay:8,expandlevel:89,expans:8,expect:8,experi:53,expert:32,expir:33,explain:[7,33,47,49,101],explan:[8,30,41,65,72],explicit:[75,96],explicitli:[29,65,67],explor:[8,57,70],expon:8,exponenti:5,expos:[39,53,75],express:[29,42,50,66],ext:8,extend:[7,59,66,75],extens:[40,57,66],extent:55,extern:[32,54,55,68],external_librari:32,extra:[6,8,9,65,70,119],extra_lay:28,extraattr:[6,108],extraattribut:8,extraattributenon:8,extract:[7,8,60,65,121,122],extract_fea_c:122,extract_fea_pi:122,extralayerattribut:[6,9,80,81],extralayeroutput:9,extrem:8,f1205:82,f120da72:110,fa0wx:110,fabric:104,face:[32,70],fact:[61,63],factor:[6,10,21],factor_s:8,factori:54,fail:[33,37,57,67,77,82,106,110],failur:[33,38],fake:48,fake_imag:64,faked_imag:48,fall:[46,62],falloc:44,fals:[6,7,8,9,10,13,21,22,23,30,31,51,56,61,62,64,73,76,80,87,88,90,93,96,97,104,108,110,121,123],false_block:[31,51,61],false_label:64,false_neg:42,false_posit:42,false_read:64,fan_in:20,fan_out:20,faq:114,far:75,fashion:65,fast:[8,37,103],faster:[8,9,26,33],fault:[28,37,68],fbd1f2bb71f4:110,fc1:[50,67,96,108],fc1_bia:50,fc1_weight:50,fc2:[50,67,108],fc3:[50,108],fc4:108,fc_act:9,fc_attr:9,fc_bias_attr:9,fc_layer:[63,74,80,82,90,108,123],fc_layer_nam:9,fc_mat:28,fc_op:74,fc_out:31,fc_output:74,fc_param_attr:9,fc_without_b:31,fclayer:96,fcop:47,fdata:90,fea:122,fea_output:122,feasibl:59,featur:[2,5,8,13,14,46,50,65,95,122],feed:[9,28,29,49,56,65,67,70,87],feed_dict:[48,67],feeder:13,feedforward:20,festiv:2,fetch:[14,33,36,62,65,77,95],fetch_op:62,few:[32,33,59,64,65,73],fewer:[8,63],fg0:8,field1:28,field2:28,field:[8,28,31,50,52,53,62,69,73,74,80],fifth:49,figur:[29,32,48,56,63,65,66,121],file:[2,7,8,13,14,15,25,28,29,32,33,35,37,38,40,41,44,45,50,53,55,64,65,68,76,95,97,119,122],file_list:2,file_nam:[82,90,122,123],file_typ:13,filenam:[2,15,35,63,80,101,102],fileoffset:44,filesystem:[40,41,44,65],fill:[8,21,33,37,63],fill_zero_grad:68,filter:[8,9,21],filter_s:[8,9,21,22],filter_size_h:21,filter_size_i:8,filter_size_w:21,filter_strid:21,find:[8,21,31,33,40,46,50,57,72],find_var:30,findop:31,findvar:[31,72],fine:[6,37,47],finish:[33,37,40,41,74,110],first:[8,21,28,29,31,33,37,40,41,45,48,49,50,56,57,61,62,63,65,68,73,74,75,97,98,119,123],first_seq:93,firstli:[7,8],firstn:13,firstseen:110,fit:[14,46,53,57,68],five:61,fix:[6,8,54,65,95,101],flag:[8,14,21],flatten0:50,flatten:[50,61,63,98],flatten_result:80,flexibl:[8,9,29,38,49,56,57,59,64,65,75],flip:15,flist:104,float16_t:46,float32:[4,13,21,46,47,48,63,64,82,97,122],float_16:21,float_to_half_rn:46,floor:[8,82],flow:[21,31,56,71],fluid:[18,20,21,22,23,25,26,101,102],fmt:82,fname:82,focu:[50,101],folder:[32,35,41,45],follow:[7,8,9,10,13,15,21,28,29,30,31,32,33,37,41,46,47,48,49,50,51,52,56,57,59,61,62,63,64,65,66,68,69,70,72,73,74,75,101,112,113,119],forbid:29,forc:63,force_load:54,forest:31,forget:[10,29],forget_bia:21,fork:8,form:[8,9,42],format:[7,13,14,15,21,25,28,30,37,46,57,75,88,95,96],former:[29,32,59],formula:[8,9,10,30],formular:8,forth:48,forward:[5,8,9,30,31,36,38,48,53,58,60,61,64,68,69,70,73,96],forward_op:30,forwardactiv:96,forwardbackward:28,forwardtest:4,found:[8,46,61,70,72],four:[7,42,49],foward:62,fp16:[46,68,76],fp32:[68,76],fp64:76,fparam:82,fpga:67,frac:21,frame:[7,68,75],framework:[29,31,42,46,50,59,61,68,70,72,74,97,101,102],free:[14,119],frequenc:14,frequent:[37,64,68,70],fresh:[40,67],friend:72,friendli:48,fritz:8,from:[2,4,7,8,9,13,14,15,20,21,28,30,31,32,33,35,36,37,38,42,45,46,47,48,49,50,51,56,57,58,60,61,63,64,65,66,67,68,69,72,75,77,92,97,98,101,102,103,110,119,121,123],from_no_sequ:[8,89],from_sequ:[8,89],from_tar:28,fromfil:[64,82,122],fromstr:82,front:50,fuction:25,fulfil:103,full:[8,33,40,56,59,119],full_matrix_project:[9,90,93],fulli:[21,65,66,119,123],fullmatrixproject:8,fullsiz:36,fully_matrix_project:9,fullyconnect:[50,63],fullyconnectedlay:96,func:[2,13,37,69],funciton:[9,21],functor:[47,50],fundament:[46,66,68],further:[8,74,119],furthermor:67,futur:[8,40,46,56,65,68,120],fvs:74,fwd_op:69,g_b0:48,g_b1:48,g_b2:48,g_block:48,g_h0:48,g_h0_bn:48,g_h0_relu:48,g_h1:48,g_h1_bn:48,g_h1_relu:48,g_h2:48,g_im:48,g_loss:48,g_optim:48,g_program:63,g_step:48,g_w0:48,g_w1:48,g_w2:48,gain:8,gamma:122,gan:29,gangliao:32,gate:[8,9],gate_act:[8,9,90],gate_activ:21,gate_attr:8,gate_bias_attr:8,gate_param_attr:8,gate_recurr:8,gather:[8,60,97],gauss:6,gaussian:20,gaussian_normal_random:48,gcc:[46,54,68,83,94,101,115,117],gcc_3:86,gce:109,gcepersistentdisk:109,gcreators_:74,gen:8,gen_proto_pi:99,gen_rand_param:82,gender:[14,111],gendrated_id:57,gener:[2,7,8,9,13,18,28,29,30,31,32,33,35,37,38,40,47,52,59,61,62,63,64,65,66,68,69,73,74,75,103,108,111,121],generated_id:57,generated_scor:57,generated_word_embed:8,generatedinput:[8,92,93],genr:111,geq:8,gereat:7,get:[2,7,8,13,14,28,30,31,32,33,37,38,40,41,44,48,50,56,57,63,67,68,69,72,74,75,84,96,101,102,104,110],get_all_op_proto:74,get_block:63,get_cloud_job:67,get_config_arg:108,get_data:[110,123],get_dict:14,get_dim:30,get_embed:14,get_float_el:30,get_grad:28,get_input_lay:96,get_model:122,get_movie_title_dict:14,get_numeric_gradi:30,get_numerical_gradi:30,get_output:30,get_sample_from_lin:80,get_shap:28,get_support:[77,86],get_symbol:50,get_tensor:30,get_vari:31,get_word_dict:14,getbatchs:96,geteigendevic:98,getenv:[29,41,104,111],gethostbynam:111,gethostnam:111,getidmap:111,getinfervartyp:52,getinput:96,getinputgrad:96,getinputvalu:96,getkerneltyp:46,getlayeroutput:28,getmat:36,getoptconfig:36,getoutputgrad:96,getoutputvalu:96,getparam:36,getparameterconfig:36,getparameterptr:96,getparameterspars:36,getparametersremot:36,getplac:[97,98],getpodlist:111,getsiz:96,gettask:37,gettranspos:96,getw:96,getweight:96,getwgrad:96,gist:9,git:[71,77,83,94,95,99,115,117],github:[9,32,48,83,94,95,99,101,102,115,117,122],give:[2,33,56,63,68],given:[8,13,21,28,38,40,47,48,57,64,66,67,70,75],glibc:[115,117],glibc_2:86,glibcxx_3:86,glide:32,global:[6,21,23,29,31,32,33,50,60,67,68,72,74,94,103],global_block:63,global_learning_r:6,global_pool:21,global_step:23,globalstat:103,globalstatinfo:103,globe:2,glorot10a:20,glorot:20,glusterf:109,gnueabihf:117,go_librari:32,go_test:32,goal:[46,49,68],gob:37,godep:32,godoc:54,goe:[8,9,33,51,72],going:[47,59,101,119],golang:32,good:[48,59,64,70,101,119],goodfellow13:8,googl:[29,68,82,101,102,115],googleapi:109,got:72,gprof:102,gprotos_:74,gpu:[6,8,11,30,41,42,46,59,60,65,67,68,70,71,77,84,86,88,94,97,103,104,108,119,122],gpu_id:[80,106,108],gpu_per_train:[65,67],gpugpu_id:105,gpukernel:68,gpuplac:97,gpustarttimestamp:25,grab:33,grad:[30,38,73,106],grad_op_class:68,grad_op_maker_:69,grad_op_typ:[68,69],grad_op_type_:69,grad_share_block_num:[105,106],grad_var_nam:30,gradient:[6,7,8,10,20,21,23,26,28,33,37,49,52,58,59,60,68,73,101,104,106,111,123],gradient_check:97,gradient_clipping_threshold:[6,80,123],gradient_evalu:7,gradient_flat:30,gradient_machin:[28,55],gradientcheck:97,gradientmachin:[4,28,55,60,111],gradientmachine_:36,gradopdescmak:[52,69],gradopdescmakerbas:69,gradopmak:69,grai:15,grain:47,gram:121,grandient:28,graph:[8,21,28,31,32,33,42,43,48,56,59,61,65,67,76],great:[66,119],greater:[8,59],greaterthan:74,green:48,grep:[84,104],gridsize3d:25,groudtruth:93,ground:[7,8],group:[9,21,22,37,50,119],group_input1:93,group_input2:93,group_input:[90,93],grouplen:14,grpc:119,gru:[8,57,123],gru_bias_attr:9,gru_decod:93,gru_decoder_with_attent:93,gru_encoder_decod:121,gru_layer_attr:9,gru_memori:9,gru_out:57,gru_siz:123,gru_step:[57,93],gru_step_lay:9,grumemori:[9,81,93],gserver:[8,96],gsizex:103,guarante:63,guard:36,guest:86,gui:101,guid:[25,44,68,110],gzip:[13,37,110],h0_bn:48,h_prev:31,hadoop:[13,29],half:46,half_to_float:46,hand:68,handi:32,handl:[13,29,41,50,60,64,65,67,72,75],handler:[28,31],happen:[37,74],hard:[57,65,66,75],harvest:123,has:[7,8,9,14,21,25,29,30,31,32,33,37,38,40,42,46,48,50,53,57,61,65,66,67,68,73,74,103,119],has_kei:28,has_selected_colum:8,hasdependentvar:62,hassubseq:90,have:[8,9,13,29,30,31,32,33,37,38,40,41,46,47,48,49,53,56,57,59,60,61,63,64,65,66,67,68,69,72,73,76,119],hdf:[13,35],head:[97,104],header:[38,53,55,68,82,122],headip:104,height:[8,13,15,31,54,64,82,96,97],height_:73,held:33,hello:29,help:[4,8,21,31,45,50,57,64,68,75,95,101],helper:[8,21,65,69,75],henc:[59,63,65,69,70,72],here:[6,7,8,9,13,29,32,33,39,45,49,50,56,64,70,74,104,119],heterogen:[65,66],heurist:[8,57,66],hidden:[8,9,58,65,81,82],hidden_a:82,hidden_b:82,hidden_dim:[21,90],hidden_out:31,hidden_s:9,hierach:92,hierarch:[8,61,63,68,90],hierarchi:68,high:[6,20,46,119],higher:[47,56,75],highest:[13,31],highli:[14,75],him:29,hint:[4,66,101],his:67,histor:47,histori:10,hl_get_sync_flag:96,hold:[29,33,37,39,46,48,50,52,67,72,74,98],holder_:98,home:[35,45,65,67,84,101,102,104,110,111],honor:37,hook2:90,hook:[6,90],hookattr:6,hookattribut:6,horizont:[8,15],host:[32,41,110],host_c:[115,116,117],hostfil:104,hostnetwork:111,hostpath:[109,110,111],hot:123,hous:[2,14,88],how:[6,8,21,29,31,33,37,45,47,50,56,57,60,65,70,74,101],howardjohnson:90,howev:[8,9,30,40,59,60,63,64,65,69,70,73,74],howto:[104,111],hpp:[46,54],html:[14,20],htod:103,http:[8,9,13,14,20,32,41,48,83,84,94,95,99,101,102,104,109,110,115,117,119,122],huber:8,huge:59,human:[8,20],hwc:15,hyper:[8,48],hyperparamet:[8,70],hyperplan:13,i1116:111,i1117:103,i386:116,iOS:116,icml2006_gravesfgs06:8,icml_rank:8,ics:14,id_input:7,id_rsa:104,idea:[32,59,64,70,101],ideal:65,ident:[8,69],identifi:[8,51],identityoffsetproject:8,identityproject:8,idmap:111,ids:[7,8,21,57,80],idx:[37,48,96],ies:45,ifels:[31,61],ifelseop:61,ight:8,ignor:[2,8,21],iil:82,illustr:[7,33,38,47,56,65,67],ilsvrc:122,im_siz:48,imag:[11,12,13,14,21,22,29,48,49,57,58,61,64,67,94,95,110,111,112,113,119,122],image_a:64,image_b:64,image_fil:64,image_h:21,image_lay:64,image_list_provid:122,image_nam:29,image_path:64,image_reader_cr:64,image_s:122,image_w:21,imagenet:[8,20,35],imagepullpolici:111,imageri:8,images_reader_cr:64,imagin:49,img2label:15,img:[2,8,9,65],img_conv_lay:9,img_pool_lay:9,imgsiz:103,imgsizei:103,imgsizex:103,immedi:[59,70],immutable_paramet:29,imperfect:68,implement:[8,9,13,20,21,23,26,31,37,38,39,40,41,47,50,51,52,54,55,57,60,62,65,66,72,74,75],implemet:36,impli:32,implicit:67,imposs:[57,119],improv:[8,66,68,101],in_fals:21,in_plac:21,in_tru:21,inarg:[4,36],inc_path:77,includ:[7,8,9,14,15,23,29,31,32,38,41,46,48,50,54,55,56,57,61,63,68,74,95,97,101,103,115,116,117],inclus:57,incorpor:8,incorrect:8,increas:[33,37,46,82],increment:[42,49],incupd:96,inde:13,independ:[8,30,38,67,72,119],index:[7,8,11,13,14,21,28,30,31,33,37,61,63,75,90],indexslot:8,indic:[7,8,21,31,38,48,56,61,69,73,75],indice_map:75,indices_map:75,individu:33,industri:[33,53,119],ineffici:60,infer:[15,29,31,33,42,51,52,54,62,63,67,68,73,88],infer_shap:63,infer_var_type_:52,inferfac:52,inferior:40,infershap:[31,63,68,97,98],infershapecontext:[97,98],infervartypefn:52,info:[7,8,14,46,56,87,90,96,104,111,119],inform:[8,14,21,28,31,41,45,50,53,56,63,65,70,72,73,101],inherit:[31,58,68],ininst:29,init:[6,20,28,31,48,56,57,65,87,88,96,104,111],init_attr:63,init_from_tar:28,init_hook:[90,123],init_model_path:[105,106,108,121,123],initi:[2,6,8,9,14,16,21,28,32,37,42,49,56,59,63,65,66,70,74,75,88,106,123],initial_max:[6,82],initial_mean:[6,8,82],initial_min:[6,82],initial_std:[6,8,82],initialize_op_attr:63,initpaddl:4,initrd:119,inlcud:9,inlin:98,inner:[8,80,90],inner_:90,inner_mem:90,inner_param_attr:9,inner_rnn_output:90,inner_rnn_st:90,inner_rnn_state_:90,inner_step:90,inner_step_impl:90,inproj_attr:8,inproj_bias_attr:8,inproj_param_attr:8,input0:98,input1:[8,9,98],input2:8,input:[2,5,7,8,9,11,13,15,21,22,28,30,31,36,40,42,46,47,48,49,50,52,56,57,59,60,62,63,64,65,66,67,68,69,72,74,75,80,81,82,87,88,89,90,92,93,96,97,98,108,111,121,123],input_conf:8,input_data:96,input_data_target:96,input_dim_idx:21,input_dtyp:21,input_featur:5,input_hassub_sequence_data:96,input_id:8,input_imag:9,input_index:96,input_label:96,input_lay:96,input_loc:8,input_nam:29,input_proj_bias_attr:9,input_proj_layer_attr:9,input_seg:75,input_seq:8,input_sequence_data:96,input_sequence_label:96,input_sparse_float_value_data:96,input_sparse_non_value_data:96,input_t:96,input_to_check:30,input_typ:[80,90,123],input_valu:30,input_var:[30,63],inputbuff:36,inputdef:96,inputgradi:69,inputlayers_:96,inputs_to_check:30,inputtyp:13,insert:[62,68,69,95],insid:[7,9,21,33,42,60,64,65,66,68,69],instal:[8,41,71,77,83,84,86,88,95,99,101,102,104,110,115,116,117],install_android:115,instanc:[8,30,31,33,35,39,43,51,56,57,59,63,65,68,69],instanti:[33,43],instead:[8,9,11,13,32,36,41,46,49,50,65],instrins:46,instruct:[31,49],int16:76,int32:[61,75,76,106],int64:[44,73,76],integ:[2,7,8,13,21,37,41,46,54,57,87,123],integer_sequ:80,integer_valu:[2,13,80,87,90,123],integer_value_sequ:[2,13,57,90,93,123],integer_value_sub_sequ:90,integr:[7,119],intellig:[20,43],inter:[8,65],interact:8,intercept:8,interchang:[49,68],interest:46,interfac:[6,8,9,20,23,25,26,28,31,37,41,45,50,60,68,69,119],intergr:8,intermedi:[45,48,58,65],intern:[8,9,20,23,28,46,101,102],internet:[32,33,119],interpret:7,interv:8,intrins:46,introduc:[8,15,31,33,48,53,70,72,74,101],intuit:[40,68],invalid:[64,72],invoc:[32,47,68],invok:[2,8,18,28,60,63,65,66,68,69,74,103],involv:57,iob:7,ioe:7,ios:116,ios_arch:116,ios_deployment_target:116,ios_development_root:116,ios_enable_bitcod:116,ios_platform:116,ios_sdk_root:116,ios_use_veclib_for_bla:116,ip_str:111,ipc:109,ips:111,ipt:[8,63,74,82,90,93],ipx:119,ipython:29,is_color:15,is_gener:121,is_loc:28,is_predict:123,is_revers:21,is_seq:[8,93],is_spars:21,is_stat:[6,82],is_target:62,is_tensor:74,is_test:[21,122],is_traget:62,is_train:[2,15],isinst:[4,87],ispodallrun:111,isspars:96,issu:[32,48,67],issue_numb:95,istag:71,item:[8,13,28,40,46,64,88,111,119],iter:[2,8,9,10,13,28,29,33,59,64,65,75],iter_multiple_input_and_param:63,its:[2,8,9,20,26,28,29,30,31,33,37,42,48,49,50,52,53,56,57,59,60,62,63,68,69,72,73,74,103],itself:[33,40,59,72],ivs:74,java:[31,54,61,68],jeremi:103,jian:20,jmlr:8,job:[14,40,65,67,68,105,106,108,109,111,122,123],job_dispatch_packag:104,job_id:14,job_mod:121,job_nam:[41,111],job_namespac:111,job_path:111,job_path_output:111,job_workspac:104,jobnam:111,jobpath:111,jobselector:111,jobserv:41,join:[33,90],jointli:9,jpg:[15,122],json:[50,110],jth:9,judg:8,juditski:59,jupyt:41,just:[5,7,8,9,14,21,32,37,38,48,52,59,60,63,64,65,68,69,70,72,73],jypyt:29,k8s:[111,119],k8s_data:111,k8s_job:29,k8s_token:29,k8s_train:111,k8s_user:29,kafka:35,kaim:20,kaimingh:122,kebilinearinterpbw:103,kebilinearinterpfw:103,keep:[8,13,15,20,33,49,57,59,63,72,74,119],keep_top_k:8,kei:[2,14,15,25,28,30,31,33,35,37,44,46,68,69,74,75,94,95,103,109,111],kept:[8,63],kera:70,kernel:[8,30,46,59,70,73,97,98],key1:106,key2:106,keyword:[63,111],kill:33,kind:[29,30,33,39,49,58,76,109,110,111],know:[29,37,53,101],known:[21,31,47,56],kriz:14,kselectedrow:73,ksimonyan:9,kube:109,kube_cluster_tl:29,kube_ctrl_start_job:29,kube_list_containers_in_job_and_return_current_containers_rank:29,kubeadm:109,kubectl:[104,109,110,111],kuberent:33,kubernet:[29,33,68,100,104,111,112,113,119],kubernetes_service_host:29,kvp:25,kwarg:[2,9,10,13,18,21,42,50,63,74,90,123],kwd:25,l1_rate:6,l1_regularization_op:70,l2_rate:6,l2_regularization_op:70,l2_sim:8,l2regular:[80,123],l93:36,label:[2,7,8,13,14,15,21,28,42,48,49,50,58,61,64,65,66,67,80,87,90,110,122,123],label_dim:[8,90,123],label_fil:64,label_lay:64,label_path:64,labelselector:111,lag:106,lake:2,lambdacost:8,lambdarank:8,lan:104,languag:[8,14,49,68,72,121],larg:[11,14,53,59,65,66,95],larger:[6,7,8],larger_than:[31,51,61],last:[7,8,9,21,56,61,89,90],last_seq:[57,90],last_time_step_output:8,lastseen:110,latenc:[8,46,65],latent:8,later:[32,68,70,98,120],latest:[8,31,32,33,40,77,84,99,110,111],latter:[59,75,101],launcher:29,layer1:[8,9,80,89],layer2:[8,80,89],layer3:8,layer:[4,6,7,9,11,13,16,20,27,28,31,36,48,49,51,58,59,61,64,65,66,68,70,74,75,80,82,87,88,89,92,93,96,122,123],layer_0:96,layer_att:81,layer_attr:[8,80,81,93,108],layer_num:[108,122],layer_s:8,layer_typ:8,layerbas:96,layerconfig:96,layergradutil:96,layerhelp:[21,63],layermap:96,layeroutout:8,layeroutput:9,layers_test:77,layout:15,lazi:[59,70],lbl:7,leaki:48,learing_r:58,learn:[6,7,8,9,10,14,29,38,40,48,49,57,59,64,65,66,67,68,70,84,103,120,122],learnabl:28,learning_method:[121,123],learning_r:[6,23,38,65,80,82,121,123],learning_rate_arg:82,learning_rate_decay_a:82,learning_rate_decay_b:82,learning_rate_schedul:82,leas:33,least:[7,33],leav:31,lecun:14,left:[8,31],left_cmd:13,left_right_flip:15,legal:74,len:[2,8,38,44,63,88,90,96,111,123],length:[8,9,13,14,15,21,38,46,53,56,57,68,75,110],leq:8,less:[8,29,119],less_than:29,let02:110,let:[7,8,29,31,40,47,49,56,57,58,69,101],level:[6,8,20,21,46,47,50,53,56,57,75,76,92],lgtest:32,lgtest_main:32,lib64:[77,84,106],lib:[55,83,101,102,115,116,117],lib_path:77,libapi:32,libari:55,libc:86,libcuda:[77,84],libgcc_:86,libgoogl:[101,102],libnvidia:[77,84],libpaddl:[54,55,68,95,101,102],libpaddle_capi:55,libpaddle_gserv:55,libpaddle_math:55,libprotobuf:82,librari:[8,32,39,55,65,106],libstdc:86,life:33,lifecycl:119,lifetim:72,lightweight:47,like:[7,8,13,14,21,31,32,33,36,41,47,48,49,50,52,59,63,64,68,69,70,72,73,75,101,119,122],limit:[8,13,21,53,57,68,70,82,103],linaro:117,line:[2,7,13,25,32,36,41,45,49,59,61,63,68,70,80,82,90,95,101,108],line_break:13,line_count:82,linear:[8,21,57,80,82,87,88],lineno:[101,102],link1:46,link2:46,link:[8,9,32,44,45,72,92,119],linux:[13,44,86,94,109,115,117],linux_x86_64:[71,77,86],lipeng:121,lipo:116,list:[1,2,7,8,9,13,15,18,21,23,28,29,31,32,37,41,43,45,48,58,60,63,67,69,72,75,80,101,102,108,122,123],listdir:104,listen:33,littl:[38,53],load:[15,29,33,48,63,65,111,122],load_and_transform:15,load_data_arg:4,load_featur:122,load_feature_c:122,load_feature_pi:122,load_imag:15,load_image_byt:15,load_missing_parameter_strategi:[105,106,108,121],load_mnist:48,load_paramet:82,loadparamet:4,loadsave_parameters_in_pserv:[36,105,106],loc:[7,20],local:[6,21,28,30,31,33,39,40,49,56,61,63,68,83,101,105,106,111],local_scop:30,localhost:[84,99,109],localip:111,localpath:45,locat:[8,28,32,75],lock:[32,33,37,38,66],lod:[21,53,56,73,75,76],lod_desc:[73,76],lod_expand:57,lod_level:[21,63,73,76],lod_tensor:[21,56,73,76],lod_tensor_arrai:21,lodtensor:[52,53,68,76],lodtensordesc:[53,73],log:[37,45,48,65,78,82,86,96,104,106,110,111],log_barrier_abstract:[105,106],log_barrier_lowest_nod:[105,106],log_barrier_show_log:[105,106],log_clip:[105,106],log_error_clip:[105,106],log_period:[106,108,110,111,123],log_period_serv:[105,106],logarithm:5,logger:[2,90],logic:[40,48,52,58,60,65,66,72,75],logist:123,logit:48,longer:[33,65],look:[2,7,31,41,49,59,63,69,70],lookahead:8,lookup:[21,52,57],lookup_t:21,loop:[30,31,64,72],loop_var:75,loss:[8,23,48,50,58,59,70,123],lot:[57,59,63,65,70,119],low:[8,20,58,75],low_rnn:56,lower:[8,46,56,57],lower_level_rnn:56,lpaddle_capi_shar:55,lpaddle_capi_whol:55,lrelu:48,lstm:[8,90,93,110,123],lstm_bias_attr:9,lstm_cell_attr:9,lstm_group:[9,90],lstm_group_input:90,lstm_input:90,lstm_last:90,lstm_layer_attr:9,lstm_nest_group:90,lstm_output:90,lstm_size:123,lstm_step:9,lstmemori:[9,81,90,93],lstmemory_group:[8,81,90],lstmemory_unit:81,ltr:8,mac:[55,115],machin:[9,14,28,48,59,65,66,70,80,92,104,119],machine_transl:93,machinelearn:8,maco:[86,94],macro:[47,69],made:[33,38],mai:[8,9,30,31,42,46,64,65,66,67,68,72],main:[4,21,50,61,68,101,102],main_program:[18,21,22,42],mainli:39,maintain:[8,31,37,59,63,68,120],majel:32,major:[46,65],make:[2,7,8,29,31,32,33,37,38,40,46,49,56,57,59,60,63,64,65,68,70,75,83,94,95,96,97,99,101,103,115,116,117,119],make_ddim:98,make_function_oper:47,make_vari:74,maker:[68,69],man:44,manag:[23,28,33,38,39,45,65,72,99],mandarin:8,mani:[9,15,32,37,48,57,60,63,68,69,72,73,74,75],manili:50,manipul:[63,69],manner:[8,59,70],manual:[58,59,65,69,82,119],manylinux1:86,manylinux1_x86_64:[71,77,86],map:[7,8,13,28,29,31,37,63,69,72,74,75,87,119],map_fn:75,map_read:13,mapper:13,mapreduc:29,mark:[48,49,56,57,66,72,101,119],market:46,mask:[6,8,21],master:[29,40,68,71,109,117],mastermind:32,mat:[54,55],mat_cache_row:36,mat_norm:36,mat_normal_shar:36,mat_param_attr:9,mat_sparse_row:36,mat_sparse_row_auto_grow:36,mat_sparse_row_id:36,mat_sparse_row_prefetch:36,mat_sparse_row_prefetch_full_s:36,mat_value_shar:36,match:[8,21,32,46,80],matchbox:119,math:[9,54,68,96,97,103],mathemat:70,matirx:8,matmul:[31,50,56,75,97],matrix:[7,8,9,13,21,28,36,54,55,96,97],matrixptr:96,matrixtyp:55,mattyp:36,max:[6,8,13,14,21,22,30,63,82,89,103,108],max_diff:30,max_id:[8,28],max_job_id:14,max_length:[8,57,93],max_movie_id:14,max_relative_error:[30,97],max_sort_s:8,max_user_id:14,maxframe_evalu:7,maxid:7,maxid_evalu:7,maxim:8,maximum:[7,8,14,31,38],maxinum:11,maxpool:8,mayb:31,md5:[14,34],mean:[6,7,8,9,10,11,13,15,20,28,32,43,50,57,62,64,65,72,80,101,106,119,122],mean_meta:122,mean_meta_224:122,mean_valu:122,mean_var_nam:8,meant:75,measur:42,mechan:[8,9,39,42,63,69],mem:[8,31,41,57,90],mem_per_p:67,mem_per_train:67,member:[8,14,29,49,50,60,63,72],memcpi:[60,103],memor:8,memori:[9,21,31,36,37,41,46,53,57,59,68,93,98,103,110],memory_boot:9,memory_nam:[8,81],memory_test:94,memory_threshold_on_load_data:[105,106],mention:[21,32,37,56,59,65,66],mere:9,merg:[8,18,38,40,42,56,60,95],mergedict:121,messag:[31,49,53,61,62,63,68,69,73,76,77,95,110],meta:122,metadata:[44,110,111],metal:[109,119],metaphor:49,metaplotlib:29,method:[2,8,10,20,23,28,30,31,40,46,48,49,50,58,63,64,66,68,72,73,75,101,102],methodolog:59,metric:[18,42],mfs:111,microarchitectur:46,microsoft:8,might:[8,31,32,61,66,101],million:14,min:[6,8,63,103,108],min_block:31,min_count:66,min_desc:31,min_pool_s:[2,80],min_word_freq:14,mind:101,mini:[8,13,18,28,31,33,42,51,56],mini_batch:64,minibatch:[8,21,31,42,49,51,61],minikub:109,minim:[23,31,48,58,66,68],minimum:8,minsizerel:[115,116,117],minu:69,minus_grad:69,minusgradop:69,minusop:69,minusopgradmak:69,minusopprotoandcheckermak:69,minut:[33,40],mip:115,mirror:32,mislead:38,miss:48,mix:[9,75,93],mixed_lay:[9,90],mixed_layer_attr:9,mixedlayertyp:8,mixtur:101,mkdir:[45,83,99,104],mkl:[68,83],mkldnn:8,mkldnn_batch_norm:8,mlp:50,mlpaper:8,mlr:20,mnist:[2,4,35,48,49,61,64,65,67,68,101,102],mnist_model:4,mnist_provid:2,mnist_random_image_batch_read:64,mnist_train:[2,64],mnist_train_batch_read:64,mnt:111,mobil:[46,68,99,114],mode:[8,25,28,46,95,111],model:[8,9,14,28,31,33,34,42,49,58,59,65,66,67,68,70,75,88,99,108,120,121,123],model_config:4,model_list:[106,108],model_path:108,model_zoo:[121,122],modifi:[8,46,50,65,70,95],modul:[2,9,14,28,47,48,57,65,75,82,97,101,102,122,123],modular:57,modulo:8,moment:[23,101],momentum:[6,21,23,72,80,87,123],momentumop:[101,102],mon:110,mono:8,month:32,moosef:109,more:[7,8,9,13,21,29,30,32,33,37,40,41,45,46,47,49,56,57,58,63,64,65,66,68,70,75,82,101,103,119],most:[8,13,21,28,29,32,40,49,50,57,59,64,65,70,101,119],mostli:[46,119],motiv:68,mount:41,mountpath:[110,111],move:[8,33,37,45,59,119],movi:[2,14],movie_categori:14,movie_id:111,movie_info:14,movie_review:14,movieinfo:14,moving_average_fract:8,mpi:104,mpirun:104,mse:[49,58,61],msra:20,much:[8,33,58,64,70,75],mul:[47,63,67,96,97],mul_grad:97,mul_op:[21,97],mul_ratio:8,mul_result:63,mulgradkernel:97,mulkernel:97,mulop:[47,97],mulopgrad:97,mulopmak:97,multi:[8,42,60,101,119,122],multi_binary_label_cross_entropi:8,multi_crop:122,multigradientmachin:60,multinomi:8,multipl:[7,8,9,13,18,21,28,29,30,37,38,40,42,47,65,66,67,68,76,101],multiple_input:63,multiple_param_attr:63,multipli:[7,8],multiprocess:13,must:[5,7,8,9,13,15,21,38,53,62,63,64,68,74,80,96,97,98,104],mutabl:98,mutable_data:[97,98],mxnet:31,my_cost:82,my_lib:104,mypaddl:[110,111],name:[2,6,7,8,9,11,15,18,21,25,28,29,30,31,33,35,36,38,41,42,46,47,50,53,55,57,61,63,67,68,71,73,74,75,76,82,87,88,90,93,96,103,108,109,110,111,112,113,119,123],name_prefix:35,namespac:[31,51,54,63,96,97,109,110,111],nativ:[8,46],natur:[37,40,57,66,75],ncall:[101,102],ncelm:8,nchw:[8,21],ndarrai:[15,28,35],ndcg:8,ndcg_num:8,ndk:115,nearest:46,nearli:30,necess:75,necessari:[8,31,38,40,42,53,57,60,63,74,75],need:[7,8,9,13,20,23,26,29,30,32,36,37,38,40,41,42,45,47,48,57,58,59,60,62,63,65,66,67,68,69,70,72,73,74,75,77,103,111,119],need_tran:82,neg:[2,7,8],neg_distribut:8,neg_overlap:8,neg_pos_ratio:8,neon:46,ner:7,nerual:21,nest:[8,13,31,61],net:[8,9,16,31,48,56,72],net_diagram:122,netop:[31,68],network:[4,6,7,8,13,20,21,26,27,28,29,30,31,33,36,42,48,50,56,58,59,64,65,66,70,72,74,76,81,87,88,90,108,111,119,121,122],network_config:108,neural:[8,9,13,20,28,29,31,33,50,56,59,65,70,72,76,88,90,92,121],neuralnetwork:60,neuron:21,never:[13,64,72,110,111],new_block_idx:63,new_stat:56,newblock:63,newest:38,newli:[46,119],newop:31,newopdesc:63,newprogram:63,newremot:65,newvardesc:63,next:[2,8,14,33,39,57,75,101],nfs:111,nfsdir:111,ngram:14,nic:[105,106,111],nil:37,nine:14,nlp:8,nltk:14,nms_threshold:8,nms_top_k:8,nmt_without_attent:80,nnz:96,no_cach:2,no_grad_set:[23,30,97],no_sequ:[2,8,87],node0:111,node1ip:104,node2ip:104,node3ip:104,node:[8,32,40,50,57,65,66,67,68,76,104,109,110,111,119],node_0:111,node_1:111,node_2:111,node_id:104,nodeattr:50,nodeentri:50,nodefil:104,nodesep:50,nohup:104,nois:[8,33,48],noisi:[8,48],non:[8,21,33,46,73],none:[2,4,6,7,8,9,10,11,15,18,20,21,22,23,28,29,30,31,42,48,50,51,56,57,58,61,63,67,74,75,93,122],nonlinear:20,norm:[9,48],norm_by_tim:8,normal:[8,9,14,20,59,110,111,122],notat:8,note:[6,8,9,11,15,18,28,29,31,36,37,41,53,64,68,98],notebook:[41,84],noth:[5,28,63,72,95],notic:69,notingradi:97,notion:75,notori:30,now:[13,32,33,48,53,59,66,68,69,70,72,92],np_arrai:13,nproc:94,nullptr:[69,72,96],num:[8,9,104,106,111],num_channel:[8,9],num_chunk_typ:7,num_class:[8,9,50],num_col_dim:21,num_filt:[8,9,21,22],num_flatten_dim:21,num_gradient_serv:[104,105,106],num_group:8,num_hidden:50,num_neg_sampl:8,num_p:[65,67],num_parameter_serv:29,num_pass:[28,87,105,106,108,110,111,123],num_per_batch:15,num_repeat:8,num_result:7,num_results_per_sampl:8,num_row:73,num_samples_process:82,num_shard:35,num_step:75,num_train:[65,67],number:[7,8,9,13,14,15,21,31,33,35,42,59,64,66,68,74,75,101],numchunktyp:7,numdevices_:108,numer:8,numeric_grad:30,numerical_grad:30,numlogicaldevices_:108,numofallsampl:7,numofwrongpredict:7,numpi:[6,13,15,21,28,35,46,48,63,64,82,83,87,97,122],numreal:36,numsampl:103,numtagtyp:7,numtimeout:37,nv_:32,nv_gpu:94,nv_librari:32,nv_test:32,nvcc:[32,46],nvidia:[46,77,84,94],obei:7,obj:[2,82,122,123],object:[2,6,8,9,13,28,29,36,42,48,50,54,58,63,67,68,70,72,103],observ:8,obtain:[40,59],obvious:[32,101],occup:111,occupi:46,occur:[14,28],occurr:31,oct:110,odd:8,off:[55,83,94,95,99,104,115,116,117,119],offer:[31,67,68,74],offici:[8,32],offlin:[33,35,119],offset:[8,36],often:[8,36,50,101],ograd:96,old:[30,38,40,57,68],omega:70,omit:[80,123],omp_num_thread:[101,102],ompi_comm_world_rank:104,on_init:2,onc:[8,33,37,42,49,59,65,66],one:[2,5,7,8,9,11,13,20,23,26,28,29,30,31,33,36,37,38,40,41,42,43,46,47,48,50,52,53,56,57,58,59,60,61,62,63,64,65,67,68,69,72,73,75,119,123],onehotcrossentropyopkernel:97,ones:[47,48,68],onli:[7,8,9,11,15,21,26,28,29,30,32,36,37,38,39,40,41,42,45,46,48,49,56,57,58,60,63,65,66,67,68,73,74,75,90,92,119],onlin:[8,10,33,35,64],only_cpu:30,onto:[21,65,66],op_:97,op_check:97,op_class:[68,74],op_desc:[52,62],op_maker_class:[68,74],op_proto:74,op_test_util:97,op_typ:68,opattrcheck:97,opcreat:74,opdesc:[31,49,61,62,63,68,69,74,76],opdescbind:[52,69],opdescbuild:31,open:[2,8,15,29,35,48,64,80,82,90,101,122],openbla:[83,84],opencv:15,openmp:101,openmpi:104,oper:[8,9,13,15,20,21,23,26,30,31,42,43,46,48,49,50,52,56,57,58,62,65,67,70,72,76,97,98],operand:46,operartor:98,operator_grad:30,operatorbas:[31,47,68,69,74,97],operatorwithkernel:97,opinfo:[52,68,69],opinfomak:52,opinfomap:69,opkernel:[97,98],opkernelkei:68,opmak:74,opproto:97,opprotoandcheckermak:[69,97],opprotomak:[74,97],opregist:74,opregistri:74,ops:[23,30,31,32,49,50,59,61,62,63,68,97,119],ops_:31,ops_test:32,opt:[29,58,62,67,74,83,111],opt_op_list:58,optestmeta:97,optim:[6,16,27,28,30,43,48,59,60,61,65,66,67,68,70,73,80,82,87,101],optimis:58,optimize_op_attr:63,optimzi:80,option:[7,8,18,21,25,29,32,48,53,61,62,63,68,73,74,76,101,115,119],optmization_op_list:58,opts_np:62,optyp:[52,74],opwithkernel:73,order:[8,9,13,15,28,43,49,53,64,70,75,101,102,111,119],ordereddict:28,org:[7,8,9,14,20,35,44,48,84,104],organ:[7,8],orient:74,origin:[8,9,13,14,30,46,48,72,75,95],other:[7,8,9,13,21,23,31,33,38,45,46,52,56,59,62,70,72,74,101,119],otherchunktyp:7,otherwis:[8,13,14,15,21,28,29,33,38,40,48,52,64],our:[29,32,48,52,59,65,66,72,75,101],out:[8,21,28,29,31,32,37,40,50,56,57,63,65,80,90,92,93,97,98,101,102],out_dir:111,out_left:8,out_mem:93,out_memori:9,out_right:8,out_size_i:8,out_size_x:8,outer:[8,90],outer_mem:90,outer_rnn_st:90,outer_rnn_state_:90,outer_step:90,outlier:8,outout_lay:28,outout_layer1:28,outout_layer2:28,output:[5,6,7,9,11,13,21,25,28,29,30,31,35,40,45,47,48,49,50,51,52,53,56,57,59,61,62,63,64,65,66,68,69,72,73,74,75,80,90,93,97,98,101,104,108,110,111,121,122,123],output_:8,output_all_step:56,output_dim_idx:21,output_dir:122,output_dtyp:21,output_fil:25,output_id:8,output_lay:[28,80,88,122],output_max_index:11,output_mem:[8,93],output_mod:25,output_nam:30,output_num:56,output_path:35,output_s:21,output_seg:75,outputbuff:36,outputgradi:69,outputh:8,outputw:8,outsid:[2,8,9,65,72],outupt:75,outv:96,over:[8,9,28,29,59,75],overal:[48,59,119],overfit:[21,70],overlap:[7,8],overlap_threshold:[7,8],overload:46,overrid:[31,33,45,67,97,98],overview:[37,38,39],overwrit:45,own:[8,38,40,50,52,58,59,65,67,70,74],paam:15,pack:[75,82],packag:[13,14,37,41,47,71,77,101,102],pad:[9,21],pad_c:8,pad_h:8,pad_w:8,padding_attr:8,padding_h:21,padding_i:8,padding_w:21,padding_x:8,paddl:[2,4,5,6,7,8,9,10,11,13,14,15,18,20,21,22,23,25,26,28,29,31,32,33,35,41,45,47,48,51,53,54,55,56,57,60,61,65,67,68,70,74,75,80,82,83,84,86,87,88,93,94,95,96,97,99,101,102,103,104,108,110,111,115,119,121,123],paddle_begin_init_param:38,paddle_dir:97,paddle_doc:99,paddle_docs_cn:99,paddle_element_typ:38,paddle_element_type_float32:38,paddle_element_type_float64:38,paddle_element_type_int32:38,paddle_element_type_int64:38,paddle_element_type_uint32:38,paddle_element_type_uint64:38,paddle_enforc:31,paddle_enforce_eq:[97,98],paddle_error:[54,55],paddle_exampl:41,paddle_finish_init_param:38,paddle_get_param:38,paddle_gradi:38,paddle_init_num_gradient_serv:104,paddle_init_param:38,paddle_init_port:104,paddle_init_ports_num:104,paddle_init_ports_num_for_spars:104,paddle_init_pserv:104,paddle_init_trainer_count:104,paddle_init_trainer_id:104,paddle_init_use_gpu:104,paddle_job:41,paddle_manylinux_devel:83,paddle_matrix:[54,55],paddle_matrix_cr:55,paddle_matrix_get_shap:54,paddle_matrix_shap:54,paddle_n:111,paddle_new_etcd_pserver_cli:38,paddle_new_pserver_cli:38,paddle_on_cloud:41,paddle_output:110,paddle_paramet:38,paddle_port:111,paddle_ports_num:111,paddle_ports_num_spars:111,paddle_process_by_paddl:111,paddle_pserver2:104,paddle_pserver_cli:38,paddle_pserver_client_releas:38,paddle_root:121,paddle_save_model:38,paddle_send_grad:38,paddle_server_num:111,paddle_source_root:121,paddle_train:[55,71,104,111],paddledev:[77,84,110,111],paddlepaddl:[8,9,13,14,15,28,32,33,35,38,39,40,41,44,45,47,48,49,51,53,56,57,58,60,63,64,67,68,72,75,76,83,84,86,88,93,94,95,101,102,103,104,112,113,115,117,119,120,121],paddlepaddlebook:84,pain:67,pair:[7,8,23,25,31,49,58,65,68],pairwis:8,pakcag:32,palceholder_just_ignore_the_embed:121,paper:[8,20,48],paper_fil:8,para:[36,121],paraconvert:121,paradigm:68,paragraph:56,paragraph_data:56,paragraph_out:56,parallel:[65,66,68,103,108,110,111],parallel_nn:[6,105,106],param:[6,8,9,13,23,30,31,38,60,63,82,98],param_attr:[8,9,21,36,63,80,82,93],param_config_proto:38,param_fil:82,param_initi:21,paramattr:[6,8,16,80,82,93],paramet:[7,9,10,11,13,18,21,23,27,30,31,32,34,36,40,43,45,48,49,50,52,53,56,58,61,64,65,72,74,75,80,82,87,88,104,106,111],parameter_block_s:[105,106],parameter_block_size_for_spars:[105,106],parameter_learning_r:6,parameter_list:[23,58],parameter_nam:[28,29],parameter_serv:29,parameter_valu:36,parameterattribut:[6,8,9,36],parameterclient2:111,parameterclient_:36,parametermap:96,parametermutex_:36,parameters_:96,parameters_and_grad:[23,58],parameterserver2:36,parameterset:29,parameterupdat:60,parameterupdater_:36,parametr:8,params_grad:58,paraphras:121,paraphrase_data:121,paraphrase_model:121,paraphrase_modeldata:121,paraspars:96,parent:[31,61,63,68],parent_:[31,72],parent_idx:63,parenthes:68,pars:[13,14,32,43,50,94],parse_config:4,parse_known_arg:111,parsefromstr:82,parser:[13,111],part:[7,8,31,40,53,61,63,65,101,119],partial:[8,28],partial_sum:8,particular:[49,53,68],particularli:20,partit:[33,35,43,65,66,68],paserv:111,pass:[2,8,13,18,21,26,28,31,33,42,48,53,58,59,60,62,63,64,68,70,72,75,80,87,95,103,106,108,110,111,123],pass_gener:8,pass_id:[28,87],pass_idx:64,pass_manu:82,passtyp:96,past:29,patch:44,path:[7,13,14,15,28,33,37,38,41,57,64,106,109,110,111,115,116,117],path_to_paddlepaddle_working_directori:99,pattern:[14,33,54,59,70],paus:[33,40],pdf:[8,9],peer:78,pem:[29,35],pend:[33,37],peopl:8,pep425tag:[77,86],per:[7,8,14,15,33,38,59,64,70],percal:[101,102],perf_test:[101,102],perform:[8,9,20,21,30,38,42,46,48,60,64,65,68,70,102,103,105],perftool:[101,102],period:[33,40,67,106],permut:21,peroid:[8,15],persist:[43,73,76],persistentvolum:109,persistentvolumeclaim:[109,111],person:[7,29],perspect:68,perturb:30,peter:67,pex:119,pfs:[35,45,67],pfsclient:35,pfspath:45,phase:[21,57,59,64,69,119],philosophi:[59,70],photo:48,physic:119,pickl:104,pid:109,piec:[8,9,43],pil:15,pillow:41,pip:[71,77,83,86,88,95,99,101,102],pipe:13,pipe_read:13,pipelin:42,pixel:[2,8,13,14],pixels_float:2,pixels_str:2,place:[2,21,33,40,65,66,68,97,98],placehold:[48,98],placement:66,plain:[7,8,13,41,53,55],plan:[33,68],platform:[31,86,97,98,115,116],pleas:[6,8,9,10,15,29,33,37,38,39,50,56,63,64,67,68,76,77,99,101,111,120],plot:29,plu:[8,30],plug:59,pnpairvalid:105,pod:[35,41,109,110,111],podip:111,podlist:111,point:[31,33,41,46,67,98,101,103,119],pointer:[31,38,50,63,68,72,98],polar:14,poli:82,pollut:40,polyak:59,ponit:50,pool3:96,pool:[9,21,27],pool_attr:9,pool_bias_attr:9,pool_layer_attr:9,pool_pad:[9,21],pool_siz:[2,8,9,21,22],pool_size_i:8,pool_strid:[9,21,22],pool_typ:[8,9,21,22],pooled_height:8,pooled_width:8,pooling_lay:[9,80,123],pooling_typ:[8,80,89,123],poolingtyp:11,pop:31,popul:38,popular:[32,48,50],port:[32,101,104,105,106,110,111],port_num:105,portabl:50,portal:99,ports_num:[104,106,111],ports_num_for_spars:[36,104,105,106,108,111],pose:33,posit:[2,7,8,9],positive_label:7,posix:109,possibl:[29,31,37,63,66,70],post:[41,44],postpon:70,potenti:[46,103],pow:82,power:[46,119],ppo_workspac:99,pprof:[101,102],pre:[8,9,14,29,38,121],pre_activ:63,pre_bia:63,pre_dictandmodel:121,pre_stat:[56,75],preambl:63,precis:[7,42,46,59],precision_evalu:7,pred:50,predetermin:8,predic:14,predict:[2,4,7,8,28,65,70,80,88,121,122,123],predict_fil:[105,106],predict_lay:28,predict_output_dir:[105,106,123],predict_sampl:4,prediction1:28,prediction2:28,prefer:109,prefetch:[36,96],prefix:[7,9,33,35,57],pregrad:96,premodel:121,prepand:63,prepar:[30,41,60,104,112],prepend:63,prepend_oper:63,preprocess:[14,15,75,121,123],present:[29,31,75,95],preserv:45,press:20,prev_batch_st:[105,106],prevent:[10,21,29,33,37,40,70,101],preview:68,previou:[8,9,28,33,45,56,57,66,101],previous:8,previous_memori:31,price:[14,68,88],primari:49,primarili:[59,70],primit:[46,75],principl:[29,32],print:[4,6,28,29,50,65,77,86,87,88,101,104],print_graphviz:50,print_s3_bucket:13,printallstatu:103,printer:7,printstatu:103,priorbox:8,prioriti:68,prite:7,privat:[31,55,63,72,73,74,75,95,98],prob:[7,28,88],probabilist:[8,121],probabl:[7,8,21,28,57],problem:[8,29,30,32,40,48,49,59,68,70],proc:84,proce:[13,33,64],procedur:[31,53],proceed:[8,20],process2:90,process:[2,6,8,9,13,29,31,35,36,37,40,42,50,53,65,70,74,80,82,90,101,111,123],process_num:13,processdata:122,processor:[46,103],prod:95,produc:[8,9,13,33,50,64],product:[8,9,21,41],productgraph:110,prof:[101,102],profil:[16,45,102,103],profl:102,proflier:103,prog:111,program:[13,18,20,21,25,29,35,38,40,49,51,58,64,65,67,68,72,101,103,111],programdesc:[43,53,62,63,67,69],programm:[63,65],progress:[33,37],proj:8,project:[8,9,41,55],promis:[8,9,57],prompt:45,prone:29,propag:[8,10,59],properti:[50,70],propos:[31,57,58,59,66,75],protect:[46,74,96,97],proto:[11,53,61,68,74,76,97],proto_:74,protobuf:[28,31,41,43,49,50,53,61,63,68,69,74,82],protocol:[7,119],protomak:97,provi:104,provid:[8,14,29,31,38,41,42,46,48,50,52,59,63,67,70,74,75,80,88,90,101,105,119,123],provis:119,prune:[8,31],ps_desir:33,pserver:[28,36,38,39,41,68,104,105,106,111],pserver_addr:38,pserver_cpu:41,pserver_id:34,pserver_mem:41,pserver_num_thread:[36,105,106],pserver_spec:28,pseudo:[29,41,69,75],pseudocod:75,psize:96,ptr:55,pub:104,pull:[32,68,71,84],purpos:[8,33,65,66,103],push:[31,111],push_back:96,put:[32,33,36,63,66],pwd:[83,84,94,95,99,115],pxe:119,py_paddl:[4,77],pybind:[31,46],pydataprovid:80,pydataprovider2:[2,4,111],pyramid:8,pyramid_height:8,python2:[101,102],python:[13,21,28,29,31,39,42,47,48,49,50,54,57,60,68,71,75,77,80,83,86,88,93,94,95,96,97,99,102,104,121,122],pythonpath:77,qualcomm:46,queri:[7,8],query_id:7,question:[8,29,66,74],queue:66,quick:[50,110],quick_start:[41,110,111,112,123],quick_start_data:110,quickli:[57,63,68],quickstart:110,quit:57,r14b:115,r_t:8,rac:8,rais:[13,50,104],rajathkmp:48,ran:[66,103],rand:[48,82,103,106,108],random:[6,8,13,20,21,35,48,60,63,64,82,97],random_crop:15,random_imag:35,randomli:[15,21,40],rang:[8,13,20,35,46,48,64,65,67,74,111,123],rank:[8,21,29,75,122,123],rank_tabl:21,rankdir:50,ranktabl:21,rapid:69,rare:2,raspberri:118,raspberrypi:117,raspbian:117,rasspberri:117,rate:[6,7,8,9,10,14,38,80,111],rather:[41,48,75],ratio:[8,106],raw:[8,13,53],rdma_tcp:[105,106],reach:33,read:[2,13,15,21,28,29,33,35,64,65,66,68,75,82,119,122],read_from_realistic_imag:29,read_from_rng:29,read_lock:34,read_mnist_imag:29,read_next_from_fil:80,read_paramet:82,read_ranking_model_data:29,readabl:[68,101],reader:[14,28,35,46,48,49,61,65,67,87,101,102,104],reader_cr:35,reader_creator_bool:64,reader_creator_random_imag:[13,64],reader_creator_random_image_and_label:[13,64],readi:[33,110,119],readlockguard:36,readm:[55,95],readwritebuffer_:36,real:[8,36,48,64],real_process:2,realist:29,realiz:[31,56],realli:70,reason:[9,29,30,33,110],recal:7,receiv:[13,33,41,56,66],recent:59,recognit:[8,122],recommend:[9,29,111],record:[13,37,74],recordio:[13,14,29,35,37,65,67],recov:[33,75],recover:68,recoveri:37,rectifi:[8,20],recurr:[56,72,90,91],recurrent_group:[9,81,90,92,93],recurrent_lay:9,recurrent_op:75,recurrentgradientmachin:[55,57,75],recurrentgroup:7,recurs:[31,32,45,68],recv:[65,66],recvparametertyp:36,red:[48,101],redirect:13,reduc:[8,21,46,66,68,101,102],reduce_by_kei:68,reduce_mean:48,refactor:[49,57,59,60,63,65,66,70,75],refer:[6,8,9,10,15,20,21,25,30,31,33,37,38,39,46,50,56,61,63,68,70,72,75,76],referenc:[8,37],refine_unknown_arg:111,reflect:37,reg:74,regard:119,region:[8,72],regist:69,register_gpu_profil:103,register_lay:96,register_op:[47,68,69,74,97],register_op_cpu_kernel:97,register_op_gpu_kernel:97,register_op_without_gradi:[68,97],register_oper:[52,69],register_tim:36,register_timer_info:103,registerop:74,registri:[41,52,110,119],regress:8,regular:[6,16,21,80,123],regularization_coeff:26,reinforc:67,rel:[9,30,40,70,97],relat:[33,40,41,46,72,101,119],relationship:69,releas:[67,71,109,115,116,117],reli:[30,57,58,59,70,101],reliabl:[33,70],relu1:50,relu2:50,relu:[8,48,50],relwithdebinfo:[101,102],remain:75,rememb:8,remot:[6,32,36,68,95,106,108],remote_ess:67,remote_sess:67,remoteparameterupdat:[36,39],remov:[13,45,57,95],ren:20,renam:[45,46],reorgan:8,repeat:[31,49,61,62,73,74,76,101],repeatedli:49,replac:[32,37,52,59,67,69],replic:65,replicaset:41,repo:[32,117],report:[37,46,65],reportdataset:37,repositori:[8,99,115],repres:[8,9,31,37,50,53,57,59,63,66,67,68,70,73,75,76],represent:[8,38,48,49,57,65,73],request:[32,33,36,40,65,68,71,110,119],requir:[7,8,23,29,33,38,40,41,45,46,50,56,59,61,62,65,66,68,70,73,74,76,99,119],res5_3_branch2c_bn:122,res5_3_branch2c_conv:122,research:[8,14,65],reserv:45,reserveoutput:96,reset:[8,18,33,42,78],reset_program:[18,42],reshap:[30,64,82],reshape_s:8,residu:122,resiz:[15,36,97,98],resize_s:15,resize_short:15,resnet_101:122,resnet_152:122,resnet_50:122,resolv:[32,95,110],resourc:67,respect:[30,46,48,56],respons:[8,36,42,48,59,60,70,110],rest:[8,31,41,44,119],restart:[33,38,110,119],restartpolici:[110,111],restor:[30,59],restrict:[70,72,101,102],result:[2,5,7,8,21,25,28,30,37,42,48,49,50,53,57,58,60,65,101,103,123],result_fil:7,resum:40,ret:13,retriev:[31,57,72,101],return_op_list:23,return_seq:9,reuqest:71,reus:[31,40,57,64,68],rev:94,revamp:65,reveal:[29,101],revers:[8,9,92,93],review:[14,95,110],reviews_electronics_5:110,rewrit:32,rgb:[8,15],rho:10,right:[8,30,31,32,41,42,68,70],rkt:[41,94],rmsprop:[59,80,123],rmspropoptim:59,rnn:[8,9,21,31,48,57,63,68,72,92,93,105],rnn_bias_attr:93,rnn_layer_attr:93,rnn_out:93,rnn_output:75,rnn_state:90,rnn_state_:90,rnn_step:8,rnn_use_batch:[105,106],rnnalgorithm:57,rnnlm:14,rnnstep:75,roadmap:75,robust:[8,20],roi:8,role:[14,29,37,38,65],rollback:63,root:[10,11,110,111,121],rot:8,round:46,routin:46,row:[7,8,13,21,36],row_id:8,rows_:73,rpc:37,rpcserver:37,rpi:117,rpi_arm_neon:117,rpi_toolchain:117,rstrip:111,rtk:119,rtype:[8,13],rule:[7,49,65],run:[29,30,31,32,33,41,42,43,46,47,48,49,50,56,58,59,61,62,65,66,67,68,72,73,77,83,84,94,95,98,99,101,102,103,104,110,111,112,113,115,119],run_test:83,runinitfunct:[103,111],runnabl:66,running_on_cloud:41,runserv:99,runtim:[2,25,31,43,52,56,67,68,76,77],runtime_table_:31,safe:41,sai:[8,49,51,64,65],same:[2,7,8,9,20,21,28,29,30,37,38,40,47,48,50,56,57,63,65,67,68,69,72,75,90,98,123],samping_id:8,sampl:[2,7,13,14,42,48,74,123],sample_fil:13,sample_id:7,sample_num:7,sample_pars:13,sampler:48,satifi:7,satisfi:[32,73],save:[2,8,13,28,33,35,37,38,41,49,50,53,59,65,73,76,110],save_dir:[106,108,110,111,123],save_only_on:[105,106],save_parameter_to_tar:28,savetxt:82,saving_period:[105,106,111],saving_period_by_batch:[105,106,108],saw:2,scalabl:68,scalar:[8,21,31,51,75],scale:[5,20,59,65,66,69,74,97],scaleop:97,scaleopmak:[68,97],scalingproject:8,scan:[37,68],scatter:8,scenario:57,schedul:[37,41,66],scheduler_factor:6,scheme:[7,10,36,70],scope:[30,43,67],score:[7,8,21,57],scp:104,script:[14,83,99,104,115],sdk:116,search:[8,33,72,93],second:[8,21,29,45,48,50,56,57,61,62,64,72,74,97,123],section:[63,66,101],see:[8,9,29,33,46,63,67,82,101],seed:[20,21,82,103,106],seem:[32,46],seen:70,segment:[7,56,75],sel_fc:8,selcet:8,select:[8,57],selected_generation_scor:57,selected_id:[8,57],selected_indic:8,selected_row:[73,76],selected_rows_desc:[73,76],selected_scor:57,selectedrow:[52,76],selectiv:8,selector:110,self:[30,42,48,50,53,58,63,75,96,97],selfnorm:8,semant:[14,29,57,71],semat:29,send:[33,38,65,66,68,74,76],send_back_parameter_typ:36,sendbackparameterspars:36,sendbackparametertyp:36,sendparameterrequest:36,sendparameterrespons:36,sens:[59,70,101],sensit:8,sent:[29,38,65,68,74,110],sentanc:80,sentenc:[2,8,14,56,57,75,90,93],sentence_input:75,sentence_last_state1:90,sentence_last_state2:90,sentiment:2,sentimental_provid:2,sentimental_train:2,separ:[7,25,38,47,59,65,69,70],seper:75,seq:[8,14,90],seq_len:75,seq_pool:[8,89],seq_silc:8,seq_text_print:7,seq_to_seq_data:121,seq_typ:13,seqenc:21,seqlastin:90,seqtext_evalu:7,seqtoseq:[8,82,121],seqtoseq_net:[8,121],sequel:2,sequenc:[2,5,7,8,9,11,13,14,21,31,43,49,58,61,75,80,87,90,92,123],sequence_conv_pool:123,sequence_group:8,sequence_layer_group:90,sequence_nest_group:8,sequence_nest_layer_group:90,sequencegen:90,sequencestartposit:8,sequencetextprint:7,sequencetyp:[2,8,87],sequenti:[8,31],seri:[9,90],serial:[28,31,37,53,60,68,76],serializ:68,serv:[46,65,68,75],server:[29,32,36,39,40,65,68,78,83,104,106,109,111,119],serverless:33,servic:[101,119],sess:[48,50,58,67],session:[50,58,62],set:[2,6,7,8,9,13,14,15,21,25,28,29,33,41,48,52,56,57,62,63,68,69,72,75,80,82,90,94,97,98,99,101,103,110,121,122,123],set_active_typ:96,set_default_parameter_nam:6,set_drop_r:96,set_float_el:30,set_input:8,set_siz:96,set_typ:96,setdatatyp:73,setdefault:97,setq:94,settotalbyteslimit:82,setup:[59,65,71,96,97,119],sever:[7,8,30,36,43,48,56,57,60,63,73,75],sexstant:119,sgd:[23,28,29,33,41,59,60,65,66,67,73,87,104,105],shall:32,shaoq:20,shape:[7,8,13,18,21,28,30,31,48,51,56,61,63,67,68,73,87],shard:[33,34,35,36,37,38,40,65,66,104],share:[8,21,32,48,55,60,63,68,70,75],shared_bia:9,shared_bias:8,shared_librari:32,shared_ptr:[54,55,72,98],shift:8,shorten:8,shorter:15,should:[6,7,8,13,15,18,20,21,23,25,26,28,29,30,31,38,41,42,43,46,47,48,52,56,57,58,59,60,61,64,65,68,69,70,73,74,75,76,92,97,99],should_be_fals:29,should_be_tru:29,should_shuffl:[2,90],show:[7,10,31,33,45,51,53,56,59,61,75,94],show_check_sparse_distribution_log:[105,106],show_layer_stat:[105,106],show_parameter_stats_period:[105,106,108,110],shown:[8,29,42,65],shrink_rnn_memori:21,shuf:80,shuffl:[13,80],side:[8,28,42,60],sigint:104,sigmod:74,sigmod_op:74,sigmod_output:74,sigmoid:[8,22,31,67,74,75],sigmoidactiv:[9,90],sign:[44,53],similar:[8,21,31,57,59,64,65,66,68,70,75,101,119],similarli:[8,13],simpl:[5,7,8,9,13,14,23,28,46,49,50,56,59,61,66,70,72,74,75,111],simple_attent:93,simple_gru:[93,123],simple_lstm:[8,81,123],simple_rnn:[8,93],simple_transform:15,simpler:60,simpli:[8,15,29,38],simplifi:[29,57,63,74],simul:116,sinc:[8,9,33,37,39,40,52,59,64,65,67,69,70,75,119],singl:[7,9,13,33,42,46,65,66,67,68,72,101],sinlg:28,sit:65,site:[32,101,102],situat:62,size:[2,7,8,9,13,14,15,21,28,33,35,36,38,46,48,53,57,59,63,64,65,67,73,74,75,80,82,87,88,90,93,96,97,98,122,123],size_a:8,size_b:8,size_t:[36,75,96],sizeof:31,skip:[64,82,95,122],sleep:111,sliceproject:8,slide:[8,10,14,33],slightli:48,slope:8,slopeinterceptlay:8,slowli:101,small:[8,14,30,43,48,57],small_messag:[105,106],smaller:[8,21,30,33,46,57],smart:72,smooth:8,snap:110,snapdragon:46,snapshot:[34,40],snippet:[47,58],sock:41,sock_recv_buf_s:[105,106],sock_send_buf_s:[105,106],socket:111,softmax:[8,9,29,31,50,51,57,61,65,66,67,80,93,96,123],softmax_param:82,softmax_param_attr:9,softmax_selfnorm_alpha:8,softmaxactiv:[90,123],softmaxoutput:50,softwar:[46,119],solid:48,solut:119,solv:[29,68],some:[6,8,13,15,21,28,29,31,32,36,37,38,40,41,46,47,48,49,56,57,58,61,62,63,64,66,68,69,72,75,119],some_c_api_funct:55,some_inst:55,some_op:[52,56,75],some_python_class:54,somecppclass:54,somedata:28,somegotyp:54,someth:[36,63,101],sometim:[8,64],someurl:13,somewhat:38,somewher:72,soon:33,sort:[8,14,75,101,111],sort_by_length:75,sourc:[8,14,30,32,45,48,53,55,57,64,68,101],source_dict_dim:[57,93],source_dict_s:57,source_language_word:[57,93],space:[7,8,46,63,66,70],space_seperated_tokens_from_dictionary_according_to_seq:7,space_seperated_tokens_from_dictionary_according_to_sub_seq:7,spars:[6,8,10,13,21,36,80,96,104,106,108,111],sparse_binary_vector:[2,13,80,87],sparse_binary_vector_sequ:13,sparse_float_vector:[2,13,87],sparse_float_vector_sequ:13,sparse_non_value_slot:13,sparse_remot:36,sparse_upd:[6,36,80],sparse_value_slot:13,sparse_vector:80,sparseparam:96,sparseprefetchrowcpumatrix:96,spatial:8,spatial_scal:8,spec:[110,111],special:[8,38,46,52,57,58,65],specialvartypeinfer:52,specif:[28,32,33,45,57,68,72],specifi:[7,8,18,21,29,30,36,37,38,41,42,43,45,48,53,63,65,67,72,74,75,99,101],speech:8,speed:[8,9,46,53,59,119],sphinx:[54,99],split:[2,8,13,40,51,57,68,75,90,104],split_count:[104,111],sqrt:[8,20],squar:[8,10,11,21,50],square_error_cost:87,squarerootnpool:8,squash:95,srand:106,src:[32,104,111],src_backward:93,src_dict:82,src_dict_path:82,src_embed:[57,93],src_forward:93,src_root:4,src_word_id:[57,93],src_word_vec:57,srl:14,ssd:8,ssh:104,ssh_server:104,sstabl:29,stabil:[8,30],stabl:[71,109],stack:[68,75],stackexchang:8,stage:[32,39,48,76,95],stale:[33,67],standalon:115,standard:[6,13,20,68,70,101],stanford:[14,30,110],star:32,start:[8,9,21,28,32,33,36,37,38,40,41,57,60,65,77,95,101,106,110,111],start_mpi_train:104,start_paddl:111,start_pass:[105,106],start_po:8,start_pserv:[105,106],startpaddl:111,startup:[21,33,41],startup_program:[18,21,22,23],stat:[103,106],state:[8,9,18,23,31,33,42,56,57,72,75,81,92,110],state_act:[8,9,90],statement:49,statfulset:111,static_cast:98,staticinput:[8,92,93],statist:[8,18,20,42],statset:103,statu:[41,57,95,103,110,111],status:110,std:[28,32,36,50,52,54,55,62,68,69,72,74,96,97,98,106],stdbuf:104,stderr:104,stdout:[13,104],step:[8,9,11,23,30,31,33,38,42,48,49,57,59,60,63,65,66,68,74,75,90,92,93,101,119],step_id:75,step_input:75,step_net:31,step_output:75,step_scop:68,stepnet:[31,56,68,72],stepout:90,still:[37,40,65,69],stirng:63,stochast:[10,33,37,40,59],stop:8,stop_gradi:21,storag:[44,46,109],store:[7,8,14,28,30,31,32,36,50,52,53,57,60,61,63,65,67,68,69,70,72,75],str:[15,18,28,41,75,108,111],straight:[61,64,73],strategi:[11,33,63,66,106],stream:13,streamid:25,street:8,strict:64,stride:[8,9,21],stride_h:21,stride_i:8,stride_w:21,stride_x:8,string:[2,7,8,13,15,25,28,31,37,45,50,53,61,62,63,67,68,69,72,73,74,76,96,97,106],strip:[82,90,101],struct:[37,38,44,46,52,55,69,74,82],structur:[31,37,48,53,57,61,63,68,73],stuff:95,stun:2,style:[8,68,74],sub:[7,8,13,29,40,48,56,60,63,65],sub_nest_seq:8,sub_sequ:[2,8,87],subclass:[23,63],subcommand:45,subgradi:10,subgraph:[48,66],submiss:65,submit:68,subnet:29,subobjectpath:110,subseq:[89,92],subsequ:8,subsequenceinput:[8,90],subset:21,succeed:[37,110],success:[8,38,110],successfulcr:110,sudo:94,suffer:30,suffix:[18,41,104],suggest:[8,32],suit:119,suitabl:73,sum:[8,10,31,34,52,63],sum_:8,summar:48,sumopgradmak:69,sumpool:[8,80],sun:20,suppli:73,support:[6,7,8,10,11,13,15,21,30,31,33,40,41,47,48,53,57,59,60,62,64,65,66,68,69,70,73,86,90,119],suppos:[9,32,47,73],suppress:[8,45],sure:101,surpass:[8,20],svs:74,swagger:44,swap_channel:122,swig:[39,54,55,83],swig_paddl:4,switchop:31,symbol:[8,31,50,55],symbols_ready_:31,symbolt:[31,68],symlink:95,sync:[33,59,70],sync_with_cpp:[101,102],syncflag:96,synchron:[33,37],syntax:[57,64],sys:122,sysroot:115,system:[31,32,33,38,40,44,47,48,65,66,80,101],t2b:121,tab:123,tabl:[7,8,21,31,52,53,73,76],tablelookup:73,tablelookupgrad:73,tablelookupop:73,tableproject:8,tag:[7,14,77,84,104],tagtyp:7,tail:57,tainer_id:111,take:[2,7,8,9,13,21,28,29,31,32,33,40,43,46,48,49,51,52,59,61,62,63,64,68,69,75,101],taken:[50,65,75],talk:38,tangl:101,tanh:[8,9,21,48,57,65,93,96],tanhactiv:[9,90],tar:[13,15,28],target:[8,14,21,23,28,31,32,48,50,58,62,65,67,68],target_dict_dim:93,target_dict_s:57,target_dictionary_dim:8,target_language_embed:8,target_language_word:93,target_link_librari:32,target_word:57,targetinlink:[8,90],task:[7,8,53,57,65,74],task_queu:37,taskentri:37,taskqueu:37,tbd:[39,90],tcp:106,tear:103,technic:33,techniqu:[21,101],tee:110,tell:[33,37,38,57,74],templat:[47,74,97,98,110,111,119],tempor:8,temporari:[18,41,59,63,120],tensor:[21,30,32,46,48,50,52,53,56,57,66,67,73,75,76,97],tensor_array_read:75,tensor_array_s:75,tensor_array_stack:75,tensor_array_unstack:75,tensor_array_writ:75,tensor_data:53,tensor_s:30,tensor_test:32,tensor_to_check:30,tensorarraydesc:75,tensordesc:[53,73],tensorflow:[31,48,51,65,66,70,75],term:[8,9,21,33,69,70],termin:110,tessorarrai:75,test100:14,test10:14,test1:35,test:[1,2,8,13,14,15,21,28,29,30,32,50,55,59,64,67,71,88,95,96,97,98,103,104,106,108,122,123],test_:97,test_all_data_in_one_period:110,test_check_grad_ingore_i:97,test_check_grad_ingore_x:97,test_check_grad_norm:97,test_compar:77,test_comparespars:77,test_comparetwonet:77,test_comparetwoopt:77,test_config_pars:77,test_data:4,test_data_dir:104,test_fcgrad:96,test_gpuprofil:103,test_layergrad:96,test_list:[2,82],test_mul_op:[83,97],test_networkcompar:77,test_pass:[105,106,108],test_period:[105,106,108],test_predict:77,test_pydataprovid:77,test_pydataprovider2:77,test_pydataproviderwrapp:77,test_recurrent_machine_gener:77,test_recurrentgradientmachin:[77,90],test_swig_api:77,test_train:77,test_traineronepass:77,test_wait:[105,106],testa:29,testb:29,testbilinearfwdbwd:103,testcas:97,testconfig:96,testfcgrad:96,testfclay:96,testlayergrad:96,testmulgradop:97,testmulop:97,testq:29,testresult:28,testutil:96,text1:45,text:[2,7,9,13,29,53,56,121],text_conv:123,text_fil:13,tflop:103,tftp:119,tgz:[14,86],than:[6,7,8,9,21,33,41,47,48,63,68,70,75,82,119],thei:[8,18,20,26,29,32,33,38,40,43,45,48,49,57,58,63,66,67,68,74,75,76,103],them:[7,8,9,15,21,29,30,32,33,36,41,43,47,52,57,64,66,68,69,72,73,74,75,76,103],themselv:32,therefor:59,therein:[8,31],theta:48,theta_d:48,theta_g:48,thi:[2,6,7,8,9,10,13,14,15,18,20,21,23,25,26,28,29,30,31,32,33,36,37,38,39,40,41,42,46,47,48,49,50,56,57,58,59,60,61,63,64,65,66,67,68,69,70,73,74,75,86,101,103,119],thin:52,thing:[2,48,65,68],think:[29,32],third:[8,33,50,101],third_parti:[8,115,116,117],those:[8,31,32,33,47,49,50,51,61],though:[75,119],thought:32,thread:[101,103],thread_local_rand_use_global_se:[105,106],threadblocks:25,threadid:108,threadloc:103,three:[7,8,30,33,42,46,49,57,58,60,61,64,122],threshold:[6,7,8,33,37,106],through:[8,21,32,33,37,39,42,58,59,67,99],throughout:43,throughput:103,thrust:68,thu:[8,40,42,50,120],tier:110,time:[8,9,11,13,28,29,30,32,33,37,40,47,52,56,57,63,64,65,66,68,69,73,74,75,76,90,101,102,103,106,110,111,119],timelin:[8,68],timeout:[33,37],timeout_sec:13,timer:103,timestamp:[8,34],timestep:[8,72],titl:[14,111],tls:44,tmp:[2,63],to_chw:15,to_no_sequ:[8,89],to_sequ:[8,89,90],to_tar:28,to_your_paddle_clone_path:99,todo:[7,13,14,31,33,37,40,57,74],toend:8,togeth:[8,9,13,28,75],token:[7,8,29,93],toler:[28,30],too:[14,30,75],took:119,tool:[99,101,111,115,117],toolchain:[101,115],top:[7,21,28,56,57,122],top_k:[7,21,57],top_level_rnn:56,topk_generated_scor:57,topk_id:57,topk_scor:57,toplevel:94,topolog:[29,33,50,53,60,65,67],topoloi:50,topolopi:28,torch:31,toronto:[8,14],tostr:82,total:[21,28,33,42,64,66,101,103,110,119],total_pass:64,tottim:[101,102],trace:[31,48],track:[33,37,50,63,95],tractabl:8,tradit:[8,31,46],traffic:65,trail:13,train100:14,train10:14,train:[1,2,6,7,8,13,14,15,20,21,31,35,37,38,40,42,48,49,53,59,60,61,62,67,68,70,73,76,78,80,84,87,93,104,106,108,110,111,112,113,121,122,123],train_arg:111,train_args_dict:111,train_args_list:111,train_conf:121,train_config_dir:111,train_data:104,train_data_dir:104,train_i:87,train_list:[2,82,104,122],train_read:87,train_x:87,trainabl:[8,53,63],trainer:[2,4,29,34,35,36,37,39,59,60,65,66,68,87,96,104,106,108,111],trainer_config:[1,2,4,110,111,123],trainer_config_help:[2,96],trainer_count:[80,88,104,105,106,108,110,111],trainer_cpu:41,trainer_cr:41,trainer_gpu:41,trainer_id:[104,106,111],trainer_intern:36,trainer_mem:41,trainer_packag:41,trainerconfighelp:82,trainerid:[40,111],trainerintern:123,trainonebatch:36,tran:96,transact:[33,37],transform:[8,9,15,21,68],transform_param_attr:9,transformed_st:9,translat:[8,9,80,121],translation_id:57,translation_scor:57,transpar:57,transpos:[8,15],transposedfullmatrixproject:8,travel:2,travers:49,travi:95,treat:[8,31,38],treatment:[38,46],tree:[8,31,63,111,117],trg_dic_siz:57,trg_embed:[57,93],trick:57,tricki:54,trigger:[40,60],trim:8,trivial:[57,75],true_block:[31,51,61],true_imag:64,true_label:64,true_neg:42,true_posit:42,true_read:64,truth:[7,8],tune:[6,101,102,105],tupl:[8,9,13,14,15,18,21,28,63,64],ture:8,turn:[8,63,64,92],tutori:[101,104,110,111,112,113,120,121],twice:[48,66],twine:71,two:[7,8,9,21,29,38,39,40,41,42,45,46,48,49,52,53,57,59,61,64,65,67,68,69,70,72,73,74,75,76,97,98,103],txt:[2,32,41,45,96,99,104,109,123],type:[2,7,8,9,11,13,14,18,21,29,31,33,36,37,40,41,44,45,46,52,53,54,55,56,57,61,62,63,64,68,69,70,73,74,76,87,88,90,93,96,97,98,108,110,122,123],type_nam:74,typedef:[38,46,54,55],typeid:74,typenam:[47,74,97,98],typic:[7,65],ubuntu:[71,86,88,101],ubyt:64,uci:14,uci_h:88,ufldl:8,uid:110,uint16_t:46,uint32:[44,53],uint64:[53,54],uint64_t:54,unawar:38,unclear:40,under:[32,37,66],underli:57,understand:[20,63,101,119],understand_senti:93,undeterminist:103,unidirect:8,unifi:[50,73],uniform:[6,8,13,20,35,48,63,64],uniform_random:63,uninstal:77,uniqu:[29,31,33,40,41,63,72],unique_nam:63,unique_name_gener:63,unique_ptr:[69,72,96],unit:[8,9,21,32,59,70],unittest:[55,77,97],unk:[73,76,121],unknown:8,unlik:[8,57],unordered_map:72,unpack:75,unrol:56,unseen:70,unseg:8,unsign:[38,46],unstack:75,unstack_from:75,unsupervis:48,unsupport:97,until:[33,38,66,72,111],untrack:95,unzip:115,updat:[6,8,10,23,33,37,38,44,46,48,56,57,58,59,60,65,72,75,95,101,102,108],update_equ:[28,87],update_hook:6,update_memori:31,update_op:58,updatecallback:96,upgrad:[77,86,120],upload:[33,41,44,71],upon:33,upper:8,upstream:[77,95],url:[13,14],usag:[7,8,9,15,28,46,51,60,104,111],use:[6,7,8,9,11,13,14,15,20,21,23,26,28,29,30,31,32,33,39,43,46,48,50,52,57,58,60,63,65,66,67,72,73,74,75,76,95,97,101,103,104,111,122],use_eigen_bla:115,use_eigen_for_bla:[115,116],use_etcd:28,use_global_stat:8,use_gpu:[4,80,87,88,104,105,106,108,110,111,122,123],use_mkldnn:8,use_nesterov:23,use_old_updat:[36,105,106],use_peephol:21,use_sparse_remote_updat:36,used:[2,7,8,9,10,11,13,14,15,20,21,25,28,29,30,31,32,33,39,40,46,48,50,56,57,59,60,63,64,65,67,68,70,72,74,75,101,103],useful:[8,9,30,46,72],usegpu:96,user:[6,8,9,13,14,15,18,20,23,25,26,28,29,30,31,32,35,37,40,41,42,45,47,48,49,50,52,57,58,59,63,64,65,66,67,68,69,70,72,74,75,101,109,119],user_id:111,user_info:14,user_nam:35,usercert:35,userinfo:14,userkei:35,usernam:[35,95,115],uses:[8,33,40,46,56,57,60,65],using:[6,8,9,13,21,28,29,31,32,33,37,38,40,41,45,46,47,48,50,52,56,58,59,61,64,65,69,70,72,74,88,97,120],usr:[77,83,84,104,106,111],usrdict:121,usrmodel:121,usual:[8,28,41,61,70,101,103],util:[103,111,119],uuid:[34,40],v28:8,v7a:115,v8a:115,valgrind:102,valid:[8,15,64,68,72],valu:[2,4,6,7,8,11,13,14,15,20,21,25,28,30,31,33,42,50,51,53,56,57,58,59,60,61,63,65,67,68,72,73,74,75,76,80,96,108,111,115,122,123],value1:106,value2:106,value_:73,value_evalu:7,value_rang:13,valueerror:[50,80],values_:75,vanilla:93,varabl:66,vardesc:[31,49,61,63,68,73],vardescbuild:31,variabl:[10,13,14,18,20,21,23,29,30,31,42,43,48,49,50,51,52,56,57,58,59,61,62,65,66,67,69,70,73,74,75,101,110],variablenamemap:97,varialbl:48,varianc:8,variant:[8,52,75],varibl:50,varienc:75,varient:75,variou:[31,46,70],varproto:74,vars_:[31,72],vartyp:[21,73,76],vartypeinfer:52,vec1:8,vec2:8,vec:82,veclib:116,vector:[8,9,13,14,21,29,31,36,38,50,51,56,57,63,68,69,73,75,123],veloc:23,vendor:32,verb:14,verbos:45,veri:[8,11,32,37,47,48,57,60,64,66,70,72,75,101],verifi:31,version:[8,9,32,41,45,48,50,51,53,57,65,71,101,103,104,105,106,116],versu:29,vertic:8,vgg:[9,22],via:[33,95,119],view:[8,53],vim:84,virtual:[52,69],virtualenv:94,visibl:[40,72],visit:28,visual:[8,57],vlog:36,volum:[99,109,110,111],volumemount:[110,111],vutbr:14,wai:[7,9,29,38,40,57,59,63,64,67,70,75],wait:[33,38,111],wangkuiyi:32,want:[2,8,29,41,42,48,59,62,64,67,70,72,75,101],warn:[28,45,77,82,111],warp:8,warpctc:8,watch:33,wbia:122,web:101,weight:[7,8,9,10,20,21,26,53,70,96,122],weight_act:9,weightlist:96,weights_:96,weights_t:96,welcom:32,well:[41,47,48,65,67,70,73],were:[7,32],wget:115,what:[6,8,32,48,57,63,65,66,74,95,101,119],wheel:86,when:[2,6,7,8,10,13,18,21,28,30,31,32,33,36,37,38,41,42,45,46,50,57,59,60,61,63,65,66,68,75,101,103,119],whenev:63,where:[8,9,10,20,21,29,31,33,40,49,56,57,59,61,65,68,70,75,101],wherea:[31,37,47,51],whether:[7,8,13,15,21,28,30,31,64,73,75],which:[6,7,8,9,13,14,15,21,28,29,30,31,32,33,35,37,38,40,41,43,46,47,48,50,52,53,56,57,58,60,61,62,63,64,65,66,67,68,69,72,73,74,75,101,102,119],while_loop:[57,75],whileloop:75,whileop:31,whl:[83,86],who:[47,49,63],whoever:38,whole:[2,7,13,48,51,54,55,56,62,74,119],whose:[8,13,30,33,40,56,68,69,74,75],why:[9,30,55,103],wide:[32,48,104],width:[7,8,13,15,36,54,64,82,96,97],wiki:[8,32,104],wikipedia:[8,14,104],wilder:2,window:[8,11,14,59,94,109],wirt:50,wise:[8,15,66,68],with_avx:[83,95,104,115,116],with_bia:74,with_c_api:[83,115,116,117],with_doc:83,with_doubl:[83,96,104],with_dso:83,with_golang:83,with_gpu:[83,94,95,104,115,116],with_mkl:83,with_profil:103,with_python:[83,104,115,116],with_rdma:[104,115,116],with_style_check:[83,95],with_swig_pi:[83,115,116],with_test:[83,95,97],with_tim:[103,104],within:[8,37],without:[7,8,23,33,38,63,64,66,68,101],wloop:75,wmt14:93,wmt_shrinked_data:14,won:90,wonder:2,word2vec:[41,80,104],word:[2,7,8,14,49,52,56,57,66,68,74,75,80,90,92,123],word_dict:[90,104,123],word_dim:[82,90,123],word_id:[2,80],word_idx:14,word_vector_dim:[8,57,93,121],words_freq_sort:14,work:[8,13,21,29,31,32,33,46,58,59,63,65,84,90,95,99,101,110,111,119],worker:[66,76],workflow:68,workspac:104,would:[28,31,32,33,40,47,48,49,58,59,60,63,64,65,66,73,75,101,119],wouldn:49,wrap:[47,48,119],wrapper:[9,32,47,59,65,69,75,103],write:[13,21,29,33,40,46,47,50,52,58,59,63,64,65,66,68,69,75,82],write_lock:34,writer:[29,63],written:[25,31,48,53,59,68,69,73,101],wrong:64,wrote:[50,66],wustl:8,wuyi:109,www:[8,14],x64:117,x86:[115,116],x86_64:[115,116],x_i:8,x_j:8,x_neg:30,x_num_col_dim:21,x_po:30,xarg:[7,77,84,96,104],xavier:20,xavieriniti:21,xcode:116,xcodebuild:116,xgbe0:106,xgbe1:106,xiangyu:20,xmap_read:13,xrang:[30,48,64,87,88,96],xxx:[29,75,122],xxxx:34,y_dim:48,y_neg:30,y_num_col_dim:21,y_po:30,y_predict:[87,88],yaml:[32,104,110,111,119],yancey1989:41,yann:14,yapf:95,yeild:28,yep:[101,102],yet:119,yield:[2,13,29,35,64,80,87,90],yin:8,yoshua:20,you:[2,6,8,9,13,28,30,41,46,72,101,119,122],your:[8,13,28,29,32,36,41,45,68,77,115,116,117,119],your_param_nam:82,your_repo:111,your_source_root:55,yuyang18:[13,14],yuyang:[101,102],z_dim:48,z_size:48,zero:[6,8,9,10,13,14,18,30,33,48,57,60,63,73,106],zhang:20,zip:[14,63,111,115],zoo:121},titles:["API","DataProvider\u7684\u4ecb\u7ecd","PyDataProvider2\u7684\u4f7f\u7528","API\u4e2d\u6587\u624b\u518c","\u57fa\u4e8ePython\u7684\u9884\u6d4b","Activation","Parameter Attribute","Evaluators","Layers","Networks","Optimizer","Pooling","Data Reader Interface and DataSets","Data Reader Interface","Dataset","Image Interface","Fluid","DataFeeder","Evaluator","Executor","Initializer","Layers","Nets","Optimizer","ParamAttr","Profiler","Regularizer","Model Configuration","Training and Inference","PaddlePaddle Design Doc","Auto Gradient Checker Design","Design Doc: Block and Scope","Required CMake Function","Design Doc: Distributed Training","\u6a21\u578b\u53c2\u6570\u68c0\u67e5\u70b9\uff08Checkpointing\uff09","\u8bad\u7ec3\u6570\u636e\u7684\u5b58\u50a8\u548c\u5206\u53d1","Alalysis of large model distributed training in Paddle","Design Doc: Master Server","Design Doc: The Client Library of Parameter Server","Design Doc: Remote Parameter Updater for Cluster Train","Design Doc: Save Model","Submit a Distributed Training Job","Evaluator Design","Executor Design Doc","FileManager\u8bbe\u8ba1\u6587\u6863","PFSClient","Design Doc: float16","Design Doc: Functions, Operators, and Layers","Design for GAN","Design Doc: Computations as a Graph","Survey on Graph","The IfElse Operator","Design Doc: InferVarType","Design Doc: Model Format","Paddle\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0","C-API \u6a21\u578b\u63a8\u65ad\u5b9e\u73b0\u6587\u6863","RNNOp design","Design: Sequence Decoder Generating LoDTensors","Optimizer Design","Averaging Parameter in PaddlePaddle","Design Doc: The C++ Class Parameters","Design Doc: PaddlePaddle Programs","Prune","Design Doc: Python API","Python Data Reader Design Doc","Design Doc: Distributed Training Architecture","Design Doc: Operation Graph Based Parameter Server","Design Doc: Session","Design Doc: Refactorization Overview","Design Doc: Gradient Operators Registration","Regularization in PaddlePaddle","PaddlePaddle\u53d1\u884c\u89c4\u8303","Design of Scope in Paddle","Design Doc: Selected Rows","Interaction between C++ and Python","Design for TensorArray","Background","\u7f16\u8bd1\u5b89\u88c5\u4e0e\u5355\u5143\u6d4b\u8bd5","\u96c6\u7fa4\u8bad\u7ec3\u4e0e\u9884\u6d4b","FAQ","\u672c\u5730\u8bad\u7ec3\u4e0e\u9884\u6d4b","\u6a21\u578b\u914d\u7f6e","\u53c2\u6570\u8bbe\u7f6e","\u4ece\u6e90\u7801\u7f16\u8bd1","\u4f7f\u7528Docker\u5b89\u88c5\u8fd0\u884c","\u5b89\u88c5\u4e0e\u7f16\u8bd1","\u4f7f\u7528pip\u5b89\u88c5","\u57fa\u672c\u4f7f\u7528\u6982\u5ff5","\u65b0\u624b\u5165\u95e8","\u652f\u6301\u53cc\u5c42\u5e8f\u5217\u4f5c\u4e3a\u8f93\u5165\u7684Layer","\u5355\u53cc\u5c42RNN API\u5bf9\u6bd4\u4ecb\u7ecd","RNN\u76f8\u5173\u6a21\u578b","Recurrent Group\u6559\u7a0b","RNN\u914d\u7f6e","\u7528Docker\u7f16\u8bd1\u548c\u6d4b\u8bd5PaddlePaddle","\u5982\u4f55\u8d21\u732e\u4ee3\u7801","\u5b9e\u73b0\u65b0\u7684\u7f51\u7edc\u5c42","\u5982\u4f55\u5199\u65b0\u7684Operator","\u5728Paddle\u4e2d\u5982\u4f55\u4f7f\u7528Eigen","\u5982\u4f55\u8d21\u732e/\u4fee\u6539\u6587\u6863","\u8fdb\u9636\u6307\u5357","Profiling the Python Code","Python\u4ee3\u7801\u7684\u6027\u80fd\u5206\u6790","GPU\u6027\u80fd\u5206\u6790\u4e0e\u8c03\u4f18","PaddlePaddle\u5206\u5e03\u5f0f\u8bad\u7ec3","\u53c2\u6570\u6982\u8ff0","\u7ec6\u8282\u63cf\u8ff0","\u8bbe\u7f6e\u547d\u4ee4\u884c\u53c2\u6570","\u4f7f\u7528\u6848\u4f8b","Kubernetes \u7b80\u4ecb","Kubernetes\u5355\u673a\u8bad\u7ec3","Kubernetes\u5206\u5e03\u5f0f\u8bad\u7ec3","<no title>","<no title>","PaddlePaddle \u6587\u6863","Android\u5e73\u53f0\u7f16\u8bd1\u6307\u5357","iOS\u5e73\u53f0\u7f16\u8bd1\u6307\u5357","Raspberry Pi\u5e73\u53f0\u7f16\u8bd1\u6307\u5357","MOBILE","Cluster bootstrapping tool survey","<no title>","\u4e2d\u6587\u8bcd\u5411\u91cf\u6a21\u578b\u7684\u4f7f\u7528","Model Zoo - ImageNet","\u5feb\u901f\u5165\u95e8\u6559\u7a0b"],titleterms:{"\u4e00\u4e9b\u7ec6\u8282\u7684\u8865\u5145":111,"\u4e0a\u4f20\u8bad\u7ec3\u6587\u4ef6":35,"\u4e0b\u8f7d\u548c\u6570\u636e\u62bd\u53d6":121,"\u4e0b\u8f7d\u6570\u636e":110,"\u4e0d\u4f7f\u7528":54,"\u4e0d\u4f7f\u7528swig\u8fd9\u79cd\u4ee3\u7801\u751f\u6210\u5668":54,"\u4e0d\u540c\u7684":81,"\u4e0d\u5bfc\u51fapaddle\u5185\u90e8\u7684\u7ed3\u6784\u4f53":54,"\u4e0d\u5f15\u7528\u5176\u4ed6\u52a8\u6001\u5e93":54,"\u4e24\u79cd\u4f7f\u7528":81,"\u4e2d\u6587\u5b57\u5178":121,"\u4e2d\u6587\u77ed\u8bed\u6539\u5199\u7684\u4f8b\u5b50":121,"\u4e2d\u6587\u8bcd\u5411\u91cf\u6a21\u578b\u7684\u4f7f\u7528":121,"\u4e2d\u6587\u8bcd\u5411\u91cf\u7684\u9884\u8bad\u7ec3\u6a21\u578b":121,"\u4e3a\u4ec0\u4e48\u8981":94,"\u4e3a\u4ec0\u4e48\u9700\u8981\u6027\u80fd\u5206\u6790":103,"\u4ec0\u4e48\u662f\u6027\u80fd\u5206\u6790":103,"\u4ec5\u4ec5\u4f7f\u7528void":54,"\u4ecb\u7ecd":[121,122],"\u4ece\u5feb\u7167\u6062\u590d":34,"\u4ece\u6e90\u7801\u7f16\u8bd1":83,"\u4ee3\u7801\u8981\u6c42":95,"\u4f18\u5316\u7b97\u6cd5":123,"\u4f7f\u7528":[95,110],"\u4f7f\u7528\u5206\u5e03\u5f0f\u8ba1\u7b97\u5e73\u53f0\u6216\u5de5\u5177":104,"\u4f7f\u7528\u52a8\u6001\u5e93\u6765\u5206\u53d1paddl":54,"\u4f7f\u7528\u6848\u4f8b":108,"\u4f7f\u7528\u6982\u8ff0":123,"\u4f7f\u7528\u6a21\u578b\u521d\u59cb\u5316\u7f51\u7edc":108,"\u4f7f\u7528\u73af\u5883\u53d8\u91cf":111,"\u4f7f\u7528\u7528\u6237\u6307\u5b9a\u7684\u8bcd\u5411\u91cf\u5b57\u5178":121,"\u4f7f\u7528\u8bf4\u660e":100,"\u4f7f\u7528\u8f6c\u6362\u5e93":35,"\u4f7f\u7528docker\u542f\u52a8paddlepaddl":84,"\u4f7f\u7528docker\u5b89\u88c5\u8fd0\u884c":84,"\u4f7f\u7528docker\u6267\u884cgpu\u8bad\u7ec3":84,"\u4f7f\u7528docker\u6784\u5efa":99,"\u4f7f\u7528fabric\u542f\u52a8\u96c6\u7fa4\u4f5c\u4e1a":104,"\u4f7f\u7528paddlepaddl":99,"\u4f7f\u7528pip\u5b89\u88c5":86,"\u4fdd\u6301\u672c\u5730\u4ed3\u5e93\u6700\u65b0":95,"\u4fee\u6539\u542f\u52a8\u811a\u672c":110,"\u4fee\u6539\u6587\u6863":99,"\u514b\u9686":95,"\u5177\u4f53\u67d0\u79cd\u7c7b\u578b\u7684\u5934\u6587\u4ef6":55,"\u5177\u4f53\u67d0\u79cd\u7c7b\u578b\u7684\u5b9e\u73b0\u6587\u4ef6":55,"\u5185\u5b58\u4e0d\u591f\u7528\u7684\u60c5\u51b5":2,"\u5185\u7f6e\u5b9a\u65f6\u5668":103,"\u5199\u68af\u5ea6\u68c0\u67e5\u5355\u5143\u6d4b\u8bd5":96,"\u51c6\u5907\u4e00\u4e2alinux\u96c6\u7fa4":104,"\u51c6\u5907\u4ea4\u53c9\u7f16\u8bd1\u73af\u5883":[115,116],"\u51c6\u5907\u6570\u636e\u96c6":104,"\u51c6\u5907\u8bad\u7ec3\u6570\u636e":111,"\u51c6\u5907\u8bad\u7ec3\u7a0b\u5e8f":104,"\u51c6\u5907openmpi\u96c6\u7fa4":104,"\u51cf\u5c11\u6570\u636e\u8f7d\u5165\u7684\u8017\u65f6":80,"\u51cf\u5c11dataprovider\u7f13\u51b2\u6c60\u5185\u5b58":80,"\u51fa\u73b0":81,"\u5206\u5757\u6587\u4ef6\u4f20\u8f93":44,"\u5206\u652f\u89c4\u8303":71,"\u521b\u5efa\u672c\u5730\u5206\u652f":95,"\u521b\u5efajob":111,"\u521b\u5efapaddl":110,"\u5220\u9664\u672c\u5730\u5206\u652f":95,"\u5220\u9664\u8fdc\u7a0b\u5206\u652f":95,"\u5229\u7528\u66f4\u591a\u7684\u8ba1\u7b97\u8d44\u6e90":80,"\u5230\u8fdc\u7a0b\u4ed3\u5e93":95,"\u5236\u4f5c\u955c\u50cf":111,"\u5236\u4f5cdocker\u955c\u50cf":110,"\u524d\u5411operator\u5355\u5143\u6d4b\u8bd5":97,"\u52a0\u8f7dpaddlepaddl":87,"\u52a0\u901f\u6267\u884c":34,"\u52a0\u901f\u8bad\u7ec3\u901f\u5ea6":80,"\u52a8\u6001\u5e93\u4e2d\u4e0d\u5d4c\u5165\u4efb\u4f55\u5176\u4ed6\u8bed\u8a00\u7684\u89e3\u91ca\u5668":54,"\u52a8\u6001\u6269\u5bb9":34,"\u5355\u5143\u6d4b\u8bd5":106,"\u5355\u53cc\u5c42rnn":90,"\u5377\u79ef\u6a21\u578b":123,"\u539f\u56e0":54,"\u539f\u56e0\u5217\u8868":54,"\u53c2\u6570\u4fe1\u606f":122,"\u53c2\u6570\u5185\u5b58":80,"\u53c2\u6570\u670d\u52a1\u5668\u548c\u5206\u5e03\u5f0f\u901a\u4fe1":106,"\u53c2\u6570\u6982\u8ff0":105,"\u53c2\u6570\u8bbe\u7f6e":82,"\u53c2\u6570\u8bfb\u53d6":122,"\u53c2\u8003":2,"\u53c2\u8003\u6587\u6863":44,"\u53c2\u8003\u8d44\u6599":103,"\u53cc\u5c42rnn":90,"\u53cc\u5c42rnn\u4ecb\u7ecd":92,"\u53cc\u5c42rnn\u7684\u4f7f\u7528":92,"\u53cd\u5411operator\u5355\u5143\u6d4b\u8bd5":97,"\u53ef\u80fd\u7684\u5185\u5b58\u6cc4\u9732\u95ee\u9898":2,"\u53ef\u80fd\u78b0\u5230\u7684\u95ee\u9898":94,"\u53ef\u9009\u529f\u80fd":121,"\u5404\u4e2a\u7248\u672c\u6700\u65b0\u7684whl\u5305":86,"\u540d\u8bcd\u89e3\u91ca":44,"\u5411\u7cfb\u7edf\u4f20\u9001\u6570\u636e":123,"\u5411\u91cf":106,"\u542f\u52a8\u4efb\u52a1":111,"\u542f\u52a8\u53c2\u6570\u670d\u52a1\u5668":104,"\u542f\u52a8\u53c2\u6570\u8bf4\u660e":104,"\u542f\u52a8\u8ba1\u7b97\u8282\u70b9":104,"\u542f\u52a8\u96c6\u7fa4\u4f5c\u4e1a":104,"\u5440":94,"\u547d\u4ee4\u884c\u53c2\u6570":123,"\u548c":89,"\u5728\u4e0d\u540c\u8bbe\u5907\u4e0a\u6307\u5b9a\u5c42":108,"\u5728docker\u4e2d\u6267\u884cpaddlepaddle\u8bad\u7ec3\u7a0b\u5e8f":84,"\u5728kubernetes\u96c6\u7fa4\u4e2d\u63d0\u4ea4\u8bad\u7ec3\u4f5c\u4e1a":104,"\u5728openmpi\u96c6\u7fa4\u4e2d\u63d0\u4ea4\u8bad\u7ec3\u4f5c\u4e1a":104,"\u5728paddle\u4e2d\u5982\u4f55\u4f7f\u7528eigen":98,"\u5728paddlepaddle\u5e73\u53f0\u8bad\u7ec3\u6a21\u578b":121,"\u57fa\u4e8edocker\u5bb9\u5668\u7684\u7f16\u8bd1\u65b9\u5f0f":115,"\u57fa\u4e8elinux\u4ea4\u53c9\u7f16\u8bd1\u73af\u5883\u7684\u7f16\u8bd1\u65b9\u5f0f":115,"\u57fa\u4e8epython\u7684\u9884\u6d4b":4,"\u57fa\u672c\u4f7f\u7528\u6982\u5ff5":87,"\u57fa\u672c\u539f\u7406":92,"\u57fa\u672c\u8981\u6c42":54,"\u5982\u4f55\u4e66\u5199\u6587\u6863":99,"\u5982\u4f55\u4f7f\u7528":81,"\u5982\u4f55\u5171\u4eab\u53c2\u6570":82,"\u5982\u4f55\u5199\u65b0\u7684oper":97,"\u5982\u4f55\u51cf\u5c11\u5185\u5b58\u5360\u7528":80,"\u5982\u4f55\u521d\u59cb\u5316\u53c2\u6570":82,"\u5982\u4f55\u52a0\u8f7d\u9884\u8bad\u7ec3\u53c2\u6570":82,"\u5982\u4f55\u52a0\u901f\u8bad\u7ec3\u901f\u5ea6":80,"\u5982\u4f55\u548c\u660e\u6587\u8fdb\u884c\u76f8\u4e92\u8f6c\u5316":82,"\u5982\u4f55\u6307\u5b9agpu\u8bbe\u5907":80,"\u5982\u4f55\u66f4\u65b0www":99,"\u5982\u4f55\u6784\u5efa\u6587\u6863":99,"\u5982\u4f55\u8bbe\u7f6e\u5b66\u4e60\u7387\u9000\u706b":82,"\u5982\u4f55\u8c03\u7528":80,"\u5982\u4f55\u8d21\u732e":99,"\u5982\u4f55\u8d21\u732e\u4ee3\u7801":95,"\u5982\u4f55\u8fdb\u884c\u6027\u80fd\u5206\u6790":103,"\u5982\u4f55\u9009\u62e9sgd\u7b97\u6cd5\u7684\u5b66\u4e60\u7387":82,"\u5b50\u5e8f\u5217\u95f4\u65e0memori":90,"\u5b50\u5e8f\u5217\u95f4\u6709memori":90,"\u5b58\u50a8\u7684\u53c2\u6570\u683c\u5f0f\u662f\u4ec0\u4e48":82,"\u5b89\u88c5":[86,123],"\u5b89\u88c5\u4e0e\u7f16\u8bd1":85,"\u5b89\u88c5\u4ea4\u53c9\u7f16\u8bd1\u5668":117,"\u5b89\u88c5\u6d41\u7a0b":85,"\u5b89\u88c5kubectl":109,"\u5b9a\u4e49operator\u7c7b":97,"\u5b9a\u4e49opkernel\u7c7b":97,"\u5b9a\u4e49protomaker\u7c7b":97,"\u5b9e\u73b0":54,"\u5b9e\u73b0\u5355\u5143\u6d4b\u8bd5":97,"\u5b9e\u73b0\u65b0\u7684\u7f51\u7edc\u5c42":96,"\u5b9e\u73b0\u65b9\u5f0f":55,"\u5b9e\u73b0\u8ba1\u7b97":98,"\u5b9e\u73b0c":[96,97],"\u5b9e\u73b0python\u5c01\u88c5":96,"\u5bfb\u627e\u6027\u80fd\u74f6\u9888":102,"\u5bfc\u51fac":54,"\u5c06\u547d\u4ee4\u53c2\u6570\u4f20\u7ed9\u7f51\u7edc\u914d\u7f6e":108,"\u5c0f\u7ed3":2,"\u5de5\u5177":103,"\u5e38\u89c1\u95ee\u9898\u548c\u89e3\u51b3\u65b9\u6cd5":86,"\u5e38\u89c1\u95ee\u9898\u89e3\u7b54":85,"\u5e76\u5b8c\u6210":95,"\u5efa\u7acb":95,"\u5f00\u53d1\u6807\u51c6":100,"\u5f00\u59cb\u5f00\u53d1":95,"\u5f02\u6b65\u968f\u673a\u68af\u5ea6\u4e0b\u964d":106,"\u5feb\u7167\u4fdd\u5b58\u7684\u8bbe\u8ba1\u5982\u4e0b":34,"\u5feb\u901f\u5165\u95e8\u6559\u7a0b":123,"\u5feb\u901f\u5b89\u88c5":88,"\u5feb\u901f\u5f00\u59cb":88,"\u6027\u80fd\u4f18\u5316":100,"\u6027\u80fd\u5206\u6790\u5c0f\u6280\u5de7":103,"\u6027\u80fd\u5206\u6790\u5de5\u5177\u4ecb\u7ecd":103,"\u6027\u80fd\u8c03\u4f18":106,"\u603b\u4f53\u6548\u679c\u603b\u7ed3":123,"\u603b\u4f53\u6d41\u7a0b":94,"\u6216\u8005\u662f":77,"\u6267\u884c\u5355\u5143\u6d4b\u8bd5":83,"\u627e\u5230\u7684pythonlibs\u548cpythoninterp\u7248\u672c\u4e0d\u4e00\u81f4":77,"\u62a5importerror":77,"\u6307\u9488\u4f5c\u4e3a\u7c7b\u578b\u7684\u53e5\u67c4":54,"\u63a5\u53e3":122,"\u63a5\u53e3\u8f93\u51fa\u591a\u4e2alayer\u7684\u9884\u6d4b\u7ed3\u679c":80,"\u63a8\u5bfc\u65b9\u7a0b":96,"\u63a8\u6d4b\u6267\u884c":34,"\u63d0\u4ea4":95,"\u63d0\u4ea4\u4ee3\u7801\u7684\u4e00\u4e9b\u7ea6\u5b9a":95,"\u63d0\u4ea4\u955c\u50cf":110,"\u642d\u5efa\u795e\u7ecf\u7f51\u7edc":87,"\u652f\u6301\u53cc\u5c42\u5e8f\u5217\u4f5c\u4e3a\u8f93\u5165\u7684layer":89,"\u652f\u6301\u7528\u6237\u81ea\u5b9a\u4e49\u7684\u6570\u636e\u9884\u5904\u7406job":35,"\u6570\u636e\u652f\u6301":106,"\u6570\u636e\u683c\u5f0f\u51c6\u5907":123,"\u6570\u636e\u7684\u51c6\u5907\u548c\u9884\u5904\u7406":121,"\u6574\u4f53\u65b9\u6848":111,"\u6587\u4ef6\u4f20\u8f93\u4f18\u5316":44,"\u6587\u4ef6\u8bbf\u95ee\u65b9\u5f0f":35,"\u6587\u4ef6\u8bbf\u95ee\u7684\u6743\u9650":35,"\u6587\u4ef6\u9884\u5904\u7406":35,"\u6587\u6863":114,"\u65b0\u624b\u5165\u95e8":88,"\u65e5\u5fd7\u4e2d\u4fdd\u5b58\u5747\u4e3a\u7f51\u7edc\u901a\u4fe1\u7c7b\u9519\u8bef":78,"\u65f6\u5e8f\u6a21\u578b":123,"\u65f6\u5e8f\u6a21\u578b\u7684\u4f7f\u7528\u573a\u666f":2,"\u65f6\u95f4\u5e8f\u5217":90,"\u65f6\u95f4\u6b65":90,"\u66b4\u9732\u63a5\u53e3\u539f\u5219":55,"\u672c\u5730\u6d4b\u8bd5":108,"\u672c\u5730\u8bad\u7ec3":108,"\u672c\u5730\u8bad\u7ec3\u4e0e\u9884\u6d4b":80,"\u672f\u8bed":34,"\u6784\u5efa\u548c\u6d4b\u8bd5":95,"\u6784\u5efapaddlepaddle\u7684android\u5f00\u53d1\u955c\u50cf":115,"\u67b6\u6784\u56fe":44,"\u67e5\u770b\u6027\u80fd\u5206\u6790\u6587\u4ef6":102,"\u67e5\u770b\u8bad\u7ec3\u7ed3\u679c":110,"\u67e5\u770b\u8f93\u51fa":111,"\u6837\u4f8b\u6570\u636e":2,"\u6846\u67b6\u751f\u6210":44,"\u6848\u4f8b\u4e00":108,"\u6848\u4f8b\u4e8c":108,"\u68c0\u67e5\u6a21\u578b\u8f93\u51fa":104,"\u68c0\u67e5\u96c6\u7fa4\u8bad\u7ec3\u7ed3\u679c":104,"\u6982\u5ff5\u7b80\u4ecb":97,"\u6982\u5ff5\u89e3\u91ca":35,"\u6982\u8ff0":[89,92,104],"\u6a21\u5757":44,"\u6a21\u578b":122,"\u6a21\u578b\u4e0b\u8f7d":122,"\u6a21\u578b\u53c2\u6570\u68c0\u67e5\u70b9":34,"\u6a21\u578b\u63a8\u65ad\u5b9e\u73b0\u6587\u6863":55,"\u6a21\u578b\u7f51\u7edc\u7ed3\u6784":123,"\u6a21\u578b\u914d\u7f6e":[81,90,100],"\u6a21\u578b\u914d\u7f6e\u7684\u6a21\u578b\u914d\u7f6e":90,"\u6ce8\u518coper":97,"\u6ce8\u610f\u4e8b\u9879":[2,97],"\u6d41\u7a0b\u4ecb\u7ecd":35,"\u6d4b\u8bd5":106,"\u6df7\u5408\u4ee3\u7801\u7684\u6027\u80fd\u5206\u6790":102,"\u7279\u5f81\u63d0\u53d6":122,"\u73af\u5883\u51c6\u5907":104,"\u751f\u6210\u5e8f\u5217":93,"\u751f\u6210\u6027\u80fd\u5206\u6790\u6587\u4ef6":102,"\u751f\u6210\u6d41\u7a0b\u7684\u4f7f\u7528\u65b9\u6cd5":92,"\u751f\u6210sparse\u6587\u4ef6":44,"\u7528\u6237\u4f7f\u7528\u6d41\u7a0b":44,"\u7528docker\u7f16\u8bd1\u548c\u6d4b\u8bd5paddlepaddl":94,"\u7684\u533a\u522b":81,"\u7684\u53c2\u6570":81,"\u7684\u65b9\u6cd5\u6709\u4f55\u533a\u522b":81,"\u76ee\u5f55\u7ed3\u6784":55,"\u76ee\u6807":44,"\u76f4\u63a5\u6784\u5efa":99,"\u76f8\u5173\u6982\u5ff5":92,"\u77e9\u9635":106,"\u793a\u4f8b1":90,"\u793a\u4f8b2":90,"\u793a\u4f8b3":90,"\u793a\u4f8b4":90,"\u793a\u4f8b\u7a0b\u5e8f":35,"\u795e\u7ecf\u5143\u6fc0\u6d3b\u5185\u5b58":80,"\u7a00\u758f\u8bad\u7ec3":108,"\u7b26\u53f7":54,"\u7b80\u4ecb":109,"\u7b80\u5355\u95e8\u63a7\u5faa\u73af\u795e\u7ecf\u7f51\u7edc":93,"\u7c7b":[54,96,97],"\u7ebf\u6027\u56de\u5f52\u5b8c\u6574\u793a\u4f8b":87,"\u7ec6\u8282\u63cf\u8ff0":106,"\u7ec8\u6b62\u96c6\u7fa4\u4f5c\u4e1a":104,"\u7ed1\u5b9apython":97,"\u7f16\u5199yaml\u6587\u4ef6":110,"\u7f16\u8bd1":97,"\u7f16\u8bd1\u4f9d\u8d56":83,"\u7f16\u8bd1\u548c\u5b89\u88c5":[115,116,117],"\u7f16\u8bd1\u548c\u6267\u884c\u5355\u5143\u6d4b\u8bd5":97,"\u7f16\u8bd1\u5b89\u88c5\u4e0e\u5355\u5143\u6d4b\u8bd5":77,"\u7f16\u8bd1\u5b89\u88c5\u540e\u6267\u884c":77,"\u7f16\u8bd1\u65b9\u6cd5":83,"\u7f16\u8bd1\u6d41\u7a0b":85,"\u7f16\u8bd1\u9009\u9879":[55,83],"\u7f16\u8bd1\u9009\u9879\u7684\u8bbe\u7f6e":83,"\u7f16\u8bd1\u9009\u9879\u8bf4\u660e":83,"\u7f16\u8bd1paddlepaddl":115,"\u7f29\u5bb9":34,"\u7f51\u7edc\u53ef\u89c6\u5316":122,"\u7f51\u7edc\u914d\u7f6e\u4e2d\u7684\u8c03\u7528":2,"\u800c\u662f\u624b\u5199\u591a\u8bed\u8a00\u7ed1\u5b9a":54,"\u80cc\u666f":54,"\u81ea\u7136\u8bed\u8a00\u5904\u7406":106,"\u83b7\u53d6paddlepaddle\u7684docker\u955c\u50cf":84,"\u8986\u76d6\u4e0d\u4e00\u81f4\u7684\u90e8\u5206":44,"\u89c2\u6d4b\u8bcd\u5411\u91cf":121,"\u8bad\u7ec3":106,"\u8bad\u7ec3\u56e0\u6b64\u9000\u51fa\u600e\u4e48\u529e":80,"\u8bad\u7ec3\u6570\u636e\u5b58\u50a8":35,"\u8bad\u7ec3\u6570\u636e\u7684\u5b58\u50a8\u548c\u5206\u53d1":35,"\u8bad\u7ec3\u6a21\u578b":[87,123],"\u8bad\u7ec3\u6d41\u7a0b\u7684\u4f7f\u7528\u65b9\u6cd5":92,"\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u51fa\u73b0":80,"\u8bbe\u7f6e\u547d\u4ee4\u884c\u53c2\u6570":107,"\u8bcd\u5411\u91cf\u6a21\u578b":123,"\u8bcd\u5411\u91cf\u6a21\u578b\u7684\u4fee\u6b63":121,"\u8bcd\u6c47\u8868":90,"\u8be6\u7ec6\u6559\u7a0b":103,"\u8bfb\u53d6\u53cc\u5c42\u5e8f\u5217\u6570\u636e":90,"\u8f6c\u6362\u5e93":35,"\u8f93\u5165":92,"\u8f93\u5165\u4e0d\u7b49\u957f":90,"\u8f93\u5165\u793a\u4f8b":92,"\u8f93\u51fa":92,"\u8f93\u51fa\u65e5\u5fd7":123,"\u8fd0\u884c\u5bb9\u5668":110,"\u8fd0\u884c\u73af\u5883\u4f9d\u8d56":86,"\u8fd0\u884cdocker":77,"\u8fd9\u4e2a\u52a8\u6001\u5e93\u4f7f\u7528c99\u6807\u51c6\u7684\u5934\u6587\u4ef6\u5bfc\u51fa\u4e00\u4e9b\u51fd\u6570":54,"\u8fdb\u884c\u8bad\u7ec3":[35,110],"\u8fdb\u9636\u6307\u5357":100,"\u9009\u62e9\u5b58\u50a8\u65b9\u6848":109,"\u901a\u7528":106,"\u903b\u8f91\u56de\u5f52\u6a21\u578b":123,"\u9047\u5230":77,"\u90e8\u7f72kubernetes\u96c6\u7fa4":109,"\u914d\u7f6e\u4e2d\u7684\u6570\u636e\u52a0\u8f7d\u5b9a\u4e49":123,"\u914d\u7f6e\u4ea4\u53c9\u7f16\u8bd1\u53c2\u6570":[115,116,117],"\u914d\u7f6e\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\u67b6\u6784":93,"\u914d\u7f6e\u7f51\u7edc":87,"\u914d\u7f6ekubectl":109,"\u914d\u7f6ekubectl\u8bbf\u95ee\u4f60\u7684kubernetes\u96c6\u7fa4":109,"\u94a9\u5b50":95,"\u9519\u8bef\u600e\u4e48\u529e":81,"\u9644\u5f55":123,"\u968f\u673a\u6570":106,"\u96c6\u7fa4\u591a\u8282\u70b9\u8bad\u7ec3":78,"\u96c6\u7fa4\u8bad\u7ec3":108,"\u96c6\u7fa4\u8bad\u7ec3\u4e0e\u9884\u6d4b":78,"\u9700\u8981\u7684\u8f6f\u786c\u4ef6":94,"\u975e\u6cd5\u6307\u4ee4":77,"\u9884\u6d4b":[122,123],"\u9884\u6d4b\u6d41\u7a0b":4,"\u9884\u6d4bdemo":4,"abstract":[65,66,67,119],"android\u5e73\u53f0\u7f16\u8bd1\u6307\u5357":115,"api\u4e2d\u6587\u624b\u518c":3,"api\u5bf9\u6bd4\u4ecb\u7ecd":90,"api\u5e93":115,"beam_search\u7684\u751f\u6210":90,"book\u4e2d\u6240\u6709\u7ae0\u8282":71,"book\u6559\u7a0b":84,"class":[48,60,63],"cmake\u6e90\u7801\u7f16\u8bd1":77,"dataprovider\u7684\u4ecb\u7ecd":1,"dataprovider\u7684\u4f7f\u7528":2,"filemanager\u8bbe\u8ba1\u6587\u6863":44,"float":80,"function":[32,47,48,63],"gpu\u548ccpu\u6df7\u5408\u4f7f\u7528":108,"gpu\u6027\u80fd\u5206\u6790\u4e0e\u8c03\u4f18":103,"gpu\u955c\u50cf\u51fa\u73b0":77,"group\u6559\u7a0b":92,"import":77,"ios\u5e73\u53f0\u7f16\u8bd1\u6307\u5357":116,"kubernetes\u5206\u5e03\u5f0f\u8bad\u7ec3":111,"kubernetes\u5355\u673a\u8bad\u7ec3":110,"mnist\u7684\u4f7f\u7528\u573a\u666f":2,"org\u5de5\u5177":99,"paddle\u52a8\u6001\u5e93\u4e2d":54,"paddle\u591a\u8bed\u8a00\u63a5\u53e3\u5b9e\u73b0":54,"paddle\u7248\u672c\u53f7\u4e3a0":77,"paddlepaddle\u5206\u5e03\u5f0f\u8bad\u7ec3":104,"paddlepaddle\u53d1\u884c\u89c4\u8303":71,"paddlepaddle\u56de\u5f52\u6d4b\u8bd5\u5217\u8868":71,"paddlepaddle\u73af\u5883\u4f9d\u8d56":86,"paddlepaddle\u7f16\u8bd1\u4f9d\u8d56":83,"pi\u5e73\u53f0\u7f16\u8bd1\u6307\u5357":117,"pod\u95f4\u901a\u4fe1":111,"pydataprovider2\u7684\u4f7f\u7528":2,"python\u4e0ec":102,"python\u4ee3\u7801\u7684\u6027\u80fd\u5206\u6790":102,"python\u63a5\u53e3":122,"python\u76f8\u5173\u7684\u5355\u5143\u6d4b\u8bd5\u90fd\u8fc7\u4e0d\u4e86":77,"python\u811a\u672c\u8bfb\u53d6\u6570\u636e":123,"return":64,"rnn\u76f8\u5173\u6a21\u578b":91,"rnn\u914d\u7f6e":93,"tensor\u4f7f\u7528\u6837\u4f8b":98,"tensor\u5230eigentensor\u7684\u8f6c\u6362":98,"tensor\u6a21\u5757":98,Abs:5,For:32,NOT:43,The:[31,38,48,49,51,52,58,60,68,69],Use:[31,61],Using:[32,38],With:41,about:48,absolut:57,accuraci:21,activ:[5,8],adadelta:10,adagrad:10,adagradoptim:23,adam:10,adamax:10,adamaxoptim:23,adamoptim:23,addto:8,aggreg:8,aggregatelevel:8,alalysi:36,algorithm:[30,33,56,62,65],all:[72,75],analysi:65,anneal:82,api:[0,3,55,58,59,63,70,74],appendix:119,applic:3,architectur:65,argument:[45,64],arrai:30,array_length:21,array_read:21,array_to_lod_tensor:21,array_writ:21,assign:21,associ:72,assumpt:119,async:106,attent:93,attribut:[6,70],auc:7,auto:30,averag:59,avg:11,backgraound:30,background:[66,67,75,76],backward:[49,68],base:[41,57,66],basepool:11,basic:119,batch:64,batch_norm:[8,21],batch_siz:64,beam:57,beam_search:8,beam_search_decod:21,becaus:82,benefit:[66,68],between:[29,63,68,74],bidirectional_gru:9,bidirectional_lstm:9,big:82,bilinear_interp:8,binari:31,bla:83,block:[31,48,49,61,63,68],block_expand:8,blockdesc:61,bootstrap:119,bottleneck:101,brelu:5,bring:119,build:[48,68],cach:2,can:72,capi:55,capi_priv:55,cast:21,challeng:[62,66],chang:57,check:[8,30],checker:30,checkpoint:[33,34,40],choos:32,chunk:7,cifar:14,classif:7,classification_error:7,classification_error_print:7,client:38,clip:8,clone:95,close:30,cluster:[39,119],cmake:32,code:[41,63,101],column_sum:7,commit:95,compar:119,comparis:63,compil:[31,46,61,68],compos:64,comput:[31,49,68,70],con:119,concat:[8,21],concept:[63,68],conclus:[40,50,119],condit:48,config:3,configur:27,conll05:14,connect:8,constantiniti:20,construct:49,content:[2,55,77,78,80,81,82,89,103],context_project:8,control:68,conv2d:21,conv2d_transpos:21,conv:8,conv_oper:8,conv_project:8,conv_shift:8,convert:[40,65,66],core:[30,63],cos_sim:[8,21],cost:8,creat:[64,67,68,72],create_arrai:21,creation:[37,59,70],creator:64,crf:8,crf_decod:8,cross_channel_norm:8,cross_entropi:21,cross_entropy_cost:8,cross_entropy_with_selfnorm_cost:8,ctc:8,ctc_error:7,cuda:[46,77,83],cudnn:83,cudnnavg:11,cudnnmax:11,current:[46,69],custom:64,data:[8,12,13,21,33,64,65],datafeed:[13,17],dataprovid:[3,106],dataset:[12,14,33,37],datatyp:13,decayedadagrad:10,decayedadagradoptim:23,decod:57,decor:64,deep:31,definit:76,demo:48,dens:40,depend:48,deploi:41,describ:58,descript:[45,68],design:[29,30,31,33,37,38,39,40,42,43,46,47,48,49,52,53,56,57,58,60,61,63,64,65,66,67,68,69,72,73,75],destroi:72,detail:36,detect:[7,8],detection_map:7,detection_output:8,develop:68,dictionari:64,differ:68,discrimin:48,discuss:[48,66],dispatch:[33,37],distribut:[29,33,36,41,65],doc:[29,31,33,37,38,39,40,43,46,47,49,52,53,60,61,63,64,65,66,67,68,69,73],docker:[41,94],doe:[43,64],dot_prod:8,dot_product_attent:9,dotmul_oper:8,dotmul_project:8,driver:77,drop_out:81,dropout:[8,21],duplic:81,dure:[57,64],dylib:55,dynam:[33,75],dynamic_lstm:21,dynet:50,eigen:98,elect:40,elementwise_add:21,elementwise_div:21,els:31,embed:[8,21],engin:48,enough:30,entri:64,environ:41,eos:8,eval:65,evalu:[7,18,42],event:[28,29],examin:101,exampl:[29,32,51,55,67],except:80,execut:[31,61,68],executor:[19,43],exp:5,expand:[8,89],expandlevel:8,explain:30,factor:8,factorization_machin:8,faq:79,fault:33,file:[31,101],fill_const:21,fill_constant_batch_size_lik:21,first_seq:[8,89],float16:46,fluid:16,fork:95,format:[31,33,53],forward:49,frame:31,framework:[30,98],from:[29,40,74],full_matrix_project:8,fulli:8,gan:48,gate:93,gated_unit:8,gener:[48,57,101,119],get_output:8,give:64,global:[61,63],gpu:106,gradient:[30,38,69],gradient_print:7,graident:30,graph:[49,50,66,68,70],group:8,gru:[9,106],gru_group:9,gru_step:8,gru_unit:9,grumemori:8,handler:[29,54],happen:40,hardwar:46,helper:63,hierarchi:31,high:[58,59,70,74],how:[30,36,59,64,68],hsigmoid:8,huber_classification_cost:8,huber_regression_cost:8,ident:5,identifi:101,identity_project:8,ifels:51,ifelseop:31,illeg:77,imag:[8,9,15,41],imagenet:122,imdb:14,img_cmrnorm:8,img_conv:8,img_conv_bn_pool:9,img_conv_group:[9,22],img_pool:8,imikolov:14,implement:[30,32,36,42,43,46,53,56,59,63,64,68,69,70],increment:21,infer:[28,80],infershap:[61,73],infervartyp:52,ingredi:29,ingress:44,init_hook:2,initi:[20,38,48],input_typ:2,insid:72,instal:119,instead:64,instruct:77,insuffici:77,interact:74,interfac:[12,13,15,30,33,38,39,58,64,67,72],intermedi:68,interpol:8,introduc:[57,75],introduct:70,isn:64,issu:[46,95],job:[33,41,110],join:8,kernel:68,kmax_sequence_scor:8,kubernet:[41,109,110],l1decayregular:26,l2_distanc:8,l2decayregular:26,lambda_cost:8,languag:31,larg:36,last_seq:[8,89],layer:[8,21,29,47,63,81],learn:[31,82],learnabl:8,less_than:21,leval:74,level:[58,59,70,74],libpaddle_capi_shar:55,libpaddle_capi_whol:55,librari:[38,46,68],limit:65,linear:5,linear_chain_crf:21,linear_comb:8,list:[34,64],local:[65,67,72],lod:57,lod_rank_t:21,lod_tensor_to_arrai:21,lodtensor:[56,57,75],lodtensordesc:76,log:5,logic:37,look:101,low:[59,70,74],lstm:[9,21,106],lstm_step:8,lstmemori:8,lstmemory_group:9,lstmemory_unit:9,machin:[8,57],macro:68,main:48,manag:32,map:[64,68],master:[33,37,41],math:8,mathemat:30,max:11,max_sequence_len:21,maxframe_print:7,maxid:8,maxid_print:7,maxout:8,mean:21,member:48,memori:[8,56,81,90,92],merge_lod_tensor:21,messag:[74,82],method:57,might:48,migrat:68,mileston:68,mini:64,minibatch:13,misc:8,mix:8,mnist:14,mobil:118,model:[3,27,29,36,38,40,48,53,57,93,122],modul:[68,77],momentum:10,momentumoptim:23,more:48,motiv:[43,53,62],movielen:14,msrainiti:20,mul:21,multi_binary_label_cross_entropy_cost:8,multibox_loss:8,multipl:64,multiplex:8,mxnet:50,name:[72,77,81],nce:8,necess:63,necessari:68,need:64,nest:56,net:22,network:[9,68,93],neural:93,nlp:[9,106],norm:[8,70],normaliniti:20,note:30,numer:30,numpi:30,nvprof:103,nvvp:103,object:33,offset:57,ones:21,onli:[64,72],op_mak:68,oper:[47,51,59,61,63,66,68,69,73,75],opinfomap:68,opkernel:68,opproto:74,ops:70,optim:[10,23,33,38,49,58,63],option:45,opwithkernel:68,order:45,org:99,origin:68,orthogon:72,out_prod:8,output:8,overview:[40,43,68,72],pack:57,packag:32,pad:8,paddl:[36,64,72,77,81,98],paddlejob:41,paddlepaddl:[29,31,59,61,65,70,71,77,99,114],parallel_nn:108,paramattr:24,paramet:[6,8,28,29,33,38,39,41,59,60,63,66,70],parameteraverageoptim:59,parent:72,part:49,partit:38,path:[40,45],penalti:70,perform:[59,101,106],persist:37,pfsclient:[44,45],pfsserver:44,placement:65,platform:77,pnpair:7,point:80,pool2d:21,pool:[8,11,89],pose:[52,69],power:8,pre:95,precision_recal:7,prefetch:64,prelu:8,print:7,pro:119,problem:[42,52,58,69],procedur:119,process:[33,38,41,58,68],profil:[25,101],program:[31,61,63],programdesc:61,project:32,propos:[52,69,70],protobuf:73,protocol:82,provid:[2,64],prune:62,pserver:40,pull:95,push:95,python:[30,41,56,58,59,63,64,65,70,74,76,101],qualiti:68,queue:[33,37],rank:7,rank_cost:8,raspberri:117,rate:82,reader:[12,13,29,64],realiz:68,recoveri:33,recurr:[8,9,81,92,93],recurrent_group:8,ref:30,refactor:68,refer:[2,65,66],regist:[52,68,74],registr:[68,69],registri:68,regular:[26,38,70],reject:82,rel:57,relat:[68,75],relu:5,remot:[39,67],repeat:8,represent:[31,68],request:95,requir:[32,48],reshap:[8,21],resiz:8,resnet:122,retri:37,reus:63,rmsprop:10,rnn:[56,75,90,106],rnnop:[31,56,68],roi_pool:8,rotat:8,row:73,row_conv:8,row_l2_norm:8,runtim:[41,65],sampl:8,sampling_id:8,save:40,scale:[8,21,33],scale_shift:8,scaling_project:8,scope:[31,56,68,72],search:57,select:[38,73],selectedrow:73,selective_fc:8,sentiment:14,separ:68,seq_concat:8,seq_reshap:8,seq_slic:8,seqtext_print:7,sequenc:[57,93],sequence_conv:21,sequence_conv_pool:[9,22],sequence_pool:21,sequencesoftmax:5,server:[33,37,38,41,66],session:[65,67],sextant:119,sgd:106,sgdoptim:23,shape:57,share:[29,72],should:72,shrink_memori:21,shuffl:64,sigmoid:[5,21],sigmoid_cross_entropy_with_logit:21,simpl:57,simple_attent:9,simple_gru2:9,simple_gru:9,simple_img_conv_pool:[9,22],simple_lstm:9,singl:64,slice:8,slice_project:8,slope_intercept:8,small_vgg:9,smooth_l1_cost:8,softmax:5,softrelu:5,softsign:5,solut:[52,62,69],spars:[38,39,40,73],split_lod_tensor:21,spp:8,squar:5,square_error_cost:[8,21],squarerootn:11,stack:31,stanh:5,start:29,statement:42,step:56,storag:70,store:33,sub_nested_seq:8,subcommond:45,submit:41,suffici:64,suitabl:32,sum:[7,11,21],sum_cost:8,sum_to_one_norm:8,summar:29,summari:53,support:[46,75,77],survei:[46,50,70,119],synopsi:45,tabl:55,table_project:8,tanh:5,task:[33,37],tecton:119,tensor:[8,68,98],tensorarrai:[57,75],tensordesc:76,tensorflow:50,text_conv_pool:9,theori:30,thi:[72,77],think:48,three:75,timelin:40,todo:[34,35],togeth:72,toler:33,too:82,tool:[32,119],topk:21,train:[28,29,33,36,39,41,58,64,65],trainer:[28,33,38,40,41],tran:8,trans_full_matrix_project:8,translat:57,transpos:21,tune:106,two:30,uci_h:14,uniform:75,uniforminiti:20,unpack:57,updat:[29,39,40],usag:[56,57,64],use:[36,64],user:33,util:7,value_print:7,vardesc:76,variabl:[63,68,72,76],version:[46,77],vgg_16_network:9,warp_ctc:8,weightdecayregular:26,what:[36,40,43],wheel:77,when:[40,72],whl:77,why:[46,59,64,68,75],wmt14:14,xavieriniti:20,zero:21,zoo:122}}) \ No newline at end of file diff --git a/doc_cn/survey/cluster_bootstrapping_tools.html b/doc_cn/survey/cluster_bootstrapping_tools.html new file mode 100644 index 0000000000000000000000000000000000000000..fd07030ba8aa6c5262e24a7086629bccba15b278 --- /dev/null +++ b/doc_cn/survey/cluster_bootstrapping_tools.html @@ -0,0 +1,370 @@ + + + + + + + + + + + Cluster bootstrapping tool survey — PaddlePaddle 文档 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + +
    +
    + Fork me on Github +
    + + English + + + + +
    + +
    +
    + + + + +
    +
    + + + +
    +
    +
    +
    +
    + +
    + + + + +
    + + + + + + + + + + + +
    +
      + +
    • Cluster bootstrapping tool survey
      • +
    +
    + +
    +
    +
    +
    + +
    +

    Cluster bootstrapping tool survey

    +
    +

    Abstract

    +

    In order to bring up a cluster from bare metal machine to a fully functional kubernetes cluster for Paddlepaddle to run, we need to utilize some tools. Here we are going to compare Sextant and Tectonic installer

    +
    +
    +

    Basic assumptions

    +

    Here are some basic assumptions before we move on to details

    +
      +
    1. You are an administrator of a bare metal machine cluster, which means:
      2. +
    +
      +
    • you have full control to each of the machines.
      • +
    • you have full control to the network which machines are connected to.
      • +
    +
      +
    1. Machines can be booted from network with PEX or iPXE
      2. +
    2. You understand the general procedure to bring up a cluster
      3. +
    +

    if your cluster is able to mark above items with checkmarks, then keep reading.

    +
    +
    +

    Comparing Sextant and Tectonic installer

    +
    +

    Sextant

    +

    Sextant is an end2end solution to bring up a bare metal cluster to a fully functional k8s cluster, it integrates DHCP, name service, PEX, cloud-config-service, docker registry services altogether.

    +
    +

    Pros

    +
      +
    1. End2End: basically all admin need to do is to config the cluster.yaml and power on the cluster.
      2. +
    2. Offline cluster configuration: Sextant has 2 phases during working with it, config time and deploy time. when admin is configuring, it requires admin’s machine has internet connectivity, which will download some images, etc. But in deploy time, it’s completely OK to go offline since all dependencies are ready during config time.
      3. +
    3. docker registry integrated.
      4. +
    4. GPU machine took care of.
      5. +
    +
    +
    +
    +

    Cons

    +
      +
    1. k8s API server is not deployed with high availability in considering by default.
      2. +
    2. No grouping support.
      3. +
    3. No API interface, a one-off service.
      4. +
    +
    +
    +

    Tectonic installer

    +

    First of all, Tectonic is not free, it requires coreos.com account as a step of installation, and free user can only create less than 10 nodes.

    +

    Tectonic is a suite of software which wraps around k8s and providing more utility regarding dev ops, ie, +Tectonic installer as it’s named, it installs Tectonic to a bare metal cluster which means it’s not totally an equivalent of Sextant. At the “booting a cluster” part, it mostly utilizes Matchbox, which is a general cluster bootstrapper.

    +

    Matchbox’s Approach is similar to Sexstant.

    +
    +
    +

    Pros

    +
      +
    1. supports grouping machines.
      2. +
    2. supports running provisioning service in rtk. (not a big deal though).
      3. +
    3. supports http/gRPC API interface.
      4. +
    4. supports multi-template.
      5. +
    +
    +
    +

    Cons

    +
      +
    1. Not an e2e solution to bring up a cluster, need a lot of extra work and other software.
      2. +
    2. Not fully supporting centOS deployment yet.
      3. +
    +
    +
    +
    +

    Conclusion

    +

    Sextant is a better solution overall for paddle cloud deploying to a bare metal cluster. It would be great if Sextant can also 1) deploy k8s api server with high availability by default; 2) not designed as a one-off service.

    +
    +
    +

    Appendix: General procedure to bring up a cluster

    +

    It’s physically impossible for a cluster admin to manually install OS and applications into cluster nodes one by one, here is what an admin would do in cloud industry:

    +
      +
    1. setup a bootstrap machine with static IP in the cluster, which has following services:
      2. +
    +
      +
    • DHCP: assigns ip address for rest of the nodes.
      • +
    • name service: to map node name to a IP
      • +
    • PXE related services: the booting related info will be delivered to newly booted machines as their IP is assigned via DHCP service, PXE service will provide further booting and installing info and image with TFTP and http protocol.
      • +
    • cluster config service: this is for providing cluster node with OS config via http
      • +
    • optional docker registry: a built-in docker registry makes the whole cluster independent from connecting internet, and speeds up software distribution.
      • +
    +
      +
    1. New node powers on, it will
      2. +
    +
      +
    • broadcast the request for an IP address
      • +
    • DHCP server assigns the IP address, and deliver the PXE booting related info to the node.
      • +
    • cluster node will request config files with booting info delivered with DHCP via the TFTP service, and in most of the cases, the config file will point to a http service for the booting image.
      • +
    • Since PXE is configured with initrd, it will utilize the cloud config service and do further installations like coreOS or K8s installations.
      • +
    • then restart the node.
      • +
    +

    For further understanding, following 2 links from Matchbox are some good readings:

    +
    + +
    +
    +
    + + +
    +
    +
    + + +
    + +
    +

    + © Copyright 2016, PaddlePaddle developers. + +

    +
    + Built with Sphinx using a theme provided by Read the Docs. + +
    + +
    +
    + +
    + +
    + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/doc_cn/tutorials/image_classification/index_cn.html b/doc_cn/tutorials/image_classification/index_cn.html deleted file mode 100644 index a6d225e259d9362f2e3e22de673e5a8bb861be46..0000000000000000000000000000000000000000 --- a/doc_cn/tutorials/image_classification/index_cn.html +++ /dev/null @@ -1,432 +0,0 @@ - - - - - - - - - - - 图像分类教程 — PaddlePaddle 文档 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    -
      - -
    • 图像分类教程
      • -
    -
    - -
    -
    -
    -
    - -
    -

    图像分类教程

    -

    在本教程中，我们将使用CIFAR-10数据集训练一个卷积神经网络，并使用这个神经网络来对图片进行分类。如下图所示，卷积神经网络可以辨识图片中的主体，并给出分类结果。 -

    Image Classification

    -
    -

    数据准备

    -

    首先下载CIFAR-10数据集。下面是CIFAR-10数据集的官方网址：

    -

    https://www.cs.toronto.edu/~kriz/cifar.html

    -

    我们准备了一个脚本，可以用于从官方网站上下载CIFAR-10数据集，转为jpeg文件并存入特定的目录。使用这个脚本前请确认已经安装了pillow及相关依赖模块。可以参照下面的命令进行安装：

    -
      -
    1. 安装pillow
      2. -
    -
    sudo apt-get install libjpeg-dev
-pip install pillow
-
    -
    -
      -
    1. 下载数据集
      2. -
    -
    cd demo/image_classification/data/
-sh download_cifar.sh
-
    -
    -

    CIFAR-10数据集包含60000张32x32的彩色图片。图片分为10类，每个类包含6000张。其中50000张图片作为训练集，10000张作为测试集。

    -

    下图展示了所有的图片类别，每个类别中随机抽取了10张图片。 -

    Image Classification

    -

    脚本运行完成后，我们应当会得到一个名为cifar-out的文件夹，其下子文件夹的结构如下

    -
    train
----airplane
----automobile
----bird
----cat
----deer
----dog
----frog
----horse
----ship
----truck
-test
----airplane
----automobile
----bird
----cat
----deer
----dog
----frog
----horse
----ship
----truck
-
    -
    -

    cifar-out下包含train和test两个文件夹，其中分别包含了CIFAR-10中的训练集和测试集。这两个文件夹下各自有10个子文件夹，每个子文件夹下存储相应分类的图片。将图片按照上述结构存储好之后，我们就可以着手对分类模型进行训练了。

    -
    -
    -

    预处理

    -

    数据下载之后，还需要进行预处理，将数据转换为Paddle的格式。我们可以通过如下命令进行预处理工作：

    -
    cd demo/image_classification/
-sh preprocess.sh
-
    -
    -

    其中preprocess.sh 调用 ./demo/image_classification/preprocess.py 对图片进行预处理

    -
    export PYTHONPATH=$PYTHONPATH:../../
-data_dir=./data/cifar-out
-python preprocess.py -i $data_dir -s 32 -c 1
-
    -
    -

    ./demo/image_classification/preprocess.py 使用如下参数：

    -
      -
    • -i 或 --input 给出输入数据所在路径；
      • -
    • -s 或 --size 给出图片尺寸；
      • -
    • -c 或 --color 标示图片是彩色图或灰度图
      • -
    -
    -
    -

    模型训练

    -

    在开始训练之前，我们需要先创建一个模型配置文件。下面我们给出了一个配置示例。注意，这里的列出的和vgg_16_cifar.py文件稍有差别，因为该文件可适用于预测。

    -
    from paddle.trainer_config_helpers import *
-data_dir='data/cifar-out/batches/'
-meta_path=data_dir+'batches.meta'
-args = {'meta':meta_path, 'mean_img_size': 32,
-        'img_size': 32, 'num_classes': 10,
-        'use_jpeg': 1, 'color': "color"}
-define_py_data_sources2(train_list=data_dir+"train.list",
-                        test_list=data_dir+'test.list',
-                        module='image_provider',
-                        obj='processData',
-                        args=args)
-settings(
-    batch_size = 128,
-    learning_rate = 0.1 / 128.0,
-    learning_method = MomentumOptimizer(0.9),
-    regularization = L2Regularization(0.0005 * 128))
-
-img = data_layer(name='image', size=3*32*32)
-lbl = data_layer(name="label", size=10)
-# small_vgg is predined in trainer_config_helpers.network
-predict = small_vgg(input_image=img, num_channels=3)
-outputs(classification_cost(input=predict, label=lbl))
-
    -
    -

    在第一行中我们载入用于定义网络的函数。

    -
    from paddle.trainer_config_helpers import *
-
    -
    -

    之后定义的define_py_data_sources2使用Python数据提供器，其中 args将在image_provider.py进行使用，该文件负责产生图片数据并传递给Paddle系统

    -
      -
    • meta: 训练集平均值。
      • -
    • mean_img_size: 平均特征图的高度及宽度。
      • -
    • img_size：输入图片的高度及宽度。
      • -
    • num_classes：类别个数。
      • -
    • use_jpeg：处理过程中数据存储格式。
      • -
    • color：标示是否为彩色图片。
      • -
    -

    settings用于设置训练算法。在下面的例子中，learning rate被设置为0.1除以batch size，而weight decay则为0.0005乘以batch size。

    -
    settings(
-   batch_size = 128,
-   learning_rate = 0.1 / 128.0,
-   learning_method = MomentumOptimizer(0.9),
-   regularization = L2Regularization(0.0005 * 128)
-)
-
    -
    -

    small_vgg定义了网络结构。这里我们使用的是一个小的VGG网络。关于VGG卷积神经网络的描述可以参考：http://www.robots.ox.ac.uk/~vgg/research/very_deep/。

    -
    # small_vgg is predined in trainer_config_helpers.network
-predict = small_vgg(input_image=img, num_channels=3)
-
    -
    -

    配置创建完毕后，可以运行脚本train.sh来训练模型。

    -
    config=vgg_16_cifar.py
-output=./cifar_vgg_model
-log=train.log
-
-paddle train \
---config=$config \
---dot_period=10 \
---log_period=100 \
---test_all_data_in_one_period=1 \
---use_gpu=1 \
---save_dir=$output \
-2>&1 | tee $log
-
-python -m paddle.utils.plotcurve -i $log > plot.png
-
    -
    -
      -
    • 这里我们使用的是GPU模式进行训练。如果你没有GPU环境，可以设置use_gpu=0。
      • -
    • ./demo/image_classification/vgg_16_cifar.py是网络和数据配置文件。各项参数的详细说明可以在命令行参数相关文档中找到。
      • -
    • 脚本plotcurve.py依赖于python的matplotlib模块。因此如果这个脚本运行失败，也许是因为需要安装matplotlib。 -在训练完成后，训练及测试误差曲线图会被plotcurve.py脚本保存在 plot.png中。下面是一个误差曲线图的示例：
      • -
    -

    Training and testing curves.

    -
    -
    -

    预测

    -

    在训练完成后，模型及参数会被保存在路径./cifar_vgg_model/pass-%05d下。例如第300个pass的模型会被保存在./cifar_vgg_model/pass-00299。

    -

    要对一个图片的进行分类预测，我们可以使用predict.sh，该脚本将输出预测分类的标签：

    -
    sh predict.sh
-
    -
    -

    predict.sh:

    -
    model=cifar_vgg_model/pass-00299/
-image=data/cifar-out/test/airplane/seaplane_s_000978.png
-use_gpu=1
-python prediction.py $model $image $use_gpu
-
    -
    -
    -
    -

    练习

    -

    在CUB-200数据集上使用VGG模型训练一个鸟类图片分类模型。相关的鸟类数据集可以从如下地址下载，其中包含了200种鸟类的照片（主要来自北美洲）。

    -

    http://www.vision.caltech.edu/visipedia/CUB-200.html

    -
    -
    -

    细节探究

    -
    -

    卷积神经网络

    -

    卷积神经网络是一种使用卷积层的前向神经网络，很适合构建用于理解图片内容的模型。一个典型的神经网络如下图所示：

    -

    Convolutional Neural Network

    -

    一个卷积神经网络包含如下层：

    -
      -
    • 卷积层：通过卷积操作从图片或特征图中提取特征
      • -
    • 池化层：使用max-pooling对特征图下采样
      • -
    • 全连接层：使输入层到隐藏层的神经元是全部连接的。
      • -
    -

    卷积神经网络在图片分类上有着惊人的性能，这是因为它发掘出了图片的两类重要信息：局部关联性质和空间不变性质。通过交替使用卷积和池化处理， 卷积神经网络能够很好的表示这两类信息。

    -

    关于如何定义网络中的层，以及如何在层之间进行连接，请参考Layer文档。

    -
    -
    -
    - - -
    -
    -
    - - -
    - -
    -

    - © Copyright 2016, PaddlePaddle developers. - -

    -
    - Built with Sphinx using a theme provided by Read the Docs. - -
    - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc_cn/tutorials/rec/ml_regression_cn.html b/doc_cn/tutorials/rec/ml_regression_cn.html deleted file mode 100644 index ac14d8c3abe50a08e3092bc943a9c71f26f018d1..0000000000000000000000000000000000000000 --- a/doc_cn/tutorials/rec/ml_regression_cn.html +++ /dev/null @@ -1,854 +0,0 @@ - - - - - - - - - - - MovieLens数据集评分回归模型 — PaddlePaddle 文档 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    -
      - -
    • MovieLens数据集评分回归模型
      • -
    -
    - -
    -
    -
    -
    - -
    -

    MovieLens数据集评分回归模型

    -

    这里我们在MovieLens数据集描述一种 余弦相似度回归 任务。 -该示例将展示paddle如何进行词向量嵌入，处理相似度回归，针对文本 -的单词级别的卷积神经网络，以及paddle如何处理多种类型的输入。 -需要注意的是，该模型网络只是用于进行demo展示paddle如何工作，而 -没有进行结构的微调。

    -

    我们非常欢迎您用PADDLEPADDLE构建更好的示例，如果您有好的建议来 -让这个示例变得更好，希望能让我们知晓。

    -
    -

    数据准备

    -
    -

    下载并解压数据集

    -

    这里我们使用 MovieLens数据集 。 -要下载和解压数据集，只需要简单的运行下面的命令即可。

    -
    cd demo/recommendation/data
-./ml_data.sh
-
    -
    -

    demo/recommendation/data/ml-1m 的目录结构为:

    -
    +--ml-1m
-        +--- movies.dat         # 电影特征
-        +--- ratings.dat        # 评分
-        +--- users.dat          # 用户特征
-        +--- README             # 数据集描述
-
    -
    -
    -
    -

    字段配置文件

    -

    字段配置文件 用来具体说明数据集的字段和文件格式， -例如，说明每个特征文件具体字段是 什么 类型。

    -

    ml-1m的字段配置文件在目录 demo/recommendation/data/config.json 中。 -其具体说明了字段类型和文件名称:

    -
      -
    1. 用户文件中有四种类型的字段: 编号，性别，年龄和职业；
      2. -
    2. 文件名称为”users.dat”，文件的分隔符为”::”。
      3. -
    -
    {
-  "user": {
-    "file": {
-      "name": "users.dat",
-      "delimiter": "::"
-    },
-    "fields": ["id", "gender", "age", "occupation"]
-  },
-  "movie": {
-    "file": {
-      "name": "movies.dat",
-      "delimiter": "::"
-    },
-    "fields": ["id", "title", "genres"]
-  }
-}
-
    -
    -
    -
    -
    -

    准备数据

    -

    你需要安装python的第三方库。 -强烈推荐使用VIRTUALENV来创造一个干净的python环境。

    -
    pip install -r requirements.txt
-
    -
    -

    预处理数据一般的命令为:

    -
    cd demo/recommendation
-./preprocess.sh
-
    -
    -

    下面介绍预处理过程具体的步骤。

    -
    -

    提取电影或用户的特征并生成python对象

    -

    在movielens 1m数据集中，电影和用户有许多的特征。 -评分文件的每一行仅仅提供电影或用户的编号来代表相应的电影或用户。 -我们首先处理电影或用户的特征文件，然后用pickle命令将特征( Meta )对象存储为文件。

    -
    -

    Meta配置文件

    -

    Meta配置文件 用来具体描述 如何 解析数据集中的每一个字段。 -该文件可以从字段配置文件生成，或是手动编辑生成。文件的格式可以 -为json或yaml格式。解析器能通过文件的扩展名自动识别文件的格式。

    -

    要将字段配置文件转化为meta配置文件，只需要运行：

    -
    cd demo/recommendation/data
-python config_generator.py config.json > meta_config.json
-
    -
    -

    生成的meta配置文件如下所示：

    -
    {
-  "meta": {
-    "movie": {
-      "fields": [
-        {
-          "type": "id", 
-          "pos": 0
-        }, 
-        {
-          "regex": {
-            "pattern": "^(.*)\\((\\d+)\\)$", 
-            "group_id": 1, 
-            "strip": true
-          }, 
-          "type": {
-            "seq_type": "sequence", 
-            "name": "embedding"
-          }, 
-          "dict": {
-            "type": "char_based"
-          }, 
-          "name": "title", 
-          "pos": 1
-        }, 
-        {
-          "type": "one_hot_dense", 
-          "dict": {
-            "delimiter": "|", 
-            "type": "split"
-          }, 
-          "name": "genres", 
-          "pos": 2
-        }
-      ], 
-      "file": {
-        "delimiter": "::", 
-        "type": "split", 
-        "name": "movies.dat"
-      }
-    }, 
-    "user": {
-      "fields": [
-        {
-          "type": "id", 
-          "pos": 0
-        }, 
-        {
-          "type": "embedding", 
-          "dict": {
-            "type": "char_based"
-          }, 
-          "name": "gender", 
-          "pos": 1
-        }, 
-        {
-          "type": "embedding", 
-          "dict": {
-            "sort": true, 
-            "type": "whole_content"
-          }, 
-          "name": "age", 
-          "pos": 2
-        }, 
-        {
-          "type": "embedding", 
-          "dict": {
-            "sort": "true", 
-            "type": "whole_content"
-          }, 
-          "name": "occupation", 
-          "pos": 3
-        }
-      ], 
-      "file": {
-        "delimiter": "::", 
-        "type": "split", 
-        "name": "users.dat"
-      }
-    }
-  }
-}
-
    -
    -

    在meta文件中有两种特征: 电影和用户。

    -
      -
    • -
      在电影文件movies.dat中
      -
        -
      • 我们仅用”::”来分隔每一行
        • -
      • pos 0 代表编号
        • -
      • -
        pos 1 特征：
        -
          -
        • name是电影名
          • -
        • 利用正则表达式来解析该特征
          • -
        • 基于字母的词嵌入特征
          • -
        • 是序列
          • -
        -
        -
        -
        • -
      • -
        pos 2 特征：
        -
          -
        • name是体裁
          • -
        • type是one hot稠密向量
          • -
        • dictionary由解析自动生成，每一个key由’|’分隔
          • -
        -
        -
        -
        • -
      -
      -
      -
      • -
    • -
      在用户文件users.dat中
      -
        -
      • 我们仅用”::”来分隔每一行
        • -
      • pos 0 代表编号
        • -
      • -
        pos 1 特征：
        -
          -
        • name是性别
          • -
        • 简单的基于字母的词嵌入
          • -
        -
        -
        -
        • -
      • -
        pos 2 特征：
        -
          -
        • name是年龄
          • -
        • 是整个的词嵌入
          • -
        • 嵌入编号会根据单词排序
          • -
        -
        -
        -
        • -
      • -
        pos 3 特征：
        -
          -
        • name是职业
          • -
        • 简单的整个词嵌入
          • -
        -
        -
        -
        • -
      -
      -
      -
      • -
    -
    -
    -
    -

    Meta文件

    -

    有了meta配置文件之后，我们可以生成 Meta文件 ，该文件是python的pickle对象， -存储着电影或用户信息。可以运行下面的命令来生成。

    -
    python meta_generator.py ml-1m meta.bin --config=meta_config.json
-
    -
    -

    meta文件 meta.bin 的结构如下：

    -
    +--+ movie
-|      +--+ __meta__
-|      |       +--+ raw_meta  # 每个特征的meta配置。列表
-|      |       |       +
-|      |       |       |     # 编号字段，我们用编号作为key
-|      |       |       +--+ {'count': 3883, 'max': 3952, 'is_key': True, 'type': 'id', 'min': 1}
-|      |       |       |
-|      |       |       |     # 电影名字段，嵌入特征字典
-|      |       |       +--+ {'dict': [ ... ], 'type': 'embedding', 'name': 'title', 'seq': 'sequence'}
-|      |       |       |
-|      |       |       |     # 体裁字段，体裁字典
-|      |       |       +--+ {'dict': [ ... ], 'type': 'one_hot_dense', 'name': 'genres'}
-|      |       |
-|      |       +--+ feature_map [1, 2] # a list for raw_meta index for feature field.
-|      |                               # it means there are 2 features for each key.
-|      |                               #    * 0 offset of feature is raw_meta[1], Title.
-|      |                               #    * 1 offset of feature is raw_meta[2], Genres.
-|      |
-|      +--+ 1 # 电影1的特征
-|      |    +
-|      |    +---+ [[...], [...]] # title ids, genres dense vector
-|      |
-|      +--+ 2
-|      |
-|      +--+ ...
-|
-+--- user
-       +--+ __meta__
-       |       +
-       |       +--+ raw_meta
-       |       |       +
-       |       |       +--+ id field as user
-       |       |       |
-       |       |       +--+ {'dict': ['F', 'M'], 'type': 'embedding', 'name': 'gender', 'seq': 'no_sequence'}
-       |       |       |
-       |       |       +--+ {'dict': ['1', '18', '25', '35', '45', '50', '56'], 'type': 'embedding', 'name': 'age', 'seq': 'no_sequence'}
-       |       |       |
-       |       |       +--+ {'dict': [...], 'type': 'embedding', 'name': 'occupation', 'seq': 'no_sequence'}
-       |       |
-       |       +--+ feature_map [1, 2, 3]
-       |
-       +--+ 1 # 用户1的特征
-       |
-       +--+ 2
-       +--+ ...
-
    -
    -
    -
    -

    分割训练/测试文件

    -

    我们将 ml-1m/ratings.dat 文件分割为训练和测试文件。分割文件的方法是：对于每位用户，我们将评分分成两部分。 -这样的话每位用户在测试文件中将与训练文件含有同样的信息。

    -

    用 separate.py 来分离训练和测试文件。

    -
    python split.py ml-1m/ratings.dat --delimiter="::" --test_ratio=0.1
-
    -
    -

    这样就会生成两个文件：ml-1m/ratings.dat.train 和 ml-1m/ratings.data.test 。 -将他们移动到目录 data ，然后进行随机打乱，再为paddle的训练过程提供文件列表。

    -
    shuf ml-1m/ratings.dat.train > ratings.dat.train
-cp ml-1m/ratings.dat.test .
-echo "./data/ratings.dat.train" > train.list
-echo "./data/ratings.dat.test" > test.list
-
    -
    -
    -
    -
    -

    神经网络结构配置

    -
    -

    训练器配置文件

    -

    网络结构如下图所示：

    -rec_regression_network -

    该示例的神经网络配置文件 trainer_config.py 如下所示：

    -
    from paddle.trainer_config_helpers import *
-
-try:
-    import cPickle as pickle
-except ImportError:
-    import pickle
-
-is_predict = get_config_arg('is_predict', bool, False)
-
-META_FILE = 'data/meta.bin'
-
-with open(META_FILE, 'rb') as f:
-    # load meta file
-    meta = pickle.load(f)
-
-settings(
-    batch_size=1600, learning_rate=1e-3, learning_method=RMSPropOptimizer())
-
-
-def construct_feature(name):
-    """
-    Construct movie/user features.
-
-    This method read from meta data. Then convert feature to neural network due
-    to feature type. The map relation as follow.
-
-    * id: embedding => fc
-    * embedding:
-        is_sequence:  embedding => context_projection => fc => pool
-        not sequence: embedding => fc
-    * one_hot_dense:  fc => fc
-
-    Then gather all features vector, and use a fc layer to combined them as
-    return.
-
-    :param name: 'movie' or 'user'
-    :type name: basestring
-    :return: combined feature output
-    :rtype: LayerOutput
-    """
-    __meta__ = meta[name]['__meta__']['raw_meta']
-    fusion = []
-    for each_meta in __meta__:
-        type_name = each_meta['type']
-        slot_name = each_meta.get('name', '%s_id' % name)
-        if type_name == 'id':
-            slot_dim = each_meta['max']
-            embedding = embedding_layer(
-                input=data_layer(
-                    slot_name, size=slot_dim), size=256)
-            fusion.append(fc_layer(input=embedding, size=256))
-        elif type_name == 'embedding':
-            is_seq = each_meta['seq'] == 'sequence'
-            slot_dim = len(each_meta['dict'])
-            din = data_layer(slot_name, slot_dim)
-            embedding = embedding_layer(input=din, size=256)
-            if is_seq:
-                fusion.append(
-                    text_conv_pool(
-                        input=embedding, context_len=5, hidden_size=256))
-            else:
-                fusion.append(fc_layer(input=embedding, size=256))
-        elif type_name == 'one_hot_dense':
-            slot_dim = len(each_meta['dict'])
-            hidden = fc_layer(input=data_layer(slot_name, slot_dim), size=256)
-            fusion.append(fc_layer(input=hidden, size=256))
-
-    return fc_layer(name="%s_fusion" % name, input=fusion, size=256)
-
-
-movie_feature = construct_feature("movie")
-user_feature = construct_feature("user")
-similarity = cos_sim(a=movie_feature, b=user_feature)
-if not is_predict:
-    outputs(mse_cost(input=similarity, label=data_layer('rating', size=1)))
-
-    define_py_data_sources2(
-        'data/train.list',
-        'data/test.list',
-        module='dataprovider',
-        obj='process',
-        args={'meta': meta})
-else:
-    outputs(similarity)
-
    -
    -

    在文件 trainer_config.py 中，我们仅仅是将每个特征种类映射到一个特征向量中，以下 -展示了如何将每个特征映射到一个向量。

    -
      -
    • id : 仅仅是简单的嵌入，然后添加一个全连接层。
      • -
    • -
      embedding :
      -
        -
      • 如果是序列，则先做嵌入，然后再做一次文本卷积网络操作， -然后得到平均采样的结果。
        • -
      • 如果不是序列，则先做嵌入，然后添加一个全连接层。
        • -
      -
      -
      -
      • -
    • -
      one_host_dense :
      -
        -
      • 仅仅是两个全连接层。
        • -
      -
      -
      -
      • -
    -

    然后我们利用多输入的:code:fc_layer 全连接层将电影的每个特征结合成一个电影特征， -并且对用户的特征做同样的操作，也得到一个用户特征。然后我们求这两个特征的余弦相似度。

    -

    在这些网络中，我们用以下的一些:ref:api_trainer_config 中的接口。

    - -
    -
    -

    数据提供脚本

    -
    from paddle.trainer.PyDataProvider2 import *
-import common_utils  # parse
-
-
-def __list_to_map__(lst):
-    ret_val = dict()
-    for each in lst:
-        k, v = each
-        ret_val[k] = v
-    return ret_val
-
-
-def hook(settings, meta, **kwargs):
-    """
-    Init hook is invoked before process data. It will set obj.slots and store
-    data meta.
-
-    :param obj: global object. It will passed to process routine.
-    :type obj: object
-    :param meta: the meta file object, which passed from trainer_config. Meta
-                 file record movie/user features.
-    :param kwargs: unused other arguments.
-    """
-    del kwargs  # unused kwargs
-
-    # Header define slots that used for paddle.
-    #    first part is movie features.
-    #    second part is user features.
-    #    final part is rating score.
-    # header is a list of [USE_SEQ_OR_NOT?, SlotType]
-    movie_headers = list(common_utils.meta_to_header(meta, 'movie'))
-    settings.movie_names = [h[0] for h in movie_headers]
-    headers = movie_headers
-    user_headers = list(common_utils.meta_to_header(meta, 'user'))
-    settings.user_names = [h[0] for h in user_headers]
-    headers.extend(user_headers)
-    headers.append(("rating", dense_vector(1)))  # Score
-
-    # slot types.
-    settings.input_types = __list_to_map__(headers)
-    settings.meta = meta
-
-
-@provider(init_hook=hook, cache=CacheType.CACHE_PASS_IN_MEM)
-def process(settings, filename):
-    with open(filename, 'r') as f:
-        for line in f:
-            # Get a rating from file.
-            user_id, movie_id, score = map(int, line.split('::')[:-1])
-
-            # Scale score to [-5, +5]
-            score = float(score) * 2 - 5.0
-
-            # Get movie/user features by movie_id, user_id
-            movie_meta = settings.meta['movie'][movie_id]
-            user_meta = settings.meta['user'][user_id]
-
-            outputs = [('movie_id', movie_id - 1)]
-
-            # Then add movie features
-            for i, each_meta in enumerate(movie_meta):
-                outputs.append((settings.movie_names[i + 1], each_meta))
-
-            # Then add user id.
-            outputs.append(('user_id', user_id - 1))
-
-            # Then add user features.
-            for i, each_meta in enumerate(user_meta):
-                outputs.append((settings.user_names[i + 1], each_meta))
-
-            # Finally, add score
-            outputs.append(('rating', [score]))
-            # Return data to paddle
-            yield __list_to_map__(outputs)
-
    -
    -

    数据提供脚本仅仅是读取meta.bin和评分文件，生成训练需要的样本。 -在脚本 dataprovider.py 中，我们需要设置：

    -
      -
    • obj.slots: 特征的类型和维度。
      • -
    • use_seq: dataprovider.py 中的数据是否为序列模式。
      • -
    • process: 返回数据的每一条样本给 paddle 。
      • -
    -

    数据提供脚本的细节文档可以参考 PyDataProvider2的使用 。

    -
    -
    -
    -

    训练

    -

    准备好数据，配置了网络，编写好数据提供脚本后，现在我们可以开始paddle训练了。

    -

    代码 run.sh 如下：

    -
    paddle train \
-    --config=trainer_config.py \
-    --save_dir=./output \
-    --use_gpu=false \
-    --trainer_count=4\
-    --test_all_data_in_one_period=true \
-    --log_period=100 \
-    --dot_period=1 \
-    --num_passes=50  2>&1 | tee 'log.txt'
-paddle usage -l log.txt -e $? -n "recommendation" >/dev/null 2>&1
-
    -
    -

    该脚本仅仅是开始一个paddle训练过程，将日志写入文件 log.txt ，然后 -打印在屏幕上。

    -

    脚本 run.sh 中的每一行命令，请参考页面 设置命令行参数 。 -这些参数的简短介绍如下：

    -
      -
    • config: 告诉paddle哪个文件是神经网络的配置文件。
      • -
    • save_dir: 告诉paddle将模型保存在: code:./output 中。
      • -
    • use_gpu: 是否使用GPU，默认为不使用。
      • -
    • trainer_count: 一台机器上面的线程数量。
      • -
    • test_all_data_in_one_period: 每一个测试周期测试一次所有数据。否则， -每个测试周期测试: code:batch_size 批次的数据。
      • -
    • log_period: 在训练了: code:log_period 批次后打印日志。
      • -
    • dot_period: 在每训练: code:dot_period 个批次后打印一个 . 。
      • -
    • num_passes: 训练至多: code:num_passes 轮。
      • -
    -

    如果训练过程启动成功的话，输出应该类似如下：

    -
    I0601 08:07:22.832059 10549 TrainerInternal.cpp:157]  Batch=100 samples=160000 AvgCost=4.13494 CurrentCost=4.13494 Eval:  CurrentEval:
-
-I0601 08:07:50.672627 10549 TrainerInternal.cpp:157]  Batch=200 samples=320000 AvgCost=3.80957 CurrentCost=3.48421 Eval:  CurrentEval:
-
-I0601 08:08:18.877369 10549 TrainerInternal.cpp:157]  Batch=300 samples=480000 AvgCost=3.68145 CurrentCost=3.42519 Eval:  CurrentEval:
-
-I0601 08:08:46.863963 10549 TrainerInternal.cpp:157]  Batch=400 samples=640000 AvgCost=3.6007 CurrentCost=3.35847 Eval:  CurrentEval:
-
-I0601 08:09:15.413025 10549 TrainerInternal.cpp:157]  Batch=500 samples=800000 AvgCost=3.54811 CurrentCost=3.33773 Eval:  CurrentEval:
-I0601 08:09:36.058670 10549 TrainerInternal.cpp:181]  Pass=0 Batch=565 samples=902826 AvgCost=3.52368 Eval:
-I0601 08:09:46.215489 10549 Tester.cpp:101]  Test samples=97383 cost=3.32155 Eval:
-I0601 08:09:46.215966 10549 GradientMachine.cpp:132] Saving parameters to ./output/model/pass-00000
-I0601 08:09:46.233397 10549 ParamUtil.cpp:99] save dir ./output/model/pass-00000
-I0601 08:09:46.233438 10549 Util.cpp:209] copy trainer_config.py to ./output/model/pass-00000
-I0601 08:09:46.233541 10549 ParamUtil.cpp:147] fileName trainer_config.py
-
    -
    -

    模型被保存在 output/ 目录中。你可以在任何时候用 Ctrl-C 来停止训练。

    -
    -
    -

    模型评估和预测

    -

    在训练了几个轮次以后，你可以对模型进行评估，得到最好轮次下的模型。运行下面命令即可：

    -
    ./evaluate.sh
-
    -
    -

    你将看到如下的信息：

    -
    Best pass is 00009,  error is 3.06949, which means predict get error as 0.875998002281
-evaluating from pass output/pass-00009
-
    -
    -

    然后，你可以预测任何用户对于任何一部电影的评价，运行下面命令即可：

    -
    python prediction.py 'output/pass-00009/'
-
    -
    -

    预测程序将读取用户的输入，然后输出预测分数。用户预测的命令行界面如下：

    -
    Input movie_id: 9
-Input user_id: 4
-Prediction Score is 2.56
-Input movie_id: 8
-Input user_id: 2
-Prediction Score is 3.13
-
    -
    -
    -
    - - -
    -
    -
    - - -
    - -
    -

    - © Copyright 2016, PaddlePaddle developers. - -

    -
    - Built with Sphinx using a theme provided by Read the Docs. - -
    - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc_cn/tutorials/semantic_role_labeling/index_cn.html b/doc_cn/tutorials/semantic_role_labeling/index_cn.html deleted file mode 100644 index 6ac8f0d2d6d8dadbca0c59503ee1c6b8952bbbaf..0000000000000000000000000000000000000000 --- a/doc_cn/tutorials/semantic_role_labeling/index_cn.html +++ /dev/null @@ -1,440 +0,0 @@ - - - - - - - - - - - 语义角色标注教程 — PaddlePaddle 文档 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    -
      - -
    • 语义角色标注教程
      • -
    -
    - -
    -
    -
    -
    - -
    -

    语义角色标注教程

    -

    语义角色标注（Semantic role labeling, SRL）是浅层语义解析的一种形式，其目的是在给定的输入句子中发现每个谓词的谓词论元结构。 SRL作为很多自然语言处理任务中的中间步骤是很有用的，如信息提取、文档自动分类和问答。 实例如下 [1]:

    -

    [ A0 He ] [ AM-MOD would ][ AM-NEG n’t ] [ V accept] [ A1 anything of value ] from [A2 those he was writing about ].

    -
      -
    • V: 动词
      • -
    • A0: 接受者
      • -
    • A1: 接受的东西
      • -
    • A2: 从……接受
      • -
    • A3: 属性
      • -
    • AM-MOD: 情态动词
      • -
    • AM-NEG: 否定
      • -
    -

    给定动词“accept”，句子中的组块将会扮演某些语义角色。这里，标签方案来自 Penn Proposition Bank。

    -

    到目前为止，大多数成功的SRL系统是建立在某种形式的句法分析结果之上的，使用了基于句法结构的预定义特征模板。 本教程将介绍使用深度双向长短期记忆（DB-LSTM）模型[2]的端到端系统来解决SRL任务，这在很大程度上优于先前的最先进的系统。 这个系统将SRL任务视为序列标注问题。

    -
    -

    数据描述

    -

    相关论文[2]采用 CoNLL-2005＆2012 共享任务中设置的数据进行训练和测试。由于数据许可的原因，演示采用 CoNLL-2005 的测试数据集，可以在网站上找到。

    -

    用户只需执行以下命令就可以下载并处理原始数据：

    -
    cd data
-./get_data.sh
-
    -
    -

    data目录会出现如下几个新的文件：

    -
    conll05st-release：the test data set of CoNll-2005 shared task 
-test.wsj.words：the Wall Street Journal data sentences
-test.wsj.props:  the propositional arguments
-feature: the extracted features from data set
-
    -
    -
    -
    -

    训练

    -
    -

    DB-LSTM

    -

    请参阅情感分析的演示以了解有关长期短期记忆单元的更多信息。

    -

    与在 Sentiment Analysis 演示中使用的 Bidirectional-LSTM 不同，DB-LSTM 采用另一种方法来堆叠LSTM层。首先，标准LSTM以正向处理该序列。该 LSTM 层的输入和输出作为下一个 LSTM 层的输入，并被反向处理。这两个标准 LSTM 层组成一对 LSTM。然后我们堆叠一对对的 LSTM 层后得到深度 LSTM 模型。

    -

    下图展示了时间扩展的2层 DB-LSTM 网络。 -

    -pic -

    -
    -
    -

    特征

    -

    两个输入特征在这个流程中起着至关重要的作用：predicate（pred）和argument（arguments）。 还采用了两个其他特征：谓词上下文（ctx-p）和区域标记（mr）。 因为单个谓词不能精确地描述谓词信息，特别是当相同的词在句子中出现多于一次时。 使用谓词上下文，可以在很大程度上消除歧义。类似地，如果它位于谓词上下文区域中，则使用区域标记 mr = 1 来表示参数位置，反之则 mr = 0。这四个简单的特征是我们的SRL系统所需要的。上下文大小设置为1的一个样本的特征如下[2]所示： -

    -pic -

    -

    在这个示例中，相应的标记句子是：

    -

    [ A1 A record date ] has [ AM-NEG n’t ] been [ V set ] .

    -

    在演示中, 我们采用上面的特征模板, 包括： argument, predicate, ctx-p (p=-1,0,1), mark 并使用 B/I/O 方案来标记每个参数。这些特征和标签存储在 feature 文件中, 用\t分割。

    -
    -
    -

    数据提供

    -

    dataprovider.py 是一个包装数据的 Python 文件。 函数 hook() 定义了网络的数据槽。六个特征和标签都是索引槽。

    -
    def hook(settings, word_dict, label_dict, **kwargs):
-    settings.word_dict = word_dict
-    settings.label_dict = label_dict
-    #all inputs are integral and sequential type
-    settings.slots = [
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(predicate_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(len(word_dict)),
-        integer_value_sequence(2),
-        integer_value_sequence(len(label_dict))]
-
    -
    -

    相应的数据迭代器如下：

    -
    @provider(init_hook=hook, should_shuffle=True, calc_batch_size=get_batch_size,
-          can_over_batch_size=False, cache=CacheType.CACHE_PASS_IN_MEM)
-def process(settings, file_name):
-    with open(file_name, 'r') as fdata:
-        for line in fdata:
-            sentence, predicate, ctx_n2, ctx_n1, ctx_0, ctx_p1, ctx_p2,  mark, label = \
-                line.strip().split('\t')
-
-            words = sentence.split()
-            sen_len = len(words)
-            word_slot = [settings.word_dict.get(w, UNK_IDX) for w in words]
-
-            predicate_slot = [settings.predicate_dict.get(predicate)] * sen_len
-            ctx_n2_slot = [settings.word_dict.get(ctx_n2, UNK_IDX)] * sen_len
-            ctx_n1_slot = [settings.word_dict.get(ctx_n1, UNK_IDX)] * sen_len
-            ctx_0_slot = [settings.word_dict.get(ctx_0, UNK_IDX)] * sen_len
-            ctx_p1_slot = [settings.word_dict.get(ctx_p1, UNK_IDX)] * sen_len
-            ctx_p2_slot = [settings.word_dict.get(ctx_p2, UNK_IDX)] * sen_len
-
-            marks = mark.split()
-            mark_slot = [int(w) for w in marks]
-
-            label_list = label.split()
-            label_slot = [settings.label_dict.get(w) for w in label_list]
-            yield word_slot, predicate_slot, ctx_n2_slot, ctx_n1_slot, \
-                  ctx_0_slot, ctx_p1_slot, ctx_p2_slot, mark_slot, label_slot
-
    -
    -

    函数 process 返回8个特征list和1个标签list。

    -
    -
    -

    神经网络配置

    -

    db_lstm.py 是在训练过程中加载字典并定义数据提供程序模块和网络架构的神经网络配置文件。

    -

    九个 data_layer 从数据提供程序加载实例。八个特征分别转换为向量，并由mixed_layer混合。 深度双向LSTM层提取softmax层的特征。目标函数是标签的交叉熵。

    -
    -
    -

    训练

    -

    训练的脚本是 train.sh，用户只需执行:

    -
      ./train.sh
-
    -
    -

    train.sh 中的内容：

    -
    paddle train \
-  --config=./db_lstm.py \
-  --use_gpu=0 \
-  --log_period=5000 \
-  --trainer_count=1 \
-  --show_parameter_stats_period=5000 \
-  --save_dir=./output \
-  --num_passes=10000 \
-  --average_test_period=10000000 \
-  --init_model_path=./data \
-  --load_missing_parameter_strategy=rand \
-  --test_all_data_in_one_period=1 \
-2>&1 | tee 'train.log'
-
    -
    -
      -
    • --config=./db_lstm.py : 网络配置文件
      • -
    • --use_gpu=false: 使用 CPU 训练（如果已安装 PaddlePaddle GPU版本并想使用 GPU 训练可以设置为true，目前 crf_layer 不支持 GPU）
      • -
    • --log_period=500: 每20个batch输出日志
      • -
    • --trainer_count=1: 设置线程数（或 GPU 数）
      • -
    • --show_parameter_stats_period=5000: 每100个batch显示参数统计
      • -
    • --save_dir=./output: 模型输出路径
      • -
    • --num_passes=10000: 设置数据遍历次数，一个pass意味着PaddlePaddle训练数据集中的所有样本被遍历一次
      • -
    • --average_test_period=10000000: 每个 average_test_period 批次对平均参数进行测试
      • -
    • --init_model_path=./data: 参数初始化路径
      • -
    • --load_missing_parameter_strategy=rand: 随机初始不存在的参数
      • -
    • --test_all_data_in_one_period=1: 在一个周期内测试所有数据
      • -
    -

    训练后，模型将保存在目录output中。 我们的训练曲线如下： -

    -pic -

    -
    -
    -

    测试

    -

    测试脚本是 test.sh, 执行:

    -
      ./test.sh
-
    -
    -

    tesh.sh 的主要部分：

    -
    paddle train \
-  --config=./db_lstm.py \
-  --model_list=$model_list \
-  --job=test \
-  --config_args=is_test=1 \
-
    -
    -
      -
    • --config=./db_lstm.py: 网络配置文件
      • -
    • --model_list=$model_list.list: 模型列表文件
      • -
    • --job=test: 指示测试任务
      • -
    • --config_args=is_test=1: 指示测试任务的标记
      • -
    • --test_all_data_in_one_period=1: 在一个周期内测试所有数据
      • -
    -
    -
    -

    预测

    -

    预测脚本是 predict.sh，用户只需执行：

    -
      ./predict.sh
-  
-
    -
    -

    在predict.sh中，用户应该提供网络配置文件，模型路径，标签文件，字典文件，特征文件。

    -
    python predict.py 
-     -c $config_file \
-     -w $best_model_path \
-     -l $label_file \
-     -p $predicate_dict_file  \
-     -d $dict_file \
-     -i $input_file \
-     -o $output_file
-
    -
    -

    predict.py 是主要的可执行python脚本，其中包括函数：加载模型，加载数据，数据预测。网络模型将输出标签的概率分布。 在演示中，我们使用最大概率的标签作为结果。用户还可以根据概率分布矩阵实现柱搜索或维特比解码。

    -

    预测后，结果保存在 predict.res 中。

    -
    -
    -
    -

    引用

    -

    [1] Martha Palmer, Dan Gildea, and Paul Kingsbury. The Proposition Bank: An Annotated Corpus of Semantic Roles , Computational Linguistics, 31(1), 2005.

    -

    [2] Zhou, Jie, and Wei Xu. “End-to-end learning of semantic role labeling using recurrent neural networks.” Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.

    -
    -
    - - -
    -
    -
    - - -
    - -
    -

    - © Copyright 2016, PaddlePaddle developers. - -

    -
    - Built with Sphinx using a theme provided by Read the Docs. - -
    - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc_cn/tutorials/sentiment_analysis/index_cn.html b/doc_cn/tutorials/sentiment_analysis/index_cn.html deleted file mode 100644 index e07d23fa65e48a25386b6bb8f5cb92f0e3a0082c..0000000000000000000000000000000000000000 --- a/doc_cn/tutorials/sentiment_analysis/index_cn.html +++ /dev/null @@ -1,549 +0,0 @@ - - - - - - - - - - - 情感分析教程 — PaddlePaddle 文档 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    -
      - -
    • 情感分析教程
      • -
    -
    - -
    -
    -
    -
    - -
    -

    情感分析教程

    -

    情感分析有许多应用场景。 一个基本的应用场景是区分给定文本的褒贬两极性，给定的文本可以是一个文档、句子、或者是一个小的文本片段。 一个简单的例子如：把用户在购物网站、旅游网站、团购网站（亚马逊、天猫、淘宝等）上发表的评论分成正面评论和负面评论两类。

    -

    情感分析也常用于基于大量评论和个人博客来监控社会媒体。 例如，研究人员分析了几个关于消费者信心和政治观点的调查，结果发现它们与同时期的Twitter消息中的情绪词频率相关 [1]。 另一个例子是通过分析每日Twitter博客的文本内容来预测股票变动 [2]。

    -

    另一方面，抓取产品的用户评论并分析他们的情感，有助于理解用户对不同公司，不同产品，甚至不同竞争对手产品的偏好。

    -

    本教程将指导您完成长期短期记忆（LSTM）网络的训练过程，以分类来自大型电影评论数据集（有时称为互联网电影数据库 (IMDB)）的句子的情感 。 此数据集包含电影评论及其相关联的类别标签，即正面和负面。

    -
    -

    数椐准备

    -
    -

    IMDB 数椐介绍

    -

    训练模型之前, 我们需要预处理数椐并构建一个字典。 首先, 你可以使用下面的脚本下载 IMDB 数椐集和Moses工具, 这是一个基于统计的机器翻译系统. 我们提供了一个数据预处理脚本，它不仅能够处理IMDB数据，还能处理其他用户自定义的数据。 为了使用提前编写的脚本，需要将标记的训练和测试样本移动到另一个路径，这已经在get_imdb.sh中完成。

    -
    cd demo/sentiment/data
-./get_imdb.sh
-
    -
    -

    如果数椐获取成功，你将在目录./demo/sentiment/data中看到下面的文件：

    -
    aclImdb  get_imdb.sh  imdb  mosesdecoder-master
-
    -
    -
      -
    • aclImdb: 从外部网站上下载的原始数椐集。
      • -
    • imdb: 仅包含训练和测试数椐集。
      • -
    • mosesdecoder-master: Moses 工具。
      • -
    -

    IMDB数据集包含25,000个已标注过的高极性电影评论用于训练，25,000个用于测试。负面的评论的得分小于等于4,正面的评论的得大于等于7，总评分10分。 运行完脚本 ./get_imdb.sh后, 我们可以看到在目录 aclImdb中的数椐集的结构如下：

    -
    imdbEr.txt  imdb.vocab  README  test  train
-
    -
    -
      -
    • train: 训练数椐集。
      • -
    • test : 测试数椐集。
      • -
    • imdb.vocab: 字典文件。
      • -
    • imdbEr.txt: 字典imdb.vocab中每个切分单词的预期评级。
      • -
    • README: 数椐说明文档。
      • -
    -

    测试集和训练集目录包含下面的文件:

    -
    labeledBow.feat  neg  pos  unsup  unsupBow.feat  urls_neg.txt  urls_pos.txt  urls_unsup.txt
-
    -
    -
      -
    • pos: 正面评价样本，包含12,500个txt文件，每个文件是一个电影评论。
      • -
    • neg: 负面评价样本，包含12,500个txt文件，每个文件是一个电影评论。
      • -
    • unsup: 未标记的评价样本，包含50,000个txt文件。
      • -
    • urls_xx.txt: 每个评论的网址。
      • -
    • xxBow.feat: 用于统计词频的Bow模型特征。
      • -
    -
    -
    -

    IMDB 数椐准备

    -

    在这个例子中，我们只使用已经标注过的训练集和测试集，且默认在训练集上构建字典，而不使用IMDB数椐集中的imdb.vocab做为字典。训练集已经做了随机打乱排序而测试集没有。 Moses 工具中的脚本tokenizer.perl 用于切分单单词和标点符号。执行下面的命令就可以预处理数椐。

    -
    cd demo/sentiment/
-./preprocess.sh
-
    -
    -

    preprocess.sh:

    -
    data_dir="./data/imdb"
-python preprocess.py -i data_dir
-
    -
    -
      -
    • data_dir: 输入数椐所在目录。
      • -
    • preprocess.py: 预处理脚本。
      • -
    -

    运行成功后目录demo/sentiment/data/pre-imdb 结构如下:

    -
    dict.txt  labels.list  test.list  test_part_000  train.list  train_part_000
-
    -
    -
      -
    • test_part_000 and train_part_000: 所有标记的测试集和训练集， 训练集已经随机打乱。
      • -
    • train.list and test.list: 训练集和测试集文件列表。
      • -
    • dict.txt: 利用训练集生成的字典。
      • -
    • labels.txt: neg 0, pos 1, 含义：标签0表示负面的评论，标签1表示正面的评论。
      • -
    -
    -
    -

    用户自定义数椐预处理

    -

    如果你执行其它的用情感分析来分类文本的任务，可以按如下的结构来准备数椐. 我们提供了脚本来构建字典和预处理数椐。所以你只用按下面的结构来组织数椐就行了。

    -
    dataset
-|----train
-|    |----class1
-|    |    |----text_files
-|    |----class2
-|    |    |----text_files
-|    |    ...
-|----test
-|    |----class1
-|    |    |----text_files
-|    |----class2
-|    |    |----text_files
-|    |    ...
-
    -
    -
      -
    • dataset: 一级目录。
      • -
    • train, test: 二级目录。
      • -
    • class1,class2,...: 三级目录。
      • -
    • text_files: 文本格式的实例文件。
      • -
    -

    所有同目录下的文本实例文件都是同级别的。 每个文本文件包含一个或者多个实例，每一行表示一个实例。 为了充分的随机打乱训练集, 在预处理含有多行数椐的文本文件时参数设置稍有不同, 执行preprocess.sh脚本时需要加上-m True参数。 tokenizer.perl 默认用来切分单记和标点符号，如果你不需要这个操作，在运行preprocess.sh时加上-t False参数即可。

    -
    -
    -
    -

    训练模型

    -

    在这步任务中,我们使用了循环神经网络（RNN）的 LSTM 架构来训练情感分析模型。 引入LSTM模型主要是为了克服消失梯度的问题。 LSTM网络类似于具有隐藏层的标准循环神经网络, 但是隐藏层中的每个普通节点被一个记忆单元替换。 每个记忆单元包含四个主要的元素: 输入门, 具有自循环连接的神经元，忘记门和输出门。 更多的细节可以在文献中找到[4]。 LSTM架构的最大优点是它可以在长时间间隔内记忆信息，而没有短时记忆的损失。在有新的单词来临的每一个时间步骤内，存储在记忆单元区块的历史信息被更新用来迭代的学习单词以合理的序列程现。

    -

    LSTM
    -
    图表 1. LSTM [3]

    -

    情感分析是自然语言理解中最典型的问题之一。 它的目的是预测在一个序列中表达的情感态度。 通常, ，仅仅是一些关键词，如形容词和副词，在预测序列或段落的情感中起主要作用。然而有些评论上下文非常长，例如 IMDB的数椐集。 我们只所以使用LSTM来执行这个任务是因为其改进的设计并且具有门机制。 首先，它能够从词级到具有可变上下文长度的上下文级别来总结表示。 第二，它可以在句子级别利用可扩展的上下文, 而大多数方法只是利用n-gram级别的知识。第三，它直接学习段落表示，而不是组合上下文级别信息。

    -

    在本演示中，我们提供两个网络，即双向LSTM和三层堆叠LSTM。

    -
    -

    双向LSTM

    -

    图2是双向LSTM网络，后面连全连接层和softmax层。

    -

    BiLSTM
    -
    图 2. Bidirectional-LSTM

    -
    -
    -

    Stacked-LSTM

    -

    图3是三层LSTM结构。图的底部是word embedding(对文档处理后形成的单词向量)。 接下来，连接三个LSTM隐藏层，并且第二个是反向LSTM。然后提取隐藏LSTM层的所有时间步长的最大词向量作为整个序列的表示。 最后，使用具有softmax激活的全连接前馈层来执行分类任务。 更多内容可查看参考文献 [5]。

    -

    StackedLSTM
    -
    图 3. Stacked-LSTM for sentiment analysis

    -

    配置

    -

    进入demo/sentiment 目录 , trainer_config.py 是一个配置文件的例子, 其中包含算法和网络配置。第一行从sentiment_net.py中导出预定义的网络。

    -

    trainer_config.py:

    -
    from sentiment_net import *
-
-data_dir  = "./data/pre-imdb"
-# whether this config is used for test
-is_test = get_config_arg('is_test', bool, False)
-# whether this config is used for prediction
-is_predict = get_config_arg('is_predict', bool, False)
-dict_dim, class_dim = sentiment_data(data_dir, is_test, is_predict)
-
-################## Algorithm Config #####################
-
-settings(
-  batch_size=128,
-  learning_rate=2e-3,
-  learning_method=AdamOptimizer(),
-  regularization=L2Regularization(8e-4),
-  gradient_clipping_threshold=25
-)
-
-#################### Network Config ######################
-stacked_lstm_net(dict_dim, class_dim=class_dim,
-                 stacked_num=3, is_predict=is_predict)
-#bidirectional_lstm_net(dict_dim, class_dim=class_dim, is_predict=is_predict)
-
    -
    -
      -
    • 数椐定义:
        -
      • get_config_arg(): 获取通过 --config_args=xx 设置的命令行参数。
        • -
      • 定义训练数椐和测试数椐提供者, 这里使用了PaddlePaddle的Python接口来加载数椐。想了解更多细节可以参考PyDataProvider部分的文档
        • -
      -
      • -
    • 算法配置:
        -
      • 使用随机梯度下降（sgd）算法。
        • -
      • 使用 adam 优化。
        • -
      • 设置batch size大小为128。
        • -
      • 设置平均sgd窗口。
        • -
      • 设置全局学习率。
        • -
      -
      • -
    • 网络配置:
        -
      • dict_dim: 获取字典维度。
        • -
      • class_dim: 设置类别数，IMDB有两个标签，即正面评价标签和负面评价标签。
        • -
      • stacked_lstm_net: 预定义网络如图3所示，默认情况下使用此网络
        • -
      • bidirectional_lstm_net: 预定义网络，如图2所示。
        • -
      -
      • -
    -

    训练

    -

    首先安装PaddlePaddle。 然后使用下面的脚本 train.sh 来开启本地的训练。

    -
    cd demo/sentiment/
-./train.sh
-
    -
    -

    train.sh:

    -
    config=trainer_config.py
-output=./model_output
-paddle train --config=$config \
-             --save_dir=$output \
-             --job=train \
-             --use_gpu=false \
-             --trainer_count=4 \
-             --num_passes=10 \
-             --log_period=20 \
-             --dot_period=20 \
-             --show_parameter_stats_period=100 \
-             --test_all_data_in_one_period=1 \
-             2>&1 | tee 'train.log'
-
    -
    -
      -
    • --config=$config: 设置网络配置。
      • -
    • --save_dir=$output: 设置输出路径以保存训练完成的模型。
      • -
    • --job=train: 设置工作模式为训练。
      • -
    • --use_gpu=false: 使用CPU训练，如果你安装GPU版本的PaddlePaddle，并想使用GPU来训练设置为true。
      • -
    • --trainer_count=4:设置线程数（或GPU个数）。
      • -
    • --num_passes=15: 设置pass，PaddlePaddle中的一个pass意味着对数据集中的所有样本进行一次训练。
      • -
    • --log_period=20: 每20个batch打印一次日志。
      • -
    • --show_parameter_stats_period=100: 每100个batch打印一次统计信息。
      • -
    • --test_all_data_in_one_period=1: 每次测试都测试所有数据。
      • -
    -

    如果运行成功，输出日志保存在路径 demo/sentiment/train.log中，模型保存在目录demo/sentiment/model_output/中。 输出日志说明如下：

    -
    Batch=20 samples=2560 AvgCost=0.681644 CurrentCost=0.681644 Eval: classification_error_evaluator=0.36875  CurrentEval: classification_error_evaluator=0.36875
-...
-Pass=0 Batch=196 samples=25000 AvgCost=0.418964 Eval: classification_error_evaluator=0.1922
-Test samples=24999 cost=0.39297 Eval: classification_error_evaluator=0.149406
-
    -
    -
      -
    • Batch=xx: 表示训练了xx个Batch。
      • -
    • samples=xx: 表示训练了xx个样本。。
      • -
    • AvgCost=xx: 从第0个batch到当前batch的平均损失。
      • -
    • CurrentCost=xx: 最新log_period个batch处理的当前损失。
      • -
    • Eval: classification_error_evaluator=xx: 表示第0个batch到当前batch的分类错误。
      • -
    • CurrentEval: classification_error_evaluator: 最新log_period个batch的分类错误。
      • -
    • Pass=0: 通过所有训练集一次称为一遍。 0表示第一次经过训练集。
      • -
    -

    默认情况下，我们使用stacked_lstm_net网络，当传递相同的样本数时，它的收敛速度比bidirectional_lstm_net快。如果要使用双向LSTM，只需删除最后一行中的注释并把“stacked_lstm_net”注释掉。

    -
    -
    -
    -

    测试模型

    -

    测试模型是指使用训练出的模型评估已标记的验证集。

    -
    cd demo/sentiment
-./test.sh
-
    -
    -

    test.sh:

    -
    function get_best_pass() {
-  cat $1  | grep -Pzo 'Test .*\n.*pass-.*' | \
-  sed  -r 'N;s/Test.* error=([0-9]+\.[0-9]+).*\n.*pass-([0-9]+)/\1 \2/g' | \
-  sort | head -n 1
-}
-
-log=train.log
-LOG=`get_best_pass $log`
-LOG=(${LOG})
-evaluate_pass="model_output/pass-${LOG[1]}"
-
-echo 'evaluating from pass '$evaluate_pass
-
-model_list=./model.list
-touch $model_list | echo $evaluate_pass > $model_list
-net_conf=trainer_config.py
-paddle train --config=$net_conf \
-             --model_list=$model_list \
-             --job=test \
-             --use_gpu=false \
-             --trainer_count=4 \
-             --config_args=is_test=1 \
-             2>&1 | tee 'test.log'
-
    -
    -

    函数get_best_pass依据分类错误率获得最佳模型进行测试。 在本示例中，我们默认使用IMDB的测试数据集作为验证。 与训练不同，它需要在这里指定--job = test和模型路径，即--model_list = $model_list。如果运行成功，日志将保存在“demo / sentiment / test.log”的路径中。例如，在我们的测试中，最好的模型是model_output / pass-00002，分类误差是0.115645，如下：

    -
    Pass=0 samples=24999 AvgCost=0.280471 Eval: classification_error_evaluator=0.115645
-
    -
    -
    -
    -

    预测

    -

    predict.py脚本提供了一个预测接口。在使用它之前请安装PaddlePaddle的python api。 预测IMDB的未标记评论的一个实例如下：

    -
    cd demo/sentiment
-./predict.sh
-
    -
    -

    predict.sh:

    -
    #Note the default model is pass-00002, you shold make sure the model path
-#exists or change the mode path.
-model=model_output/pass-00002/
-config=trainer_config.py
-label=data/pre-imdb/labels.list
-cat ./data/aclImdb/test/pos/10007_10.txt | python predict.py \
-     --tconf=$config\
-     --model=$model \
-     --label=$label \
-     --dict=./data/pre-imdb/dict.txt \
-     --batch_size=1
-
    -
    -
      -
    • cat ./data/aclImdb/test/pos/10007_10.txt : 输入预测样本。
      • -
    • predict.py : 预测接口脚本。
      • -
    • --tconf=$config : 设置网络配置。
      • -
    • --model=$model : 设置模型路径。
      • -
    • --label=$label : 设置标签类别字典，这个字典是整数标签和字符串标签的一个对应。
      • -
    • --dict=data/pre-imdb/dict.txt : 设置字典文件。
      • -
    • --batch_size=1 : 设置batch size。
      • -
    -

    注意应该确保默认模型路径model_output / pass-00002存在或更改为其它模型路径。

    -

    本示例的预测结果：

    -
    Loading parameters from model_output/pass-00002/
-./data/aclImdb/test/pos/10014_7.txt: predicting label is pos
-
    -
    -

    我们真诚地感谢您的关注，并欢迎您来参与贡献。

    -
    -
    -

    参考文档

    -

    [1] Brendan O’Connor, Ramnath Balasubramanyan, Bryan R. Routledge, and Noah A. Smith. 2010. From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series. In ICWSM-2010.
    -[2] Johan Bollen, Huina Mao, Xiaojun Zeng. 2011. Twitter mood predicts the stock market, Journal of Computational Science.
    -[3] Alex Graves, Marcus Liwicki, Santiago Fernan- dez, Roman Bertolami, Horst Bunke, and Ju ̈rgen Schmidhuber. 2009. A novel connectionist system for unconstrained handwriting recognition. IEEE Transactions on Pattern Analysis and Machine In- telligence, 31(5):855–868.
    -[4] Zachary C. Lipton, A Critical Review of Recurrent Neural Networks for Sequence Learning, arXiv:1506.00019.
    -[5] Jie Zhou and Wei Xu; End-to-end Learning of Semantic Role Labeling Using Recurrent Neural Networks; ACL-IJCNLP 2015.

    -
    -
    - - -
    -
    -
    - - -
    - -
    -

    - © Copyright 2016, PaddlePaddle developers. - -

    -
    - Built with Sphinx using a theme provided by Read the Docs. - -
    - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc_cn/tutorials/text_generation/index_cn.html b/doc_cn/tutorials/text_generation/index_cn.html deleted file mode 100644 index 767e22662e74ee171f3cbea466b02915d30e230b..0000000000000000000000000000000000000000 --- a/doc_cn/tutorials/text_generation/index_cn.html +++ /dev/null @@ -1,602 +0,0 @@ - - - - - - - - - - - 文本生成教程 — PaddlePaddle 文档 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    - -
    -
    - Folk me on Github -
    - - English - - - - -
    - -
    -
    - - - - -
    -
    - - - -
    -
    -
    -
    -
    - -
    - - - - -
    - - - - - - - - - - - -
    -
      - -
    • 文本生成教程
      • -
    -
    - -
    -
    -
    -
    - -
    -

    文本生成教程

    -

    在语言生成领域中，“序列到序列”（sequence to sequence）的方法已被证明是一种强大的模型。它可以被应用于进行机器翻译（machine translation）、query改写（query rewriting）、图像描述（image captioning）等等。

    -

    本篇教程将会指导你通过训练一个“序列到序列”的神经网络机器翻译（NMT）模型来将法语翻译成英语。

    -

    我们遵循 Neural Machine Translation by Jointly Learning to Align and Translate 这篇文章，其中详细说明了模型架构，以及在WMT-14数据集上得到良好表现的训练过程。本篇教程在PaddlePaddle中重现了这一良好的训练结果。

    -

    我们感谢@caoying的pull request，其中定义了模型架构和solver配置。

    -
    -

    数据准备

    -
    -

    下载与解压缩

    -

    从该链接 http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/ 下载WMT-14数据集，然后解压，并将Develop和Test数据分别放入不同的文件夹。

    - -

    在Linux下，只需要简单地运行以下命令。否则你需要自己下载、解压、拆分到不同文件夹、并且分别重命名文件后缀。

    -
    cd demo/seqToseq/data
-./wmt14_data.sh
-
    -
    -

    我们会发现数据集 wmt14 中包含如下表所示的3个文件夹。

    - ------ - - - - - - - - - - - - - - - - - - - - - - - - -
    folder nameFrench-English parallel corpora filenumber of total filesize
    train_dataccb2_pc30.src, ccb2_pc30.trg, etc123.55G
    test_datantst1213.src, ntst1213.trg21636k
    gen_datantst14.src, ntst14.trg2864k
    -
      -
    • 每个文件夹都包含法语到英语的平行语料库
      • -
    • XXX.src 是原始法语文件；XXX.trg 是目标英语文件
      • -
    • XXX.src 和 XXX.trg 的行数应该一致
      • -
    • 每行都是一个法语或者英语的句子
      • -
    • XXX.src 和 XXX.trg 中任意第i行的句子之间都有着一一对应的关系
      • -
    -
    -
    -

    用户自定义数据集

    -

    如果你想进行诸如语义转述（Paraphrasing）等其他“序列到序列”的任务，你只需要按照如下方式组织数据，并将它们放在demo/seqToseq/data目录下：

    -
    dataset
-  train
-    file1.src file1.trg
-    file2.src file2.trg
-    ......
-  test
-    file1.src file1.trg
-    file2.src file2.trg
-    ......
-  gen
-    file1.src file1.trg
-    file2.src file2.trg
-    ......
-
    -
    -
      -
    • 一级目录：数据集文件夹名称
      • -
    • 二级目录：train、test和gen这三个文件夹是固定的
      • -
    • 三级目录：源语言到目标语言的平行语料库文件
        -
      • XXX.src 是源语言的文件，XXX.trg 时目标语言的文件
        • -
      • 文件中的每行都必须是一个句子
        • -
      • XXX.src 和 XXX.trg 中任意第i行的句子之间都必须有着一一对应的关系
        • -
      -
      • -
    -
    -
    -
    -

    数据预处理

    -
    -

    预处理工作流程

    -
      -
    • 将每个源语言到目标语言的平行语料库文件合并为一个文件：
        -
      • 合并每个 XXX.src 和 XXX.trg 文件为 XXX
        • -
      • XXX 中的第i行 = XXX.src 中的第i行 + ‘\t’ + XXX.trg中的第i行
        • -
      -
      • -
    • 创建训练数据的“源字典”和“目标字典”，每个字典都有DICTSIZE个单词，包括：
        -
      • 词频最高的（DICTSIZE - 3）个单词
        • -
      • 3个特殊符号
        • -
      • <s>：序列的开始
        • -
      • <e>：序列的结束
        • -
      • <unk>：未包含在字典中的单词
        • -
      -
      • -
    -
    -
    -

    预处理命令和结果

    -

    对数据集进行预处理的基本命令是：

    -
    cd demo/seqToseq/
-python preprocess.py -i INPUT [-d DICTSIZE] [-m]
-
    -
    -
      -
    • -i INPUT：输入的原始数据集路径
      • -
    • -d DICTSIZE：指定的字典单词数，如果没有设置，字典会包含输入数据集中的所有单词
      • -
    • -m --mergeDict：合并 “源字典”和“目标字典”，使得两个字典有相同的上下文
      • -
    -

    你将会看到如下消息：

    -
    concat parallel corpora for dataset
-build source dictionary for train data
-build target dictionary for train data
-dictionary size is XXX
-
    -
    -

    然后你只需要运行以下命令：

    -
    python preprocess.py -i data/wmt14 -d 30000
-
    -
    -

    这将花费数分钟的时间，并且将预处理好的数据集存放在demo/seqToseq/data/pre-wmt14目录下。目录结构如下：

    -
    train test gen train.list test.list gen.list src.dict trg.dict# Text generation Tutorial #
-
    -
    -
      -
    • train, test, gen：分别包含了法语到英语的平行语料库的训练数据、测试数据和生成数据。文件夹中的每个文件的每一行包含两部分，首先是法语序列，然后是对应的英语序列。
      • -
    • train.list, test.list, gen.list：分别为train，test，gen文件夹中的文件列表
      • -
    • src.dict, trg.dict：源（法语）/目标（英语）字典，每个字典包含总共30000个单词：29997个最高频单词和3个特殊符号
      • -
    -
    -
    -
    -

    模型训练

    -
    -

    简介

    -

    神经网络机器翻译（NMT）旨在建立一个可以被协同调至最优翻译效果的单神经元网络。近期提出的NMT模型通常都属于编解码模型（encoder–decoder models）的一种。编解码模型将一个源语句编码为一个定长的向量，然后解码器通过这个向量生成一个目标语句。

    -

    在这个任务中，我们使用了一个编解码模型的扩展，它同时学习排列(align)与翻译。每当模型在翻译过程中生成了一个单词，它就会在源语句中搜索出最相关信息的位置的集合。解码器根据上下文向量预测出一个目标单词，这个向量与源中搜索出的位置和所有之前生成的目标单词有关。如想了解更多详细的解释，可以参考 Neural Machine Translation by Jointly Learning to Align and Translate。

    -

    这个模型对于编解码模型来说，最不同的特色是它并没有将输入语句编码为一个单独的定长向量。相反，它将输入语句编码为向量的序列，其中每个向量对应输入语句中的一个元素。然后在解码被翻译的语句时，会自适应地从这些向量中选择一个子集出来。这使得NMT模型得以解放出来，不必再将任意长度源语句中的所有信息压缩至一个定长的向量中。该模型在长语句翻译的场景下效果提升更加明显，在任意长度语句翻译的场景下都可以观察到其效果的提升。 -

    -
    Figure 1. Encoder-Decoder-Attention-Model

    -
    -
    -

    使用PaddlePaddle训练模型

    -

    我们在训练之前需要常见一个模型配置文件，这里是一个例子demo/seqToseq/translation/train.conf。前三行import了定义network，job_mode和attention_mode的python函数。

    -
    from seqToseq_net import *
-is_generating = False
-
-### Data Definiation
-train_conf = seq_to_seq_data(data_dir = "./data/pre-wmt14",
-                             is_generating = is_generating)
-
-### Algorithm Configuration
-settings(
-    learning_method = AdamOptimizer(),
-    batch_size = 50,
-    learning_rate = 5e-4)
-
-### Network Architecture
-gru_encoder_decoder(train_conf, is_generating)
-
    -
    -
      -
    1. Data Definiation：在示例中我们定义了一个序列到序列的训练和测试数据。它返回train_conf作为配置，其输入参数如下：
      2. -
    -
      -
    • data_dir：训练数据和测试数据的目录
      • -
    • is_generating：这个配置是否用来生成，这里设置为False
      • -
    -
      -
    1. Algorithm Configuration：在示例中我们使用SGD训练算法（默认），和ADAM学习方法，指定batch_size为50，learning_rate为5e-4
      2. -
    2. Network Architecture：在示例中我们使用attention版本的GRU编解码网络。它包括了一个双向的GRU作为编码器和解码器，它模拟了解码翻译过程中在源语句中的搜索。
      3. -
    -
    -
    -

    训练模型的命令与结果

    -

    写完模型配置之后，我们可以通过以下命令来训练模型：

    -
    cd demo/seqToseq/translation
-./train.sh
-
    -
    -

    train.sh 的内容如下所示：

    -
    paddle train \
---config='translation/train.conf' \
---save_dir='translation/model' \
---use_gpu=false \
---num_passes=16 \
---show_parameter_stats_period=100 \
---trainer_count=4 \
---log_period=10 \
---dot_period=5 \
-2>&1 | tee 'translation/train.log'
-
    -
    -
      -
    • config: 设置神经网络的配置文件
      • -
    • save_dir: 设置保存模型的输出路径
      • -
    • use_gpu: 是否使用GPU训练，这里设置为使用CPU
      • -
    • num_passes: 设置passes的数量。paddle中的一条pass表示训练数据集中所有的样本一次
      • -
    • show_parameter_stats_period: 这里每隔100个batch显示一次参数统计信息
      • -
    • trainer_count: 设置CPU线程数或者GPU设备数
      • -
    • log_period: 这里每隔10个batch打印一次日志
      • -
    • dot_period: 这里每个5个batch打印一个点”.”
      • -
    -

    训练的损失函数默认每隔10个batch打印一次，你将会看到如下消息：

    -
    I0719 19:16:45.952062 15563 TrainerInternal.cpp:160]  Batch=10 samples=500 AvgCost=198.475 CurrentCost=198.475 Eval: classification_error_evaluator=0.737155  CurrentEval: classification_error_evaluator=0.737155
-I0719 19:17:56.707319 15563 TrainerInternal.cpp:160]  Batch=20 samples=1000 AvgCost=157.479 CurrentCost=116.483 Eval: classification_error_evaluator=0.698392  CurrentEval: classification_error_evaluator=0.659065
-.....
-
    -
    -
      -
    • AvgCost：从第0个batch到当前batch的平均cost
      • -
    • CurrentCost:：当前batch的cost
      • -
    • classification_error_evaluator(Eval)：从第0个评估到当前评估中，每个单词的预测错误率
      • -
    • classification_error_evaluator(CurrentEval)：当前评估中，每个单词的预测错误率
      • -
    -

    当classification_error_evaluator的值低于0.35时，模型就训练成功了。

    -
    -
    -
    -

    文本生成

    -
    -

    简介

    -

    一般而言，NMT模型受制于源语句的编码，并且通过给出当前目标单词来预测下一个目标单词。在训练过程中，当前单词在相比之下总是被当作真值（ground truth）。在生成过程中，当前单词是解码器最后一步的输出，这来自于PaddlePaddle的内存中。

    -

    而且，我们使用集束搜索（Beam Search）来生成序列。集束搜索使用广度优先搜索来构建搜索树。对于树的每一层，生成当前层的所有后继状态，并将它们按照启发代价（heuristic cost）升序排列。但是这种方法在每层只保存预设数量的最优状态（这个数量称为beam size）。

    -
    -
    -

    预训练的模型

    -

    我们在拥有50个节点的集群中训练模型，每个节点有两个6核CPU。我们在5天里训练了16个pass，其中每条pass花费了7个小时。model_dir中有16个子目录，每个里面都包含202MB的全部的模型参数。然后我们发现pass-00012的模型有着最高的BLEU值27.77（参考文献BLEU: a Method for Automatic Evaluation of Machine Translation）。要下载解压这个模型，只需在linux下运行如下命令：

    -
    cd demo/seqToseq/data
-./wmt14_model.sh
-
    -
    -
    -
    -

    使用PaddlePaddle生成模型

    -

    在翻译法语句子之前，我们需要创建模型配置文件。这里是一个例子demo/seqToseq/translation/gen.conf。前三行import了定义network，job_mode和attention_mode的python函数。

    -
    from seqToseq_net import *
-is_generating = True
-
-################## Data Definiation #####################
-gen_conf = seq_to_seq_data(data_dir = "./data/pre-wmt14",
-                           is_generating = is_generating,
-                           gen_result = "./translation/gen_result")
-
-############## Algorithm Configuration ##################
-settings(
-  learning_method = AdamOptimizer(),
-  batch_size = 1,
-  learning_rate = 0)
-
-################# Network configure #####################
-gru_encoder_decoder(gen_conf, is_generating)
-
    -
    -
      -
    1. Data Definiation：在示例中我们定义了一个序列到序列的生成数据。它返回gen_conf作为配置，其输入参数如下：
      2. -
    -
      -
    • data_dir：生成数据的目录 - - is_generating：这个配置是否用来生成，这里设置为True - - gen_result：保存生成结果的文件
      • -
    -
      -
    1. Algorithm Configuration：在生成过程中我们使用SGD训练算法，并指定batch_size为1（每次生成1个序列），learning_rate为0
      2. -
    2. Network Architecture：本质上与训练模型一样
      3. -
    -
    -
    -

    生成模型的命令与结果

    -

    写完模型配置之后，我们可以通过以下命令来进行从法语到英语的文本翻译：

    -
    cd demo/seqToseq/translation
-./gen.sh
-
    -
    -

    gen.sh 的内容如下所示。与训练模型不同的是，这里有一些不同的参数需要指定：

    -
    paddle train \
---job=test \
---config='translation/gen.conf' \
---save_dir='data/wmt14_model' \
---use_gpu=true \
---num_passes=13 \
---test_pass=12 \
---trainer_count=1 \
-2>&1 | tee 'translation/gen.log'
-
    -
    -
      -
    • job：设置任务的模式为测试
      • -
    • save_dir：存储模型的路径
      • -
    • num_passes and test_pass：从test_pass到（num_passes - 1）加载模型参数，这里只加载 data/wmt14_model/pass-00012
      • -
    -

    你将会看到这样的消息：

    -
    I0706 14:48:31.178915 31441 GradientMachine.cpp:143] Loading parameters from data/wmt14_model/pass-00012
-I0706 14:48:40.012039 31441 Tester.cpp:125]  Batch=100 samples=100 AvgCost=0
-I0706 14:48:48.898632 31441 Tester.cpp:125]  Batch=200 samples=200 AvgCost=0
-...
-
    -
    -

    然后在demo/seqToseq/translation/gen_result中的生成结果如下所示：

    -
    0
-0       -11.1314         The <unk> <unk> about the width of the seats while large controls are at stake <e>
-1       -11.1519         The <unk> <unk> on the width of the seats while large controls are at stake <e>
-2       -11.5988         The <unk> <unk> about the width of the seats while large controls are at stake . <e>
-
-1
-0       -24.4149         The dispute is between the major aircraft manufacturers about the width of the tourist seats on the <unk> flights , paving the way for a <unk> confrontation during the month of the Dubai <unk> . <e>
-1       -26.9524         The dispute is between the major aircraft manufacturers about the width of the tourist seats on the <unk> flights , paving the way for a <unk> confrontation during the month of Dubai &apos; s <unk> . <e>
-2       -27.9574         The dispute is between the major aircraft manufacturers about the width of the tourist seats on the <unk> flights , paving the way for a <unk> confrontation during the month of Dubai &apos; s Dubai <unk> . <e>
-...
-
    -
    -
      -
    • 这是集束搜索的结果，其中beam size是3
      • -
    • 第一行的“0”和第6行的“1”表示生成数据的序列id
      • -
    • 其他六行列出了集束搜索的结果
        -
      • 第二列是集束搜索的得分（从大到小）
        • -
      • 第三列是生成的英语序列
        • -
      -
      • -
    • 有两个特殊标识：
        -
      • <e>：序列的结尾
        • -
      • <unk>：不包含在字典中的单词
        • -
      -
      • -
    -
    -
    -

    BLEU评估

    -

    对机器翻译的人工评估工作很广泛但也很昂贵。一篇论文 BLEU: a Method for Automatic Evaluation of Machine Translation 展示了一种方法，当需要快速或者频繁的评估时，使用自动的替补来替代经验丰富的人工评判。Moses 是一个统计学的机器翻译系统，我们使用其中的 multi-bleu.perl 来做BLEU评估。运行以下命令来下载这个脚本：

    -
    cd demo/seqToseq/translation
-./moses_bleu.sh
-
    -
    -

    由于标准的翻译结果已经下载到这里data/wmt14/gen/ntst14.trg，我们可以运行以下命令来做BLEU评估。

    -
    cd demo/seqToseq/translation
-./eval_bleu.sh FILE BEAMSIZE
-
    -
    -
      -
    • FILE：生成的结果文件
      • -
    • BEAMSIZE：集束搜索中的扩展广度
      • -
    -
    -
    -
    - - -
    -
    -
    - - -
    - -
    -

    - © Copyright 2016, PaddlePaddle developers. - -

    -
    - Built with Sphinx using a theme provided by Read the Docs. - -
    - -
    -
    - -
    - -
    - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/doc_cn/tutorials/index_cn.html b/doc_cn/v1_api_tutorials/README.html similarity index 75% rename from doc_cn/tutorials/index_cn.html rename to doc_cn/v1_api_tutorials/README.html index f8a9d85f3a6a79dd571d2b80b95f94ba34cde5cc..cdf896f6d2cc91364d77d670b6df5cbc34cdce87 100644 --- a/doc_cn/tutorials/index_cn.html +++ b/doc_cn/v1_api_tutorials/README.html @@ -8,7 +8,7 @@ - 完整教程 — PaddlePaddle 文档 + <no title> — PaddlePaddle 文档 @@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -85,6 +85,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -108,9 +109,10 @@ @@ -173,7 +195,7 @@
      -
    • 完整教程
      • +
    • <no title>
    @@ -182,28 +204,10 @@
    -
    -

    完整教程

    -
    - -
    -
    -

    常用模型

    -
    - -
    -
    -
    +

    The tutorials in v1_api_tutorials are using v1_api currently, and will be upgraded to v2_api later. +Thus, v1_api_tutorials is a temporary directory. We decide not to maintain it and will delete it in future.

    +

    Please go to PaddlePaddle/book and +PaddlePaddle/models to learn PaddlePaddle.

    diff --git a/doc_cn/tutorials/embedding_model/index_cn.html b/doc_cn/v1_api_tutorials/embedding_model/index_cn.html similarity index 89% rename from doc_cn/tutorials/embedding_model/index_cn.html rename to doc_cn/v1_api_tutorials/embedding_model/index_cn.html index 6c5d4fd3f9f23404f6c898154b73d3428f177d10..8758614bae46642563a0060f53fddeb47ef86ce3 100644 --- a/doc_cn/tutorials/embedding_model/index_cn.html +++ b/doc_cn/v1_api_tutorials/embedding_model/index_cn.html @@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -85,6 +85,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -108,9 +109,10 @@ @@ -288,7 +310,7 @@ python extract_para.py –preModel PREMODEL –preDict PREDICT –us
  • --init_model_path: 初始化模型的路径配置为data/paraphrase_modeldata/paraphrase_model
  • --load_missing_parameter_strategy：如果参数模型文件缺失，除词向量模型外的参数将使用正态分布随机初始化
    • -

    如果用户想要了解详细的数据集的格式、模型的结构和训练过程，请查看 Text generation Tutorial.

    +

    如果用户想要了解详细的数据集的格式、模型的结构和训练过程，请查看 Text generation Tutorial.

    diff --git a/doc_cn/tutorials/imagenet_model/resnet_model_cn.html b/doc_cn/v1_api_tutorials/imagenet_model/resnet_model_cn.html similarity index 93% rename from doc_cn/tutorials/imagenet_model/resnet_model_cn.html rename to doc_cn/v1_api_tutorials/imagenet_model/resnet_model_cn.html index ec3503aa1ed1f91b527122e07adfc2d081530c12..35ceaad74fe1533398880a197588bb722025a74f 100644 --- a/doc_cn/tutorials/imagenet_model/resnet_model_cn.html +++ b/doc_cn/v1_api_tutorials/imagenet_model/resnet_model_cn.html @@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -85,6 +85,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -108,9 +109,10 @@ diff --git a/doc_cn/tutorials/quick_start/index_cn.html b/doc_cn/v1_api_tutorials/quick_start/index_cn.html similarity index 78% rename from doc_cn/tutorials/quick_start/index_cn.html rename to doc_cn/v1_api_tutorials/quick_start/index_cn.html index de21b7f5bc325493d6a9098762aaee2ba0e35cbc..abba7f9b86bb7fe6b92ce7c4005ea093b3a511d1 100644 --- a/doc_cn/tutorials/quick_start/index_cn.html +++ b/doc_cn/v1_api_tutorials/quick_start/index_cn.html @@ -62,7 +62,7 @@
    - Folk me on Github + Fork me on Github
    English @@ -85,6 +85,7 @@
  • 进阶指南
  • API
  • FAQ
    • +
  • MOBILE
    • @@ -108,9 +109,10 @@ @@ -256,147 +278,11 @@
  • process：PaddlePaddle调用process函数来读取数据。每次读取一条数据后，process函数会用yield语句输出这条数据，从而能够被PaddlePaddle 捕获 (harvest)。

    • dataprovider_bow.py 文件给出了完整例子：

    -
     1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
-24
-25
-26
-27
-28
-29
-30
-31
-32
-33
-34
-35
-36
-37
-38
-39
-40
-41
-42
-43
-44
-45
-46
-47
-48
-49
-50
    # initializer is called by the framework during initialization.
-# It allows the user to describe the data types and setup the
-# necessary data structure for later use.
-# `settings` is an object. initializer need to properly fill settings.input_types.
-# initializer can also store other data structures needed to be used at process().
-# In this example, dictionary is stored in settings.
-# `dictionay` and `kwargs` are arguments passed from trainer_config.lr.py
-def initializer(settings, dictionary, **kwargs):
-    # Put the word dictionary into settings
-    settings.word_dict = dictionary
-
-    # setting.input_types specifies what the data types the data provider
-    # generates.
-    settings.input_types = {
-        # The first input is a sparse_binary_vector,
-        # which means each dimension of the vector is either 0 or 1. It is the
-        # bag-of-words (BOW) representation of the texts.
-        'word': sparse_binary_vector(len(dictionary)),
-        # The second input is an integer. It represents the category id of the
-        # sample. 2 means there are two labels in the dataset.
-        # (1 for positive and 0 for negative)
-        'label': integer_value(2)
-    }
-
-
-# Delaring a data provider. It has an initializer 'data_initialzer'.
-# It will cache the generated data of the first pass in memory, so that
-# during later pass, no on-the-fly data generation will be needed.
-# `setting` is the same object used by initializer()
-# `file_name` is the name of a file listed train_list or test_list file given
-# to define_py_data_sources2(). See trainer_config.lr.py.
-@provider(init_hook=initializer, cache=CacheType.CACHE_PASS_IN_MEM)
-def process(settings, file_name):
-    # Open the input data file.
-    with open(file_name, 'r') as f:
-        # Read each line.
-        for line in f:
-            # Each line contains the label and text of the comment, separated by \t.
-            label, comment = line.strip().split('\t')
-
-            # Split the words into a list.
-            words = comment.split()
-
-            # convert the words into a list of ids by looking them up in word_dict.
-            word_vector = [settings.word_dict.get(w, UNK_IDX) for w in words]
-
-            # Return the features for the current comment. The first is a list
-            # of ids representing a 0-1 binary sparse vector of the text,
-            # the second is the integer id of the label.
-            yield {'word': word_vector, 'label': int(label)}
-
    -

    详细内容请参见 DataProvider的介绍 。

    配置中的数据加载定义

    在模型配置中通过 define_py_data_sources2 接口来加载数据：

    -
     1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
    dict_file = "./data/dict.txt"
-word_dict = dict()
-with open(dict_file, 'r') as f:
-    for i, line in enumerate(f):
-        w = line.strip().split()[0]
-        word_dict[w] = i
-
-is_predict = get_config_arg('is_predict', bool, False)
-trn = 'data/train.list' if not is_predict else None
-tst = 'data/test.list' if not is_predict else 'data/pred.list'
-process = 'process' if not is_predict else 'process_predict'
-define_py_data_sources2(
-    train_list=trn,
-    test_list=tst,
-    module="dataprovider_emb",
-    obj=process,
-    args={"dictionary": word_dict})
-
    -

    以下是对上述数据加载的解释：

    • data/train.list,data/test.list: 指定训练数据和测试数据
      • @@ -413,7 +299,7 @@
      ../../_images/PipelineNetwork_cn.jpg
      -

      我们将以最基本的逻辑回归网络作为起点，并逐渐展示更加深入的功能。更详细的网络配置连接请参考 Layers 。 +

      我们将以最基本的逻辑回归网络作为起点，并逐渐展示更加深入的功能。更详细的网络配置连接请参考 api_trainer_config_helpers_layers 。 所有配置都能在 源代码 的 demo/quick_start 目录下找到。

      逻辑回归模型

      @@ -656,7 +542,7 @@ Momentum, RMSProp，AdaDelta，AdaGrad，ADAM，Adamax等，这里采用Adam优
    -

    这里只简单介绍了单机训练，如何进行分布式训练，请参考 运行分布式训练 。

    +

    这里只简单介绍了单机训练，如何进行分布式训练，请参考 cluster_train 。

    预测