* update api format
test=develop

* update API.spec
test=develop

test=develop

* add MultiSlotStringDataGenerator for speedup of string based user input data

* test=develop, add some comments for Program.clone

* test=develop, add API.spec

* test=develop, refine comments

* refine Program doc and clone doc

* test=develop, refine doc

*  add cuda_is_available
test=develop

* Fix api.spec
test=develop

* fix api doc
test=develop

test=develop

Fix dygraph show style for FluidDoc.

* update pe reduce config
test=develop

*  drop the local_exe_scopes of the previous parallel_executor
test=develop

* add is_runnning in communicator, test=develop

* add WITH_COVERAGE option, default OFF

test=develop

* add coverage for python sdk

test=develop

* fix code style

* fix COVERAGE_FILE path

test=develop

* remove coverage package

test=develop

* test = develop, run coverage as module

* fix prepare context redundant code problem, optimize executor by caching create_varaiables
test=develop

* supports collective training in executor

* make fetch_list runable with variables, add more unittest for use_program_cache
test=develop

* fix comment
test=develop

* use unique name for nccl_id

* supports output to stream in program_to_code

* insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code

* set op role in collective training

* add collective op role

* remove orig file

* add build optimizer by strategy

* add collective strategy

* refine collective strategy

* add multi-process role maker

* refine strategy building factory so that we can easily plugin more strategy

* scale loss grad in collective sgd transpiler

* add support for distributed fc

* code format

* revert some features for dist fc

* add support for distributed fc training

* fix prepare context redundant code problem, optimize executor by caching create_varaiables
test=develop

* supports collective training in executor

* make fetch_list runable with variables, add more unittest for use_program_cache
test=develop

* use unique name for nccl_id

* supports output to stream in program_to_code

* insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code

* set op role in collective training

* add collective op role

* fix comment
test=develop

* remove orig file

* add build optimizer by strategy

* add collective strategy

* refine collective strategy

* add multi-process role maker

* refine strategy building factory so that we can easily plugin more strategy

* scale loss grad in collective sgd transpiler

* add support for distributed fc

* code format

* revert some features for dist fc

* add support for distributed fc training

* test=develop
add collective op unittest standard

* test=develop
remove the test_collective directory

* test=develop
remove the test_collective directory

* remove slicegather test

* code format for reducescatter

* update attr of shard_index_op

* Modify macro nccl_helper

* remove test without distribute

* macro collective_helper

* marcro update

* test=develop
update support python3.5

* test=develop change gpu memory use to 0.1 when test

* test=develop
update ut equal func

* test=develop
set flags to 1.5

* test=develop fix pickle dumple  py35

* test=develop
fix divide in slice and add sync_comm_stream
update atol and rtol to 1e-05
rm shard_index op and test
modify read input from file to read from memory
remove origin_program in framework and add i/o in c_sync_calc_stream

* test=develop update unittest sync operator I/O

* Simplify multi_box_head API in detection.py and remove assign op.

* Update lamb optimizer

test=develop, test=document_preview

* Regenerate api spec

test=develop, test=document_preview

Socket can't be pickled.
test=develop

* test=develop, add_install_check_for_multigpu

* test=develop, refine code to use cuda_devices

* sequnce mask support max length tensor input; test=develop

* add rnn_impl.py; test=develop

* add basic gru lstm unittest; test=develop

* fix api spec; test=develop

* fix sequence_mask op bug;
test=develop
test=document_preview

* change +-*x to elmentwise_op; test=develop

* add mkl flag; test=develop

* fix rnn impl bug; test=develop

* update api spec; test=develop

* fix doc bug; test=develop

* fix lstm bugs; test=develop

* Revert "Add multi gpu install check (#18229)"

This reverts commit 61ed06b2.

* test=develop, start ci

* optimize communicator merge sparse gradient test=develop

* revert multithread selected rows merge add test=develop

* follow comment test=develop

test=develop

* fix default value of fluid.memory_optimize
test=develop

* fix api.spec
test=develop

test=develop

* improve doc of lstm, sequence_enumerate, softmax_with_cross_entropy, space_to_depth APIs, test=develop

* update API.spec, test=develop

test=develop

* fix paddle cloud role maker bug

test=develop, test=document_preview

test=develop
test=document_preview

* refine core cmake warning and print more info

test=develop

* fix comments

test=develop

refine StaticRNN api doc
test=develop
test=document_preview

1. fix the bug that out_put_var in SaveSelectedRows would be empty string
2. use merge_sparse_lookup_table to replace sum op for load_persistables_for_inference
3. fix the bug in _clone_var_in_block_ when the var is SELECTED_ROWS.