* fix prepare context redundant code problem, optimize executor by caching create_varaiables
test=develop

* supports collective training in executor

* make fetch_list runable with variables, add more unittest for use_program_cache
test=develop

* fix comment
test=develop

* use unique name for nccl_id

* supports output to stream in program_to_code

* insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code

* set op role in collective training

* add collective op role

* remove orig file

* add build optimizer by strategy

* add collective strategy

* refine collective strategy

* add multi-process role maker

* refine strategy building factory so that we can easily plugin more strategy

* scale loss grad in collective sgd transpiler

* add support for distributed fc

* code format

* revert some features for dist fc

* add support for distributed fc training

* fix prepare context redundant code problem, optimize executor by caching create_varaiables
test=develop

* supports collective training in executor

* make fetch_list runable with variables, add more unittest for use_program_cache
test=develop

* use unique name for nccl_id

* supports output to stream in program_to_code

* insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code

* set op role in collective training

* add collective op role

* fix comment
test=develop

* remove orig file

* add build optimizer by strategy

* add collective strategy

* refine collective strategy

* add multi-process role maker

* refine strategy building factory so that we can easily plugin more strategy

* scale loss grad in collective sgd transpiler

* add support for distributed fc

* code format

* revert some features for dist fc

* add support for distributed fc training

* test=develop
add collective op unittest standard

* test=develop
remove the test_collective directory

* test=develop
remove the test_collective directory

* remove slicegather test

* code format for reducescatter

* update attr of shard_index_op

* Modify macro nccl_helper

* remove test without distribute

* macro collective_helper

* marcro update

* test=develop
update support python3.5

* test=develop change gpu memory use to 0.1 when test

* test=develop
update ut equal func

* test=develop
set flags to 1.5

* test=develop fix pickle dumple  py35

* test=develop
fix divide in slice and add sync_comm_stream
update atol and rtol to 1e-05
rm shard_index op and test
modify read input from file to read from memory
remove origin_program in framework and add i/o in c_sync_calc_stream

* test=develop update unittest sync operator I/O

* Simplify multi_box_head API in detection.py and remove assign op.

* Update lamb optimizer

test=develop, test=document_preview

* Regenerate api spec

test=develop, test=document_preview

Socket can't be pickled.
test=develop

* test=develop, add_install_check_for_multigpu

* test=develop, refine code to use cuda_devices

* sequnce mask support max length tensor input; test=develop

* add rnn_impl.py; test=develop

* add basic gru lstm unittest; test=develop

* fix api spec; test=develop

* fix sequence_mask op bug;
test=develop
test=document_preview

* change +-*x to elmentwise_op; test=develop

* add mkl flag; test=develop

* fix rnn impl bug; test=develop

* update api spec; test=develop

* fix doc bug; test=develop

* fix lstm bugs; test=develop

* Revert "Add multi gpu install check (#18229)"

This reverts commit 61ed06b2.

* test=develop, start ci

* optimize communicator merge sparse gradient test=develop

* revert multithread selected rows merge add test=develop

* follow comment test=develop

test=develop

* fix default value of fluid.memory_optimize
test=develop

* fix api.spec
test=develop

test=develop

* improve doc of lstm, sequence_enumerate, softmax_with_cross_entropy, space_to_depth APIs, test=develop

* update API.spec, test=develop

test=develop

* fix paddle cloud role maker bug

test=develop, test=document_preview

test=develop
test=document_preview

* refine core cmake warning and print more info

test=develop

* fix comments

test=develop

refine StaticRNN api doc
test=develop
test=document_preview

1. fix the bug that out_put_var in SaveSelectedRows would be empty string
2. use merge_sparse_lookup_table to replace sum op for load_persistables_for_inference
3. fix the bug in _clone_var_in_block_ when the var is SELECTED_ROWS.

(1) use channel instead of vector/BlockingQueue in Dataset，to keep same with existing implementation, and make code more readable and flexible (dataset single output channel or multi output channel). one previous memory out of limit problem is cause by not release memory after training.
(2) add Record because MultiSlotType costs too much memory (80B)，fix memory out of limit problem.
(3) add Channel, Archive in paddle/fluid/framework
(4) change dataset from shared_ptr to unique_ptr in pybind
(5) move create/destroy readers from trainer to dataset
(6) move shuffle from datafeed to dataset. dataset holds memory, datafeed is only for load data and feed data to network.
(7) fix thread num bug of Dataset when filelist size < thread num
(8) support set_queue_num in InMemoryDataset

* improve the doc of DataFeeder and default_main_program

* update API.spec, test=develop

* test=develop, add add_multi_gpu_install_check

* test=develop, refine warning doc

* test=develop, refine warning doc

* test=develop, refine warning doc

* test=develop, support multi cpu

* test=develop, find right num of cuda device

* test=develop, find right num of cuda device

* test=develop, fix multigpu processing and fix type bug in dygraph

* test=develop, fix multigpu processing and fix type bug in dygraph

* set src_idx > 0, test=develop

* add unittest and cu, test=develop

* add more print function for timeout issue, make timeout value larger

* fix relative import error in python3
* fix debug string info

* [DOC] Fix comment code of API create_py_reader_by_data. test=develop, test=document_preview

* Fix code style of API comment. test=develop,test=document_preview

Fix code style of API comment. test=develop,test=document_preview

* update api spec of api create_py_reader_by_data

* remove default config code. test=develop

* remove useless code. test=develop

* update create_py_reader_by_data api. test=develop

* fix_create_lod_tensor, test=develop

* remove program_guard import,test=develop

* fix windows numpy default int32 error, test=develop

* fix slice op bug; test=develop

* fix variabel test bug; test=develop

* remove slice while true; test=develop