* fix prepare context redundant code problem, optimize executor by caching create_varaiables
test=develop

* supports collective training in executor

* make fetch_list runable with variables, add more unittest for use_program_cache
test=develop

* fix comment
test=develop

* use unique name for nccl_id

* supports output to stream in program_to_code

* insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code

* set op role in collective training

* add collective op role

* remove orig file

* add build optimizer by strategy

* add collective strategy

* refine collective strategy

* add multi-process role maker

* refine strategy building factory so that we can easily plugin more strategy

* scale loss grad in collective sgd transpiler

* add support for distributed fc

* code format

* revert some features for dist fc

* add support for distributed fc training

* fix prepare context redundant code problem, optimize executor by caching create_varaiables
test=develop

* supports collective training in executor

* make fetch_list runable with variables, add more unittest for use_program_cache
test=develop

* use unique name for nccl_id

* supports output to stream in program_to_code

* insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code

* set op role in collective training

* add collective op role

* fix comment
test=develop

* remove orig file

* add build optimizer by strategy

* add collective strategy

* refine collective strategy

* add multi-process role maker

* refine strategy building factory so that we can easily plugin more strategy

* scale loss grad in collective sgd transpiler

* add support for distributed fc

* code format

* revert some features for dist fc

* add support for distributed fc training

* test=develop
add collective op unittest standard

* test=develop
remove the test_collective directory

* test=develop
remove the test_collective directory

* remove slicegather test

* code format for reducescatter

* update attr of shard_index_op

* Modify macro nccl_helper

* remove test without distribute

* macro collective_helper

* marcro update

* test=develop
update support python3.5

* test=develop change gpu memory use to 0.1 when test

* test=develop
update ut equal func

* test=develop
set flags to 1.5

* test=develop fix pickle dumple  py35

* test=develop
fix divide in slice and add sync_comm_stream
update atol and rtol to 1e-05
rm shard_index op and test
modify read input from file to read from memory
remove origin_program in framework and add i/o in c_sync_calc_stream

* test=develop update unittest sync operator I/O

add prior_box quantization code

add scale algo rules for prior box

test=develop

* - Reusing of reuder used in elementwise_add_mkldnn

- Added MKL-DNN sum prim reusing

test=develop

- Compilation fixes

test=develop

- Yet another compilation fix

test=develop

- Yet another compilation fix

test=develo

- Yet another linking fix

test=develop

- Final compilation fix

test=develop

- lint fixes

test=develop

- Lint fixes

test=develop

* - Fixes after review

test=develop

* Simplify multi_box_head API in detection.py and remove assign op.

* Update lamb optimizer

test=develop, test=document_preview

* Regenerate api spec

test=develop, test=document_preview

* fix softrelu doc

test=develop

* update API doc

test=develop

* sequnce mask support max length tensor input; test=develop

* add rnn_impl.py; test=develop

* add basic gru lstm unittest; test=develop

* fix api spec; test=develop

* fix sequence_mask op bug;
test=develop
test=document_preview

* change +-*x to elmentwise_op; test=develop

* add mkl flag; test=develop

* fix rnn impl bug; test=develop

* update api spec; test=develop

* fix doc bug; test=develop

* fix lstm bugs; test=develop

* optimize communicator merge sparse gradient test=develop

* revert multithread selected rows merge add test=develop

* follow comment test=develop

* Use TensorCopySync for sequence_unpad op

test=develop

* Fix the tensor memory alloc bug

test=develop

* clean build_strategy
test=develop

* DataBalanceOpHandle has been removed
test=develop

* debug

* update build_strategy.
test=develop

1. fix the bug that out_put_var in SaveSelectedRows would be empty string
2. use merge_sparse_lookup_table to replace sum op for load_persistables_for_inference
3. fix the bug in _clone_var_in_block_ when the var is SELECTED_ROWS.

* set src_idx > 0, test=develop

* add unittest and cu, test=develop

* fix slice op bug; test=develop

* fix variabel test bug; test=develop

* remove slice while true; test=develop

* fix spelling errors; test=develop

* Update API.spec

update md5

* Update API.spec

* change the order of api;test=develop

* test=develop
fix type error of std::pow in sigmoid_focal_loss_op.cu and sigmoid_focal_loss_op.h

* test=develop
fix wrong code stype in sigmoid_focal_loss_op.cu and sigmoid_focal_loss_op.h

* fix py_reader iterable bug, test=develop

* move data from buffered_reader,test=develop

* Update backward.py:
     - If there is no input grad var in all outputs of previous ops, do not append this op into graph.
     - Only apply this stragety when double backward.
* Update some double backward op.
* Update sum_op to judge whether a tensor is empty by numel or IsInitialized().

* test=develop
add detection output for supporting retinanet

* test=develop
add test_layers.py

* test=develop
add API.spec

* test=develop
alter test_retinanet_detection_output.py

* test=develop
alter round 2

* test=develop
alter retinanet_detection_output

* test=develop
alter paddle/fluid/API.spec

* test=devlop
alter detection.py

* test=develop
alter retinanet_detection_output

* test=develop
alter paddle/fluid/API.spec

* test=develop
alter detection.py

* test=develop
alter API.spec

* test=develop
alter retinanet_detection_output

* test=develop
alter paddle/fluid/API.spec

* test=develop
alter python/paddle/fluid/tests/unittests/test_retinanet_detection_output.py

* test=develop
alter python/paddle/fluid/tests/unittests/test_retinanet_detection_output.py

* test=develop
fix grammer error

* test=develop
fix grammer error

* test=develop
fix grammer error

* test=develop
alter python/paddle/fluid/tests/unittests/test_layers.py

* test=develop
alter paddle/fluid/API.spec

* test=develop
add sigmoid_focal_loss for supporting retinanet

* test=develop
add test_layers

* test=develop
add API.spc

* test=develop
alter sigmoid_focal_loss_op.cc

* test=develop
alter detection.py

* test=develop
alter API.spec

* test=develop
alter round 1

* test=develop
alter simooid_focal_loss

* test=develop
alter sigmoid_focal_loss_op.cc

* test=develop
alter test_layers.py

* test=develop
alter paddle/fluid/API.spec

* test=develop
alter sigmoid_focal_loss_op.cu

* test=develop
alter paddle/fluid/operators/detection/sigmoid_focal_loss_op.cc

* Update generate_proposal_labels_op to support CascadeRCNN.

* test=develop add target assign for retinanet

* test=develop
run ci

* test=developp
add test_layers

* test=develop
add APi.spec

* test=develop
alter round 1

* test=develop
alter rpn_target_assign_op.cc

* test=develop
alter test_rpn_target_assign_op.py

* test=develop
alter rpn_target_assign_op.cc

* test=develop

alter API.spec

* test=develop
alter paddle/fluid/operators/detection/rpn_target_assign_op.cc

* test=develop
alter rpn_target_assign_op.cc

* test=develop
alter python/paddle/fluid/layers/detection.py

* test=develop
alter paddle/fluid/API.spec

* fix code bug
test=develop

test=develop

test=develop

* refractor the function ConvFwdPrimitiveDesc
test=develop

* change according to review
test=develop

* use pointer way without boost::optional
test=develop

* pass vector to function by reference instead of raw vector
test=develop

* change pointer to shared_ptr
test=develop

* added unit test for QAT FP32 & INT8 comparison

test=develop

* enabled other models and updated filenames

test=develop

* added accuracy check and multiple batch handling

test=develop

* removed quantization_mkldnn_pass.py

test=develop

* cleanup

test=develop

* updated model paths

test=develop

* renamed tests without MKL-DNN

test=develop

* fix reusing mkldnn pool2d primitive

test=develop

* add performance measuring

test=develop

* fix accuracy statistics

test=develop

* removed non-mkldnn tests

test=develop

* added conv2d_depthwise->conv2d mkldnn transformation

test=develop

* format update

test=develop

* fixed creating key for pool2d grad

test=develop

* added pass

* Fix the accuracy issue while using float precision to get the scale.

test=develop

* Fix the format issue when 'X' is not nchw.

test=develop

* removed output comparing and changed number of images

test=develop

* cmake and comment fix

test=develop

* updated acc threshold for QAT comparison tests

test=develop

* added OMP_NUM_THREADS setting

test=develop

* enable all QAT INT8 tests

test=develop

* restored upstream version of a file

test=develop

* modified directory names

test=develop

test=develop

Optimize the concat and split cuda implementation for cases when the number of inputs/outputs is less than 5. (#17979)

test=develop

* fix save/load in Fleet
* add UT framework of Fleet

* Fix the GetExpectedKernelType of add_position_encoding_op.
test=develop

* Fix the doc of lstm_unit outputs in nn.py.
test=develop

* test=develop
The scatter op has a calc bug when the indices has same index, the scatter op use overwrite mode to calculate the same index, fix this bug by using the accumulate mode to calculate the same index.At the same time, the gather op has the same bug when the op calc the grad. And we use the lib of open-blas and eigen to optimize the time cost in accumulate mode.

* test=develop
Fix some code format problem, and the same time add the test case in gather and scatter op

Repair error prompt: Users are prompted to check whether the model or parameter files are damaged when loading parameters are wrong.

Fix bug in sequence_unpad op, when allocate the output memory do not match actual memory, check memory failed. Fix this bug by allocating the output memeory in correct code position.

* add deformable psroi pooling

* test=develop

* test=develop

* test=develop
modify format

* fix bug

* test=develop run ci

* test=develop
add API.spec

* add test_layers.py

* run ci again

* test=develop
run ci again

* run ci again

* test=develop
run ci again

* test=develop
run ci again

* test=develop
run ci again

* add space between two lines

* test=develop
add space between two lines

* test=develop
add space between lines

* test=develop
modify comment in nn.py

* test=develop
add space between two lines

* test=develop
add space between two lines

* update API.spec

* run ci again

* test=develop
run ci again

* rerun ci

* test=develop
rerun ci

* change input shape

* run ci

* test=develop
run ci

* modify format of nn.py

* test=develop

* test=develop

* test=develop
update API.spec

* test=develop
fix API doc

* modify API comment

* modift API comment

* test=develop
update API.spec

* test=develop
modify comment

* test=develop
modift comment

* test=develop
modift comment

* test=develop
update API.spec

* test=develop
modify comment

* test=develop
add inference in nn.py

* test=develop
update API.spec

* test=develop
resolve confict

* test=develop
update API.spec

* add unfold op
test=develop

* fix divide bug in python3 when calculating output width and height
test=develop

* add name=None in python api, move redundant code into inline function

* try to trigger ci for this code
test=develop

* - removed is_reusing_

* - Added TID to keys for reusing apart from softmax PD

* - compilation fix

* - Yet another compilation fix

* - Batch Norm and Conv adapted

* - Fix to softmax MT

* - Fixes to MT code of MKL-DNN

* - Lint fixes

test=develop

* update anakin-engine interfaces for content-dnn

test=develop

* support only-gpu mode of Anakin

modify eltwise parse

test=develop

* modification for thread-safe

test=develop

* Integrated template instance

test=develop

* increase template parameters

test=develop

* support MLU predictor

test=develop

* update anakin cmake files

test=develop

* update TargetWrapper::set_device

* update the initialization of anakin subgraph

test=develop

* use the default constructor of base class

test=develop

fix merge conflict of 'Remove attribute in Allocator::Allocate' and elementwise_add_mkldnn_op (#17949)

test=develop

* remove attribute in Allocator::Allocate, test=develop

* fix travis ci error, test=develop