* add distributed_strategy

* register int/int64_t/float16 in  pow/square kernel,test=develop

* add abs/square/exp type,test=develop

* fix test_conv2d_ngraph for grad diff, test=develop

* register NoNeedBufferVarsInference for max_pool_grad_op, test=develop

* refine error message, test=develop

* fix numpy, test=develop

* disable test conv2d_ngraph_op, test=develop
Co-authored-by: NZhang Ting <709968123@qq.com>

* Add the dynamic load of nvrtc, and support runtime compiling of CUDA kernel using nvrtc.
test=develop

* Call CUDA driver api to launch the kernel compiled by nvrtc.
test=develop

* Disable for mac and windows.
test=develop

* Refine the codes to support manually specified num_threads and workload_per_thread.
test=develop

* Refine the CUDA kernel to support large dims.
test=develop

* Add DeviceCodePool to manage all device codes.

* Add the first implementation fusion_group op.

* Add unit-test for fusion_group op.

* Add the check of result.

* Add the check of nvrtc in unit-test.
test=develop

* Add comment to explain the inputs, outputs and features of fusion_group op.
test=develop

* Disable fusion_group op for mac and windows.
test=develop

* Make the compiling of device code return status instead of hanging up.
test=develop

* Add the check of whether there is CUDA driver library, and do not core dump when failing to call the CUDA driver API.

* Unify fusion_group_op's input and output names.
test=develop

* Add the check of CUDA driver library in unittest.
test=develop

* Refine the calling of PADDLE_ENFORCE.
test=develop

* test=develop fix generate_proposal_labesl op

* update error information about batch_norm_grad

* update bn,test=develop

* fix integer overflow in match_matrix test=develop

* fix integer overflow in match_matrix test=develop

* fix typo test=develop

* Refine multihead kernel, align block to 32

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* Refine log comments

test=develop
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* add shuffle batch op, test=develop, test=document_preview

* fix size_t conflict and check_output test=develop, test=document_preview

* fix bug test=develop, test=document_preview

* add unittest of shuffle_batch layer test=develop, test=document_preview

* fix py coverage and op input type, test=develop, test=document_preview

* fix py coverage, test=develop

* fix en doc, test=develop

* move to contrib test=develop

* add unique_name test=develop

* invoke shuffle_batch in contrib.layers test=develop

* make reverse op support negative axis

* fix compile error in CUDA10 test=develop

* remove double in pad2d test=develop

* fix stat shape back in global auc scenario
* add UT to cover global auc

* add register op_data_type test=develop

* fix register bug in isfinite op test=develop

* rm int int64_t in pad2d gradKernel  test=develop

* optimize adam speed by removing _finish_update test=develop

* fix SparseAdamFunctor param list test=develop

* Remove scale_op in expect_list of adam_op test=develop

* fix test optimizer loss assert error test=develop

* fix test optimizer loss assert error test=develop

* modify PADDLE_ENFORCE usage test=develop

* fix op_type in lamb_op.cc test=develop

* fix errors ostream format bug test=develop

* add betaPowOut in ngraph op test=develop

* fix ngraph::op api for gcc8 test=develop

* clean code test=develop

* modify struct into class test=develop

* remove code of beta1Tensor in lamb_op test=develop

Update iou_similarity op to support non-normalized bbox

test=develop

* test=develop, speed dense calc & communication

test=develop

1. Make while_op accept GPU conditional data
2. Add more complex test cases for while_loop API

* Fix softmax cuda bug

* Refine multihead log and softmax logic

* yolo_box OP add Attr(clip_bbox). test=develop

* fix elementwise_pow bug on integer, test=develop

* use llrint to support elementwise_pow_grad, test=develop

* add some tests, test=develop

* revert grad functor, test=develop

* rename paddle throw error macro, test=develop

* fix new error use case, test=develop

* Add reshape int8 op

test=develop

* Change test to CPUPlace

test=develop

* Correct tests

test=develop

* add fake init for the trainer, fix large memory hold in the trainer
* do not merge recv vars from a remote endpoint, test=develop
* add recv and save op, merge slice var in one op, save memory
* remove hsigmoid with pull sparse, test=develop

test=develop