* add sync communicator and implement

* Add TopK Op Grad CPU&GPU Kernel test=develop

* Add TopK Op Grad, modify grad op maker test=develop

* Add TopK Op Grad, modify grad op maker test=develop

* Add TopK Op Grad, modify PADDLE_ENFORCE test=develop

* Add TopK Op Grad, modify PADDLE_THROW test=develop

* Add TopK Op Grad, modify unittest test=develop

* fix ngraph top k op unittest test=develop

test=release/1.7

* [cherry-pick] :Refine the error log about runtime batch and max_batch_size. #22535
test=release/1.7

* fix comments
test=release/1.7

* 1. load model from memory

2. scale is no longer added when saving inference model

test=develop

* raise ci coverage, test=develop

* supports saving weights to memory. test=develop

* raise ci coverage, test=develop

* fix PADDLE_ENFORCE messages, test=develop

* Add support for dynamic_decode(while) training. test=develop

* Fix assign_op and tensor_array_read_write_op after solving conflict. test=develop

* Fix test_rnn_decode_api.py. test=develop

* Refine docs for apis in rnn.py. test=develop

* Adjust outputs of dynamic_decode. test=develop

* Remove the force_cpu update in assign_op. test=develop

* Remove the force_cpu update in assign_op. test=develop

* Make RNNCell.get_initial_states support batch_dim_idx argument. test=develop

* Rename _create_array_outof_while as _create_array_out_of_while in rnn.py.

test=release/1.7

* Introduce Ernie NLP

* Fix error regarding incorrect attr type

test=release/1.7
Co-authored-by: NWojciech Uss <wojciech.uss@intel.com>

* Add weight quantization in post_training_quanzitaion (#22445)

* [cherry-pick]Support int16 for Tensor (#22423)

* add int16 support, test=develop, test=release/1.7
Co-authored-by: NLeo Chen <chenqiuliang@baidu.com>

[cherry-pick] #22509

支持不依赖nccl进行编译。

多卡下，如果没有打开WITH_NCCL开关编译，多卡不能通信，则只能选择一张卡使用

[cherry-pick] #22484

支持不依赖nccl进行编译。

多卡下，如果没有打开WITH_NCCL开关编译，则只能使用单卡

* optimize interpolate op.
test=develop

Co-authored-by: NManjunath Bhat <manjunathbhat9920@gmail.com>

[cherry-pick] #22384 and #22371

22384增加了WITH_NCCL开关

22371修改了fluid依赖lite的commit id

* Fix GEO-SGD init & send Bug (#22375)

* test=develop, fix geo Send & Init

* test=release/1.7,test=develop, cherry-pick 8f36c395

test=release/1.7

* Enable quantize to reorder to nchw as well

* Correct FC MKL-DNN input dim requirements to accept 3D

* Improve DNNL FC format, error and 3D input handling

* Improve error checking in FC

* Improve PADDLE_ENFORCE messages in fc-related files

* Remove data layout attribute from obligatory pass args

* Fix message in fc_mkldnn_pass to be logically correct

* add half_async in the communicator
* fix DistributedStrategy

* test=develop, bug fix for sparse recorder

* test Optimizer in dygraph, test=develop

* add optest for Optimizer in dygraph, test=develop

* fix adagrad optimizer, test=develop

* fix dpsgd optimizer, test=develop

* fix test_optimizer.py, test=develop

* fix dpsgd optimizer, this op only support cpu, test=develop

* add optest for optimizer, test=develop

* add description for dpsgd, test=develop

* add rmsprop to white_list in unused_var_check.cc, test=develop

* polish code style, test=develop

* polish code style, test=develop

* delete seed attribute for DpsgdOptimizer, test=develop

* change testing to debugging, test=develop

stack's wait cost a lot of cpu time, use cuda kernel to do memory copy
will reduce cpu time.
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

1. Add a new input named batch_roi_nums for prroi_pool_op. batch_roi_nums includes the number of roi for each image in batch when rois is Tensor. This information is saved in rois's lod when rois is LoDTensor.
2. add grad check to prroi_pool_op and solve unnormal X grad diff in CPU.

windows conv_fusion failed as no kernel， explicit declare lambda
Signed-off-by: Nzhaoyuchen <zhaoyuchen01@baidu.com>

* add special way to add distribute vars， Update Pyramid hash op

* add erf op and python interface.

* add fp16 support for erf op.

* add unitests for erf op and its python interface.

* support elu activation double grad,test=develop

* delete the code commit in .cc,test=develop

* fix relu test unpass, test=develop

* add elu double grad kernel and unit test

* add caculate dX in elu double grad functor, test=develop

* update the commit code,test=develop

* add gelu plugin

* align trt bert with gpu

* add support for fused fc with relu,

* add unittest for bert trt

* add distributed_strategy

* register int/int64_t/float16 in  pow/square kernel,test=develop

* add abs/square/exp type,test=develop