* add fuse_bn_add_act pass

* add compile limit for paddle enforce

* polish elementwise_op_function.cu.h

* fix failed unittest

* fix windows compile failed

* detail polish

* revert no type constructor

* fix strided_slice_op's GetExpectedKernelType when input tensor is at CUDAPinnedPlace

* add unittest for tensors in cuda pinned place

* skip test for cuda pinned place on cpu machines

* fix generate_proposal_labels in cascade-rcnn series model, test=develop

* fix example code & unittest, test=develop

* update code from review comments, test=develop

* fill_constant supports nan and inf

* add ut

* add stack pool2d roi_align xpu op,test=kunlun

* error message opt, test=kunlun

* add xpu unittest,test=kunlun

* skip check grad,test=kunlun

* fix boostget , test=kunlun

* error message opt for XPU, test=kunlun

* add rois_num for roi_align xpu OP, test=develop

* lookup_table_v2_op_xpu report errors;test=kunlun

* lookup_table_v2_op_xpu report errors;test=kunlun

* lookup_table_xpu op report errors;test=kunlun

* add adam xpu op;test=kunlun

* reset lookup

* change adam wrong;test=kunlun

* fix error message,test=kunlun

* fix, test=kunlun

Fix diag OP bug on Windows Python3.8 ，remove the std::min

* Still has bugs.

* Fixed allclose_op bug, which cannot deal with some cases of fp64 inputs.

* improved CUDA kernel performance.

* Changed CUDA code.

* Fixed a bug in cuda kernel which cannot deal with large dimension input, and added an unittest for it.

* Add a test case for float32 input.

* fix multinomial doc

* fix multinomial error message

* little doc change

* fix Categorical class doc

* optimize format of error message

* fix CPU Kernel error message format

* fix isinf and isnan error in WindowsOPENBLAS CI

* delete inf and nan

* add manual_seed in sample code

* little error message change

* change error message to InvalidArgument

* add full point for error message and add manual_seed in CPU environment

* add stack pool2d roi_align xpu op,test=kunlun

* error message opt, test=kunlun

* add xpu unittest,test=kunlun

* skip check grad,test=kunlun

* fix boostget , test=kunlun

* error message opt for XPU, test=kunlun

* Add truncated_gaussian_random_op XPU kernel

* Add truncated_gaussian_random_op XPU kernel, test=kunlun

* little change, test=kunlun

* change boost_get to BOOST_GET_CONST

* change boost_get to BOOST_GET_CONST, test=kunlun

* little change, test=kunlun

* use Generator to generate random number and optimize format, test=kunlun

* little change, test=kunlun

* add TODO, test=kunlun

* Add gaussian_random XPU kernels

* commit kunlun, test=kunlun

* new version, test=kunlun

* change boost_get to BOOST_GET_CONST, test=kunlun

* use Generator to generate random number and optimize format, test=kunlun

* add TODO, test=kunlun

* support uniform_random op on Baidu Kunlun

* change dtype of attr shape from int to int64_t

* kunlun ci, test=kunlun

* new version, test=kunlun

* change boost_get to BOOST_GET_CONST

* change boost_get to BOOST_GET_CONST, test=kunlun

* use Generator to generate random number and optimize format

* run Kunlun CI, test=kunlun

* add TODO, test=kunlun

* polish error message,test=kunlun

* polish error,test=kunlun

* polish error,test=kunlun

* polish error,test=kunlun

* addd

* add cast_op_xpu, test=kunlun

* fix bug for cast_op_xpu,test=kunlun

* add concat_op_xpu, test=kunlun

* slove conflicts, test=kunlun

* fix bug,test=kunlun

* add assign_op_xpu, test=kunlun

* fix bug,test=kunlun

* test=kunlun;test=develop

* fix concat bug,test=kunlun

* fix check_dygraph set in test_concat_op_xpu.py,test=kunlun

* fix error message,test=kunlun
Co-authored-by: Nmapingshuo <mps2012@yeah.net>

* Incorporate cudnn_lstm into LSTM api.
test=develop

* Make coalesce_tensor support alignment optionally.
test=develop

* Reorganize RNN apis. test=develop

* Fix cudnn rnn layout conversion.
test=develop

* Add sequence_length support for RNN cudnn implement.
Add optional init_h and init_c gradient for cudnn_lstm_op.
test=develop

* Use create_parameter for rnn cudnn impl.
test=develop

* Move `self._flat_weight = self.create_parameter()` in RNNBase to main_program.
test=develop

* Update RNN api unittest to use set_device.
test=develop

* Fix set_place for unit tests of RNN apis.
test=develop

* Fix use_align in coalesce_tensor_op.
test=develop

* Adjust RNN apis arguments according to comments.
test=develop

* Polish documents for SimpleRNN apis.
test=develop

* Refine random seed in cudnn_lstm_op.
Expose rnn params from sublayers to RNN.
test=develop

* Fix RNN saving for jit.save.
Refine cudnn_lstm dropout behavior.
test=develop

* Fix doc of GRU. test=develop

* Use ShareDataWith to avoid copying for cudnn_lstm_op test.
test=develop

* Remove updates on cudnn_lstm temporarily.
test=develop

* Use ShareDataWith to avoid copying for cudnn_lstm_op test.
test=develop

* Refine random seed in cudnn_lstm_op.
test=develop

* Fix test_lstm by adjust ConcreteProgram buffer getter.
test=develop

* Use create_parameter instead of create_var for rnn._flat_weight for static graph usage.
test=develop

* Remove W input for cudnn_lstm to pass unused_var_check.
test=develop

* Add test_predict for RNN unit tests coverage.
test=develop

* Fix code style of rnn.
test=develop

* Fix F.rnn usage in rnn.py.
test=develop

* test=kunlun;

Add elementwise XPU OP kernel for KUNLUN core, including (but still cannot process common broadcast):

    * elementwise_div op
    * elementwise_max op
    * elementwise_mul op (with grad op)
    * elementwise_sub op (with grad op)

* 0.05->0.01

* add xpu error message description;test=kunlun

* conv dilated mkldnn support: forward and backward pass

* add mkldnn conv_transpose dilation UT
test=develop

* remove unnecessary PADDLE_ENFORCE

* add int8 and bf16 dilated conv UT

* update according to reviews

* add size method for large scale

* add large scale UT

* add ut for checkpoint

* test=develop, fix geo sgd communicator

* test=develop, gloo_init_method

* test=develop, bug fix for gloo http_init

* add reduce xpu op test=develop;test=kunlun

* add reduce xpu op test=develop;test=kunlun

* add reduce xpu op test=develop;test=kunlun

* add reduce xpu op test=develop;test=kunlun

* add reduce xpu op test=develop;test=kunlun

* add tensor clone

* fix unittest test_var_base

1. support channel last in BatchNorm*d (#27875)
2. fix a bug in batch_norm_op cuda kernel by extracting ResizeToChannelFist(Last), TransToChannelFirst(Last) to operators/layer_utils.h