As the title

* base changes for split op

* 90% of split functionality added

* full fp32 functionality

* added bf16 test

* added submemory caching

* added bf test to static mode whitelist

* minor change

* enabled split op for inference

* minor fix

* minor fix

* elastic unitest

* rename demo

* new api diagonal, test=develop

* add new api diagonal, test=develop

* new api diagonal, test=develop

* add new api paddle.diagonal, test=develop

* use framework::stride replace ComputeDimStride

* replace cudaMalloc/cudaMemcpy by TensorFormVector in cudaKernel and cudaGradKernel

* perfect funciton: when attr(offset) is exceed attr(axis1) or attr(axis2), set the diagonal dim is 0

* fix RP-Mac-CI bug: replace framework::stride() by ComputDimStride.

* perfect code-block

* perfect code of python API diagonal

* api supports dtype of float16 and bool

* api supports dtype of float16 and bool

* modify unittest code

* modify unittest code

* perfect dtype describe

* perfect code-block

* using argparse to handle selections

* 2 TODOs

* 先不更改pipeline配置，这里强制改成GPU版本

* sorted the all_names

* exec gpu sample codes tests incrementally

* get all apis from the pr.spec file

* condition with WITH_GPU

WITH_GPU == ON

save

* delete the useless codes

* delete the useless codes.

test=document_fix

* echo the diff result

test=document_fix

* dont reuse the variables

* rename fun to _func not work. put it into the skiplist

https://github.com/PaddlePaddle/Paddle/commit/038ffc795025170e8cda74bcd473b46301b9a1c0
test=document_fix

* skip it in check api approvals

test=document_fix

save

* skip the private _variables

* print signatures wrong. now rename it to _func

test=document_fix

* transform complex scale to tensor

* add test_case for complex scalar

* modify import paddle

* dygraph post training quantization

* refine the ptq config

* refine ptq quantizer

* Add oneDNN AXPY handler.

* Add fallback for small tensors.

* Fix ifdefs

* Remove unnecessary namespace prefixes and add missing headers.

* Guard handler_axpy with proper ifdefs.

* Compilation of this function is possible only when Paddle is not build
with CUDA nor HIP.

* Move AXPY handler code to separate files.

* Use oneDNN AXPY handler in SGD op.

* Use axpy handler only when Paddle is built with oneDNN.

* Add test for SUM BF16 with big rows.

* Fix SFINAE rules for elementwise_add_to.

* Add test case for SGD with big rows.

* update

* update
Co-authored-by: NAdam Osewski <adam.osewski@intel.com>

* use BatchMatMul

* replace TensorCopy with ShareDataWith

* remove check fp16 grad

* fix format

* add grad_check

* fix grad check

* Combine amp and qat
* add unit test

* del py2 code2

* fix test timeout

* elastic etcd ready

* enable npu alignment

* support flatten_params/grads

* support clip by global norm

* remove memset in coalesce_tensor_op

* fix npu kernel of sum op when input is one tensor

* add ut for flatten_param_grads+regularizer

* fix ut

* fix typo

* Add bf16 support for save and load ops

* Add bf16 test condition

* Add matmul and chagne fluid.io to paddle.static

* Reduce the test duration

* add atan2_op

* fix

* support non-tensor type

* fix unittest failed

* add unittest with prune

* rm unused code

* coverage

* fix two or

* Add lookup_table_v2 BF16

* Reuse lookup table UT

* Change op_type to op_version

* Remove check_dygraph

* Remove skip_check_grad_ci

* - Draft of implementation of refactoring

- compilation fix

* - Fixes after review

* - Removed unnecessary comment