* opt dygraph python API, test=develop

* Fix unbind bug in manipulation.py

* add num_iters in fit/evalate, test=develop

* Fix test_jit_save_load random failure.

* Since CI is not activated, recommit the code.

* delete temp file.

* added prelu bf16/fp32 fwd/bwd kernel

* add delta score, scale show

* so parser

* windows

* windows

* add gpu implementation of shuffle batch
test=develop

* add thrust cuda patches
test=develop

* fix macro guard

* fix shuffle batch compile on windows/hip

* fix hip compilation error

* refine CMakeLists.txt

* fix windows compile error

* try to fix windows CI compilation error

* fix windows compilation again

* fix shuffle_batch op test on Windows

* fix interpolate for shape[i]=0, test=develop

* fix test_trilinear_interp_v2 random failure, test=develop

* pipeline adaptive for heterps;test=develop
* fix finalize hang;test=develop
* add is_compiled_with_heterps for dataset;test=develop
* fix hashtable core when pass ins_num=0;test=develop

* reduce sum op default fp32, add into amp black list

* reduce_sum default fp32 can avoid return inf when the sum value large than 65504

* change Add to AddN in sum npu op

* add AddInputNames

* change fp16 to fp32 because numpy has accuracy loss in fp16 adding

* delete check

* fix runner error

* refine ptq according comments
* reuse the module to calculate kl threshold

* add amp support for pp

* add amp untest

* fix hdfs and fleet_util for supporting save/load infer;test=develop

* Save all scales to target ops
* Move quant layers to paddle.nn.quant

* delete useless GELU in gelu npu op

* add description

* fix format

* add check_grad in gelu unittest

* refine the old code

* support moving_average_abs_max and per_channel_abs_max

* Add moving_average_abs_max_scale op

* Convert the test program

* add random and prevent deadlock

* dygraph sharding

* update unitest hybrid_parallel_communicate_group