* Support NPU OP hard_swish and hard_swish_grad

* Support NPU OP hard_swish and hard_swish_grad

* add the unittest to compare the result between npu ans cpu

* format the prompt of exception

* replace Min and Max op by ClipByValue op

* fix the precision problem for fp16

* Using HardtanhGrad to improve performace

* fix dice_loss bug

* add p_norm_op_npu

* remove p_norm_grad op

* update

* OP dot: refactor CPU kernels and get better loop performance.

* Minor fix on code format.

* Fixed minor errors.

* Add new API: einsum

* Update the Einsum unit test.

One case failed with matmul_v2, where the dtype is int64:

a = np.arange(2 * 3 * 1).reshape(2, 3, 1)
b = np.arange(1)
paddle.einsum("...i, ...i", a, b)

* Test cases in test_einsum test floating point dtypes only.

As of now Paddle only supports float/double dtypes in matmul, which is
one of building blocks of this Einsum implementation. We decide not to
test einsum against other dtypes.

* Polish format.

* More formatting.

* Format...

* Einsum: improve test coverage.

* Einsum: bug fixes and more testcases for testing error messages

* Einsum: fix format..

* Einsum: fixed typo and format.

* Einsum: format again...

* Einsum: applied suggested changes.

* Einsum API: improve API documentation.

* Einsum API: apply suggested changes.

* Einsum API: Add dygraph only note.

* Einsum API: Add dygraph only note.

* Einsum API: fixed unittest.

* Fix a bug : can't load more than one custom op module

* Fix a bug : can't load more than one custom op module

* add test for load multiple modules of custom c++ op

* add config for Coverage CI

* Fix safety-bug of functional.linear

* Fix safety-bug of functional.linear

* Fix safety-bug of functional.linear

* Fix safety-bug of functional.linear

* add recompute for pp

* add recompute offload

* add recompute partition

* [NPU] Support npu op expand_v2 and expand_v2_grad

* [NPU] Support npu op expand_v2 and expand_v2_grad

* [NPU] Support npu op expand_v2 and expand_v2_grad

* update test_expand_v2_op_npu.py

* update test_expand_v2_op_npu.py

* modify expand_v2_op_npu.cc

* modify expand_v2_op_npu.cc

* add state_dict and load_state_dict and unittest for class GradScaler

* refine unittest for coverage of load_state_dict

* refine comments of code-block

* refine some comments

* refine state_dict code and unittest

* add #require gpu, xpu for GradScaler get/set example code

* add #require gpu, xpu for GradScaler get/set example code

* refine example code

* refine unittest for state_dict

* refine unittest for state_dict

* fix bug of DataLoader in TestGradScalerStateDict

* add flag FLAGS_cudnn_deterministic

* add set_value_grad op

* add unittest.

* polish unittest.

* polish code.

* support cuda kernel

* polish code according to CI

* polish code.

* polish code

* remove *.pyc

* polish code.

* add unittest to improve coverage.

* polish code.

* add while read_from_array write_to_array npu op

* optimize unittest

* add momentum_op_npu and test

* update

* fix hang

* add reduce_mean_op_npu and test

* remove skip.If

* update

* add batch_norm_op_npu and tests

* remove skip.If

* fix bug

* add auto_parallel apis

* add save/load for pipelineparallel

* add save/load

* add exp and exp_grad npu op

* modify support register type

* remove empty line and remove exp_grad support data type int/int64

* move exp and epx_grad kernel to activation_op_npu.cc, delete attrs

* move code to activation_op_npu.cc

* Add NPU kernel for TopKV2 op

* deleted unnecessary cache file static_mode_white_list.cpython-37.pyc

* A draft for error checking

* A commit with accuracy error for float32 data

* Modify codes according to the review comments

* Modify codes according to the review comments

* fix npu compile error, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* Update flatten_op_npu.cc

* Update flatten_op_npu.cc
Co-authored-by: Nqili93 <qili93@qq.com>

* [NPU] add squared_l2_norm squared_l2_norm and tests

* [NPU] replace Square&ReduceSumD with SquareSumV1

* Support npu kernel for fill_any_like op

* modify the description of exception

* remove useless template element

* remove useless decorator

* fix the code format error

* fix npu compile error, test=develop

* add fill constant batch size lilke op npu,test=develop
Co-authored-by: Nqili93 <qili93@qq.com>

* Support npu kernel for tile op

* modify according to the comments

* fix compute function

* Support npu kernel for expand_as_v2 op

* mofify the registry data type name

* fix test unit

* fix npu compile error, test=develop

* fix compute function
Co-authored-by: Nqili93 <qili93@qq.com>

* Fix error of HSigmoidLoss

* update unittest

* update unittest

* add one_hot_op and tests

* update

* make code clear

* add broadcast supporting for elementwise_add

* add broadcast supporting for elementwise_add

* add more tests

* remove the redundant code

* update

* fix place error in unittest

* remove skip.If