* add save/load for pipelineparallel

* add save/load

* add exp and exp_grad npu op

* modify support register type

* remove empty line and remove exp_grad support data type int/int64

* move exp and epx_grad kernel to activation_op_npu.cc, delete attrs

* move code to activation_op_npu.cc

* Add NPU kernel for TopKV2 op

* deleted unnecessary cache file static_mode_white_list.cpython-37.pyc

* A draft for error checking

* A commit with accuracy error for float32 data

* Modify codes according to the review comments

* Modify codes according to the review comments

* fix npu compile error, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* [NPU] Support npu kernel for flatten_contiguous_range op, test=develop

* Update flatten_op_npu.cc

* Update flatten_op_npu.cc
Co-authored-by: Nqili93 <qili93@qq.com>

* [NPU] add squared_l2_norm squared_l2_norm and tests

* [NPU] replace Square&ReduceSumD with SquareSumV1

* Support npu kernel for fill_any_like op

* modify the description of exception

* remove useless template element

* remove useless decorator

* fix the code format error

* fix npu compile error, test=develop

* add fill constant batch size lilke op npu,test=develop
Co-authored-by: Nqili93 <qili93@qq.com>

* Support npu kernel for tile op

* modify according to the comments

* fix compute function

* Support npu kernel for expand_as_v2 op

* mofify the registry data type name

* fix test unit

* fix npu compile error, test=develop

* fix compute function
Co-authored-by: Nqili93 <qili93@qq.com>

* Fix error of HSigmoidLoss

* update unittest

* update unittest

* add one_hot_op and tests

* update

* make code clear

* add broadcast supporting for elementwise_add

* add broadcast supporting for elementwise_add

* add more tests

* remove the redundant code

* update

* fix place error in unittest

* remove skip.If

* fix npu compile error, test=develop

* [NPU] Support npu kernel for atan and atan_grad op, test=develop

* [NPU] Support npu kernel for atan and atan_grad op, test=develop
Co-authored-by: Nqili93 <qili93@qq.com>

* [NPU] add reduce_prod

* [NPU] delete check_dygraph=False

* [NPU] delete skipIf

* add attrs support or check

* [NPU] delete extra codes for test_reduce_max_op_npu

* [NPU] add attr out_dtype

* fix paddle.optimizer test=document_fix

* fix paddle.optimizer test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix nn formula bugs test=document_fix

* fix nn formula bugs test=document_fix

* fix nn formula bugs test=document_fix

* add eye npu op

* remove useless headers

* code style

* Update eye_op_npu.cc

* Update eye_op_npu.cc

* remove useless code in test file

* code style check

* change Copyright to 2021

* add test case and do some fix

* fix

* update code

* fix for CI

* return

* fix

* fix ut

* decrease gpu memory consumption

* remove exclusive

* [NPU] Support npu op: (1) cos (2) cos_grad

* Update test_cos_op_npu.py

* Update activation_op_npu.cc

* rm redundant {1}

* Add relu6 and relu6_grad npu op

* fixed pre-commit-config.yaml

* fixed for CI

* first test version

* add test exec;

* add data transfer; test=develop

* add new exec head;

* add memcpy; test=develop

* add python fetch

* add new test

* add graph node; test=develop

* remove useless new executor test; test=develop

* remove gperf dependency; test=develop

* fix compile bugs; test=develop

* remove useless code; test=develop

* remove useless code; test=develop

* add uni test; test=develop

* polish code; test=develop

* polish code; test=develop

* add interpreter cmakefile; test=develop

* remove useless code; test=develop

* integrated gast library

* integrated gast library

* fix unittest and remove ast2.py

* remove 'gast' from __all__ in __init__.py

* add copyright in other files

* fix copyright

* support bool dtype for paddle.sum

* Support Mixed Precision training in @to_static

* fix block.vars logic

* fix GPU training loss diff

* remove unused code