* set scipy and numpy version suit for py3.6

* pynacl1.5.0 which is needed by PyGithub built failed in python36, change it to 1.4.0 also works;test=document_fix;test=windows_ci

* np.corrcoef support parameter since 1.20

* delete test code

* [NPU] add multinomial op

* fix place

* deal with cann version

* fix for old operator

* change another way

* add nan_inf_utils for eager

* support check nan and inf

* add unittest for coverage

* adapt faster tokenizer op

* add eager test

* add unittest

* add OneCycleLR

* add missing total_steps

* try

* update

* fix conflict bug

* fix typo

* update doc

* update and polish

* Refactor

* update

* change end_lr to end_learning_rate

* support construct scalar using non-cpu tensor

* fix bugs when run unittest

* fix compile bugs

* fix bugs when run ci

* fix compile bugs

* fix bugs when move copy

* perfect unit test

* perfect unittest

* update according to comment

* int_array supports constructed by gpu tensor

* add some test

* polish code

* adjust full api

* add unittest

* add unittest
Co-authored-by: NYuanRisheng <yuanrisheng@baidu.com>

* Refactor test_tensordot

* Add test_static

* Fix CI errors

* add nvsmi

* collect gpu info to log

* fix unitest

* rm ret_type

* add cinn pass to program

* remove build_cinn_pass ut

* polish ut, add ut

* guard ut with is_compiled_with_cinn

* enable ut test_build_cinn_pass_resnet

* Use tempfile to place the temporary files

* Revise test_bert to use tempfile for temporary files

* Use tempfile for test_transformer

* Fix test_dataset file race

* add tile double_grad yaml and test case

* add expand double yaml and test case

* add clip dobule grad yaml and test case

* add concat dobule grad yaml and test case

* Add int8 scales gathering pass for convolution

* Fix typo

* Add unittest

* Add corrected unit test

* Change test name

* Remove enabling mkldnn in test

* Speed up test

* Change max examples

* Add functional test

* Change test name

* Add new test case

* Rename pass

* add conv2d_transpose_double_grad yaml

* add test_conv_transpose double_grad test case

* update to popart v2.5.0

* use a specific version of sdk2.5.0

* support diff batch for sparse of eagerreducer

* fix

* change logic for optimize

* modifty

* [Auto Parallel] Refactor the engine api and parallelizer

* [Auto Parallel] Fix the default dist op for the slice op

* [Auto Parallel] Fix the format of planer.py

* [Auto Parallel] Fix a bug

* Readd conv_affine_channel fuse pass as mkldnn pass

* Fix formatting

* Add new test to parallel_UT_rule.py

* Fix Coverage and Windows CI issues

* Revert "Fix Coverage and Windows CI issues"

This reverts commit f33459846385c9fd51c07f9f44e7ff283a652637.

* Fix CI errors

* Remove unnecessary conv_eltwise_add_affine_channel fuse pass

* Remove test from parallel_UT_rule.py

* [Eager] fix sharding under eager mode

* [Eager] fix several sharding test under eager mode

* Recover using _test_eager_guard

* Ensured fleet.init under legacy

* Ensured fleet.init under legacy

* Fix CI issue, re-definition strategy and call fleet.init() in stage2_offload

* Modified dygraph_group_sharded_api.py, move fleet.init to a better line

* added fill_constant kernel

* CI fix

* ci fix

* switched from nan to zero memory

* CI FIX

* ci fixes

* CI rerun

* ci fix

* minor change

* CI rerun

* Create elementwise_heaviside_op.cc

* add ElementwiseHeavisideFunctor

* Create test_elementwise_heaviside_op.py

* 增加heaviside的python接口

* add heaviside in white list

* 增加heaviside的签名

* 增加heaviside的核函数

* 增加heaviside梯度的核函数

* 增加heaviside梯度的注册

* 调整代码格式

* Update elementwise_sig.cc

* add heaviside in __all__

* Update heaviside docs

* Update math.py

* Update math.py

* Update math.py

* fix adamw unittest

* tiny fix

* fix param name