* add adam/sharedadam optimzier for gpups;edit optimizer struct;test=develop

* use yapf to format all python file

* yapf exclude two unittests file for they rely on writing and reading file, and format will break them

* disable diff_py_file because too many diff files cause command following failed

* add fused_bias_dropout_residual_ln op and layer.

* nanmedian op

* 修改cuda kernel的bug

* 修复count_if在其他硬件平台不兼容

* 修复某些cpu硬件不兼容

* 修复某些cpu硬件不兼容

* 修复isnan判断

* 兼容numpy低版本不支持全部nan的情况

* 兼容numpy低版本不支持全部nan的情况

* fix code example

* fix api comment error

* 修改反向传播逻辑以及c++处理逻辑

* 完成修改建议

* typo pre_dim

* update en docs, test=document_fix

* remove numpy in en doc, test=document_fix

* add r,test=document_fix

* 添加api到all

* follow advice from chenwhql

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

* approve amp for intermediate_dygraph

* add amp_utils for intermediate_dygraph

* add amp needcast check for mlu & npu

* test unittest

* add SetGradNode for set_stop_gradient && add checktensor for GradientHooks

* refine code

* refien unittest of imperative_amp for new dygraph

* inplace api skip amp

* add test_imperative_qat_amp for intermediate amp

* refine code

* refine test_amp ci strategy

* refine unittest code

* refine amp_utils code

* refine amp getpromotetype for some special op

* refine unittest code

* remove

* pass

* more pass

* add cpu kernel of lstsq

* update

* modify code style

* modify unittest

* remove support for complex

* add test for conv_elementwise_add2_act_fuse_pass and conv_elementwise_add_act_fuse_pass

* Add conv_eltwiseadd_bn_fuse_pass test and fix test_conv_elementwise_addX_act_fuse_pass

* add tests for conv_act_mkldnn_fuse_pass

* add test for conv_bias_mkldnn_fuse_pass

* update code

* add conv_act_mkldnn_fuse_pass for relu, relu6, swish, leaky_relu

* update test

* update

* update bug

* update

* update pattern_detector

* fix test_conv_eltwiseadd_bn_fuse_pass

* add diff display notest;test=windows_ci_inference

* fix

* remove test_conv_act_mkldnn_fuse_pass.py

* ifix

* add layer_norm_fuse_pass test case

* restore cmakelist code

* Merge branch 'develop' into layer_norm_fuse_pass

* Merge branch 'develop' into layer_norm_fuse_pass

* add bad case test

* make some test higher paralle level

* modify slightly acoodring ci test:40100-40107

* modify slightly acoodring ci test:40124-40134

* modify slightly acoodring ci test:40138-40150

* modify slightly acoodring ci test:40153-40165

* modify slightly acoodring ci test:40153-40165

* add fc_elementwise_layernorm_fuse_pass

* fix name conflictn

* rebuild CI

* fix Ran Programs=0 bug

* make some non_parallel unittest parallel execute

* delete duplicate ut

* add test_unsqueeze2_eltwise_fuse_pass

* fix name conflictn

* rebuild CI

* fix ut retry

* for pure fp16

* opt topk

* modify for wincheck-inference case

* modify according to zhouwei's comment

* modify according to zhouwei's comment 2nd time
Co-authored-by: Nzhangkaihuo <zhangkaihuo@baidu.com>

* modify unittest's parallel num

* modify unittest's parallel num

* add randint like

* rm .cc .cu

* Update unity_build_rule.cmake

* try to make test pass

* use python

* update

* update randint_like

* rename test_randint_like_op -> test_randint_like

* update

* update randint like docs

* update randint like

* update

* update

* add bool

* update randint like test

* update

* update

* Support EMA in Paddle2.x and Fleet

* update

* update

* update

* modify ut of ema

* modify docs

* modify bugs

* update

* update

* update

* modify ut

* Change uts to nightly mode

* remove test_trt_pool_op from parallel_UT_rule.py,test=document_fix

* Add model for ResNet50 for Dy2stat AMP training

* fix timeout

* fix dataloader

* polish windows compile for Ninja, fix random compile fail

* polish windows compile for Ninja, fix random compile fail

* Modify the search order of dynamic library

* Modify the search order of dynamic library

* fix ninja compilation bug on windows

* polish windows ci

* polish windows ci

* add complex template file

* add numtraits for complex template

* add complex template type register

* modify specify template of complex

* modify specify template of complex

* modify specify template of complex

* modify specify template of complex

* make TensorCheckerVisitor support complex type

* fix operator= error

* add complex template

* add complex template type

* add complex template type to pyarray transform

* add complex template type to pyarray transform

* remove complex type for dlpack register

* set dlpack supprot complex type

* set dlpack supprot complex type

* set dlpack supprot complex type

* remove explict for complex constructor

* add complex unit test file

* remove ut from parallel_ut list

* remove some timeout ut

* fix windows CI

* fix windows CI