* try to add fwd op input dtypes

* refactor base impl

* return tmp_ins after dygraph prepare data

* fix typo found in debug

* polish comment & add complex net test

* revert detail change

* fix unittest failed

* add complex kernel condition control

* fix xpu test failed & polish comment

* polish details by review comments

* add support for complex grad accumulated

* add unittest for coverage

* update test dtype

* remove useless blank line

* support some shape in matmul and cast

* modify matmul

* feat: support check_nan_inf for kunlun device

* support kunlun stack

* minor

* oneps (3/4)
Co-authored-by: NMrChengmo <cmchengmo@163.com>
Co-authored-by: Nmalin10 <malin10@baidu.com>
Co-authored-by: Nchengmo <chengmo@baidu.com>

1. Type of index: int, slice(step must be 1).

2. Type of value: 
 (1) int32, int64, float32, bool; 
 (2) numpy.array(int32, int64, float32, bool);<Note: float64 is not supported>
 (3) paddle.Tensor(int32, int64, float32, float64, bool);

* remove duplicate ut reload

* remove duplicate ut define in cmakelist

* 　add heter box

* add trainer, worker, wrapper...

* format

* for ci

* format

* remove boost get

* boost & copyright

* rename

* 　rename

* format

* format

* format
Co-authored-by: Nyaoxuefeng6 <yaoxuefeng@baidu.com>

* - Tensor copy fix to oneDNN tensors

* - Fixes after review

* test=develop, optimize roll_op_cuda_kernel

* Using VisitDataTypeTiny and put CastOP after ReduceOP, test=develop

* remove changes of reduce_op.h, test=develop

* add conj op for complex types

* add conj for complex types

* add more test case

* add conj_op test

* modify conj api and impl

* add complex type for fill_constant_op xpu

* add setConstant for complex type

* remove complex conj test file

* user define grad for test_conj_op

* add test case for static mode of conj api

* modify conj doc

* change input args name to x

* remove useless codes

* conj support real types

* add conj test case for real number

* delete no need to calculate inputs in dygraph op_test

* delete no need to calculate inputs in dygraph op_test

* modify grad of mul for complex types

* fix the grads of inputs args order not match bug

* change the grad of div when complex types

* fix the grads of inputs args order not match bug

* add nearest_interp_v2 on kunlun

* add nearest_interp_v2 on kunlun

* fix fleet for multi-stream

* fix memcpy for ncclid

* use sync to solve move operation

Add Retry Logic to CublasHandlerHolder to avoid random unittest failure.

* add double grad for conv_transpose

* Test compilation time with less parallel count, notest, test=windows_ci

* optimize rules of Unity Build, notest, test=windows_ci, test=windows_op

* limit parallel counts used only on GPU, test=develop

* remove limit of argument /m:8 on Windows, test=develop

* - Reimplemented elementwise_add grad

- lint

* - fix after review

* - Fix to fix after review

fix isfinite_v2_op OpProtoAndCheckerMaker AddComment bug 

* add conj op for complex types

* add conj for complex types

* add more test case

* add conj_op test

* modify conj api and impl

* add complex type for fill_constant_op xpu

* add setConstant for complex type

* remove complex conj test file

* user define grad for test_conj_op

* add test case for static mode of conj api

* modify conj doc

* change input args name to x

* remove useless codes

* conj support real types

* add conj test case for real number

* enable_use_gpu has higher priority than FLAGS

* update.