* add erfinv API, test=develop

* fix gradient accuracy error, test=develop

* fix cuda compilation error on Windows, test=develop

* fix M_2_SQRTPI undeclared identifier on Windows, test=develop

* add align for WorkQueue

* add spinlock

* merge develop

* merge

* Add EventsWaiter

* Revert "Add EventsWaiter"

This reverts commit e206173aa9be7401b83a53581627bfaf557c8fb2.

* update EventsWater

* fix

* split workqueue files

* add more tests

* fix

* bugfix

* bugfix

* update
Co-authored-by: Nliutiexing <liutiexing@google.com>

* add empty and empty_like kernel in pten

* add empty dev_api

* support external stream.

* update

* update

* update

* add mkldnn conv_elementwise_add_mkldnn_fuse_pass ut

* update mkldnn conv_elementwise_add_mkldnn_fuse_pass ut

* update conv_elementwise_add_mkldnn_fuse_pass ut

* update conv_elementwise_add_mkldnn_fuse_pass ut

* update conv_elementwise_add_mkldnn_fuse_pass ut

* restrict conv2d data_format in conv_elementwise_add_mkldnn_fuse_pass

* update conv_elementwise_add_mkldnn_fuse_pass OpCompat

* update conv_elementwise_add_mkldnn_fuse_pass ut

* update ut

* add new API: paddle.clone;Tensor.element_size;nn.utils.parameters_to_vector

* fix comment

* add flatten2_matmul squeeze2_matmul reshape2_matmul test case

* modify skip func to ignore_pass_case func

* rebuild CI

* add test_xx_matmul_fuse_pass timeout

* add test_map_xx_pass timeout

* add max_duration of test cast

* add trt skip

* add timeout

* del commented code

* optimize gelu backward

* optimize gelu backward

* optimize code

* Number to expression

* Replacement number

* add mkldnn reshape_transpose_matmul fuse pass ut and op version check

* update reshape_transpose_matmul_mkldnn_fuse_pass ut

* update ut

* update mkldnn batch_norm_activation fuse pass ut

* update ut

* update mkldnn batch_norm_act_fuse_pass ut

* update batch_norm_act_fuse_pass ut

* update ut

* move flatten

* fix bugs of test

* modify header file

* add copy declare

* fix compile bugs

* CE fix

* more format

* add inplace_map for trace_op in pybind

* fix inplace problem of setitem

* refactor the param format  of trace_op
Co-authored-by: Npangyoki <pangyoki@126.com>

* update seqconv_eltadd_relu_fuse_pass ut

* update ut

* update ut

* update ut

* update squared_mat_sub_fuse_pass ut

* update ut

* update ut

* rename cuda to gpu

* revert CMake change

* resolve conflit

* rename other cuda to gpu

* poish details

* relu forward opt

* add gelu functor

* optimize code

* add seqpool_cvm_concat_fuse_pass ut

* rename ut name

* mean first version

* fix scalar mean

* add fp16 dtype for api

* add mkldnn conv_transpose_bias fuse pass ut

* update conv_transpose_bias_mkldnn_fuse_pass ut

* update conv_transpose_bias_mkldnn_fuse_pass ut

* update conv_transpose_bias_mkldnn_fuse_pass ut

* restrict conv2d data_format in conv_transpose_bias_mkldnn_fuse_pass

* update ut timeout setting

* update ut

* add pten conj kernel

* modify conj_kernel file path

* add defined cuda macro to cuda/conj_kernel.h

* support FP16 for more ops

* add amp list tests

* refine reduce_mean_grad

* fix OP benchmark ci

* fix fp16 reduce_mean

* updat ut, but still have some problems

* remove mean/reduce_mean fp16 kernel

softmax_with_cross_entropy optimization with soft label. This PR includes optimization of
    "SoftmaxWithCrossEntropySoftLabel" : compute log_softmax and then compute loss.
    "CrossEntropySoftLabel" : compute loss with softmax as input.
These optimization includes following technics:
    read data to buffer with vectorization
    compute max and sum in warp
    fixed loop size with macro
Performance (computation time):
    softmax_with_cross_entropy_0 (forward) : -40.1%
    softmax_with_cross_entropy_0 (backward): -41%

* add matmul_scale matmul_v2_scale fuse pass

* add scaletensor judge

* modify var name

* add timeout notest;test=coverag

* fix error commit

* fix use_mkldnn attr

* fix use_mkldnn attr

* add multi_tensor for momentum and clear_grads for optimizer

* fix bug for dygraph

* add unittest

* refine comment

* add param_group

* refine regularizaiton logic

* del clear_grads

* add clear_grads

* add dispensable check of None

* refine clear_grad

* fix build bug

* refine code by comment

* refine code

* add multi tensor check

* refine param_group update

* add multi tensor for static mode

* refine comments

* delete useless comma for momentum

* refine comment for momentum

* refine code by commment

* fix bugs when run reshape

* fix ci bug