提交 · 4d3b7d7d587deecea68c0add417a03eec5f9e61d · PaddlePaddle / Paddle

02 6月, 2022 8 次提交

[XPUPS] modify BKCL comm op register (#43028) · 1bfbcfaf

由 Fan Zhang 提交于 6月 02, 2022

* Adapt XPUPS - 1st version - 3.24

* Adapt XPUPS - update XPU PushSparse -  2nd version - 3.24

* Adapt XPUPS - add XPU PullSparseOp - 3nd version - 3.25

* refactor heter comm kernel

* update. test=develop

* Adapt XPUPS - modify by compilation - 4th version - 3.27

* update calc_shard_offset. test=develop

* update xpu kernel. test=develop

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* heter_comm update

* heter_comm update

* update calc_shard_offset. test=develop

* heter_comm update

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* fix. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* Adapt XPUPS - use WITH_XPU_KP and modify wrapper kernel function - 5th version - 3.30

* update. test=develop

* update pslib.cmake

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* Adapt XPUPS - modify by kp compilation  - 6th version - 3.30

* update. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* fix. test=develop

* fix. test=develop

* used by minxu

* update heter_comm_inl

* fix. test=develop

* Adapt XPUPS - modify by kp compilation  - 7th version - 3.30

* fix. test=develop

* add optimizer kernel. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 3.31 update

* Adapt XPUPS - update kp compilation path  - 8th version - 3.31

* add optimizer kernel. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update heter_comm_kernel.kps 3.31

* fix. test=develop

* fix. test=develop

* update heter_comm_kernel.kps 3.31

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update heter_comm.h 3.31

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update hashtable. test=develop

* update. test=develop

* Adapt XPUPS - update by kp compilation  - 9th version - 4.1

* update hashtable. test=develop

* fix. test=develop

* update hashtable 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 10th version - 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update. test=develop

* modify by compilation 4.1

* update. test=develop

* update. test=develop

* fix. test=develop

* modify by compilation 4.1

* update. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* modify by compilation 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* modify by compilation 4.1 19:30

* fix. test=develop

* update ps_gpu_wrapper.kps 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 11th version - 4.1

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 12nd version - 4.2

* fix. test=develop

* fix. test=develop

* modify by compilation 4.2

* 4.2 update

* fix. test=develop

* template init. test=develop

* update 4.6

* fix. test=develop

* template init. test=develop

* 4.6 modify by compilation

* hashtable template init. test=develop

* hashtable template init. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 13nd version - 4.7

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 4.11 update

* fix. test=develop

* fix. test=develop

* 4.11 update

* update by pre-commit

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 4.12 update

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 14th version - 4.13

* 4.13 update

* 4.14 update

* 4.14 update

* 4.14 update

* 4.14 modify by merged latest compilation

* retry CI 4.14

* 4.15 pass static check

* 4.15 modify by gpups CI

* 3.16 update by gpups CI - modify ps_gpu_wrapper.h

* 4.16 update

* 4.16 pass xpu compile

* 4.16 retry CI

* 4.16 update

* Adapt XPUPS - adapt BKCL comm for XPUPS - 4.24

* update by compilation

* Adapt XPUPS - register PSGPUTrainer for XPUPS - 4.25

* update device_worker_factory

* Adapt XPUPS - split heter_ps into .cu and .cc - 4.27

* Adapt XPUPS - register pull_box_sparse op under XPU_KP - 4.28

* update

* 5.7 modify ps_gpu_wrapper pull_sparse

* 5.11 update ps_gpu_wrapper CopyKeysKernel

* 5.13 modify calc_shard_offset_kernel & fill_shard_key_kernel

* modify fill_dvals_kernel & PullCopy & c_sync_calc_stream - 5.18

* modify PushCopy & fill_shard_grads_kernel & register push_box_sparse - 5.19

* Adapt XPUPS - modify BKCL comm op register - 5.26

* Adapt XPUPS - modify BKCL comm op register - 5.27

* Adapt XPUPS - modify BKCL comm op register - 5.27v2

* Adapt XPUPS - modify BKCL comm op register - 5.27v3

* Adapt XPUPS - modify c_comm_init_all_op to adapt BKCL init - 5.30

* Adapt XPUPS - modify c_comm_init_all_op to adapt BKCL init v2 - 5.30

* Adapt XPUPS - modify c_comm_init_all_op to adapt BKCL init v3 - 5.30

* Adapt XPUPS - modify c_comm_init_all_op to adapt BKCL init v4 - 5.31
Co-authored-by: Nzmxdream <zhangminxu01@baidu.com>

1bfbcfaf

Z
Support head_dim = 96 in fused_multi_transformer for PLATO-XL (#43120) · 990c5e7f
由 Zhang Zheng 提交于 6月 02, 2022
```
* Support head_dim = 96 in fused_multi_transformer in PLATO-XL

* add notes
```
990c5e7f
[MLU]add mlu kernel for squeeze and squeeze2 (#43094) · 041000c2
由光明和真理提交于 6月 02, 2022
```
Co-authored-by: Nliupeiyu <liupeiyu@cambricon.com>
```
041000c2
C

add concat_grad mlu kernel. (#43117) · fe911a51
由 Chenxiao Niu 提交于 6月 02, 2022

fe911a51
Z
Delete inplace strategy in group_norm_fwd (#43137) · 0cb9dae5
由 Zhang Zheng 提交于 6月 02, 2022
```
* Delete inplace strategy in group_norm_fwd

* fix
```
0cb9dae5
G

fix the bug of margin cross entropy loss for eager mode (#43161) · 67163fb4
由 Guoxia Wang 提交于 6月 02, 2022

67163fb4
L
Extend forward fast layer_norm kernel to support more dimensions. (#43118) · 85baa3c0
由 Li Min 提交于 6月 02, 2022
```
* extend forward fast_ln_kernel to support more column values.
```
85baa3c0

Support CUDA Graph for partial graph in dygraph mode (#42786) · d05b940a

由 sneaxiy 提交于 6月 02, 2022

* support CUDAGraph for partial graph

* add ut

* fix ci

* fix ut again because of eager mode

* fix kunlun ci

* fix win ci

d05b940a

01 6月, 2022 2 次提交
- G
  
  support nccl api for bfloat16, required >= cudnn 10.1, nccl >= 2.10.3 (#43147) · 67b9b51b
  由 Guoxia Wang 提交于 6月 01, 2022
  
  67b9b51b
- S
  Make fuse_gemm_epilogue support transpose_x and transpose_y (#40558) · 048b0013
  由 sneaxiy 提交于 6月 01, 2022
```
* support weight transpose

* add ut

* add template

* fix transpose error

* fix transpose_comment

* add api tests

* add skipif

* add doc
```
  048b0013
31 5月, 2022 7 次提交

Remove mkldnn attributes from base ops (#42852) · 4b89120b

由 Sławomir Siwek 提交于 5月 31, 2022

* remove attrs from base op

* fix typos

* remove brelu

* undo removing code related to matmul

* remove whitespaces

* undo changes in matmul

* remove empty line

4b89120b

C

[MLU] add mlu kernel for abs op (#43099) · cb195fa0
由 cambriconhsq 提交于 5月 31, 2022

cb195fa0
A
[NPU] fix arg_max and reduce_max (#42887) · f9e55dee
由 Aganlengzi 提交于 5月 31, 2022
```
* fix arg_max and reduce_max

* add arg_max ut
```
f9e55dee

【PaddlePaddle Hackathon 2】16 新增 API RRelu (#41823) · 21e1d10f

由 thunder95 提交于 5月 31, 2022

* rrelu逻辑部分

* unregistered op kernel (unresolved)

* commit before merge

* 丰富测试用例

* 修复rrelu-sig的bug

* 修复cpu环境测试

* 修改拼写错误

* 修改code format

* 尝试优化测试用例timeout的问题

* 优化测试用例

* 移除seed, 优化随机函数

* update en doc for rrelu

* fix rrelu en docs, test=document_fix

* add paper link for en docs, test=document_fix

* udpate en doc

* add r,test=document_fix

21e1d10f

L
Fix the underflow of fp16 fake quantize operators (#43088) · 0ae8a2d6
由 Leo Chen 提交于 5月 31, 2022
```
Co-authored-by: NRyan Jeng <rjeng@nvidia.com>
```
0ae8a2d6
L
Rename dropout is test (#43098) · 67497119
由 Li Min 提交于 5月 31, 2022
```
* replace dropout_is_test with is_test.
* improve atol on a100.
```
67497119

OneDNN md-in-tensor refactoring part 5: Memory descriptor enabled for... · 12d8a567

由 jakpiase 提交于 5月 30, 2022

OneDNN md-in-tensor refactoring part 5: Memory descriptor enabled for elementwises, reductions and expand_v2 ops (#43036)

* enabled md in elementwises, reductions and expand_v2

* CI fix for invalid numpy copy

* fixed formatting

* CI rerun

* changes after review

12d8a567

30 5月, 2022 6 次提交

C

[mlu] add one_hot_v2 mlu kernel (#43025) · 13a21cf7
由 Chenxiao Niu 提交于 5月 30, 2022

13a21cf7
L
Add fused_bias_dropout_residual_ln op and layer. (#43062) · dceccd9d
由 Li Min 提交于 5月 30, 2022
```
* add fused_bias_dropout_residual_ln op and layer.
```
dceccd9d
C

Implement fused_gate_attention operator for AlphaFold. (#42018) · fdcdbec5
由 crystal 提交于 5月 30, 2022

fdcdbec5

【PaddlePaddle Hackathon 2】15 新增 API Nanmedian (#42385) · f87fa3c0

由 thunder95 提交于 5月 30, 2022

* nanmedian op

* 修改cuda kernel的bug

* 修复count_if在其他硬件平台不兼容

* 修复某些cpu硬件不兼容

* 修复某些cpu硬件不兼容

* 修复isnan判断

* 兼容numpy低版本不支持全部nan的情况

* 兼容numpy低版本不支持全部nan的情况

* fix code example

* fix api comment error

* 修改反向传播逻辑以及c++处理逻辑

* 完成修改建议

* typo pre_dim

* update en docs, test=document_fix

* remove numpy in en doc, test=document_fix

* add r,test=document_fix

* 添加api到all

* follow advice from chenwhql

f87fa3c0

C

[MLU]add mlu kernel for log_softmax op (#43040) · 586f9429
由 cambriconhsq 提交于 5月 30, 2022

586f9429
W
[Dy2St]Fix cond_block_grad error when handle no need grad vras (#43034) · cd3d0911
由 WangZhen 提交于 5月 30, 2022
```
* Fix cond_block_grad error when handle no need grad vras

* Add comment and UT
```
cd3d0911

28 5月, 2022 1 次提交
- S
  [Bug Fix]Fix global_scatter/global_gather in ProcessGroup (#43027) · 8cc2e28c
  由 ShenLiang 提交于 5月 28, 2022
```
* fix alltoall

* rename utest
```
  8cc2e28c
27 5月, 2022 2 次提交

[Phi] Change optional tensor from `optional<const Tensor&>` to `optional<Tensor>` (#42939) · 6d78524c

由 zyfncg 提交于 5月 27, 2022

* refactor the optional tensor

* remove optiona<MetaTensor> in InferMeta

* fix bug

* fix optional<vector<Tensor>>

* fix bug

* fix rmsprop

* fix amp of eager_gen

* polish code

* fix deleted code

* fix merge conflict

* polish code

* remove is_nullopt_

* fix merge conflict

* fix merge conflict

6d78524c

R
Support memory stats for CPU (#42945) · 21f11d35
由 Ruibiao Chen 提交于 5月 27, 2022
```
* Support memory stats for CPU

* Add UTs

* Fix typos

* Fix typos
```
21f11d35

26 5月, 2022 2 次提交
- Y
  
  move instance_norm_double_grad (#43021) · b2b78cd4
  由 YuanRisheng 提交于 5月 26, 2022
  
  b2b78cd4
- Y
  [Phi]Refactor InstanceNormKernel and InstanceNormGradKernel (#42978) · cc272afb
  由 YuanRisheng 提交于 5月 26, 2022
```
* move instance_norm

* change mutable_data

* fix compile bugs
```
  cc272afb
25 5月, 2022 4 次提交

J
OneDNN md-in-tensor refactoring part 4: Memory descriptor enabled for more ops (#42946) · 657abd51
由 jakpiase 提交于 5月 25, 2022
```
* added support for md in more ops

* fixed typo
```
657abd51

fix maybe-uninitialized warning (#42902) · f1f79b0d

由 Leo Chen 提交于 5月 25, 2022

* fix maybe-uninitialized warning

* fix compile

* fix xpu compile

* fix npu compile

* fix infer compile

* fix compile

* fix compile

f1f79b0d

F

[MLU] adapt coalesce_tensor op for mlu (#42873) · cbb24136
由 fwenguang 提交于 5月 25, 2022

cbb24136

[EinsumOp] Optimize the backward speed of EinsumOp (#42663) · 71b046cd

由 xiongkun 提交于 5月 25, 2022

* change logic for optimize

* modifty

* optimize the backward speed of EinsumOp

* add cache optimizer for einsum op

* EinsumOp: fix new dygraph mode error

* fix bug

* change Cache->InnerCache

* fix code

* fix

* add nan inf utils for einsum op

* add as_extra

* Compatible with v2.3 EinsumOp

* remove dispensable

71b046cd

24 5月, 2022 2 次提交

Y
[Phi]Move grad_add op kernel into phi and delete elementwise_add_op file (#42903) · 4d7a9eef
由 YuanRisheng 提交于 5月 24, 2022
```
* move grad_add

* fix unittest bugs

* fix compile bugs
```
4d7a9eef

[XPUPS] Modify XPU Kernel (#42745) · f8931c97

由 Fan Zhang 提交于 5月 24, 2022

* Adapt XPUPS - 1st version - 3.24

* Adapt XPUPS - update XPU PushSparse -  2nd version - 3.24

* Adapt XPUPS - add XPU PullSparseOp - 3nd version - 3.25

* refactor heter comm kernel

* update. test=develop

* Adapt XPUPS - modify by compilation - 4th version - 3.27

* update calc_shard_offset. test=develop

* update xpu kernel. test=develop

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* heter_comm update

* heter_comm update

* update calc_shard_offset. test=develop

* heter_comm update

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* fix. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* Adapt XPUPS - use WITH_XPU_KP and modify wrapper kernel function - 5th version - 3.30

* update. test=develop

* update pslib.cmake

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* Adapt XPUPS - modify by kp compilation  - 6th version - 3.30

* update. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* fix. test=develop

* fix. test=develop

* used by minxu

* update heter_comm_inl

* fix. test=develop

* Adapt XPUPS - modify by kp compilation  - 7th version - 3.30

* fix. test=develop

* add optimizer kernel. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 3.31 update

* Adapt XPUPS - update kp compilation path  - 8th version - 3.31

* add optimizer kernel. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update heter_comm_kernel.kps 3.31

* fix. test=develop

* fix. test=develop

* update heter_comm_kernel.kps 3.31

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update heter_comm.h 3.31

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update hashtable. test=develop

* update. test=develop

* Adapt XPUPS - update by kp compilation  - 9th version - 4.1

* update hashtable. test=develop

* fix. test=develop

* update hashtable 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 10th version - 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update. test=develop

* modify by compilation 4.1

* update. test=develop

* update. test=develop

* fix. test=develop

* modify by compilation 4.1

* update. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* modify by compilation 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* modify by compilation 4.1 19:30

* fix. test=develop

* update ps_gpu_wrapper.kps 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 11th version - 4.1

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 12nd version - 4.2

* fix. test=develop

* fix. test=develop

* modify by compilation 4.2

* 4.2 update

* fix. test=develop

* template init. test=develop

* update 4.6

* fix. test=develop

* template init. test=develop

* 4.6 modify by compilation

* hashtable template init. test=develop

* hashtable template init. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 13nd version - 4.7

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 4.11 update

* fix. test=develop

* fix. test=develop

* 4.11 update

* update by pre-commit

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 4.12 update

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 14th version - 4.13

* 4.13 update

* 4.14 update

* 4.14 update

* 4.14 update

* 4.14 modify by merged latest compilation

* retry CI 4.14

* 4.15 pass static check

* 4.15 modify by gpups CI

* 3.16 update by gpups CI - modify ps_gpu_wrapper.h

* 4.16 update

* 4.16 pass xpu compile

* 4.16 retry CI

* 4.16 update

* Adapt XPUPS - adapt BKCL comm for XPUPS - 4.24

* update by compilation

* Adapt XPUPS - register PSGPUTrainer for XPUPS - 4.25

* update device_worker_factory

* Adapt XPUPS - split heter_ps into .cu and .cc - 4.27

* Adapt XPUPS - register pull_box_sparse op under XPU_KP - 4.28

* update

* 5.7 modify ps_gpu_wrapper pull_sparse

* 5.11 update ps_gpu_wrapper CopyKeysKernel

* 5.13 modify calc_shard_offset_kernel & fill_shard_key_kernel

* modify fill_dvals_kernel & PullCopy & c_sync_calc_stream - 5.18

* modify PushCopy & fill_shard_grads_kernel & register push_box_sparse - 5.19
Co-authored-by: Nzmxdream <zhangminxu01@baidu.com>

f8931c97

23 5月, 2022 2 次提交

[Internal reviewing] NHWC fix to am_vocoder model for oneDNN 2.6 (#42729) · d414af94

由 Jacek Czaja 提交于 5月 23, 2022

* - prototype of reimplemented fixes

* - compilation fixes

* - compilation fix

* - cosmetic info

* - hopefully fix

* - compilation fix

* - supported for nested blocking of cache clearing

* - fix

* - Unit test to changes

* - Compilation fix to windows (hopefully)

* - Moved resetting layout to ResetBlob

* - fixes after review

d414af94

remove is_init_py of RandomGenerator, and use Global RandomGenerator by default (#42876) · 3b488bae
由 zhouweiwei2014 提交于 5月 23, 2022
```
* remove is_init_py of RandomGenerator, and use Global Generator if not OP seed

* fix comment
```
3b488bae

20 5月, 2022 4 次提交

W

fix fused_attention_op cacheKV InferShape (#42900) · 7306d1fb
由 WangXi 提交于 5月 20, 2022

7306d1fb
Y

move activation kernel (#42880) · 191c441a
由 YuanRisheng 提交于 5月 20, 2022

191c441a

[Hackathon No.5] tril_indices OP (#41639) · 75db5b86

由 xiaoguoguo626807 提交于 5月 20, 2022

* add tril_indices cpu kernal

* modify tril_indice cpu op

* modify bug

* modify bug

* add tril_indices python api

* add tril_indices python api

* resolve conflict

* add tril_indices test

* modify details

* add tril_indices.cu

* pythonapi pass

* save tril_indices

* CPU tril_indices pass

* delete vlog

* modify test_tril_indices_op.py

* delete tril_indices_kernel.cc.swp

* delete tril_indice.cu

* modify code style

* add newline in creation.py

* modify creation.py linux newline

* delete annotation

* check code style

* check .py style add final_state??

* modify code style

* add gpu_tril_indices

* modify gpu_compiled_juage

* modify gpu judge

* code style

* add test example

* modify english document

modify english document

modify english document

modify document

modify document

* modify pram name

* modify pram name

* modify pram

* reduce test ex

75db5b86

Y
merge dymf branch (#42714) · 3f619290
由 yaoxuefeng 提交于 5月 20, 2022
```
merge dymf branch
```
3f619290

PaddlePaddle / Paddle 大约 1 年 前同步成功

PaddlePaddle / Paddle
大约 1 年前同步成功