提交 · b056c9cb87151920e7e15f9e45048154fa6a024e · 机器未来 / Paddle

08 6月, 2022 1 次提交

Fix wrong reduce_dims in fused_gate_attention and optimize the memory usage. (#43216) · 10f8637c

由 Yiqun Liu 提交于 6月 08, 2022

* Polish codes and memory usage for fused_gate_attention.

* Fix wrong reduce_dims in fused_gate_attention when computing gradient of nonbatched_bias.

10f8637c

07 6月, 2022 8 次提交
- S
  Matmul post-ops for fuses (#43198) · 5434d663
  由 Sławomir Siwek 提交于 6月 07, 2022
```
* add method for post ops

* format code

* change post-ops pattern

* code style
```
  5434d663
- S
  
  Optimized the performance of activation op in XPU2 (#43187) · d5afc1ba
  由 shixingbo 提交于 6月 07, 2022
  
  d5afc1ba
- S
  Add use_master_acc_grad for DistributedFusedLamb (#43266) · 601d7a35
  由 sneaxiy 提交于 6月 07, 2022
```
* add use_master_acc_grad

* add ut
```
  601d7a35
- Q
  [MLU]support cast double type (#43058) · 42dd0f1b
  由 qipengh 提交于 6月 07, 2022
```
* [MLU]support cast double type

* [MLU]fix cast test
```
  42dd0f1b
- L
  Transpose optimization with assitant of Chengdu Supercomputing Center and... · 71a63f0a
  由 limingshu 提交于 6月 07, 2022
```
Transpose optimization with assitant of  Chengdu Supercomputing Center and auto_tune operation (#42704)
```
  71a63f0a
- N
  
  [XPU KP]Add xpu register, any, amax, amin op test (#43204) · aec49361
  由 niuliling123 提交于 6月 07, 2022
  
  aec49361
- S
  
  fix dropout (#43234) · a2020d0c
  由 sneaxiy 提交于 6月 07, 2022
  
  a2020d0c
- Z
  
  Supoort more dimensions in forward fast layer_norm kernel (#43226) · d9f8636c
  由 Zhang Zheng 提交于 6月 07, 2022
  
  d9f8636c
06 6月, 2022 1 次提交
- N
  
  Replace ReduceAmax/Amax.part.cu with KP (#43202) · 39903f72
  由 niuliling123 提交于 6月 06, 2022
  
  39903f72
05 6月, 2022 2 次提交
- S
  
  revert modification for format PR passing CI; not sort headers for windows CI test failed (#43200) · 58d2949d
  由 Sing_chan 提交于 6月 05, 2022
  
  58d2949d
- S
  
  【code format check upgrade】 step2：clang-format (#42840) · a3730dc8
  由 Sing_chan 提交于 6月 05, 2022
  
  a3730dc8
04 6月, 2022 1 次提交
- S
  
  【code format check upgrade】 step2：cmake-format (#43057) · 92568edb
  由 Sing_chan 提交于 6月 04, 2022
  
  92568edb
02 6月, 2022 9 次提交

Add generate_proposals_v2 op and expend function of gather op for kunlun. *test=kunlun (#43162) · ff22a9c4

由 Leo Guo 提交于 6月 02, 2022

* Add generate_proposals_v2 op and unittest for kunlun. *test=kunlun

* Add the assign op to xpu2_op_list and expand the function of gather op. Add the unit-test of generate_proposals_v2. *test=kunlun

ff22a9c4

[XPUPS] modify BKCL comm op register (#43028) · 1bfbcfaf

由 Fan Zhang 提交于 6月 02, 2022

* Adapt XPUPS - 1st version - 3.24

* Adapt XPUPS - update XPU PushSparse -  2nd version - 3.24

* Adapt XPUPS - add XPU PullSparseOp - 3nd version - 3.25

* refactor heter comm kernel

* update. test=develop

* Adapt XPUPS - modify by compilation - 4th version - 3.27

* update calc_shard_offset. test=develop

* update xpu kernel. test=develop

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* heter_comm update

* heter_comm update

* update calc_shard_offset. test=develop

* heter_comm update

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* fix. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* Adapt XPUPS - use WITH_XPU_KP and modify wrapper kernel function - 5th version - 3.30

* update. test=develop

* update pslib.cmake

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* Adapt XPUPS - modify by kp compilation  - 6th version - 3.30

* update. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* fix. test=develop

* fix. test=develop

* used by minxu

* update heter_comm_inl

* fix. test=develop

* Adapt XPUPS - modify by kp compilation  - 7th version - 3.30

* fix. test=develop

* add optimizer kernel. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 3.31 update

* Adapt XPUPS - update kp compilation path  - 8th version - 3.31

* add optimizer kernel. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update heter_comm_kernel.kps 3.31

* fix. test=develop

* fix. test=develop

* update heter_comm_kernel.kps 3.31

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update heter_comm.h 3.31

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update hashtable. test=develop

* update. test=develop

* Adapt XPUPS - update by kp compilation  - 9th version - 4.1

* update hashtable. test=develop

* fix. test=develop

* update hashtable 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 10th version - 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update. test=develop

* modify by compilation 4.1

* update. test=develop

* update. test=develop

* fix. test=develop

* modify by compilation 4.1

* update. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* modify by compilation 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* modify by compilation 4.1 19:30

* fix. test=develop

* update ps_gpu_wrapper.kps 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 11th version - 4.1

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 12nd version - 4.2

* fix. test=develop

* fix. test=develop

* modify by compilation 4.2

* 4.2 update

* fix. test=develop

* template init. test=develop

* update 4.6

* fix. test=develop

* template init. test=develop

* 4.6 modify by compilation

* hashtable template init. test=develop

* hashtable template init. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 13nd version - 4.7

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 4.11 update

* fix. test=develop

* fix. test=develop

* 4.11 update

* update by pre-commit

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 4.12 update

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 14th version - 4.13

* 4.13 update

* 4.14 update

* 4.14 update

* 4.14 update

* 4.14 modify by merged latest compilation

* retry CI 4.14

* 4.15 pass static check

* 4.15 modify by gpups CI

* 3.16 update by gpups CI - modify ps_gpu_wrapper.h

* 4.16 update

* 4.16 pass xpu compile

* 4.16 retry CI

* 4.16 update

* Adapt XPUPS - adapt BKCL comm for XPUPS - 4.24

* update by compilation

* Adapt XPUPS - register PSGPUTrainer for XPUPS - 4.25

* update device_worker_factory

* Adapt XPUPS - split heter_ps into .cu and .cc - 4.27

* Adapt XPUPS - register pull_box_sparse op under XPU_KP - 4.28

* update

* 5.7 modify ps_gpu_wrapper pull_sparse

* 5.11 update ps_gpu_wrapper CopyKeysKernel

* 5.13 modify calc_shard_offset_kernel & fill_shard_key_kernel

* modify fill_dvals_kernel & PullCopy & c_sync_calc_stream - 5.18

* modify PushCopy & fill_shard_grads_kernel & register push_box_sparse - 5.19

* Adapt XPUPS - modify BKCL comm op register - 5.26

* Adapt XPUPS - modify BKCL comm op register - 5.27

* Adapt XPUPS - modify BKCL comm op register - 5.27v2

* Adapt XPUPS - modify BKCL comm op register - 5.27v3

* Adapt XPUPS - modify c_comm_init_all_op to adapt BKCL init - 5.30

* Adapt XPUPS - modify c_comm_init_all_op to adapt BKCL init v2 - 5.30

* Adapt XPUPS - modify c_comm_init_all_op to adapt BKCL init v3 - 5.30

* Adapt XPUPS - modify c_comm_init_all_op to adapt BKCL init v4 - 5.31
Co-authored-by: Nzmxdream <zhangminxu01@baidu.com>

1bfbcfaf

Z
Support head_dim = 96 in fused_multi_transformer for PLATO-XL (#43120) · 990c5e7f
由 Zhang Zheng 提交于 6月 02, 2022
```
* Support head_dim = 96 in fused_multi_transformer in PLATO-XL

* add notes
```
990c5e7f
[MLU]add mlu kernel for squeeze and squeeze2 (#43094) · 041000c2
由光明和真理提交于 6月 02, 2022
```
Co-authored-by: Nliupeiyu <liupeiyu@cambricon.com>
```
041000c2
C

add concat_grad mlu kernel. (#43117) · fe911a51
由 Chenxiao Niu 提交于 6月 02, 2022

fe911a51
Z
Delete inplace strategy in group_norm_fwd (#43137) · 0cb9dae5
由 Zhang Zheng 提交于 6月 02, 2022
```
* Delete inplace strategy in group_norm_fwd

* fix
```
0cb9dae5
G

fix the bug of margin cross entropy loss for eager mode (#43161) · 67163fb4
由 Guoxia Wang 提交于 6月 02, 2022

67163fb4
L
Extend forward fast layer_norm kernel to support more dimensions. (#43118) · 85baa3c0
由 Li Min 提交于 6月 02, 2022
```
* extend forward fast_ln_kernel to support more column values.
```
85baa3c0

Support CUDA Graph for partial graph in dygraph mode (#42786) · d05b940a

由 sneaxiy 提交于 6月 02, 2022

* support CUDAGraph for partial graph

* add ut

* fix ci

* fix ut again because of eager mode

* fix kunlun ci

* fix win ci

d05b940a

01 6月, 2022 2 次提交
- G
  
  support nccl api for bfloat16, required >= cudnn 10.1, nccl >= 2.10.3 (#43147) · 67b9b51b
  由 Guoxia Wang 提交于 6月 01, 2022
  
  67b9b51b
- S
  Make fuse_gemm_epilogue support transpose_x and transpose_y (#40558) · 048b0013
  由 sneaxiy 提交于 6月 01, 2022
```
* support weight transpose

* add ut

* add template

* fix transpose error

* fix transpose_comment

* add api tests

* add skipif

* add doc
```
  048b0013
31 5月, 2022 7 次提交

Remove mkldnn attributes from base ops (#42852) · 4b89120b

由 Sławomir Siwek 提交于 5月 31, 2022

* remove attrs from base op

* fix typos

* remove brelu

* undo removing code related to matmul

* remove whitespaces

* undo changes in matmul

* remove empty line

4b89120b

C

[MLU] add mlu kernel for abs op (#43099) · cb195fa0
由 cambriconhsq 提交于 5月 31, 2022

cb195fa0
A
[NPU] fix arg_max and reduce_max (#42887) · f9e55dee
由 Aganlengzi 提交于 5月 31, 2022
```
* fix arg_max and reduce_max

* add arg_max ut
```
f9e55dee

【PaddlePaddle Hackathon 2】16 新增 API RRelu (#41823) · 21e1d10f

由 thunder95 提交于 5月 31, 2022

* rrelu逻辑部分

* unregistered op kernel (unresolved)

* commit before merge

* 丰富测试用例

* 修复rrelu-sig的bug

* 修复cpu环境测试

* 修改拼写错误

* 修改code format

* 尝试优化测试用例timeout的问题

* 优化测试用例

* 移除seed, 优化随机函数

* update en doc for rrelu

* fix rrelu en docs, test=document_fix

* add paper link for en docs, test=document_fix

* udpate en doc

* add r,test=document_fix

21e1d10f

L
Fix the underflow of fp16 fake quantize operators (#43088) · 0ae8a2d6
由 Leo Chen 提交于 5月 31, 2022
```
Co-authored-by: NRyan Jeng <rjeng@nvidia.com>
```
0ae8a2d6
L
Rename dropout is test (#43098) · 67497119
由 Li Min 提交于 5月 31, 2022
```
* replace dropout_is_test with is_test.
* improve atol on a100.
```
67497119

OneDNN md-in-tensor refactoring part 5: Memory descriptor enabled for... · 12d8a567

由 jakpiase 提交于 5月 30, 2022

OneDNN md-in-tensor refactoring part 5: Memory descriptor enabled for elementwises, reductions and expand_v2 ops (#43036)

* enabled md in elementwises, reductions and expand_v2

* CI fix for invalid numpy copy

* fixed formatting

* CI rerun

* changes after review

12d8a567

30 5月, 2022 6 次提交

C

[mlu] add one_hot_v2 mlu kernel (#43025) · 13a21cf7
由 Chenxiao Niu 提交于 5月 30, 2022

13a21cf7
L
Add fused_bias_dropout_residual_ln op and layer. (#43062) · dceccd9d
由 Li Min 提交于 5月 30, 2022
```
* add fused_bias_dropout_residual_ln op and layer.
```
dceccd9d
C

Implement fused_gate_attention operator for AlphaFold. (#42018) · fdcdbec5
由 crystal 提交于 5月 30, 2022

fdcdbec5

【PaddlePaddle Hackathon 2】15 新增 API Nanmedian (#42385) · f87fa3c0

由 thunder95 提交于 5月 30, 2022

* nanmedian op

* 修改cuda kernel的bug

* 修复count_if在其他硬件平台不兼容

* 修复某些cpu硬件不兼容

* 修复某些cpu硬件不兼容

* 修复isnan判断

* 兼容numpy低版本不支持全部nan的情况

* 兼容numpy低版本不支持全部nan的情况

* fix code example

* fix api comment error

* 修改反向传播逻辑以及c++处理逻辑

* 完成修改建议

* typo pre_dim

* update en docs, test=document_fix

* remove numpy in en doc, test=document_fix

* add r,test=document_fix

* 添加api到all

* follow advice from chenwhql

f87fa3c0

C

[MLU]add mlu kernel for log_softmax op (#43040) · 586f9429
由 cambriconhsq 提交于 5月 30, 2022

586f9429
W
[Dy2St]Fix cond_block_grad error when handle no need grad vras (#43034) · cd3d0911
由 WangZhen 提交于 5月 30, 2022
```
* Fix cond_block_grad error when handle no need grad vras

* Add comment and UT
```
cd3d0911

28 5月, 2022 1 次提交
- S
  [Bug Fix]Fix global_scatter/global_gather in ProcessGroup (#43027) · 8cc2e28c
  由 ShenLiang 提交于 5月 28, 2022
```
* fix alltoall

* rename utest
```
  8cc2e28c
27 5月, 2022 2 次提交

[Phi] Change optional tensor from `optional<const Tensor&>` to `optional<Tensor>` (#42939) · 6d78524c

由 zyfncg 提交于 5月 27, 2022

* refactor the optional tensor

* remove optiona<MetaTensor> in InferMeta

* fix bug

* fix optional<vector<Tensor>>

* fix bug

* fix rmsprop

* fix amp of eager_gen

* polish code

* fix deleted code

* fix merge conflict

* polish code

* remove is_nullopt_

* fix merge conflict

* fix merge conflict

6d78524c

R
Support memory stats for CPU (#42945) · 21f11d35
由 Ruibiao Chen 提交于 5月 27, 2022
```
* Support memory stats for CPU

* Add UTs

* Fix typos

* Fix typos
```
21f11d35

机器未来 / Paddle 与 Fork 源项目一致

机器未来 / Paddle
与 Fork 源项目一致