提交 · 80992253b3f3268a3641f5bd5fc478d544062a97 · 机器未来 / Paddle

20 4月, 2022 1 次提交

[cherry-pick] Cherry pick pr of new-exec (#42009) · 80992253

由 Leo Chen 提交于 4月 20, 2022

* [new-exec] shrink downstream map (#41471)

* shrink downstream map

* shrink last live ops of var

* add comment

* fix bug

* add dependency for send/recv to support pp parallel (#41652)

* [new-exec] clear the scope listener after run (#41947)

* clear the listener after run

* only sync variables in program

* refine code

* fit for lod_tensor_blocking_queue

80992253

19 4月, 2022 4 次提交

Z
[XPUPS]add rename for heter_ps.cu (#41922) (#41968) · 13202ff7
由 zmxdream 提交于 4月 19, 2022
```
* add rename for heter_ps.cu

* update. test=develop

* update. test=develop

* fix. test=develop
```
13202ff7
Y
[Cherry-pick 2.3] Autotune the workspace and kernel choosing of conv (#41833) · b4adbe5c
由 Yiqun Liu 提交于 4月 19, 2022
```
Cherry-pick #40338 #41741 #41313
```
b4adbe5c

[cherry-pick] XPUPS Adaptation (#41917) · a9d8b947

由 Fan Zhang 提交于 4月 19, 2022

* XPUPS Adaptation (#40991)

* Adapt XPUPS - 1st version - 3.24

* Adapt XPUPS - update XPU PushSparse -  2nd version - 3.24

* Adapt XPUPS - add XPU PullSparseOp - 3nd version - 3.25

* refactor heter comm kernel

* update. test=develop

* Adapt XPUPS - modify by compilation - 4th version - 3.27

* update calc_shard_offset. test=develop

* update xpu kernel. test=develop

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* heter_comm update

* heter_comm update

* update calc_shard_offset. test=develop

* heter_comm update

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* fix. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* Adapt XPUPS - use WITH_XPU_KP and modify wrapper kernel function - 5th version - 3.30

* update. test=develop

* update pslib.cmake

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* Adapt XPUPS - modify by kp compilation  - 6th version - 3.30

* update. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* fix. test=develop

* fix. test=develop

* used by minxu

* update heter_comm_inl

* fix. test=develop

* Adapt XPUPS - modify by kp compilation  - 7th version - 3.30

* fix. test=develop

* add optimizer kernel. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 3.31 update

* Adapt XPUPS - update kp compilation path  - 8th version - 3.31

* add optimizer kernel. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update heter_comm_kernel.kps 3.31

* fix. test=develop

* fix. test=develop

* update heter_comm_kernel.kps 3.31

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update heter_comm.h 3.31

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update hashtable. test=develop

* update. test=develop

* Adapt XPUPS - update by kp compilation  - 9th version - 4.1

* update hashtable. test=develop

* fix. test=develop

* update hashtable 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 10th version - 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update. test=develop

* modify by compilation 4.1

* update. test=develop

* update. test=develop

* fix. test=develop

* modify by compilation 4.1

* update. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* modify by compilation 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* modify by compilation 4.1 19:30

* fix. test=develop

* update ps_gpu_wrapper.kps 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 11th version - 4.1

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 12nd version - 4.2

* fix. test=develop

* fix. test=develop

* modify by compilation 4.2

* 4.2 update

* fix. test=develop

* template init. test=develop

* update 4.6

* fix. test=develop

* template init. test=develop

* 4.6 modify by compilation

* hashtable template init. test=develop

* hashtable template init. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 13nd version - 4.7

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 4.11 update

* fix. test=develop

* fix. test=develop

* 4.11 update

* update by pre-commit

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 4.12 update

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 14th version - 4.13

* 4.13 update

* 4.14 update

* 4.14 update

* 4.14 update

* 4.14 modify by merged latest compilation

* retry CI 4.14

* 4.15 pass static check

* 4.15 modify by gpups CI

* 3.16 update by gpups CI - modify ps_gpu_wrapper.h

* 4.16 update

* 4.16 pass xpu compile

* 4.16 retry CI

* 4.16 update
Co-authored-by: Nzmxdream <zhangminxu01@baidu.com>

* modify ps_gpu_wrapper.cc

* update
Co-authored-by: Nzmxdream <zhangminxu01@baidu.com>

a9d8b947

cinn_launch_op: optimize the overhead of preparing variables before executing... · dab7dfbf

由 TeFeng Chen 提交于 4月 19, 2022

cinn_launch_op: optimize the overhead of preparing variables before executing cinn compiled program (#41777) (#41910)

cherry-pick #41777
* optimize preparation overhead before executing cinn compiled program

dab7dfbf

18 4月, 2022 3 次提交

[cherry-pick]XPUPS add support for kunlun2 (#41916) · 3a2fb4cf

由 zmxdream 提交于 4月 18, 2022

* [XPUPS]add support for kunlun2 (#40985)


[XPUPS]add support for kunlun2
Co-authored-by: NWorgenZhang <frank08081993@gmail.com>

* [XPUPS]fix hashtable_kernel.kps (#41790)

* refactor heter comm kernel

* update. test=develop

* update calc_shard_offset. test=develop

* update xpu kernel. test=develop

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* update. test=develop

* fix. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* fix. test=develop

* fix. test=develop

* add optimizer kernel. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update hashtable. test=develop

* update. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update. test=develop

* update. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* template init. test=develop

* hashtable template init. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix hashtable_kernel. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop
Co-authored-by: NWorgenZhang <frank08081993@gmail.com>

* [XPUPS]modify xpu_kp.cmake with HETERPS&PSLIB (#41760)

* modify xpu_kp.cmake with HETERPS&PSLIB

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop
Co-authored-by: NWorgenZhang <frank08081993@gmail.com>

3a2fb4cf

[Phi]Reduce kernels into multiply files (#41747) (#41854) · 688f4ec0

由 chentianyu03 提交于 4月 18, 2022

* split reduce_kernel

* rm reduce_kernel in cmake

* split reduce_grad kernels

* fix cmake build error

* format code

* fix standalone_executor_test error

688f4ec0

[Cherry-pick] Organize the API of custom operators (#41882) · 897911fc

由 Chen Weihang 提交于 4月 18, 2022

* [Phi&CustomOp] Remove deprecated enum PlaceType for custom op & add warning (#41647)

* remove old custom op placetype

* replace dist  placetype using

* add with gpu macro

* fix mutable_data error

* fix set value error

* add comment

* remove all is initialized using (#41766)

* remove inner_place using (#41768)

* polish tensor depreacted method warning (#41807)

* [CustomOp] Fix PlaceType related compat error (#41826)

* fix place type related compat error

* fix test failed

* remove dll decl

* revert place type change

* add dll decl

* resolve conflict

897911fc

15 4月, 2022 2 次提交

[cherry-pick] fix some bug in Phi (#41622, #41646) (#41815) · b7244512

由 zyfncg 提交于 4月 15, 2022

* fix data transform problem for cudnn backend (#41622)

* Fix problem of infermeta with vector output (#41646)

* remove stack_grad infershape

* fix bug of output with null

* fix bug

b7244512

the one ps proto (#41659) (#41789) · 8287e330

由 wangguanqun 提交于 4月 15, 2022

* the one ps proto (#41659)

* the one ps proto

* the one ps proto

* fix

* fix

* fix

* fix windows ci

* fix windows ci

* add dependency

* add dependency

* fix bug of ps_py_proto cant find path for the folder not created (#41793)
Co-authored-by: NSing_chan <51314274+betterpig@users.noreply.github.com>

8287e330

14 4月, 2022 1 次提交
- C
  [CustomOp]Add new method for custom double grad (#41538) (#41781) · 76d5483a
  由 Chen Weihang 提交于 4月 14, 2022
```
* add new method for custom double grad

* add tanh double grad unittest

* change year

* revert tensor init method
```
  76d5483a
13 4月, 2022 1 次提交
- A
  Revert "[Phi] Migrate Adam and AdamW into Phi (#40351)" (#41712) · 8663376f
  由 Aurelius84 提交于 4月 13, 2022
```
* Revert "[Phi] Migrate Adam and AdamW into Phi (#40351)"

This reverts commit 56cd3407.

* add infermeta
```
  8663376f
11 4月, 2022 3 次提交

Unittest recover (#41431) (#41590) · 74626f66

由 zhaocaibei123 提交于 4月 11, 2022

* update name

* update name

* fix test

* fix fleet bind

* update name

* update name

* fix test

* fix gpups wrapper

* remove Push/Pull/Load/Save with context in client and wrapper base class

* fix

* fix

* remove some interface

* fix

* remove

* code style

* recover

* fix

* remove code unused

* remove some unused table & accessor & CommonDenseTable => MemoryDenseTable

* fix

* fix

* fix

* recover

* remove unused code

* recover unittest

* fix

* remove

* fix

* remove code unuseful

* remove

* fix

* recover

* remove
Co-authored-by: Nesythan <esythan@126.com>
Co-authored-by: Nesythan <esythan@126.com>

74626f66

L
[cherry-pick 2.3]fix bug when TruncatedNormal cannot fall back in cpu (#41595) · f275bf6d
由 Liu-xiandong 提交于 4月 11, 2022
```
fix bug when TruncatedNormal cannot fall back in cpu
```
f275bf6d
L
[new-exec] fix bug that no thread is waked up when adding task to threadpool (#41567) (#41575) · 53aef69c
由 Leo Chen 提交于 4月 11, 2022
```
* fix bug that no thread is waked up when adding task to threadpool

* fix typo
```
53aef69c

06 4月, 2022 1 次提交
- A
  [IPU] remove paddle_ipu shared library (#41307) · 229e91bf
  由 Allen Guo 提交于 4月 06, 2022
```
* remove paddle_ipu shared library

* fix unique_name
```
  229e91bf
05 4月, 2022 2 次提交

Z
Fix bug of data transform in inference executor (#41349) · 91212104
由 zyfncg 提交于 4月 05, 2022
```
* fix bug of data transform in inference executor

* fix bug
```
91212104

[new-exec] enable the new standalone executor by default (#41179) · 93ea1297

由 Leo Chen 提交于 4月 05, 2022

* enable new executor by default

* enable stream safe allocator

* test=document_fix;test=coverage

* do not use scope in op kernel

* fit empty program for new executor

* fix communication depend

* fix test_sync_batch_norm

* skip unsupported place

* refine datatransfer

* fit for dirtributed program

* fix dependencpy

* fix some ut

93ea1297

04 4月, 2022 2 次提交

S
conv + elementwise_add refactor (#41286) · e5e0b726
由 Sławomir Siwek 提交于 4月 04, 2022
```
* DRY

* change nodes names

* add const prefix

* change asX to as_x in all files
```
e5e0b726

Add dropout yaml (#41355) · 1c7001e7

由 hong 提交于 4月 04, 2022

* add dropout slice yaml

* remove useless code

* fix infer shape error

* skip infrt compile for dropout

1c7001e7

03 4月, 2022 1 次提交

[Phi]Concat grad (#41112) · 3f57ef7a

由 chentianyu03 提交于 4月 03, 2022

* add concat_grad kernel

* fix error

* remove comment code

* fix outs nullptr error

* change to phi header

* add concat_grad declare for standalone_executor_test

3f57ef7a

02 4月, 2022 5 次提交
- L
  
  [new-exec] fit empty program for new executor (#41328) · e0ccaeaf
  由 Leo Chen 提交于 4月 02, 2022
  
  e0ccaeaf
- W
  [Paddle inference] support new quant_model (#41049) · 1b58ce14
  由 Wangzheee 提交于 4月 02, 2022
```
* paddle inference support new quant_model
```
  1b58ce14
- L
  [KP] fix bug in phi static graph mode (#41269) · d0f46aac
  由 Liu-xiandong 提交于 4月 02, 2022
```
* [KP] fix bug in phi static graph mode

* modify the useless code
```
  d0f46aac
- Z
  统一ps refine (#41234) · b3270adf
  由 zhaocaibei123 提交于 4月 02, 2022
```
* update name

* update name

* fix test

* fix fleet bind

* update name

* update name

* fix test

* fix gpups wrapper

* remove Push/Pull/Load/Save with context in client and wrapper base class

* fix

* fix
Co-authored-by: Nesythan <esythan@126.com>
```
  b3270adf
- L
  
  [new-exec] support to enable mkldnn by flags (#41274) · cb124156
  由 Leo Chen 提交于 4月 02, 2022
  
  cb124156
01 4月, 2022 6 次提交

L
fix mac c++ version (#41172) · a2c01db1
由 liutiexing 提交于 4月 01, 2022
```
* fix mac c++ version

* update

* fix apple systems
```
a2c01db1

[Phi] Move softmax with cross entropy kernel into phi (#40832) · e6ec98fe

由 Chen Weihang 提交于 4月 01, 2022

* add cross_entropy_with_softmax phi kernel

* remove softmax_with_cross_entropy kernel

* add softmax_with_cross_entropy grad kernel

* remove original op kernel

* refine cross entropy impl

* fix pointer error

* revert kernel cu change

* fix xpu failed

* fix cinn failed

* fix npu failed

* add forward sig

* add check_nan_inf for pt kernel

* remove repeat cmake item

* fix unittest error

e6ec98fe

[Phi]Interploatd kernels into phi (#40855) · d65a7a46

由 chentianyu03 提交于 4月 01, 2022

* add interploate cpu kernel

* fix nullptr bug

* add interpolate gpu kernel

* fix unit test error

* remove raw kernels

* add cuda kernel impl

* add infermeta

* recover accidentally deleted kernels in interpolate op

* fix grad x_grad name error

* remove interpolate_v2_op.h

* rm unused codes

* fix xpu build error

* fix build error

* fix namespace error

* add register header for nup

* fix infermeta error

* modify by review

* add the missing args in test_trt_convert_nearest_interp_v2

d65a7a46

[GPUPS]fix CMakeLists with pslib (#41225) · 4da4265a

由 zmxdream 提交于 4月 01, 2022

* fix cmake. test=develop

* fix. test=develop

* fix dep for graphs_ps_gpu. test=develop

* update. test=develop

* update. test=develop

4da4265a

A

[custom kernel] support fallback (#41212) · 9c2a9afd
由 Aganlengzi 提交于 4月 01, 2022

9c2a9afd
L
[new-exec] move WaitEvent/RecordEvent into try-catch (#41222) · 5dae6da0
由 Leo Chen 提交于 4月 01, 2022
```
* move WaitEvent/RecordEvent into try-catch

* refine supportNpu
```
5dae6da0

31 3月, 2022 8 次提交

heter & multi-cloud brpc communication (#40965) · 2f41f389

由 ziyoujiyi 提交于 3月 31, 2022

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

2f41f389

Z
[Phi] Rename ScalarArray to IntArray (#40975) · e559fe41
由 zyfncg 提交于 3月 31, 2022
```
* rename scalar_array to int_array

* update cmake

* fix conflict

* remove useless log
```
e559fe41

[new-exec] fit mkldnn op (#41058) · 02cf6764

由 Leo Chen 提交于 3月 31, 2022

* fix bug that some op has no op_role attr

* add mkldnn support for new executor

* fit for mkldnn data_transfer

* fit for mkldnn data_transfer

02cf6764

Maintain old profiler (#41132) · a6bf2218

由 chenjian 提交于 3月 31, 2022

* no

* maintain old profiler

* exclude new python record events for old profiler

* maintain old profiler

* maintain

* maintain old profiler

* maintain

* fix cmakes

a6bf2218

add flatten2,reshape2,squueze2_trt_fuse_pass test cast (#41031) · 7ef69202

由 heliqi 提交于 3月 31, 2022

* add flatten2,reshape2,squueze2_trt_fuse_pass  test cast

* add flatten2,reshape2,squueze2_trt_fuse_pass  test cast

* add flatten2,reshape2,squueze2_trt_fuse_pass  test cast

7ef69202

W
[phi] move yolov3_loss to phi (#40944) · fb93bd5c
由 wuyefeilin 提交于 3月 31, 2022
```
* mv yolov3_loss op to phi

* fix as review

* update operator.h
```
fb93bd5c

fix load bug and add distributed strategy from pslib (#40883) · 47383dca

由 wangguanqun 提交于 3月 31, 2022

* fix load bug and add distributed strategy from pslib

* add unittest

* use cvm config

* trainer and worker config

* add unittest

* add unittest

* add test

* code style

47383dca

L
add depend when doing fuse_all_optimizer on program (#41178) · 3b00dc92
由 Leo Chen 提交于 3月 31, 2022
```
* fix dependency of fused optimizer

* add ut
```
3b00dc92

机器未来 / Paddle 与 Fork 源项目一致

机器未来 / Paddle
与 Fork 源项目一致