提交 · 1bf08eca09e745370a024783d5c53f011917bef5 · wmsofts / Paddle

26 4月, 2022 1 次提交

optimize graph_engine pybind (#42192) · 1bf08eca

由 seemingwang 提交于 4月 26, 2022

* extract sub-graph

* graph-engine merging

* fix

* fix

* fix heter-ps config

* test performance

* test performance

* test performance

* test

* test

* update bfs

* change cmake

* test

* test gpu speed

* gpu_graph_engine optimization

* add dsm sample method

* add graph_neighbor_sample_v2

* Add graph_neighbor_sample_v2

* fix for loop

* add cpu sample interface

* fix kernel judgement

* add ssd layer to graph_engine

* fix allocation

* fix syntax error

* fix syntax error

* fix pscore class

* fix

* change index settings

* recover test

* recover test

* fix spelling

* recover

* fix

* move cudamemcpy after cuda stream sync

* fix linking problem

* remove comment

* add cpu test

* test

* add cpu test

* change comment

* combine feature table and graph table

* test

* test

* pybind

* test

* test

* test

* test

* pybind

* pybind

* fix cmake

* pybind

* fix

* fix

* add pybind

* add pybind

* optimize pybind

* test

* fix pybind

* fix
Co-authored-by: NDesmonDay <908660116@qq.com>

1bf08eca

24 4月, 2022 1 次提交

combine graph_table and feature_table in graph_engine (#42134) · 0e0f7da6

由 seemingwang 提交于 4月 24, 2022

* extract sub-graph

* graph-engine merging

* fix

* fix

* fix heter-ps config

* test performance

* test performance

* test performance

* test

* test

* update bfs

* change cmake

* test

* test gpu speed

* gpu_graph_engine optimization

* add dsm sample method

* add graph_neighbor_sample_v2

* Add graph_neighbor_sample_v2

* fix for loop

* add cpu sample interface

* fix kernel judgement

* add ssd layer to graph_engine

* fix allocation

* fix syntax error

* fix syntax error

* fix pscore class

* fix

* change index settings

* recover test

* recover test

* fix spelling

* recover

* fix

* move cudamemcpy after cuda stream sync

* fix linking problem

* remove comment

* add cpu test

* test

* add cpu test

* change comment

* combine feature table and graph table

* test

* test

* pybind

* test

* test

* test

* test

* pybind

* pybind

* fix cmake

* pybind

* fix

* fix

* add pybind

* add pybind
Co-authored-by: NDesmonDay <908660116@qq.com>

0e0f7da6

21 4月, 2022 1 次提交

[XPUPS]add hashtable interface (#41987) · 6becabaa

由 zmxdream 提交于 4月 21, 2022

* add hashtable interface. test=develop

* update. test=develop

* update. test=develop

* fix. test=develop

* fix optimizer config for xpups. test=develop

* fix. test=develop

* fix. test=develop

6becabaa

19 4月, 2022 2 次提交

Cpu gpu graph engine (#41942) · 4f461ab9

由 seemingwang 提交于 4月 19, 2022

* extract sub-graph

* graph-engine merging

* fix

* fix

* fix heter-ps config

* test performance

* test performance

* test performance

* test

* test

* update bfs

* change cmake

* test

* test gpu speed

* gpu_graph_engine optimization

* add ssd layer to graph_engine

* fix allocation

* fix syntax error

* fix syntax error

* fix pscore class

* fix

* recover test

* recover test

* fix spelling

* recover

* fix

* fix linking problem

* remove comment

4f461ab9

[XPUPS]add rename for heter_ps.cu (#41922) · afaf58ac

由 zmxdream 提交于 4月 19, 2022

* add rename for heter_ps.cu

* update. test=develop

* update. test=develop

* fix. test=develop

afaf58ac

18 4月, 2022 1 次提交
- T
  
  fix concurrent_unordered_map (#41820) · 360f0ae4
  由 Thunderbrook 提交于 4月 18, 2022
  
  360f0ae4
17 4月, 2022 1 次提交

XPUPS Adaptation (#40991) · 0ef3ef28

由 Fan Zhang 提交于 4月 17, 2022

* Adapt XPUPS - 1st version - 3.24

* Adapt XPUPS - update XPU PushSparse -  2nd version - 3.24

* Adapt XPUPS - add XPU PullSparseOp - 3nd version - 3.25

* refactor heter comm kernel

* update. test=develop

* Adapt XPUPS - modify by compilation - 4th version - 3.27

* update calc_shard_offset. test=develop

* update xpu kernel. test=develop

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* heter_comm update

* heter_comm update

* update calc_shard_offset. test=develop

* heter_comm update

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* fix. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* Adapt XPUPS - use WITH_XPU_KP and modify wrapper kernel function - 5th version - 3.30

* update. test=develop

* update pslib.cmake

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* Adapt XPUPS - modify by kp compilation  - 6th version - 3.30

* update. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* fix. test=develop

* fix. test=develop

* used by minxu

* update heter_comm_inl

* fix. test=develop

* Adapt XPUPS - modify by kp compilation  - 7th version - 3.30

* fix. test=develop

* add optimizer kernel. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 3.31 update

* Adapt XPUPS - update kp compilation path  - 8th version - 3.31

* add optimizer kernel. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update heter_comm_kernel.kps 3.31

* fix. test=develop

* fix. test=develop

* update heter_comm_kernel.kps 3.31

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update heter_comm.h 3.31

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update hashtable. test=develop

* update. test=develop

* Adapt XPUPS - update by kp compilation  - 9th version - 4.1

* update hashtable. test=develop

* fix. test=develop

* update hashtable 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 10th version - 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update. test=develop

* modify by compilation 4.1

* update. test=develop

* update. test=develop

* fix. test=develop

* modify by compilation 4.1

* update. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* modify by compilation 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* modify by compilation 4.1 19:30

* fix. test=develop

* update ps_gpu_wrapper.kps 4.1

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 11th version - 4.1

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 12nd version - 4.2

* fix. test=develop

* fix. test=develop

* modify by compilation 4.2

* 4.2 update

* fix. test=develop

* template init. test=develop

* update 4.6

* fix. test=develop

* template init. test=develop

* 4.6 modify by compilation

* hashtable template init. test=develop

* hashtable template init. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 13nd version - 4.7

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 4.11 update

* fix. test=develop

* fix. test=develop

* 4.11 update

* update by pre-commit

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* 4.12 update

* fix. test=develop

* Adapt XPUPS - update by kp compilation  - 14th version - 4.13

* 4.13 update

* 4.14 update

* 4.14 update

* 4.14 update

* 4.14 modify by merged latest compilation

* retry CI 4.14

* 4.15 pass static check

* 4.15 modify by gpups CI

* 3.16 update by gpups CI - modify ps_gpu_wrapper.h

* 4.16 update

* 4.16 pass xpu compile

* 4.16 retry CI

* 4.16 update
Co-authored-by: Nzmxdream <zhangminxu01@baidu.com>

0ef3ef28

15 4月, 2022 2 次提交

gpu_graph engine optimization+ (#41455) · ce72690c

由 seemingwang 提交于 4月 15, 2022

* extract sub-graph

* graph-engine merging

* fix

* fix

* fix heter-ps config

* test performance

* test performance

* test performance

* test

* test

* update bfs

* change cmake

* test

* test gpu speed

* gpu_graph_engine optimization

* add ssd layer to graph_engine

* fix allocation

* fix syntax error

* fix syntax error

* fix pscore class

* fix

* recover test

* recover test

* fix spelling

* recover

* fix

ce72690c

[XPUPS]fix hashtable_kernel.kps (#41790) · ef6ff4ef

由 zmxdream 提交于 4月 15, 2022

* refactor heter comm kernel

* update. test=develop

* update calc_shard_offset. test=develop

* update xpu kernel. test=develop

* update args of calc_shard_offset

* update. test=develop

* remove customGradMerger

* update. test=develop

* update. test=develop

* fix. test=develop

* update. test=develop

* update. test=develop

* update optimizer kernel

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* update. test=develop

* fix. test=develop

* fix. test=develop

* add optimizer kernel. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix kunlun not support size_t. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update hashtable. test=develop

* update. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* update. test=develop

* update. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* template init. test=develop

* hashtable template init. test=develop

* fix. test=develop

* fix. test=devlop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix hashtable_kernel. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop

* fix. test=develop
Co-authored-by: NWorgenZhang <frank08081993@gmail.com>

ef6ff4ef

13 4月, 2022 1 次提交
- Z
  [XPUPS]add support for kunlun2 (#40985) · c9c03e7b
  由 zmxdream 提交于 4月 13, 2022
```
[XPUPS]add support for kunlun2
Co-authored-by: NWorgenZhang <frank08081993@gmail.com>
```
  c9c03e7b
01 4月, 2022 1 次提交

[GPUPS]fix CMakeLists with pslib (#41225) · 4da4265a

由 zmxdream 提交于 4月 01, 2022

* fix cmake. test=develop

* fix. test=develop

* fix dep for graphs_ps_gpu. test=develop

* update. test=develop

* update. test=develop

4da4265a

28 3月, 2022 1 次提交

delete commonsparsetable and communicator from gpups (#40973) · 630f5b89

由 wangguanqun 提交于 3月 28, 2022

* trainer and worker

* delete commonsparsetable from gpups

* delete vlog

* codestyle

* delete communicator from gpups

630f5b89

24 3月, 2022 1 次提交

test gpu graph engine's performance (#40775) · 83ae1619

由 seemingwang 提交于 3月 24, 2022

* extract sub-graph

* graph-engine merging

* fix

* fix

* fix heter-ps config

* test performance

* test performance

* test performance

* test

* test

* update bfs

* change cmake

83ae1619

21 3月, 2022 1 次提交

[GpuPs] Update graph sampling method (#40085) · 78fad09b

由 Siming Dai 提交于 3月 21, 2022

* gpu ps graph engine

* remove logs

* Add neighbor sampling method

* Add actual_sample_size and offset for sampling

* Delete Chinese comment

* Fix code style
Co-authored-by: seemingwang <zsasuke@qq.com>

78fad09b

17 3月, 2022 1 次提交
- S
  merge cpu and gpu graph engines (#40597) · 31776199
  由 seemingwang 提交于 3月 17, 2022
```
* extract sub-graph

* graph-engine merging

* fix

* fix

* fix heter-ps config
```
  31776199
04 3月, 2022 1 次提交
- T
  Dump cpu xingneng (#40068) · 5496a7ab
  由 Thunderbrook 提交于 3月 04, 2022
```
* dump cpu

* code format
```
  5496a7ab
21 2月, 2022 1 次提交
- S
  gpu ps graph engine (#39699) · 05982c10
  由 seemingwang 提交于 2月 21, 2022
```
* gpu ps graph engine

* remove logs
```
  05982c10
20 2月, 2022 1 次提交

[PTen->Phi PR1] Change pten dirname and namespace to phi (#39748) · dcfe1986

由 Chen Weihang 提交于 2月 20, 2022

* rename pten dir to phi

* rename namespace to phi

* rename infrt pten dir to phi

* resolve conflict

* rename pten to phi in cmake

* revert all infrt change

* change needed files

* fix infrt failed

* fix inference failed

dcfe1986

25 1月, 2022 2 次提交

Y

reconstruct directory of ps (#39191) · 2bf9b844
由 yaoxuefeng 提交于 1月 25, 2022

2bf9b844

[Move selected_rows PR #3] Change the relationship of [include/Cmake]. (#39128) · 2bafd338

由 Weilong Wu 提交于 1月 25, 2022

* Added selected_rows and rw_lock to pten

* Renamed the unit test target to fix CI

* Removed Class SelectedRows in Fluid, changed include/cmake relationship, use pten::SelectedRows in Fluid

* Remove rw_lock.h,rw_lock_test.cc in fluid

* Use pten::RWLock and pten::AutoRDLock, fix CI

* Use pten::SelectedRows

* Use pten::SelectedRows

* Fix to pass NPU CI

* Use pten::SelectedRows, to pass NPU CI

* To fix NPU CI

* To fix NPU CI again

2bafd338

29 12月, 2021 2 次提交
- Y
  add hashtable dynamic mf support (#38493) · d9c174d1
  由 yaoxuefeng 提交于 12月 29, 2021
```
add hashtable dynamic mf support
```
  d9c174d1
- Y
  add dynamic mf size api (#38436) · 7411dab5
  由 yaoxuefeng 提交于 12月 29, 2021
```
add dynamic mf size api
```
  7411dab5
23 12月, 2021 1 次提交
- Y
  add mem pool (#37127) · 745477fe
  由 yaoxuefeng 提交于 12月 23, 2021
```
add mem pool
```
  745477fe
20 12月, 2021 1 次提交
- 石
  
  changes the call AllocShared to Alloc, test=develop (#38258) · bb0713b2
  由石晓伟提交于 12月 20, 2021
  
  bb0713b2
03 12月, 2021 1 次提交
- R
  refine structure for cuda and rocm (#37202) · a6d2fddb
  由 ronnywang 提交于 12月 03, 2021
```
* refine structure for cuda and rocm

* update

* update

* update

* update
```
  a6d2fddb
29 11月, 2021 1 次提交
- T
  [HeterPs] fix allocation (#37476) · 27a5f52b
  由 Thunderbrook 提交于 11月 29, 2021
```
* auc temp

* cuballocator

* code format

* code format
```
  27a5f52b
08 11月, 2021 1 次提交
- T
  add refrence (#36994) · ac1d3571
  由 Thunderbrook 提交于 11月 08, 2021
```
* add refrence

* format
```
  ac1d3571
19 10月, 2021 1 次提交
- D
  
  [heterps]edit shrink and unseenday logit for pslib (#36194) · 9e494472
  由 danleifeng 提交于 10月 19, 2021
  
  9e494472
01 9月, 2021 1 次提交
- T
  [HeterPs] merge dense && data norm && g2sum (#35029) · a647b80a
  由 Thunderbrook 提交于 9月 01, 2021
```
* merge dense

* log level

* tensor copy sync

* format
```
  a647b80a
26 7月, 2021 1 次提交
- D
  【HETERPS】edit cuda remote_streams (#34276) · 539d7185
  由 danleifeng 提交于 7月 26, 2021
```
* psgpu:edit cuda remote_streams; test=develop
```
  539d7185
19 7月, 2021 1 次提交
- Y
  
  fix pull&push dim (#34217) · 2891ce64
  由 yaoxuefeng 提交于 7月 19, 2021
  
  2891ce64
05 7月, 2021 1 次提交
- D
  【HeterPS】fix hdfs and fleet_util for supporting save/load/infer (#33903) · 2ef6188b
  由 danleifeng 提交于 7月 05, 2021
```
* fix hdfs and fleet_util for supporting save/load infer;test=develop
```
  2ef6188b
16 6月, 2021 1 次提交
- T
  
  add delta score, scale show (#33492) · a50d1296
  由 Thunderbrook 提交于 6月 16, 2021
  
  a50d1296
10 5月, 2021 1 次提交
- T
  [pslib] pslib with cmake (#32800) · fbbc3394
  由 Thunderbrook 提交于 5月 10, 2021
```
* pslib with cmake

* heter util

* vlog

* heter server test

* add dtor

* cmake
```
  fbbc3394
28 4月, 2021 1 次提交

[PsCore] solve Brpc dep (#32632) · 4ead9a5a

由 Thunderbrook 提交于 4月 28, 2021

* Revert "Revert "[PsCore] optimize performance of large kv (#32535)" (#32599)"

This reverts commit 809ac036.

* brpc dep

4ead9a5a

27 4月, 2021 1 次提交
- T
  Revert "[PsCore] optimize performance of large kv (#32535)" (#32599) · 809ac036
  由 tianshuo78520a 提交于 4月 27, 2021
```
This reverts commit 4b7242b0.
```
  809ac036
26 4月, 2021 1 次提交
- T
  [PsCore] optimize performance of large kv (#32535) · 4b7242b0
  由 Thunderbrook 提交于 4月 26, 2021
```
* optimize pull sparse

* optimize pull sparse

* change macro

* format
```
  4b7242b0
15 4月, 2021 1 次提交

heterps support pscore (#32093) · 9f8c8f96

由 Thunderbrook 提交于 4月 15, 2021

* pscore support heterps

* fleet cmake

* fleet wrapper

* macro

* solve conflict

* solve conflict

* add unitest

* paddle enforce

* unitest

* unitest

* unitest

9f8c8f96

02 3月, 2021 2 次提交

[ROCM] update fluid operators for rocm (part5), test=develop (#31258) · 65bcaeb0

由 Qi Li 提交于 3月 02, 2021

* [ROCM] update fluid operators for rocm (part5), test=develop

* address review comments, test=develop

* fix typo, test=develop

65bcaeb0

topo and memory performance for heterps (#30440) · d1075df2

由 danleifeng 提交于 3月 02, 2021

* topo and memory performance for heterps; test=develop
* add trainwithprofiler in heter trainier; test=develop

d1075df2

wmsofts / Paddle 与 Fork 源项目一致

wmsofts / Paddle
与 Fork 源项目一致