提交 · 7d6d38486449eb243f9d87d56084fbf2511c4a28 · Crayon鑫 / Paddle

18 2月, 2022 2 次提交

[AMP] support GPU BF16 amp for dygraph (#39029) · 7d6d3848

由 zhangbo9674 提交于 2月 18, 2022

* support dtype param for auto_cast

* add amp_dtype for tracer

* add unsupported bf16 list

* support bf16 amp for O2

* refine python interface for bfloat16

* refine code

* refine code

* refine unittest

* refine code

* refine code

* add bf16 o1

* refine code by comment

* add gradient accumulator

* add recompute

7d6d3848

B
Fix sharding group (#39668) · bc3ca678
由 Baibaifan 提交于 2月 18, 2022
```
* fix_sharding_group

* fix_sharding_group
```
bc3ca678

17 2月, 2022 1 次提交
- B
  
  optimizer sharding paramters (#39581) · 18c6f40b
  由 Baibaifan 提交于 2月 17, 2022
  
  18c6f40b
16 2月, 2022 1 次提交

sync/geo test ok & fix heter_worker program ok (#39511) · b2986bab

由 ziyoujiyi 提交于 2月 16, 2022

* delete gloo connect retry

* the_one_ps dirs reconstruct

* .

* .

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* the one ps dirs modify

* the one ps dirs modify

* the one ps dirs modify

* the one ps dirs modify

* refactor ps optimize

* refactor ps optimize

* refactor ps optimize

* .

* .

* .

* .

* .

* .

* refactor theoneps

* the_one_ps

* add ps pass unittest

* add ps pass unittest

* ps unitest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* add cpu_async_ps_mode test

* add cpu_async_ps_mode test

* add cpu_async_ps_mode test

* ps unittest ready

* ps unittest ready

* solve dist_pass init conflict

* solve import CommContext error

* unittest ok

* implement AllocateFrom

* solve setup.py.in conflict

* solve conflict

* solve conflict

* solve conflict

* .

* .

* cpu-async-ps minimize test ok & gpu minimize test ok

* add heter 2stage unittest

* add heter 2stage unittest

* add heter 2stage unittest

* sync/geo test ok & fix heter_worker program ok

* .
Co-authored-by: Nzkh2016 <zhangkaihuo@baidu.com>

b2986bab

11 2月, 2022 1 次提交

统一 ps 开发 - python (#39431) · 22c67d14

由 ziyoujiyi 提交于 2月 11, 2022

* delete gloo connect retry

* the_one_ps dirs reconstruct

* .

* .

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* the one ps dirs modify

* the one ps dirs modify

* the one ps dirs modify

* the one ps dirs modify

* refactor ps optimize

* refactor ps optimize

* refactor ps optimize

* .

* .

* .

* .

* .

* .

* refactor theoneps

* the_one_ps

* add ps pass unittest

* add ps pass unittest

* ps unitest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* add cpu_async_ps_mode test

* add cpu_async_ps_mode test

* add cpu_async_ps_mode test

* ps unittest ready

* ps unittest ready

* solve dist_pass init conflict

* solve import CommContext error

* unittest ok

* implement AllocateFrom

* solve setup.py.in conflict

* solve conflict

* solve conflict

* solve conflict

* .

* .

* cpu-async-ps minimize test ok & gpu minimize test ok
Co-authored-by: Nzkh2016 <zhangkaihuo@baidu.com>

22c67d14

09 2月, 2022 1 次提交
- B
  
  optimize sharding stage3 offload (#39397) · b292dfb8
  由 Baibaifan 提交于 2月 09, 2022
  
  b292dfb8
08 2月, 2022 2 次提交

ps optimize refactor (#38982) · 196dbfc2

由 ziyoujiyi 提交于 2月 08, 2022

* delete gloo connect retry

* the_one_ps dirs reconstruct

* .

* .

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* create the_one_ps dirs

* the one ps dirs modify

* the one ps dirs modify

* the one ps dirs modify

* the one ps dirs modify

* refactor ps optimize

* refactor ps optimize

* refactor ps optimize

* .

* .

* .

* .

* .

* .

* refactor theoneps

* the_one_ps

* add ps pass unittest

* add ps pass unittest

* ps unitest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* ps unittest frame

* add cpu_async_ps_mode test

* add cpu_async_ps_mode test

* add cpu_async_ps_mode test

* ps unittest ready

* ps unittest ready

* solve dist_pass init conflict

* solve import CommContext error

* unittest ok

* implement AllocateFrom

* solve setup.py.in conflict

* solve conflict

* solve conflict

* solve conflict

* .

* .
Co-authored-by: Nzkh2016 <zhangkaihuo@baidu.com>

196dbfc2

B

optimize sharding stage3 (#39334) · 23d559dd
由 Baibaifan 提交于 2月 08, 2022

23d559dd

30 1月, 2022 1 次提交
- Z
  geo memory sparse table (#39250) · 9b3b53ba
  由 zhaocaibei123 提交于 1月 30, 2022
```
* geo depends

* add memory geo table

* fix
```
  9b3b53ba
28 1月, 2022 2 次提交

[PSLIB] Add Metrics Module, Support User-defined Add Metric (#38789) · 2e6be886

由 Fan Zhang 提交于 1月 28, 2022

* [PSLIB] Add Metrics Module, Support User-defined Add Metric

* [PSLIB] Modify According to CI

* [PSLIB] Modify According to CI

* [PSLIB] Modify According to CI

* [PSLIB] Modify According to CI Coverage

* [PSLIB] Modify According to CI

* [PSLIB] Modify According to CI

* [PSLIB] Modify According to CI

* [PSLIB] Modify According to CI

* [PSLIB] Modify According to CI

* [PSLIB] Modify According to CI Coverage

* [PSLIB] Modify According to CI Coverage

* [PSLIB] Modify According to CI Coverage

* modify role_maker

* update CMakeLists.txt

2e6be886

B

fix_stage2_minimize (#39285) · 90f44c6f
由 Baibaifan 提交于 1月 28, 2022

90f44c6f

25 1月, 2022 3 次提交
- B
  
  fix_stage3_fp16 (#39171) · 8bb509d5
  由 Baibaifan 提交于 1月 25, 2022
  
  8bb509d5
- H
  [Dygraph] Support param groups in grad_clip (#39175) · b0cca48e
  由 Haohongxiang 提交于 1月 25, 2022
```
* support param groups in grad_clip

* update

* modify for review
```
  b0cca48e
- K
  
  restore gloo (#39163) · faf517b2
  由 kuizhiqing 提交于 1月 25, 2022
  
  faf517b2
24 1月, 2022 2 次提交
- B
  
  Add sharding stage3 offload (#38989) · 46823104
  由 Baibaifan 提交于 1月 24, 2022
  
  46823104
- B
  
  fix sharding stage2 unittest (#39112) · f4623876
  由 Baibaifan 提交于 1月 24, 2022
  
  f4623876
18 1月, 2022 1 次提交
- K
  
  fix http gloo bug (#39017) · a998c077
  由 kuizhiqing 提交于 1月 18, 2022
  
  a998c077
17 1月, 2022 2 次提交
- W
  
  fix benchmark in paddlerec (#38278) · 1dbc8632
  由 wangguanqun 提交于 1月 17, 2022
  
  1dbc8632
- S
  Add NoReduce mode for ParallelExecutor (#38969) · e50d883e
  由 sneaxiy 提交于 1月 17, 2022
```
* add no reduce mode for pe

* add NoReduce ut
```
  e50d883e
14 1月, 2022 1 次提交
- B
  
  Add dygraph sharding stage3 (#38052) · 4c77a908
  由 Baibaifan 提交于 1月 14, 2022
  
  4c77a908
06 1月, 2022 1 次提交
- M
  
  [Paddle-ASP]Asp sharding (#37725) · aec6e8a9
  由 minghaoBD 提交于 1月 06, 2022
  
  aec6e8a9
30 12月, 2021 1 次提交
- Z
  
  [PSCore]Fix test fleet base 2 (#38588) · 04496d89
  由 zmxdream 提交于 12月 30, 2021
  
  04496d89
21 12月, 2021 3 次提交
- Y
  
  [fleet_executor] Python side fleet executor and task node (#38290) · a4afb97a
  由 Yuang Liu 提交于 12月 21, 2021
  
  a4afb97a
- G
  
  fix recompute no grad warning (#38293) · 2005b98b
  由 Guoxia Wang 提交于 12月 21, 2021
  
  2005b98b
- H
  optimize performance of offload in dygraph sharding stage2 (#38064) · f74ebd8a
  由 Haohongxiang 提交于 12月 21, 2021
```
* update

* fix bugs

* modify code style

* fix bugs of _get_global_group
```
  f74ebd8a
20 12月, 2021 1 次提交
- Y
  
  [fleet_executor] Remove runtime graph, all scheduler on python side (#38261) · 2f188341
  由 Yuang Liu 提交于 12月 20, 2021
  
  2f188341
19 12月, 2021 1 次提交
- B
  
  Integration sharding stage2 function (#38151) · 327e5050
  由 Baibaifan 提交于 12月 19, 2021
  
  327e5050
17 12月, 2021 3 次提交
- C
  Add mcmc of planner, of update cost model and relaunch (#38177) · 1bb2c68a
  由 caozhou 提交于 12月 17, 2021
```
* add planner

* add planner

* add cost model update

* add relaunch updation

* update process_group

* fix error

* add unitest

* update unitest

* update cost model

* avoid api problem
```
  1bb2c68a
- Y
  
  [fleet_executor] run time graph on python side (#38164) · fc701369
  由 Yuang Liu 提交于 12月 17, 2021
  
  fc701369
- W
  
  fix bind failed with Address already in use (#38174) · 446a62e8
  由 WangXi 提交于 12月 17, 2021
  
  446a62e8
09 12月, 2021 2 次提交

support offload in sharding stage2 (#37904) · dfed4a63

由 Haohongxiang 提交于 12月 09, 2021

* merge latest develop branch

* fix bugs

* update

* fix bugs for unittest

* modify for less use of gpu mem

* fix bugs of using _reset_grad_inplace_version

* update

* update

* modify for CI-Coverage

* retrick all CIs

dfed4a63

W
default accessor and multi table config (#37714) · a9e0d28c
由 wangguanqun 提交于 12月 09, 2021
```
* default accessor and multi table config

* add unittest

* add unittest

* delete print
```
a9e0d28c

07 12月, 2021 2 次提交

Buf fix for reset grad inplace version (#37811) · cf586021

由 Zhanlue Yang 提交于 12月 07, 2021

* Debug

* Fixed issue with reset_grad_inplace_version when used with clear_gradient & cross-batch accumulation

* Rearranged interfaces

* Fixed ci issues

cf586021

[Auto para] Relaunch with auto mapping function (#37326) · 506e79d1

由 Yulong Ao 提交于 12月 07, 2021

* [Auto Parallel]  Add the unified cluster representation

* [Auto Parallel] Add the graph class for physical mapping

* [Auto Parallel] Add the simple physical mapper

* Set the timeout of the mapper

* Merge the upstream develop unittests cmake files

* Fix a bug of the process group

* Remove mapper unittest from platforms which is not GPU

* Move the instantiation of process group after resharding

* Add the local id for devices

* Update the rank mapping format

* [Auto Parallel] Relaunch with the rank mapping file

* Remove the unnecessary json file

* Avoid entering get_device_proc_info for auto mapping

* Correct the mapper unit test

* Add some comments

* Remove the related files about mapping

* Update the unittest for auto mapping

* Remove unused rank_mapping unittest

* Improve the unittest coverage

* Improve the unittest coverage

* Improve the unittest of relaunch

* Fix the unittest problem in CI

* Improve the unittest of relaunch

* Remove unnecessary statements

* Update the unittest cmakefile

* Correct the cmakefile of auto parallel unittests

* Modify codes based on the new elastic change

* Use the GPUs exclusively in the unittest

* Correct the cmakefile

* Set the timeout of the unittest

506e79d1

06 12月, 2021 2 次提交
- B
  
  sharding_stage2_pfp16 (#37836) · 828f87ae
  由 Baibaifan 提交于 12月 06, 2021
  
  828f87ae
- K
  
  heter for collective (#37613) · 1bdb8578
  由 kuizhiqing 提交于 12月 06, 2021
  
  1bdb8578
02 12月, 2021 2 次提交
- X
  
  fix fleet elastic bug (#37781) · a710abee
  由 xiayanming 提交于 12月 02, 2021
  
  a710abee
- B
  
  Add dygraph sharding stage2 (#37707) · 20e19776
  由 Baibaifan 提交于 12月 02, 2021
  
  20e19776
01 12月, 2021 1 次提交
- Z
  [heterps]fix launch_utils.py. test=develop (#37752) · 1a1aeff6
  由 zmxdream 提交于 12月 01, 2021
```
* fix launch_utils.py. test=develop

* fix launch_utils.py. test=develop
```
  1a1aeff6
30 11月, 2021 1 次提交

[Auto Parallel] elastic support auto parallel re-launch (#37523) · 5440d2f9

由 xiayanming 提交于 11月 30, 2021

* [Auto Parallel] elastic support auto parallel re-launch

* [Auto Parallel] elastic support auto parallel re-launch

* fix ci issue

* fix ci issue

* fix rank mapping unittest

* fix rank mapping unittest

* fix ci issue

* fix ci issue

* fix ci issue

* fix ci issue

* fix ci issue

* fix ci issue

* fix ci issue

* fix ci issue

* fix ci issue

* fix ci issue

* fix ci issue

* fix ci issue

* fix ci issue

5440d2f9

Crayon鑫 / Paddle 与 Fork 源项目一致

Crayon鑫 / Paddle
与 Fork 源项目一致