提交 · 34a11688c4b3dd98fe23d6222fc4c54e7ac74fe3 · Greenplum / DeepSpeed

25 1月, 2023 1 次提交
- L
  
  Change zero_grad() argument to match pytorch (#2741) · 34a11688
  由 loadams 提交于 1月 24, 2023
  
  34a11688
09 1月, 2023 1 次提交
- J
  [Bug Fixed] use torch.cuda.is_available() (#2661) · 323c266c
  由 JackieWu 提交于 1月 09, 2023
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
```
  323c266c
21 6月, 2022 1 次提交
- K
  fix import errors (#2026) · 735406e5
  由 Karim Foda 提交于 6月 20, 2022
```
Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>
```
  735406e5
11 6月, 2022 1 次提交

DeepSpeed comm backend v1 (#1985) · 36ad3119

由 Ammar Ahmad Awan 提交于 6月 10, 2022

Co-authored-by: NQuentin Anthony <qganthony@yahoo.com>
Co-authored-by: NAmmar Ahmad Awan <ammar.awan@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

36ad3119

12 5月, 2022 1 次提交
- J
  
  Fairseq support (#1915) · 50893458
  由 Jeff Rasley 提交于 5月 11, 2022
  
  50893458
08 2月, 2022 1 次提交
- O
  Move param_shapes to model files (#1732) · 135a6256
  由 Olatunji Ruwase 提交于 2月 07, 2022
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  135a6256
23 1月, 2022 1 次提交
- A
  
  Add codespell to pre-commit checks (#1717) · 4cf970e6
  由 Alex Hedges 提交于 1月 22, 2022
  
  4cf970e6
15 1月, 2022 1 次提交

[ZeRO] Default disable elastic ckpt in stage 1+2 and reduce CPU memory... · 3293cf72

由 Jeff Rasley 提交于 1月 14, 2022

[ZeRO] Default disable elastic ckpt in stage 1+2 and reduce CPU memory overhead during ckpt load (#1525)
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

3293cf72

30 9月, 2021 1 次提交

Big science related changes (#1407) · e2fdd254

由 Jeff Rasley 提交于 9月 29, 2021

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
Co-authored-by: NShaden Smith <shaden.smith@microsoft.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NSamyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: Neltonzheng <eltonz@microsoft.com>
Co-authored-by: NStas Bekman <stas00@users.noreply.github.com>

e2fdd254

17 8月, 2021 1 次提交

DeepSpeed MoE (#1310) · f2843244

由 Ammar Ahmad Awan 提交于 8月 16, 2021

Co-authored-by: NAlex Muzio <Alex.Muzio@microsoft.com>
Co-authored-by: NAmmar Ahmad Awan <ammar.awan@microsoft.com>
Co-authored-by: NConglong Li <conglong.li@gmail.com>
Co-authored-by: NFelipe Cruz Salinas <Andres.Cruz@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
Co-authored-by: NReza Yazdani <reyazda@microsoft.com>
Co-authored-by: NSamyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: NShaden Smith <shaden.smith@microsoft.com>
Co-authored-by: NYoung Jin Kim <youki@microsoft.com>
Co-authored-by: Nbapatra <bapatra@microsoft.com>
Co-authored-by: NSamyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: NShaden Smith <shaden.smith@microsoft.com>
Co-authored-by: NYoung Jin Kim <youki@microsoft.com>

f2843244

24 5月, 2021 1 次提交

Quantization + inference release (#1091) · ed3de0c2

由 Reza Yazdani 提交于 5月 24, 2021

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
Co-authored-by: Neltonzheng <eltonz@microsoft.com>
Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: NReza Yazdani <reyazda@microsoft.com>
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: NElton Zheng <eltonz@microsoft.com>
Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: Neltonzheng <eltonz@microsoft.com>
Co-authored-by: NArash Ashari <arashari@microsoft.com>
Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NReza Yazdani <reyazda@microsoft.com>
Co-authored-by: Nniumanar <60243342+niumanar@users.noreply.github.com>
Co-authored-by: Neltonzheng <eltonz@microsoft.com>
Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: NReza Yazdani <reyazda@microsoft.com>
Co-authored-by: NArash Ashari <arashari@microsoft.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Nniumanar <60243342+niumanar@users.noreply.github.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
Co-authored-by: Neltonzheng <eltonz@microsoft.com>
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: NArash Ashari <arashari@microsoft.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Nniumanar <60243342+niumanar@users.noreply.github.com>

ed3de0c2

21 4月, 2021 1 次提交

1-bit LAMB optimizer (#970) · 67a48aaa

由 Conglong Li 提交于 4月 20, 2021

1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed.
Author: @conglongli, @awan-10, @samyam, Hanlin Tang, Yuxiong He
Paper: https://arxiv.org/abs/2104.06069Co-authored-by: Nsdtblck <46172032+sdtblck@users.noreply.github.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

67a48aaa

11 3月, 2021 1 次提交
- S
  less scary overflow notice (#833) · 29853c3e
  由 Stas Bekman 提交于 3月 10, 2021
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  29853c3e
10 9月, 2020 1 次提交
- S
  Pipeline parallel training engine. (#392) · 65c2f974
  由 Shaden Smith 提交于 9月 09, 2020
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  65c2f974
02 9月, 2020 1 次提交

Sparse attn + ops/runtime refactor + v0.3.0 (#343) · e5bbc2e5

由 Jeff Rasley 提交于 9月 01, 2020

* Sparse attn + ops/runtime refactor + v0.3.0
Co-authored-by: NArash Ashari <arashari@microsoft.com>
Co-authored-by: NArash Ashari <arashari@microsoft.com>

e5bbc2e5

05 6月, 2020 1 次提交

Add log util (#230) · e1ad8803

由 Chunyang Wen 提交于 6月 05, 2020

* Add log util

* replace all occurrences of print and logging

* address format

* disable propagate to avoid duplicate log

e1ad8803

12 5月, 2020 1 次提交
- O
  Support dynamic loss scale args in fp16 optimizers (#212) · 0be026e3
  由 Olatunji Ruwase 提交于 5月 11, 2020
```
* Support dynamic loss scale args in fp16 optimizers

* Update names
```
  0be026e3
03 4月, 2020 1 次提交
- K
  
  add representation to optimizer (#181) · 3637b86b
  由 kouml 提交于 4月 04, 2020
  
  3637b86b
11 3月, 2020 1 次提交

Enhancement: Ability to load checkpoint without loading the optimizer… (#128) · 936117b5

由 Samyam Rajbhandari 提交于 3月 10, 2020

* Enhancement: Ability to load checkpoint without loading the optimizer states. Unittest testing saving and loading checkpoint with fused, unfused and zero optimizer. The unitest takes about 165s

936117b5

15 2月, 2020 1 次提交
- J
  Fix issue with empty grads for non-fused optimizers (#83) · 807480a0
  由 Jeff Rasley 提交于 2月 14, 2020
```
bug fixes for adamw/lamb and corresponding tests
```
  807480a0
04 2月, 2020 1 次提交
- S
  Add files via upload · ec79b239
  由 Samyam Rajbhandari 提交于 2月 03, 2020
```
Different Optimizers in DeepSpeed.
```
  ec79b239

Greenplum / DeepSpeed 上一次同步 大约 1 年

Greenplum / DeepSpeed
上一次同步大约 1 年