提交 · 8630f73e09084cc29a5516656c882dc5174e108c · Greenplum / DeepSpeed

06 4月, 2023 5 次提交
- V
  Remove Extra Bracket (#3101) · 8630f73e
  由 Vincent Hellendoorn 提交于 4月 05, 2023
```
This was breaking the link to the Pyramid MoE paper on Arxiv
```
  8630f73e
- E
  fix example symlink about DeepSpeed+AzureML (#3127) · 951d4df7
  由 Earlee 提交于 4月 06, 2023
```
Co-authored-by: NLogan Adams <114770087+loadams@users.noreply.github.com>
```
  951d4df7
- L
  Update skip on torch in tests (#3136) · ab1f32de
  由 Logan Adams 提交于 4月 05, 2023
```
* Replace old torch version checks with existing function

* Clean up formatting
```
  ab1f32de
- S
  [ci] `nv-transformers-v100` - use the same torch version as transformers CI (#3096) · 20ed15be
  由 Stas Bekman 提交于 4月 05, 2023
```
Co-authored-by: NLogan Adams <114770087+loadams@users.noreply.github.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  20ed15be
- S
  Update engine.py (#2826) · bcb03531
  由 Stas Bekman 提交于 4月 05, 2023
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
```
  bcb03531
05 4月, 2023 6 次提交

Enable autoTP for bloom (#3035) · 6ba0024d

由 Wang, Yi 提交于 4月 05, 2023

Signed-off-by: NWang, Yi A <yi.a.wang@intel.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

6ba0024d

O

Use generic O_DIRECT (#3115) · 514b020b
由 Olatunji Ruwase 提交于 4月 05, 2023

514b020b

Optimize Softmax Kernel (#3112) · e73de8ce

由 Molly Smith 提交于 4月 04, 2023

* Simplify kernel

* Coalesce memory attempt 1. Logits divergence.

* Logits fix?

* sync after every global mem access

* template on iterations. Down to 8.3% cuda time for 8k tokens

* Up to 64 iterations

* Add alibi/mask check

* fp32

* Revert builder.py

* naming. precommit

* Revert "naming. precommit"

This reverts commit 150eb7d9.

* naming. spacing

* Spacing. simplify checks

* remove bsyncs

* missed bsyncs

* precommit

e73de8ce

M

fix CI badges (#3138) · f2c9a827
由 Michael Wyatt 提交于 4月 04, 2023

f2c9a827
O

Fix launch issue (#3137) · fcb868e2
由 Olatunji Ruwase 提交于 4月 04, 2023

fcb868e2
L

Update SD triton version in requirements-sd.txt (#3135) · fcc0d9c0
由 Lev Kurilenko 提交于 4月 04, 2023

fcc0d9c0

01 4月, 2023 3 次提交
- J
  
  [docs] fix blog links · 62db3c0d
  由 Jeff Rasley 提交于 3月 31, 2023
  
  62db3c0d
- Z
  VL MoE Blog (#3120) · 1242d8b4
  由 Zhewei Yao 提交于 3月 31, 2023
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  1242d8b4
- M
  
  added full apache license (#3119) · 6c4782b0
  由 Michael Wyatt 提交于 3月 31, 2023
  
  6c4782b0
31 3月, 2023 1 次提交
- M
  Update DeepSpeed copyright license to Apache 2.0 (#3111) · b361c727
  由 Michael Wyatt 提交于 3月 30, 2023
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  b361c727
30 3月, 2023 2 次提交

O
Make fp32 default communication data type (#2970) · 261d6370
由 Olatunji Ruwase 提交于 3月 30, 2023
```
* Make fp32 default communication data type

* Fix asserts
```
261d6370

♻

️ replace deprecated functions for communication (#2995) · a6317eb5

由 Mayank Mishra 提交于 3月 30, 2023

* 💩 drop dead code

* ♻ replace has_all_gather_base with has_all_gather_into_tensor

* ♻ remove deprecated _all_gather_base

* ♻ remove deprecated _reduce_scatter_base

* 🎨 reformat files

* 🔧 fix _six

* Trigger CI

* Trigger CI

* Trigger CI

* 🎨 formatting

* incorporate suggestion

* incorporate suggestion

---------
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

a6317eb5

29 3月, 2023 1 次提交

Disable Stage 1&2 CPUAdam pathways (#3097) · 4b6d7c15

由 Michael Wyatt 提交于 3月 28, 2023

* disable CPUAdam pathways in optimizer copy/step

* Update stage_1_and_2.py

---------
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

4b6d7c15

28 3月, 2023 1 次提交

Fix comms benchmark import issues and support MPI/slurm launching (#2932) · 9726bd46

由 Quentin Anthony 提交于 3月 27, 2023

* Fix benchmark import issues and support MPI launching with pure torch.dist

* Formatting

* Update comms benchmark README

* Formatting

* Added better error handling and support MPI torch.dist backend

* Update formatting versions

* Formatting again

* Trigger CI

---------
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

9726bd46

27 3月, 2023 1 次提交
- J
  
  update formatter version and style settings (#3098) · 91d63e02
  由 Jeff Rasley 提交于 3月 27, 2023
  
  91d63e02
24 3月, 2023 6 次提交
- L
  Move cuda check into utils (#3074) · b3ec1c97
  由 Logan Adams 提交于 3月 23, 2023
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  b3ec1c97
- M
  pre-commit check for torch.cuda in code (#2981) · 090d49e7
  由 Ma, Guokai 提交于 3月 24, 2023
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
```
  090d49e7
- O
  Empty ZeRO3 partition cache (#3060) · e80ae088
  由 Olatunji Ruwase 提交于 3月 23, 2023
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  e80ae088
- M
  Goodbye Torch 1.8 (#3082) · 5cdf3593
  由 Michael Wyatt 提交于 3月 23, 2023
```
* bump torch18 -> torch19
* fix gptj

---------
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  5cdf3593
- S
  allow list (#3042) · 5c2a81c2
  由 Satpal Singh Rathore 提交于 3月 23, 2023
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  5c2a81c2
- F
  Fix nebula in save_16bit_model issue (#3023) · a78d6b89
  由 FreyaRao 提交于 3月 24, 2023
```
Co-authored-by: NQinghuan Rao <qinghuanrao@microsoft.com>
```
  a78d6b89
22 3月, 2023 7 次提交
- C
  Softmax Scheduling Cleanup (#3046) · 1286e374
  由 Connor Holmes 提交于 3月 22, 2023
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  1286e374
- M
  Remove bf16 from inference config dtye enum (#3010) · 27e1b02d
  由 Molly Smith 提交于 3月 22, 2023
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  27e1b02d
- M
  fix return prev key and value , added strides to from_blob (#2828) · 871c8a3f
  由 Mor Zusman 提交于 3月 22, 2023
```
Co-authored-by: NMor Zusman <morz@ai21.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  871c8a3f
- J
  
  [CI] follow-up fixes (#3072) · 36677588
  由 Jeff Rasley 提交于 3月 21, 2023
  
  36677588
- M
  Assert mp_size is factor of model dimensions (#2891) · 9ea0fdc2
  由 Molly Smith 提交于 3月 21, 2023
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  9ea0fdc2
- L
  Several fixes to unblock CI (#3047) · 4e068623
  由 Logan Adams 提交于 3月 21, 2023
```
Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  4e068623
- Q
  
  [docs] add MCR-DL paper to readme/docs (#3066) · b38b3036
  由 Quentin Anthony 提交于 3月 21, 2023
  
  b38b3036
18 3月, 2023 1 次提交
- S
  
  Fix Broken Links (#3048) · f1e4fb0b
  由 Satpal Singh Rathore 提交于 3月 18, 2023
  
  f1e4fb0b
16 3月, 2023 1 次提交
- J
  
  update email info · bbfd0a6a
  由 Jeff Rasley 提交于 3月 15, 2023
  
  bbfd0a6a
15 3月, 2023 5 次提交
- Q
  Improve loss overflow logs (#3008) · ac2c9ffa
  由 Quentin Anthony 提交于 3月 15, 2023
```
* Improve overflow logs

* Trigger CI

---------
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
```
  ac2c9ffa
- J
  Convert model parameters from generator to list. (#3017) · 94f7da26
  由 Joe Mayer 提交于 3月 15, 2023
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
```
  94f7da26
- S
  [logger] implement warning_once (#3021) · 50a49e42
  由 Stas Bekman 提交于 3月 14, 2023
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
```
  50a49e42
- J
  adding attribute checks for bf opt with zero (#3022) · d7c925e4
  由 Joe Mayer 提交于 3月 14, 2023
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
```
  d7c925e4
- S
  
  Update torch_checkpoint_engine.py (#3019) · e355863b
  由 Stas Bekman 提交于 3月 14, 2023
  
  e355863b

Greenplum / DeepSpeed 上一次同步 大约 1 年

Greenplum / DeepSpeed
上一次同步大约 1 年