提交 · 67a48aaa8906878b2ce244319e219155c85de46c · Greenplum / DeepSpeed

21 4月, 2021 5 次提交

由 Conglong Li 提交于 4月 20, 2021

1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed.
Author: @conglongli, @awan-10, @samyam, Hanlin Tang, Yuxiong He
Paper: https://arxiv.org/abs/2104.06069Co-authored-by: Nsdtblck <46172032+sdtblck@users.noreply.github.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

67a48aaa

S

remove debug prints: (#986) · eecef309
由 Stas Bekman 提交于 4月 20, 2021

eecef309
S
make bold+italic work without escaping _ (#775) · 835b4c87
由 Stas Bekman 提交于 4月 20, 2021
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
```
835b4c87

[ZeRO Infinity] Allow Init to take a dict for the deepspeed config (#983) · 35251023

由 Sean Naren 提交于 4月 20, 2021

* Add check to see if json file is already loaded

* Update doc

* Address review

* Remove doc comment
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

35251023

J

add option to force multi-node launcher mode (#977) · 9e0dab40
由 Jeff Rasley 提交于 4月 20, 2021

9e0dab40

20 4月, 2021 1 次提交
- S
  
  assert no Z2/Z3 with pipeline and fix some docs links (#980) · fbece50b
  由 Shaden Smith 提交于 4月 19, 2021
  
  fbece50b
19 4月, 2021 2 次提交
- J
  
  bump to 0.3.15 · 2c2a7f31
  由 Jeff Rasley 提交于 4月 19, 2021
  
  2c2a7f31
- J
  
  [docs] zero-inf updates · 5f570bbc
  由 Jeff Rasley 提交于 4月 19, 2021
  
  5f570bbc
20 4月, 2021 1 次提交
- S
  ZeRO-Infinity docs (#979) · 11279ae4
  由 Shaden Smith 提交于 4月 19, 2021
```
* zinf tutorial

* more megatron integration docs

* ZInf + tiling docs
```
  11279ae4
19 4月, 2021 5 次提交
- J
  
  [docs] zero-inf updates · 598e50f9
  由 Jeff Rasley 提交于 4月 19, 2021
  
  598e50f9
- J
  
  [docs] add ZeRO-Inf news items · 1a741957
  由 Jeff Rasley 提交于 4月 19, 2021
  
  1a741957
- S
  ZeRO-Infinity tutorial additions (#978) · 3c47d09c
  由 Shaden Smith 提交于 4月 19, 2021
```
* zinf tutorial

* more megatron integration docs
```
  3c47d09c
- J
  
  revert zero-inf change to launcher · 72a30c1e
  由 Jeff Rasley 提交于 4月 18, 2021
  
  72a30c1e
- J
  ZeRO-Infinity (#976) · 0d4a54a0
  由 Jeff Rasley 提交于 4月 18, 2021
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NSamyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
```
  0d4a54a0
17 4月, 2021 1 次提交
- O
  Fix ZeRO-3 UnboundLocalError (#968) · 2805c393
  由 Olatunji Ruwase 提交于 4月 16, 2021
```
* Fix UnboundLocalError

* Get full partition size
```
  2805c393
15 4月, 2021 3 次提交

update lr scheduler doc for doing per step or epoch update (#913) · c83e49f9

由 Cheng Li 提交于 4月 14, 2021

* update lr scheduler doc for doing per step or epoch update

* work

* trigger build
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

c83e49f9

[zero] faster flatten/unflatten (cpp version) (#910) · 8b8ed2a7

由 Stas Bekman 提交于 4月 14, 2021

* faster flatten/unflatten with apex

* switch to cpp flatten/unflatten

* style

* better comment

* missing import

* switch to build ops at run time

* fixes
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

8b8ed2a7

S

document how to override ~/.cache/torch_extensions (#959) · 7003d447
由 Stas Bekman 提交于 4月 14, 2021

7003d447

14 4月, 2021 3 次提交
- S
  [config] turn exponential notation back on for config dump (#955) · c87118b0
  由 Stas Bekman 提交于 4月 14, 2021
```
* e-notation for large floats

* handle ints too

* readability

* handle bool
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
```
  c87118b0
- S
  
  fix double linear override; spelling (#954) · adac058a
  由 Stas Bekman 提交于 4月 14, 2021
  
  adac058a
- T
  
  Delete check of pdsh (#941) · e6999ebd
  由 Takuya Makino 提交于 4月 14, 2021
  
  e6999ebd
09 4月, 2021 2 次提交
- J
  
  add pypi badge · 112ebff6
  由 Jeff Rasley 提交于 4月 08, 2021
  
  112ebff6
- J
  
  bump to 0.3.14 · 0cd9abf2
  由 Jeff Rasley 提交于 4月 08, 2021
  
  0cd9abf2
08 4月, 2021 9 次提交
- R
  Supporting different hidden dimensions for transformer kernels-v2 (#934) · e721cb69
  由 Reza Yazdani 提交于 4月 07, 2021
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  e721cb69
- S
  docs (#909) · 31699291
  由 Stas Bekman 提交于 4月 07, 2021
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  31699291
- S
  Samyamr/stage 3 skip modules without parameters (#867) · 7b46d11f
  由 Samyam Rajbhandari 提交于 4月 07, 2021
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  7b46d11f
- S
  [zero doc] fix misspelled param (#878) · f19cf671
  由 Stas Bekman 提交于 4月 07, 2021
```
We really really really need those params to be validated...
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  f19cf671
- S
  improved readability + typos (#895) · 5ca86ae4
  由 Stas Bekman 提交于 4月 07, 2021
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  5ca86ae4
- S
  [benchmarks] flatten/unflatten benchmarks (#919) · a128f34e
  由 Stas Bekman 提交于 4月 07, 2021
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  a128f34e
- S
  fix cpu_adam memory leak on deepspeed re-use in the same process (#896) · c79184eb
  由 Stas Bekman 提交于 4月 07, 2021
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NReza Yazdani <reyazda@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  c79184eb
- S
  [zero3] GatheredParameters can now handle a list of params (#884) · 6d94afb5
  由 Stas Bekman 提交于 4月 07, 2021
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  6d94afb5
- S
  Fix for fragmented linear inputs in ZeRO 3 Linear layers where reshap… (#881) · b5f56b2c
  由 Samyam Rajbhandari 提交于 4月 07, 2021
```
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  b5f56b2c
07 4月, 2021 1 次提交
- T
  
  Add space in help string (#926) · ce14cf1a
  由 Takuya Makino 提交于 4月 07, 2021
  
  ce14cf1a
03 4月, 2021 2 次提交

A
Add link to AML examples. (#916) · ab5534fc
由 Ammar Ahmad Awan 提交于 4月 02, 2021
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
ab5534fc

disable pipe test (#915) · 8db4fdf8

由 Jeff Rasley 提交于 4月 02, 2021

This test has been giving us trouble for a bit, seeing nondeterministic failures, skipping for now to not break out CI. Need to revisit soon though.

8db4fdf8

02 4月, 2021 1 次提交

zero.Init() clarification (#880) · 5d721e09

由 Stas Bekman 提交于 4月 01, 2021

* zero.Init() clarification

clarify that if `model.half()` can't fit into gpu memory `zero.Init()` is a must.

this proposal is via @samyam's clarification shared elsewhere.

Thank you.

* style

* add clarity

* style
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

5d721e09

31 3月, 2021 4 次提交

J

[website] we're hiring! · c814abda
由 Jeff Rasley 提交于 3月 30, 2021

c814abda
J

[website] We're hiring! + integration posts · c6b497df
由 Jeff Rasley 提交于 3月 30, 2021

c6b497df
J

We're hiring! + integration posts · 8c9e16eb
由 Jeff Rasley 提交于 3月 30, 2021

8c9e16eb

Bump kramdown from 2.3.0 to 2.3.1 in /docs (#905) · c0422642

由 dependabot[bot] 提交于 3月 30, 2021

Bumps [kramdown](https://github.com/gettalong/kramdown) from 2.3.0 to 2.3.1.
- [Release notes](https://github.com/gettalong/kramdown/releases)
- [Changelog](https://github.com/gettalong/kramdown/blob/master/doc/news.page)
- [Commits](https://github.com/gettalong/kramdown/commits)
Signed-off-by: Ndependabot[bot] <support@github.com>
Co-authored-by: Ndependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

c0422642

Greenplum / DeepSpeed 上一次同步 大约 1 年

Greenplum / DeepSpeed
上一次同步大约 1 年