提交 · f5cce75e7061a6736f3021fd5a5a0c744704ed4f · Greenplum / DeepSpeed

17 9月, 2020 3 次提交

S
Overflow fix (#416) · f5cce75e
由 Shaden Smith 提交于 9月 16, 2020
```
* Switches fused_optimizer overflow calculation
```
f5cce75e

由 Olatunji Ruwase 提交于 9月 16, 2020

* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

7d91be97

O
Add Linear warmup+decay lr schedule (#414) · 0e942df0
由 Olatunji Ruwase 提交于 9月 16, 2020
```
Update lr schedule unit tests
```
0e942df0

16 9月, 2020 2 次提交
- J
  
  add cpu-adam, reformat, add colors (#413) · a9e8325f
  由 Jeff Rasley 提交于 9月 15, 2020
  
  a9e8325f
- J
  
  fix bug related to stitching reduced grads across communication partitions (#318) · 55ed1057
  由 Jeff Rasley 提交于 9月 15, 2020
  
  55ed1057
15 9月, 2020 1 次提交
- J
  pytest skips for tests requiring certain ops (#411) · 91b4a93d
  由 Jeff Rasley 提交于 9月 15, 2020
```
* add pytest skips around tests that require certain ops to be installed
```
  91b4a93d
14 9月, 2020 1 次提交
- S
  
  scales throughput by logging freq (#408) · 473ff985
  由 Shaden Smith 提交于 9月 13, 2020
  
  473ff985
12 9月, 2020 1 次提交
- J
  Revert "supporting different intermediate sizes other than 4 * hidden_dim (#389)" (#404) · 4ac9bf60
  由 Jeff Rasley 提交于 9月 11, 2020
```
This reverts commit e549be60.
```
  4ac9bf60
11 9月, 2020 11 次提交
- R
  supporting different intermediate sizes other than 4 * hidden_dim (#389) · e549be60
  由 RezaYazdaniAminabadi 提交于 9月 11, 2020
```
* supporting different intermediate sizes other than 4*hidden_dim

* run precommit

* uncommnet the unit tests
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  e549be60
- S
  
  readthedocs upgrade (#402) · c82756cd
  由 Shaden Smith 提交于 9月 10, 2020
  
  c82756cd
- J
  
  remove old pt file · 15ca99c6
  由 Jeff Rasley 提交于 9月 10, 2020
  
  15ca99c6
- O
  Update ZeRO-Offload blog post link (#401) · d15015e9
  由 Olatunji Ruwase 提交于 9月 10, 2020
```
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  d15015e9
- J
  
  Update news site with press release link · 5dc4d6c7
  由 Jeff Rasley 提交于 9月 10, 2020
  
  5dc4d6c7
- J
  
  Update _config.yml · ea92ed29
  由 Jeff Rasley 提交于 9月 10, 2020
  
  ea92ed29
- J
  
  Update README.md · 96935958
  由 Jeff Rasley 提交于 9月 10, 2020
  
  96935958
- J
  
  bump DSE and doc tweak · 4b1df25a
  由 Jeff Rasley 提交于 9月 10, 2020
  
  4b1df25a
- J
  
  only add 1bit adam reqs if mpi is installed, update cond build for cpu-adam (#400) · 240ea97b
  由 Jeff Rasley 提交于 9月 10, 2020
  
  240ea97b
- J
  
  update docker image and bump DSE · b29229bf
  由 Jeff Rasley 提交于 9月 10, 2020
  
  b29229bf
- S
  Website edits (#398) · 6bb5c69f
  由 Shaden Smith 提交于 9月 10, 2020
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
  6bb5c69f
10 9月, 2020 16 次提交

J

Update README.md · 7baf3c3a
由 Jeff Rasley 提交于 9月 10, 2020

7baf3c3a

Landing page updates (#395) · a8a8b3d2

由 Jeff Rasley 提交于 9月 10, 2020

Co-authored-by: NShaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>

a8a8b3d2

A
Adding sparse attention news index item (#376) · c76769c4
由 Arash Ashari 提交于 9月 10, 2020
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
c76769c4
M
Minjiaz/zero offload (#382) · 59ce90d0
由 Minjia Zhang 提交于 9月 10, 2020
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
59ce90d0
A
Sparse attention: updating code tag in documentation (#394) · be4b94be
由 Arash Ashari 提交于 9月 10, 2020
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
be4b94be
J

fix for 16GB v100 nodes (#393) · b1d4bd73
由 Jeff Rasley 提交于 9月 10, 2020

b1d4bd73

ZeRO tutorials (#384) · 2dea61f2

由 Olatunji Ruwase 提交于 9月 10, 2020

Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

2dea61f2

J

Add openmpi to dockerfile · c0d5424f
由 Jeff Rasley 提交于 9月 10, 2020

c0d5424f

Fix datatype issue with sparse attention softmax (#363) · dca0b784

由 Jeff Rasley 提交于 9月 10, 2020

Fixes a dataype issue with softmax where the number of blocks being sent to the Triton kernel source was a torch.Tensor but should have been a python integer. On some environments (e.g., conda) this resulted in triton not knowing how to serialize the input (and crashing in our tests). Once switching to the correct datatype that triton expects this seems to have solved the issue.
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>

dca0b784

A
Update documentation for 1-bit Adam (#388) · 093f09ff
由 Ammar Ahmad Awan 提交于 9月 10, 2020
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
093f09ff
S
Pipeline parallel training engine. (#392) · 65c2f974
由 Shaden Smith 提交于 9月 09, 2020
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
65c2f974

ZeRO-Offload release (#391) · 41db1c2f

由 Jeff Rasley 提交于 9月 09, 2020

* ZeRO-Offload (squash) (#381)
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NReza Yazdani <reyazda@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
Co-authored-by: NJie <37380896+jren73@users.noreply.github.com>
Co-authored-by: NArash Ashari <arashari@microsoft.com>
Co-authored-by: NReza Yazdani <reyazda@microsoft.com>
Co-authored-by: NSamyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Narashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>
Co-authored-by: NRezaYazdaniAminabadi <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: NReza Yazdani <reyazda@microsoft.com>
Co-authored-by: NSamyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>

41db1c2f

J

Update test triggers to exclude docs · 79093d74
由 Jeff Rasley 提交于 9月 09, 2020

79093d74
A
fixing a link issue with SA tutorial (#387) · 161e8e60
由 Arash Ashari 提交于 9月 09, 2020
```
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
```
161e8e60

Add 1-bit Adam support to DeepSpeed (#380) · 01726ce2

由 Ammar Ahmad Awan 提交于 9月 09, 2020

* 1-bit adam (#353)
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
Co-authored-by: NYour Name <you@example.com>
Co-authored-by: Ntanghl1994 <htang14@ur.rochester.edu>
Co-authored-by: NHank <tanghl1994@gmail.com>
Co-authored-by: Nroot <root@node2x12b.cs.rochester.edu>
Co-authored-by: NAmmar Ahmad Awan <awan.ammar@microsoft.com>

01726ce2

J

temp disable model tests · 234bba0c
由 Jeff Rasley 提交于 9月 09, 2020

234bba0c

09 9月, 2020 1 次提交
- A
  
  adding sparse attention to feature index page (#377) · b73894de
  由 Arash Ashari 提交于 9月 08, 2020
  
  b73894de
07 9月, 2020 1 次提交

Update Sparse Attention Tutorial (#357) · 9dadf38d

由 Arash Ashari 提交于 9月 06, 2020

* adding BingSqaud e2e test

* updating the draft test; bring final step under try section

* finalizinf test for base deepspeed and deepspeed with ZeRO

* applying the comment (thanks Jeff); fixed formatting

* update Sparse Attention Tutorial

* fixed few issues and applied comments for better organization and readability

* updated sparse attention tutorial with making how to use section incremental; applying more comments
Co-authored-by: Narashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>

9dadf38d

06 9月, 2020 3 次提交
- O
  
  Update installation instructions (#362) · 9e83ef21
  由 Olatunji Ruwase 提交于 9月 06, 2020
  
  9e83ef21
- S
  
  Move code quality tests to Azure-hosted agents. (#368) · 4d4eafbe
  由 Shaden Smith 提交于 9月 05, 2020
  
  4d4eafbe
- A
  
  fixed a typo; this was fixed before but seems like it has been lost in the refactor (#364) · a64b0abb
  由 Arash Ashari 提交于 9月 05, 2020
  
  a64b0abb

Greenplum / DeepSpeed 上一次同步 大约 1 年

Greenplum / DeepSpeed
上一次同步大约 1 年