- 10 9月, 2020 12 次提交
-
-
由 Arash Ashari 提交于
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Jeff Rasley 提交于
-
由 Olatunji Ruwase 提交于
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com> Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
Fixes a dataype issue with softmax where the number of blocks being sent to the Triton kernel source was a torch.Tensor but should have been a python integer. On some environments (e.g., conda) this resulted in triton not knowing how to serialize the input (and crashing in our tests). Once switching to the correct datatype that triton expects this seems to have solved the issue. Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
-
由 Ammar Ahmad Awan 提交于
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Shaden Smith 提交于
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Jeff Rasley 提交于
* ZeRO-Offload (squash) (#381) Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com> Co-authored-by: NReza Yazdani <reyazda@microsoft.com> Co-authored-by: NJeff Rasley <jerasley@microsoft.com> Co-authored-by: NJie <37380896+jren73@users.noreply.github.com> Co-authored-by: NArash Ashari <arashari@microsoft.com> Co-authored-by: NReza Yazdani <reyazda@microsoft.com> Co-authored-by: NSamyam Rajbhandari <samyamr@microsoft.com> Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com> Co-authored-by: Narashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com> Co-authored-by: NRezaYazdaniAminabadi <44502768+RezaYazdaniAminabadi@users.noreply.github.com> Co-authored-by: NReza Yazdani <reyazda@microsoft.com> Co-authored-by: NSamyam Rajbhandari <samyamr@microsoft.com> Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
-
由 Jeff Rasley 提交于
-
由 Arash Ashari 提交于
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Ammar Ahmad Awan 提交于
* 1-bit adam (#353) Co-authored-by: NJeff Rasley <jerasley@microsoft.com> Co-authored-by: NYour Name <you@example.com> Co-authored-by: Ntanghl1994 <htang14@ur.rochester.edu> Co-authored-by: NHank <tanghl1994@gmail.com> Co-authored-by: Nroot <root@node2x12b.cs.rochester.edu> Co-authored-by: NAmmar Ahmad Awan <awan.ammar@microsoft.com>
-
由 Jeff Rasley 提交于
-
- 09 9月, 2020 1 次提交
-
-
由 Arash Ashari 提交于
-
- 07 9月, 2020 1 次提交
-
-
由 Arash Ashari 提交于
* adding BingSqaud e2e test * updating the draft test; bring final step under try section * finalizinf test for base deepspeed and deepspeed with ZeRO * applying the comment (thanks Jeff); fixed formatting * update Sparse Attention Tutorial * fixed few issues and applied comments for better organization and readability * updated sparse attention tutorial with making how to use section incremental; applying more comments Co-authored-by: Narashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>
-
- 06 9月, 2020 3 次提交
-
-
由 Olatunji Ruwase 提交于
-
由 Shaden Smith 提交于
-
由 Arash Ashari 提交于
-
- 04 9月, 2020 2 次提交
-
-
由 Shaden Smith 提交于
-
由 Arash Ashari 提交于
* adding link to Sparse Attention in Navigation page
-
- 03 9月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
-
- 02 9月, 2020 3 次提交
-
-
由 Jeff Rasley 提交于
Remove llvm/cmake install for now, causing pyyaml issues
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
* Sparse attn + ops/runtime refactor + v0.3.0 Co-authored-by: NArash Ashari <arashari@microsoft.com> Co-authored-by: NArash Ashari <arashari@microsoft.com>
-
- 01 9月, 2020 7 次提交
-
-
由 Shaden Smith 提交于
-
由 Jeff Rasley 提交于
-
由 Samyam Rajbhandari 提交于
Renaming config files to gas3
-
由 Samyam Rajbhandari 提交于
-
由 Samyam Rajbhandari 提交于
-
由 Samyam Rajbhandari 提交于
* Adding gradient accumulation support for ZeRO Stage 2. Changing all Megatron-LM tests to also test gradient accumulation * Gradient Accumulation support for Stage 2. Model tests added to test the feature * formatting * Update deepspeed_light.py removing comment * Update ds_config_func_bs8_zero1.json reverting this file back. Its not needed for this PR * defining baseline prefix Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Samyam Rajbhandari 提交于
* Update deepspeed_checkpointing.py * formatting Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
- 29 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
-
- 28 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
* Create CODEOWNERS
-
- 19 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
* turn off multi-node launch if only 1 node
-
- 14 8月, 2020 3 次提交
-
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
* update fan out flag for pdsh
-
- 13 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
-
- 12 8月, 2020 1 次提交
-
-
由 Shaden Smith 提交于
-
- 11 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
* add fix and tests for get_lr from lr_scheduler before training starts
-