- 04 9月, 2020 2 次提交
-
-
由 Shaden Smith 提交于
-
由 Arash Ashari 提交于
* adding link to Sparse Attention in Navigation page
-
- 03 9月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
-
- 02 9月, 2020 3 次提交
-
-
由 Jeff Rasley 提交于
Remove llvm/cmake install for now, causing pyyaml issues
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
* Sparse attn + ops/runtime refactor + v0.3.0 Co-authored-by: NArash Ashari <arashari@microsoft.com> Co-authored-by: NArash Ashari <arashari@microsoft.com>
-
- 01 9月, 2020 7 次提交
-
-
由 Shaden Smith 提交于
-
由 Jeff Rasley 提交于
-
由 Samyam Rajbhandari 提交于
Renaming config files to gas3
-
由 Samyam Rajbhandari 提交于
-
由 Samyam Rajbhandari 提交于
-
由 Samyam Rajbhandari 提交于
* Adding gradient accumulation support for ZeRO Stage 2. Changing all Megatron-LM tests to also test gradient accumulation * Gradient Accumulation support for Stage 2. Model tests added to test the feature * formatting * Update deepspeed_light.py removing comment * Update ds_config_func_bs8_zero1.json reverting this file back. Its not needed for this PR * defining baseline prefix Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Samyam Rajbhandari 提交于
* Update deepspeed_checkpointing.py * formatting Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
- 29 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
-
- 28 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
* Create CODEOWNERS
-
- 19 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
* turn off multi-node launch if only 1 node
-
- 14 8月, 2020 3 次提交
-
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
* update fan out flag for pdsh
-
- 13 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
-
- 12 8月, 2020 1 次提交
-
-
由 Shaden Smith 提交于
-
- 11 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
* add fix and tests for get_lr from lr_scheduler before training starts
-
- 09 8月, 2020 1 次提交
-
-
由 Shaden Smith 提交于
-
- 08 8月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
Add webinar on-demand links and update readme
-
由 Shaden Smith 提交于
The parenthesis alter the evaluation of the assert() and it will always evaluate to True.
-
- 01 8月, 2020 1 次提交
-
-
由 Emmanuel Kahembwe 提交于
mpu object is bound to the class instance.. the if statement uses `self.mpu' but just `mpu` is called in the following lines.. This raises a NameError
-
- 29 7月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
* fix nv_peer_mem version in dockerfile * fix security issue, remove pillow dependency (this is only needed for cifar example which has its own requirements.txt)
-
由 Emmanuel Kahembwe 提交于
-
- 28 7月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
-
- 25 7月, 2020 2 次提交
-
-
由 Shaden Smith 提交于
-
由 Jeff Rasley 提交于
-
- 24 7月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
* updates to amp to support grad clip and grad accumulation * zero grad using optimizer if in amp mode
-
- 23 7月, 2020 1 次提交
-
-
由 Olatunji Ruwase 提交于
* Avoid deadlock for unsynchronized non-zero checkpointing * Fix formatting issues Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
- 22 7月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
-
由 Arash Ashari 提交于
-
- 16 7月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
* empty grad fix * add unit tests for empty grad
-
由 Olatunji Ruwase 提交于
-
- 14 7月, 2020 1 次提交
-
-
由 Olatunji Ruwase 提交于
* Support saving and loading ZeRO checkpoints on different data parallelism degree. * Fix formatting * Support checkpoint with varying GPU count in ZeRO stage 1 * Fix formatting * Formatting fixes * Update model tests * Remove pprint * Minor fix * Fix formatting * Update model tests Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-