- 01 9月, 2020 5 次提交
-
-
由 Samyam Rajbhandari 提交于
Renaming config files to gas3
-
由 Samyam Rajbhandari 提交于
-
由 Samyam Rajbhandari 提交于
-
由 Samyam Rajbhandari 提交于
* Adding gradient accumulation support for ZeRO Stage 2. Changing all Megatron-LM tests to also test gradient accumulation * Gradient Accumulation support for Stage 2. Model tests added to test the feature * formatting * Update deepspeed_light.py removing comment * Update ds_config_func_bs8_zero1.json reverting this file back. Its not needed for this PR * defining baseline prefix Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Samyam Rajbhandari 提交于
* Update deepspeed_checkpointing.py * formatting Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
- 29 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
-
- 28 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
* Create CODEOWNERS
-
- 19 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
* turn off multi-node launch if only 1 node
-
- 14 8月, 2020 3 次提交
-
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
* update fan out flag for pdsh
-
- 13 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
-
- 12 8月, 2020 1 次提交
-
-
由 Shaden Smith 提交于
-
- 11 8月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
* add fix and tests for get_lr from lr_scheduler before training starts
-
- 09 8月, 2020 1 次提交
-
-
由 Shaden Smith 提交于
-
- 08 8月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
Add webinar on-demand links and update readme
-
由 Shaden Smith 提交于
The parenthesis alter the evaluation of the assert() and it will always evaluate to True.
-
- 01 8月, 2020 1 次提交
-
-
由 Emmanuel Kahembwe 提交于
mpu object is bound to the class instance.. the if statement uses `self.mpu' but just `mpu` is called in the following lines.. This raises a NameError
-
- 29 7月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
* fix nv_peer_mem version in dockerfile * fix security issue, remove pillow dependency (this is only needed for cifar example which has its own requirements.txt)
-
由 Emmanuel Kahembwe 提交于
-
- 28 7月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
-
- 25 7月, 2020 2 次提交
-
-
由 Shaden Smith 提交于
-
由 Jeff Rasley 提交于
-
- 24 7月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
* updates to amp to support grad clip and grad accumulation * zero grad using optimizer if in amp mode
-
- 23 7月, 2020 1 次提交
-
-
由 Olatunji Ruwase 提交于
* Avoid deadlock for unsynchronized non-zero checkpointing * Fix formatting issues Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
- 22 7月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
-
由 Arash Ashari 提交于
-
- 16 7月, 2020 2 次提交
-
-
由 Jeff Rasley 提交于
* empty grad fix * add unit tests for empty grad
-
由 Olatunji Ruwase 提交于
-
- 14 7月, 2020 2 次提交
-
-
由 Olatunji Ruwase 提交于
* Support saving and loading ZeRO checkpoints on different data parallelism degree. * Fix formatting * Support checkpoint with varying GPU count in ZeRO stage 1 * Fix formatting * Formatting fixes * Update model tests * Remove pprint * Minor fix * Fix formatting * Update model tests Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Jeff Rasley 提交于
* add amp docs
-
- 12 7月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
* add amp support for deepspeed (non-ZeRO) * tests for amp mode
-
- 07 7月, 2020 1 次提交
-
-
由 Olatunji Ruwase 提交于
* Load non-DeepSpeed checkpoints into ZeRO optimizer * Handle parameters smaller than DP * Formatting fixes * Handle empty partitions * Fix perf bug Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
- 30 6月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
-
- 26 6月, 2020 1 次提交
-
-
由 Jeff Rasley 提交于
-
- 24 6月, 2020 2 次提交
-
-
由 Conglong Li 提交于
* syntax/typo fix * add README for documentation * fix links * update navigation * typo fix * docs readme fix
-
由 Olatunji Ruwase 提交于
* Load non-DeepSpeed checkpoints into ZeRO optimizer * Handle parameters smaller than DP * Formatting fixes
-
- 20 6月, 2020 2 次提交
-
-
由 Shaden Smith 提交于
-
由 Samyam Rajbhandari 提交于
* Removing handle_overflow debugging code in deepspeed_utils.py * Removing handle_overflow debugging code in deepspeed_zero_optimizer.py Removing unnecessary overflow handle code. Not sure why it was there in the first place.
-