- 08 6月, 2021 3 次提交
-
-
由 Stas Bekman 提交于
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
由 Stas Bekman 提交于
* fix missed subclassed partitioning bug * fix on exit Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
由 Stas Bekman 提交于
* allow minor cuda version differences * cleanup * typo
-
- 04 6月, 2021 1 次提交
-
-
由 Stas Bekman 提交于
fixes: s/micro_batch_per_gpu/train_micro_batch_size_per_gpu/ Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
- 03 6月, 2021 3 次提交
-
-
由 Stas Bekman 提交于
-
由 Reza Yazdani 提交于
-
由 Reza Yazdani 提交于
* Change the sparse attention API to be compatible with latest changes on the triton side * remove compatibility checks for CUDA 11 * Update requirements-sparse_attn.txt Co-authored-by: NArash Ashari <arashari@microsoft.com>
-
- 28 5月, 2021 1 次提交
-
-
由 Reza Yazdani 提交于
-
- 26 5月, 2021 1 次提交
-
-
由 Reza Yazdani 提交于
-
- 25 5月, 2021 8 次提交
-
-
由 eltonzheng 提交于
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Jeff Rasley 提交于
* delay imports for replace policies and fix missing req * fix issue with _orig_layer_class always being None
-
由 Jeff Rasley 提交于
-
由 Jeff Rasley 提交于
* update with inference refs * updates
-
由 Reza Yazdani 提交于
* Fix Inference and Quantization tutorial links * fix inference api * use correct attention scaling Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Jeff Rasley 提交于
-
由 Reza Yazdani 提交于
-
由 Reza Yazdani 提交于
-
- 24 5月, 2021 1 次提交
-
-
由 Reza Yazdani 提交于
Co-authored-by: NJeff Rasley <jerasley@microsoft.com> Co-authored-by: Neltonzheng <eltonz@microsoft.com> Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com> Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com> Co-authored-by: NReza Yazdani <reyazda@microsoft.com> Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com> Co-authored-by: NElton Zheng <eltonz@microsoft.com> Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com> Co-authored-by: Neltonzheng <eltonz@microsoft.com> Co-authored-by: NArash Ashari <arashari@microsoft.com> Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com> Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com> Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com> Co-authored-by: NReza Yazdani <reyazda@microsoft.com> Co-authored-by: Nniumanar <60243342+niumanar@users.noreply.github.com> Co-authored-by: Neltonzheng <eltonz@microsoft.com> Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com> Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com> Co-authored-by: NReza Yazdani <reyazda@microsoft.com> Co-authored-by: NArash Ashari <arashari@microsoft.com> Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com> Co-authored-by: Nniumanar <60243342+niumanar@users.noreply.github.com> Co-authored-by: NJeff Rasley <jerasley@microsoft.com> Co-authored-by: Neltonzheng <eltonz@microsoft.com> Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com> Co-authored-by: NArash Ashari <arashari@microsoft.com> Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com> Co-authored-by: Nniumanar <60243342+niumanar@users.noreply.github.com>
-
- 22 5月, 2021 1 次提交
-
-
由 Meng, Peng 提交于
* fix Reduce Scatter default value * Update constants.py Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
- 21 5月, 2021 1 次提交
-
-
由 Olatunji Ruwase 提交于
* Align fp16 param wap buffers * Integrating swap buffer manager for fp16 params * Support swapping misaligned fp16 parameters * Support swap into unaligned fp16 buffer
-
- 20 5月, 2021 3 次提交
-
-
由 Mark 提交于
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 dependabot[bot] 提交于
Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.11.0 to 1.11.4. - [Release notes](https://github.com/sparklemotion/nokogiri/releases) - [Changelog](https://github.com/sparklemotion/nokogiri/blob/main/CHANGELOG.md) - [Commits](https://github.com/sparklemotion/nokogiri/compare/v1.11.0...v1.11.4) Signed-off-by: Ndependabot[bot] <support@github.com> Co-authored-by: Ndependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-
由 Jeff Rasley 提交于
-
- 19 5月, 2021 1 次提交
-
-
由 Olatunji Ruwase 提交于
* Align fp16 param wap buffers * Integrating swap buffer manager for fp16 params * Support swapping misaligned fp16 parameters
-
- 16 5月, 2021 1 次提交
-
-
由 Olatunji Ruwase 提交于
* Round robin partitioning to improve ZeRO-2 Offload CPU copy * Formatting fixes * Fix index issues in debug dumps * Remove debug prints * Code cleanup * Remove unintended stage3.py changes * Add TODO
-
- 14 5月, 2021 5 次提交
-
-
由 Shaden Smith 提交于
* is not -> != * Use pytest-randomly to seed unit tests.
-
由 Olatunji Ruwase 提交于
-
由 Stas Bekman 提交于
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
由 Stas Bekman 提交于
* [configure_distributed_model] improve assert This PR changes the 2 asserts to actually print the names of the params that are wrong. e.g.: ``` fp16 is enabled but the following parameters have dtype that is not fp16: wav2vec2.masked_spec_embed ``` * style Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
由 Olatunji Ruwase 提交于
* Fix docstring * Make screenshots clickable for easier viewing * Navigation menu in alphabetical order; More clicable screenshots * Rename 1Cycle doc * Tweak naming * Remove no longer used flag * ZeRO3 Offload release * Single GPU results * Rearrange figures * Single GPU text * tweak intro * zero3-offload section * Add asynchronous i/o docs
-
- 13 5月, 2021 2 次提交
-
-
由 Cheng Li 提交于
* use the original function's name as the key to old_functions dict * update profile output format * print at global rank 0 * add flops calculation in bwd pass using time from ds timers * improve aggregated profiling out to show all depth * print samples/second * update readme and examples * update docs * fix typo and reorder printing * fix format
-
由 William Buchwalter 提交于
* rename train_step_batch_size to train_micro_batch_size_per_gpu * clarify batch_size related doc
-
- 11 5月, 2021 1 次提交
-
-
由 Olatunji Ruwase 提交于
* Validate env; remove write size from logs * Performance scripts for auto-tunining/auto-generating aio params of deepspeed config. * Formatting fixes * Address feedback
-
- 08 5月, 2021 1 次提交
-
-
由 Olatunji Ruwase 提交于
* Unused parameters assert should be disabled by default * Fix message * Invert assert logic in unit test * Change option for ignoring unused parameters Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
- 07 5月, 2021 1 次提交
-
-
由 Mark Saroufim 提交于
* Added community tutorials to README * Update README.md
-
- 06 5月, 2021 1 次提交
-
-
由 Olatunji Ruwase 提交于
* NVMe intra-request validation should be on entire file Optimizer swap buffer sizes should be aligned * Add fix message for missing aio lib error.
-
- 05 5月, 2021 1 次提交
-
-
由 Jeff Rasley 提交于
Previously we would bump release version then push release, but then any unreleased commits in master are versioned to the previous release which isn't correct. Instead we will now push to PyPI then bump to next version for un-released commits.
-
- 04 5月, 2021 2 次提交
-
-
由 Stas Bekman 提交于
* fix assert The current assert "Model must initialized in fp16 mode for ZeRO Stage 3." needs TLC - I rewrote it completely to match its cousen assert, so now we have 2 consistent matching asserts: - f"fp16 is enabled but one or several model parameters have dtype that is not fp16" - f"fp16 is not enabled but one or several model parameters have dtype of fp16" * remove f
-
由 janEbert 提交于
Fix #1032
-
- 03 5月, 2021 1 次提交
-
-
由 Cheng Li 提交于
-