- 13 3月, 2023 1 次提交
-
-
由 Adam Moody 提交于
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
- 11 3月, 2023 1 次提交
-
-
由 Lev Kurilenko 提交于
This PR fixes Meta Tensor checkpoint loading for OPT models where the SD keys start with `model.`.
-
- 10 3月, 2023 1 次提交
-
-
由 Jeff Rasley 提交于
-
- 09 3月, 2023 1 次提交
-
-
由 Jeff Rasley 提交于
Co-authored-by: NLogan Adams <114770087+loadams@users.noreply.github.com>
-
- 08 3月, 2023 4 次提交
-
-
由 Rahil Bathwal 提交于
Co-authored-by: NRajhans Samdani <rajhans@gmail.com>
-
由 Jeff Rasley 提交于
-
由 noabauma 提交于
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Ma, Guokai 提交于
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com> Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
- 07 3月, 2023 3 次提交
-
-
由 Olatunji Ruwase 提交于
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Lev Kurilenko 提交于
-
由 Molly Smith 提交于
* check kernel injection supported models * Clarify why user should use kernel injection
-
- 02 3月, 2023 2 次提交
-
-
由 Molly Smith 提交于
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 mzl 提交于
* MPICH support * MPICH changes * MPICH changes * MPICH changes * MPICH changes * accelerator runtime modifications * Accelerator runtime changes * Accelerator runtime modifications * Remove redundant print from single node * Move hostfile to tmp * Code cleanup for MPICH class * Code cleanup, rm whitespace * Removing mpiexec environment check details * Not needed tmp hostfile as pass directly * Remove debugging comments * rm print statement * Revert comm changes as WA not needed * Use MPICHRunner name for class * Use MPICHRunner as class name * No need to use args.force_multi and args.launcher . This should be set in deepspeedexamples gpt-3.6b .sh script as: $launcher=MPICH run_cmd=" deepspeed --hostfile=${hostfile_ds} --num_nodes ${NUM_WORKERS} --num_gpus ${NUM_GPUS_PER_WORKER} --launcher=${launcher} --force_multi pretrain_gpt2.py $@ ${gpt_options}" * Adhere to code pattern * Rm empty lines in MPICHRunner class * Uncomment check for num nodes and workers when used hostfile_deepspeed in gpt-3.6b.sh * pass MPICH hostfile through launcher_args in gpt-3.6b.sh * Clean code and remove args hostfile * fix merge * fix merge --------- Co-authored-by: NAbhilash Majumder <30946547+abhilash1910@users.noreply.github.com> * clean up and fix format * add ut --------- Co-authored-by: NAbhilash Majumder <30946547+abhilash1910@users.noreply.github.com> Co-authored-by: NAmmar Ahmad Awan <ammar.awan@microsoft.com> Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
- 01 3月, 2023 3 次提交
-
-
由 Sam Foreman 提交于
Updates `deepspeed/monitor/monitor.py` to instantiate objects with correct configs Relevant issue: https://github.com/microsoft/DeepSpeed/issues/2853Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
由 Molly Smith 提交于
-
由 Farzan Taj 提交于
* #1213: Fix CPUAdam for when `vendor_id_raw` is not provided * formatting (yapf) fix --------- Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
- 28 2月, 2023 4 次提交
-
-
由 Heyang Qin 提交于
This PR updates the replace_fn function when loading inference checkpoints. The container will now be passed to the load_model_with_checkpoint() so we can call load_params() from there. load_params() is also updated to access the variables in the policy.
-
由 Mayank Mishra 提交于
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
由 Olatunji Ruwase 提交于
* Enable tensor fragments for zero 2 * Update deepspeed/utils/tensor_fragment.py Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * Update deepspeed/utils/tensor_fragment.py Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * Support offload * Support multi-gpu * Cleanup * WIP * Update deepspeed/runtime/zero/stage3.py Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * Support padding * Update deepspeed/runtime/zero/stage3.py Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * z3 optimizer state support; aligned api * Support frozen z3 params * Unit tests * Check NVMe offload capability * Formatting * Docs * More docs * More docs * Update docs/code-docs/source/zero3.rst Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * More docs * Update docs/code-docs/source/zero3.rst Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * More docs * More docs * Update docs/code-docs/source/zero3.rst Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * Update deepspeed/utils/tensor_fragment.py Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * More docs * Support unsharded fp32 grad * Remove debug prints * Fix off-by-one detection of empty grads * Update deepspeed/utils/tensor_fragment.py Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * Update deepspeed/utils/tensor_fragment.py Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * Update deepspeed/utils/tensor_fragment.py Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * Update deepspeed/runtime/zero/stage3.py Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> * Fix off-by-one error * Skip ranks with no gradient data * Formatting * Add license * Fix license --------- Co-authored-by: NStas Bekman <stas00@users.noreply.github.com> Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>
-
由 Jeff Rasley 提交于
Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com> Co-authored-by: NConglong Li <conglong.li@gmail.com> Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
- 24 2月, 2023 4 次提交
-
-
由 Olatunji Ruwase 提交于
-
由 Yasyf Mohamedali 提交于
* Remove deprecated `torch._six` imports Closes #2845. * Support older versions of PyTorch as well. --------- Co-authored-by: NJeff Rasley <jerasley@microsoft.com> Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
-
由 Molly Smith 提交于
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Heyang Qin 提交于
* Check device count before running dist tests * fixing format for "Check device count before running dist tests" * Check device count against max world size * Check GPU count before launching dist tests * double-check GPU actually exists --------- Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com> Co-authored-by: NJeff Rasley <jerasley@microsoft.com> Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>
-
- 23 2月, 2023 1 次提交
-
-
由 Molly Smith 提交于
* Modify table for compatible web format * Add tutorial links to navigation * Add news bit to main readme * Update docs/_tutorials/automatic-tensor-parallelism.md Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com> --------- Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>
-
- 22 2月, 2023 4 次提交
-
-
由 Olatunji Ruwase 提交于
* Make z3 respect comm dtype * Support fp32 comm dtype * Remove obsolete assert * Code cleanup
-
由 Conglong Li 提交于
* data efficiency library update * data efficiency library update * data efficiency update * data efficiency update
-
由 Molly Smith 提交于
Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>
-
由 Logan Adams 提交于
-
- 18 2月, 2023 4 次提交
-
-
由 Michael Wyatt 提交于
-
由 Michael Wyatt 提交于
* add auto-generated PR for private repo * change variable names
-
由 Jeff Rasley 提交于
-
由 Michael Wyatt 提交于
-
- 17 2月, 2023 2 次提交
-
-
由 Reza Yazdani 提交于
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com> Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
由 Lev Kurilenko 提交于
Co-authored-by: NReza Yazdani <reyazda@microsoft.com> Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com> Co-authored-by: NHeyang Qin <heyangqin@microsoft.com> Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>
-
- 16 2月, 2023 2 次提交
-
-
由 Ammar Ahmad Awan 提交于
Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>
-
由 Molly Smith 提交于
* Fix auto TP for duplicate modules with different gems * precommit and comments * Comment * Combine gem list of same named modules * remove duplicates from gem_list before updating policy * Add module attribute with name variation for ProphetNet --------- Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
-
- 15 2月, 2023 3 次提交
-
-
由 Michael Wyatt 提交于
* fix overlapping checkpoint names in unit tests * remove running cpu-only on master merge
-
由 Michael Wyatt 提交于
* don't run tests in parallel * make AsyncIO test sequential
-
由 Michael Wyatt 提交于
* fix permissions issue with pip upgrade * install to .local instead of use sudo * upgrade pip in venv * Update action.yml * fix typos
-