* Fixes for asymmetric quantization

* addtional offset to further improve accuracy

* put the 0.5 into offset rather than applying it later

* update unit test for quantization

* fix format

* attempt to fix format

---------
Co-authored-by: NConnor Holmes <connorholmes@microsoft.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

resutls -> results
Co-authored-by: NLogan Adams <114770087+loadams@users.noreply.github.com>

update link to PL docs talking about DeepSpeed integration

Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>

* Update torch version check in building sparse_attn

* Update triton check

* Fix error message

* Format fixes

* Test with triton 2

* Change requirements back

* fix to just prevent 2.0.0

* Fix formatting

---------
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

* support nesting zero.Init() and dynamically defined module

* throw an error if model class defined in zero.Init is not wrapped

* fix check on new classes that are not wrapped in zero.Init()

* add tests of nesting zero.Init() and dynamically defined classes

* fix tests for zero.Init

* fix style

---------
Co-authored-by: NLogan Adams <114770087+loadams@users.noreply.github.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

* update docs to reflect changes in deepspeed-chat training script

* add blogs to ignored changes in unit tests

* add fallback path for kernels used in megatron

* temporary numactl WA for SPR 56core

* adapt core allocation according to number of ranks

* add switch to turn on numactl

* detect number of cores on the system

* allow select a subset of the cores on the system to bind

* remove unneeded changes

* use current_env to set OMP_NUM_THREADS in subprocess

* add test for ds_arguments

* change --bind_cores_to_rank option to store_true

* add test for parse_range_list

* add comment for parse range list

* add test for parse range list, rewrite parse_range_list

* fix format error

* fix format

* add -m parameter to numactl when necessary

* Check KMP_AFFINITY to avoid conflict with numactl

* fix format

* negative case for parse_range_list

* detect whether numactl is installed before use numactl to bind cores

* check numactl with package manager of distro

---------
Co-authored-by: Nsdp <sdp@aia-sdp-spr-108864.jf.intel.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

* Update AMD workflows

* Update MI200 test flow to use torch latest

* Update tolerances to values that pass (will fix before completing PR)

* Revert chyanges to atol

* Rename workflows

* Fix CI badges

* feat: Add support for `NamedTuple` when sharding parameters [#3029]

* Formatting

---------
Co-authored-by: NAlexander van Eck <alexander.vaneck@paige.ai>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

Co-authored-by: NConglong Li <conglong.li@gmail.com>

Co-authored-by: NAmmar Ahmad Awan <ammar.awan@microsoft.com>

Fix microsoft/DeepSpeed#3163

Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>

Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: Nyaozhewei <zheweiy@berkeley.edu>
Co-authored-by: NAmmar Ahmad Awan <ammar.awan@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
Co-authored-by: NConnor Holmes <connorholmes@microsoft.com>
Co-authored-by: NLok Chand Koppaka <lokoppak@microsoft.com>
Co-authored-by: NMasahiro Tanaka <81312776+tohtana@users.noreply.github.com>
Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>

Co-authored-by: NSamyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: NReza Yazdani <reyazda@microsoft.com>
Co-authored-by: NSamyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: NConglong Li <conglong.li@gmail.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
Co-authored-by: NZhewei Yao <zheweiyao@gmail.com>

Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

* Persist params in zero.Init

* Disable debug prints

* Formatting

* Avoid offloading persisted params

* Simplify world_size=1

* Formatting

* Remove pdb

* Restructure

* Formating

* Formatting

* Apply persistence only if ds_config available

* Fix typo

* add util function for getting pydantic config default values

---------
Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>

This reverts commit 0f8b5da6.

Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NBing Xie <bingxie@BINGHYPC014.redmond.corp.microsoft.com>
Co-authored-by: NShaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

* fix copyright script and add replace-copyright script

cross attention kwargs and vae config for diffusers 0.14.0

* submit changes

* update format

* fix fomrat

* revert

* test

* add top

* treat z1 as z2

* revert

---------
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

This reverts commit 1ec34e54.

This was breaking the link to the Pyramid MoE paper on Arxiv