* Fix the half-precision version of CPU-Adam

* remove unexpected return

* fix the increase width (fp32/fp16)

* support fp16 tests for cpu-adam

* fix the fp16 data-loading

* change unit-test for fp16 check & slight change to parameter size

* fix for numpy error
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>

Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

* Split parameter offload from z3

* Format fixes

* Bug fixes

* Cleanup

* Remove dead code

* Retain prefetched params until last use

* Unit tests fixes

* fix to catch assert error for inference test imports

* fix wrong syntax

* changed to sequential inf tests

* fix for lm_eval import

* added environment check fixture

* added expected torch and cuda version

* check various version depth for cuda/torch
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NReza Yazdani <reyazda@microsoft.com>
Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>
Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com>

Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

Co-authored-by: NQuentin Anthony <qganthony@yahoo.com>
Co-authored-by: NAmmar Ahmad Awan <ammar.awan@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Add '-S' argument to pdsh command to return the largest error code from the ssh sessions

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

* added unit test for various HF model families and tasks

* formatting

* added missing import

* fixed broken pytest global vars

* modified test to conform to other test structure

* removed gpt-j. it cannot run on V100s (OOM)
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

* Fix do not updated sparse grads

* Remove call .data for sparse grads
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.13.4 to 1.13.6.
- [Release notes](https://github.com/sparklemotion/nokogiri/releases)
- [Changelog](https://github.com/sparklemotion/nokogiri/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sparklemotion/nokogiri/compare/v1.13.4...v1.13.6)

---
updated-dependencies:
- dependency-name: nokogiri
  dependency-type: indirect
...
Signed-off-by: Ndependabot[bot] <support@github.com>
Co-authored-by: Ndependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

* DeepSpeed needs to start cleaning up

* Remove debug prints