* utilize shorter tests for MII

* use cached torch download

* rework zero++ unit tests

* formatting

---------
Co-authored-by: NHeyangQin <heyangqin@microsoft.com>

* update docs to reflect changes in deepspeed-chat training script

* add blogs to ignored changes in unit tests

Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

add reusable workflow that sets up fresh venv for each test and prints relevant environment info

* remove any cupy install when setting up environments

* revert previous changes to run on cu111 runners

* fix for when no cupy is installed

* remove cupy uninstall for workflows not using latest torch version

* update to cu116 for inference tests

* fix pip uninstall line

* move python environment list to after DS install

* remove cupy uninstall

* re-add --forked

* fix how we get cupy version (should be based on nvcc version)

* check only major CUDA version in CI

* update expected torch latest version

* pin torch latest to 1.12 until issues with 1.13 are resolve

* wrong expected torch version

* Update nv-torch18-v100.yml

* remove forked from pytest option due to cuda re-initialization errors

* removed expected torch version from inference tests, causing errors currently

* fix various bugs that popped up

* move all tests over to cu111 runners, cu113 runners having problems

* batch of refactored tests

* more test refactoring

* fp16 test refactor

* more refactors

* added DistributedFixture class

* applied DistributedFixture to first batch of tests as a trial

* added DistributedFixture test and documentation

* last tests

* fixes for refactored tests

* remove subdirs in workflow files

* fix pytest syntax error

* fix another syntax error

* update imports

* use DistFixture with elastic checkpoint test

* missing import

* update to shared class tmpdir for elastic test

* moved test files

* avoid duplicate test file name

* last refactor and moving test files

* formatting

* fix broken import

* testing forked AMD tests

* update abstract method

* use blob storage for accelerate and transformers tests

* upgrade torch for acclerate CI
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: NJeff Rasley <jerasley@microsoft.com>
Co-authored-by: NSam Ade Jacobs <samjacobs@microsoft.com>

Refactor Distributed unit tests

Fix for distributed tests
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>

Co-authored-by: NReza Yazdani <reyazda@microsoft.com>
Co-authored-by: NMichael Wyatt <michaelwyatt@microsoft.com>
Co-authored-by: NReza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com>

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>

* added concurrency to github actions

* fixed problem where one workflow can cancel another workflow

* added tmp file to help test concurrency policy on CI

* removed tmp file to finish testing concurrency policy

Co-authored-by: NJeff Rasley <jerasley@microsoft.com>