* fix(wyh):model test and policy/entry test

* fix(wyh):delect qacd

* fix(wyh):test serial entry onpolicy

* add_dqfd

* Is_expert to is_expert

* modify according to the last commnets

* value_gamma; done; marginloss; sqil compatibility

* finally shorten the code, revise config

* revise config, style

* add_readme/two_more_config

* correct format
Co-authored-by: Nniuyazhe <niuyazhe@sensetime.com>

* fix(wyh):test rl_utils code

* fix(wyh):modify rl utils bug ppo adv batch B,A

* fix(wyh):style

* fix(wyh):fix bug

* feature(nyz): add gym-hybrid hybrid action space env

* style(nyz): update readme for gym_hybrid env

* slime volley env in dizoo, first commit

* fix bug in slime volley env

* modify volley env to satisfy ding 1v1 requirements; add naive self-play and league training pipeline(evaluator is not finished, now use a very naive one)

* adopt volley builtin ai as default eval opponent

* polish(nyz): polish slime_volley_env and its test

* feature(nyz): add slime_volley vs bot ppo demo

* feature(nyz): add battle_sample_serial_collector and adapt abnormal check in subprocess env manager

* feature(nyz): add slime volley self-play demo

* style(nyz): add slime_volleyball env gif and split MARL and selfplay label

* feature(nyz): add save replay function in slime volleyball env
Co-authored-by: Nzlx-sensetime <zhaoliangxuan@sensetime.com>
Co-authored-by: Nniuyazhe <niuyazhe@sensetime.com>

* added experience replay and n-step

* implementing distributional q value

* added distributional q-value

* added overview in qac_dist and d4pg

* derived D4PG from DDPG

* fixed a bug when action shape >1

* benchmark D4PG mujoco + minor fixs

-entry for DDPG mujoco
-entry for D4PG mujoco
-config for D4PG mujoco
-fixed style D4PG code
-unittests for QAC distributional

* formatted code

* minor updates (read description)

-added d4pg seria_entry test
-updated comments in QACDIST
-added d4pg in commander register
-added q_value in d4pg return dict
-added priority update in d4pg entry
-added assertion in QACDIST

* feature(zym): add offlineRL algo Discrete CQL.

* feature(zym): add offlineRL algo Discrete CQL; add hdf5 dataset for offlineRL.

* style(nyz): add ctools.pysc2 import in smac env

* feature(nyz): add smac docker build(enable docker, smac docker)

* fix(nyz): fix if condition syntax in deploy(enable docker, smac docker)

* fix(nyz): fix if condition syntax in deploy(enable docker, smac docker)

* fix(nyz): remove cache layer in smac docker(enable docker, smac docker)

* feature(nyz): use self-hosted runner in docker smac deploy(enable docker, smac docker)

* feature(nyz): build smac docker manually(enable docker, smac docker)

* feature(nyz): use docker buildx as default tool in smac and add SC2Map in setup(enable docker, smac docker)

* feature(nyz): add __init__.py in smac env maps(enable docker, smac docker)

* feature(nyz): add docker_mujoco build and upgrada numpy version to 1.20.0(enable docker)

* fix(nyz): fix numpy version compatibility bug and add -y option in apt-get(enable docker)

* fix(nyz): add libosmesa6-dev in Dockerfile.env(enable docker)

* fix(nyz): add permanent env variable about mujoco(enable docker)

* fix(nyz): change sh source to .(enable docker)

* fix(nyz): set env variable in bashrc(enable docker)

* fix(nyz): fix pip typo(enable docker)

* fix(nyz): add env in dockerfile(enable docker)

* add wqmix

* update annotation

* reformate

* update annotation

* update config

* fix annotation

* update as review

* fix as review

* add 5m6m MMM MMM2 config

* reformate

* fix(pu): fix rnn reset bug and add unittest

* fix(pu): fix rnn reset bug in centrally-weighted wqmix

* style(pu): yapf format and let WQMIXPolicy extend QMIXPolicy

* fix(pu): fix wqmix policy extend bug

* test(pu): add unittest test_wqmix

* fix(pu): fix mixer key bug in particle config

* feature(pu): add cooperative_navigation_wqmix_config

* style(pu): yapf format

* test(pu): change nn.Identity() to nn.Sequential()

* fix(pu): fix unittest bug in test_wqmix

* feature(nyz): move atari_py to ale-py and polish standard docker build(enable docker)

* fix(nyz): fix atari env import bug(enable docker)

* feature(nyz): add autorom install in docker(enable docker)

* feature(nyz): split base and env docker build(enable docker)

* fix(nyz): fix docker env source image bug(enable docker)

* add md ppo

* add doc string

fix(xjx): fix the catch statments that will never succeed in test networks; fix silence method (#71)

* style(nyz): add docker deploy workflow(enable docker)

* style(nyz): fix docker push info(enable docker)

* style(nyz): modify org name and image default name rule(enable docker)

* style(nyz): change default version to date(enable docker)

* fix(hansbug): try support spawn backend

* fix(hansbug): try fix the xxxxing problem in interaction spawn support && reformat the code style

* fix(nyz): disable silence decorator for spawn context interaction test
Co-authored-by: HansBug <killog@126.com>