* feature(wyh):mappo and ippo win rate and time

* feature(wyh):mappo and ippo epymarl win rate and time

* feature(wyh):smac epymarl commit id
Co-authored-by: N卫昱宏 <SENSETIME\weiyuhong@cn0214000504l.domain.sensetime.com>

* fix(wyh): plot function

* fix(wyh): plot function pytest

* fix(wyh):plot function modify comments

* feature(wyh):plot style
Co-authored-by: Nweiyuhong <weiyuhong@sensetime.com>

* start implementing bsuite env

* add bsuite env

* Implemented

* removed unused file

* added cartpole_swing environment

* Update test_bsuite_env.py

* added env in readme and in setup.py

* Create bsuite.png

* update md_dqn

* update offpolicy ppo

* add rainbow md policy

* format code

* del ppo; leave to future updates

* add doc string; fix rainbow returns

* test rnd

* fix mz config

* fix config

* fix(pu): fix r2d2

* fix(puyuan): fix r2d2

* feature(puyuan): add minigrid r2d2 config

* polish minigrid config

* modified as review

* fix(pu): fix bugffor compatibility

* polish(pu): add annotations and polish slice operation

* style(pu): run format.sh

* style(pu): correct yapf format

* fix(pu): fix config

* fix(pu): fix done slice bug and lstm reset bug

* style(pu): format config

* polish(pu): polish config params for cartpole, lunarlander and minigrid

* polish(pu): polish minigrid config params

* Update r2d2.py

* polish(pu): polish rnn reset problem

* fix(pu): fix merge error

* polish(pu): polish cartpole config

* polish(nyz): polish cartpole r2d2 config for faster convergence

* test(nyz): enable r2d2 algotest
Co-authored-by: Nniuyazhe <niuyazhe@sensetime.com>

* fix_formatted_config_bug_eval

* fix(wyh):add config pytest

* feature(nyz): add trueskill as league metric, naive elo calculator, fix game_env info bug

* fix(nyz): fix league player mutate bug

* fix(nyz): fix league unittest bug

* feature(nyz): add elo ranking in league metric env

* polish(nyz): modify fixed eval policy and trueskill init

* add_scheduler_module

* fix_change_range_and_factor

* cooldown_counter_bug_fix

* add_div_mode

* code_format_fixed

* fix_pr_bug

* add_unnitest_module

* add_patience_test

* polish(nyz): polish scheduler design and fix league mode scheduler bug

* fix(nyz): fix merge test_metric.py bug
Co-authored-by: Nniuyazhe <niuyazhe@sensetime.com>
Co-authored-by: N李可 <like2@CN0014008466M.local>

* fix_mappo_bug_masknan_and_dict_cannot_unsqueeze

* squeeze_bug

* test dijob

* test: wait for dijob Succeeded phase, and read coordinator logs

* test: update wait condition

* ci: update algo_test.yaml and flake check

* test: move kubernetes package to where it will be used

* feat: add k8s launcher

* feat: install kubectl when install k3d

* feat: add orchestrator launcher and a test case

* ci: install kubernetes related package and cli

* style: format code

* style: flake check code

* test k8s launcher

* ci: change back to unit test

* feat: delete cert manager when delete orchestrator

* style: flake8 check

* feat: merge k8s-launcher with k8s-helper

1. merge k8s-launcher with k8s-helper
2. move kubernetes package import to where it will be used
3. hack/install-k8s-tools.sh -> ding/scripts/install-k8s-tools.sh

* feature(nyz): add resnet for cv sl task

* feature(nyz): add imagenet classification dataset and adapt compile config for sl

* feature(nyz): add naive image training entry demo

* style(nyz): polish image cls train log

* polish(nyz): polish multi gpu training setting

* feature(nyz): add nn training bp and update async execution

* feature(nyz): add distributed sampler for different dist backend

* fix(nyz): fix compile config collector and buffer compatibility problem

* style(nyz): correct yapf format

* fix(nyz): fix env manager compile config compatibility bug

* refactor(nyz): abstarct ISerialEvaluator and rename serial evaluation implementation

* refactor(nyz): refactor collector name

* feature(nyz): add metric evaluator and image cls acc metric eval demo

* fix(nyz): fix cuda and multi gpu bug in image cls demo

* env-list

* env-list-fix-grammmer

* env-only-test

* modify-gif

* modify-gif-pendulum

* modify-gif-delect-maze

* feature(zym): add pybullet env info; add entropy type in sac.

* feature(zym): add cql; add serial entry for offlineRL.

* feature/polish(zym): add generation entry in mujoco env for offlineRL; polish cql/serial entry for offlineRL.

* feature(lj): add d4rl env for offlineRL.

* polish(zym): polish cql.

* feature/polish(zym): add dataset registry; polish offlineRL pipeline.

* fix(zym): fix bug in d4rl/mujoco config; fix bug in dataset for offlineRL.

* style(zym): add pybulletgym and d4rl requirements in setup.

* fix/polish(zym): support str in NaiveRLDataset; polish cql.

* polish(zym): polish command policy.

* feature(zym): add cql in pendulum env; add unittest/algotest for cql.

* fix(zym): fix cql bug in unittest/algotest for cql.

* enable user to use any model generated here

* delete irelevant package

* add test

* bash format.sh to reformat style

* test rnd

* fix mz config

* fix config

* fix(pu): fix r2d2

* feature(puyuan): add minigrid r2d2 config

* polish minigrid config

* modified as review

* fix(pu): fix bugffor compatibility

* polish(pu): add annotations and polish slice operation

* style(pu): run format.sh

* style(pu): correct yapf format

* feature(zlx): Add tb in naive buffer; modify tb in advanced buffer

* feature(zlx): naive_buffer tb, fix bug in valid_count update

* feature(nyz): add trueskill as league metric, naive elo calculator, fix game_env info bug

* fix(nyz): fix league player mutate bug

* fix(nyz): fix league unittest bug

* feature(nyz): add elo ranking in league metric env

* polish(nyz): modify fixed eval policy and trueskill init

* feature(nyz): add init main player in evaluation and fix stop_value bug

* style(nyz): rename test_league_metric to avoid pyc cache bug