* feature(zym): add pybullet env info; add entropy type in sac.

* feature(zym): add cql; add serial entry for offlineRL.

* feature/polish(zym): add generation entry in mujoco env for offlineRL; polish cql/serial entry for offlineRL.

* feature(lj): add d4rl env for offlineRL.

* polish(zym): polish cql.

* feature/polish(zym): add dataset registry; polish offlineRL pipeline.

* fix(zym): fix bug in d4rl/mujoco config; fix bug in dataset for offlineRL.

* style(zym): add pybulletgym and d4rl requirements in setup.

* fix/polish(zym): support str in NaiveRLDataset; polish cql.

* polish(zym): polish command policy.

* feature(zym): add cql in pendulum env; add unittest/algotest for cql.

* fix(zym): fix cql bug in unittest/algotest for cql.

* enable user to use any model generated here

* delete irelevant package

* add test

* bash format.sh to reformat style

* test rnd

* fix mz config

* fix config

* fix(pu): fix r2d2

* feature(puyuan): add minigrid r2d2 config

* polish minigrid config

* modified as review

* fix(pu): fix bugffor compatibility

* polish(pu): add annotations and polish slice operation

* style(pu): run format.sh

* style(pu): correct yapf format

* feature(zlx): Add tb in naive buffer; modify tb in advanced buffer

* feature(zlx): naive_buffer tb, fix bug in valid_count update

* feature(nyz): add trueskill as league metric, naive elo calculator, fix game_env info bug

* fix(nyz): fix league player mutate bug

* fix(nyz): fix league unittest bug

* feature(nyz): add elo ranking in league metric env

* polish(nyz): modify fixed eval policy and trueskill init

* feature(nyz): add init main player in evaluation and fix stop_value bug

* style(nyz): rename test_league_metric to avoid pyc cache bug

* add modifiled predator_prey env

* add collision_ratio

* add readme and cfg for modified_predator_prey env

* add readme imgs for modified_predator_prey

* check format

* fix format

* add sqil

* conceal all the personal info

* revise according to the comments

* correct_format

* add_comment to hardcodes part

* pass flake8

* add force_reproducibility = True; device, ex_model

* check format

* fix weight bug

* update acer qbert result

* fix flake8 format problem

* update space qbert config

* update as review
Co-authored-by: Nshenziju <simonshen2011@foxmail.com>

* init runable ppo

* init overcooked env

* overcooked ppo in place

* runable ppo with shaped rewards

* modified config

* feature(nyz): modify win rate calculation with draws

* remove redundant code, modified baseline model

* Update __init__.py

* Update config.py

* modify temp_config_file.close() position in config.py to work in windows os

* remove redundant comments and rename files

* fix name bug and use namedlist

* add simple readme and remove redundant comments from copies

* resolve threads

* remove debug comments
Co-authored-by: Nniuyazhe <niuyazhe314@outlook.com>