* update comments for ES

* check dependence on paddle or torch

* update readme

* update readme#2

* users can still use parl.remote when no DL framework was found

* yapf

* add maddpg example

* format with yapf

* fix coding style

* fix coding style

* unittest without import multiagent env

* update maddpg code

* update maddpg readme

* add copyright comments

* update parl.maddpg without import gym

* update NeurlIPS2018.gif to NeurlIPS2019.gif

* update readme and comments

* fix paddle version bug

* add gym dependence (introduced by MADDPG)

* recall

* torch benchmark policy gradient

* refine comments and use native api

* add maddpg example

* format with yapf

* fix coding style

* fix coding style

* unittest without import multiagent env

* update maddpg code

* update maddpg readme

* add copyright comments

* add offline q learning

* Update README.md

* update

* yapf

* add sac

* Training pipeline of NeurIPS2019-Learn-to-Move-Challenge

* fix grammar mistakes

* release 1.2.1

* copyright

* fix bug

* refine README

* refine README

* fix typo

* Update README.md

* Update README.md

* Update reward calculation in QuickStart

* update

* yapf

* Update train.py

remove create_actors thread in train.py

* Update GA3C train.py

* add TD3

* update

* yapf.....

* Update train.py

* update dqn readme

* update merge.png

* make job run task in a separate process

* fix typo

* add more debug info in xparl client

* refine control flow of different processes in xparl job

* refine control flow of different processes in xparl job

* remove tsinghua source

* remove tsinghua source

* remove unnecessary logic

* fix typo

* refine comments and some logic

* fix bug, `decay=0` means totally synchronize weights of source model to target model

* add save_param in docs and quickstart

* Update train.py

* add pytorch a2c

* add set/get_weights test & copyright

* yapf....

* Update model_base_test_torch.py

* update

* Delete banma.py

* Update model_base_test_torch.py

* update

* Update model.py

* update torch tests

* Update model_base_test_torch.py

* final submit models of NeurIPS2019 challenge

* update readme

* fix yapf

* refine comment