* stop using pickle cloud to transmit the remote class

* add VisualDL as summary logging backend

* yapf

* copyright

* using visualdl as the default logging tool

* only support linux at the moment

* yapf

* yapf

* remove dependency on six

* unittest

* yapf

* ping the master before connection

* yapf

* fix comments

* remove the useless library

* install ping for the docker environment

* remove protobuf intallation

* remove evokit test

* replace tensorboard with summary to support VDL in the future

* unittest

* rename keys for record

* yapf

* Update utils.py

* Update utils.py

* state to obs

* yapf & update softlink in offline-q-learning

* first version; unittest in parl/remote is passed

* add windows local unittest shell; fix some incompatible problem

* fix save api

* refine comments and remove log of xparl stop

* fix bug

* Update scripts.py

* Update utils.py

* Update agent_base_test.py

* remove version 1.3 warnings

* update

* yapf

* add algorithms test

* Update algs_test.py

* Update algs_test.py

add SAC DDPG TD3 tests

* yapf

* release v1.2.3

* change dep of liftsim a2c

* fix a2c cannot run in paddle 1.6.0

* fix impala compatibility

* yapf

* fix compatibility issue with the newest paddle

* remove logging lines

* resolve the compatibility issue with the newest paddle

* yapf
Co-authored-by: Nrobot <zenghongsheng@baidu.com>

* fix compatibility issue with the newest paddle

* remove logging lines
Co-authored-by: Nrobot <zenghongsheng@baidu.com>

* mac compatibility

* refine import trick

* update comments for ES

* check dependence on paddle or torch

* update readme

* update readme#2

* users can still use parl.remote when no DL framework was found

* yapf

* add maddpg example

* format with yapf

* fix coding style

* fix coding style

* unittest without import multiagent env

* update maddpg code

* update maddpg readme

* add copyright comments

* update parl.maddpg without import gym

* update NeurlIPS2018.gif to NeurlIPS2019.gif

* update readme and comments

* torch benchmark policy gradient

* refine comments and use native api

* add maddpg example

* format with yapf

* fix coding style

* fix coding style

* unittest without import multiagent env

* update maddpg code

* update maddpg readme

* add copyright comments

* add sac

* Training pipeline of NeurIPS2019-Learn-to-Move-Challenge

* fix grammar mistakes

* release 1.2.1

* copyright

* fix bug

* refine README

* refine README

* fix typo

* Update README.md

* Update README.md

* add TD3

* update

* yapf.....

* Update train.py

* make job run task in a separate process

* fix typo

* add more debug info in xparl client

* refine control flow of different processes in xparl job

* refine control flow of different processes in xparl job

* remove tsinghua source

* remove tsinghua source

* remove unnecessary logic

* fix typo

* refine comments and some logic

* fix bug, `decay=0` means totally synchronize weights of source model to target model

* add pytorch a2c

* add set/get_weights test & copyright

* yapf....

* Update model_base_test_torch.py

* update

* Delete banma.py

* Update model_base_test_torch.py

* update

* Update model.py

* update torch tests

* Update model_base_test_torch.py

* add Double & Dueling DQN

* yapf......................

* update

* Update train.py

* torch test env

* Update build.sh

* update torch unit test

* git commit -m torchdqn

* yapf

* fix bugs

* fix bugs

* fix bugs

* yapf

* remove fstring format

* torch_test yapf

* yapf

* Add torch in unittest.requirements

* update torch_unittest

* Torch and FLUID conflict problem in __init__.py

* Unittest fail for torch when both torch and fluid exists.

* cluster_test fail in the unittest, add timeout seconds.

* Torch backend for PARL

* add sleep time for unit test send_job_test.py

* Unit test for send_job_test.py

* use multiple try for unit test

* Fix compatibility for python2.7.

* fix send_job_test.py bugs

* check file exist before send_job_test.py

* Modify send_job_test.py

* Limit impala to single GPU training

* refine comment of scheduler

* refine comment

* clear the directory for the function

* add the unit test

* unit test bug

* yapf