* fix compatibility issue with the newest paddle

* remove logging lines

* resolve the compatibility issue with the newest paddle

* yapf
Co-authored-by: Nrobot <zenghongsheng@baidu.com>

* add SGD and Adam Optimizer for DeepES

* update deepes readme

* add warning when input different size in the same param update()

* add error return in update(), add optimizer.cc

* separate SGD and Adam, optimizer type in config is not case sensitive

* delete optimizer.cc

* config optimizer in deepes.proto

* more readable

* update maddpg readme, fixed gym version

* add tutorial of deepes, written with numpy, less than 100lines

* modify learning_rate as an arugment of Agent

* liftsim a2c baseline

* update readme

* compatible with different os

* empty

* refine comments

* remove unnecessary assertion; add tensorboard guide

* remove unnecessary assertion

* update parl dependence of A2C

* update comments for ES

* check dependence on paddle or torch

* update readme

* update readme#2

* users can still use parl.remote when no DL framework was found

* yapf

* fix paddle version bug

* add gym dependence (introduced by MADDPG)

* recall

* add maddpg example

* format with yapf

* fix coding style

* fix coding style

* unittest without import multiagent env

* update maddpg code

* update maddpg readme

* add copyright comments

* add offline q learning

* Update README.md

* update

* yapf

* add sac

* Training pipeline of NeurIPS2019-Learn-to-Move-Challenge

* fix grammar mistakes

* release 1.2.1

* copyright

* fix bug

* refine README

* refine README

* fix typo

* Update README.md

* Update README.md

* Update reward calculation in QuickStart

* update

* yapf

* Update train.py

remove create_actors thread in train.py

* Update GA3C train.py

* add TD3

* update

* yapf.....

* Update train.py

* update dqn readme

* update merge.png

* make job run task in a separate process

* fix typo

* add more debug info in xparl client

* refine control flow of different processes in xparl job

* refine control flow of different processes in xparl job

* remove tsinghua source

* remove tsinghua source

* remove unnecessary logic

* fix typo

* refine comments and some logic

* fix bug, `decay=0` means totally synchronize weights of source model to target model

* add save_param in docs and quickstart

* Update train.py

* final submit models of NeurIPS2019 challenge

* update readme

* fix yapf

* refine comment

* add Double & Dueling DQN

* yapf......................

* update

* Update train.py

* add dygraph pg

* update acc. comments

* update comments

* Limit impala to single GPU training

* refine comment of scheduler

* refine comment

* fix minor probmels in the docs

* typo

* remove pip source

* fix monitor

* add performance of A2C

* Update README.md

* modify logger for GPU detection

* add learning curve for ES

* add learning curve for ES

* support new APIs of the cluster

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* rename learner.py

* Update README.md

* Update README.md

* Update README.cn.md

* Update README.md

* Update README.cn.md

* Update README.md

* ES example

* refine settings

* fix yapf

* refine documentation; remove csv logger

* fix bug

* merge learner.py and train.py; add version requirements of gym and atari_py

* unify actor num

* add new_alg.rst

* rename LiftSim_demo as LiftSim_baseline

* Update new_alg.rst

* Update new_alg.rst

* add liftsim baseline

* yapf

* yapf...

* modify acc. comments

* yapf

* yapf..........

* yapf!

why is yapf on paddle different from that on my mac!!!!!

* first pr

* start a worker when the master is started.

* First PR & Fix logger bugs.

* update docs for a2c, impala and ga3c

* update doc

* yapf modification

* update logger

* yapf correct

* yapf

* setup.py

* old setup.py

* worker 86

* new feature: save params

* add unittest for save()/retore()

* add an example demonstrating the usage

* rename the variable

* yapf

* fix comment

* fix some problems of tensorboard

* yapf

* fix the compatibility issue

* fix the comment issue

* support paddle 1.5.1 and replace PE with compiler

* yapf&copyright

* yapf

* fix the teamcity problem

* fix the teamcity problem

* fix comment

* only support paddle 1.5.1

* Cmake

* fix comment

* fix the compatibility issue

* fix the comment issue

* remove previous algorithms; use the forward function in parl.Model

* remove abundant lines

* yapf

* intra-version: move parl.framework into parl.core.fluid

* add folder: parl.core

* remove former test folders

* yapf

* yapf0.24

* make the quickstart more compact

* remove args in the main function

* yapf

* add gif

* remove render

* Update README.md

* Update README.md

* Update README.md

* Update README.cn.md

* Update README.md

* Update README.md

* Update README.cn.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* refine A2C example

* fix unittest in python2; fix codestyle

* fix codestyle

* refine comment

* add A2C benchmark; add more information in PyPI homepage

* filter picture in PyPI homepage