* Training pipeline of NeurIPS2019-Learn-to-Move-Challenge

* fix grammar mistakes

* release 1.2.1

* copyright

* fix bug

* refine README

* refine README

* fix typo

* Update README.md

* Update README.md

* Update reward calculation in QuickStart

* update

* yapf

* Update train.py

remove create_actors thread in train.py

* Update GA3C train.py

* add TD3

* update

* yapf.....

* Update train.py

* update dqn readme

* update merge.png

* make job run task in a separate process

* fix typo

* add more debug info in xparl client

* refine control flow of different processes in xparl job

* refine control flow of different processes in xparl job

* remove tsinghua source

* remove tsinghua source

* remove unnecessary logic

* fix typo

* refine comments and some logic

* fix bug, `decay=0` means totally synchronize weights of source model to target model

* add save_param in docs and quickstart

* Update train.py

* final submit models of NeurIPS2019 challenge

* update readme

* fix yapf

* refine comment

* add Double & Dueling DQN

* yapf......................

* update

* Update train.py

* add dygraph pg

* update acc. comments

* update comments

* Limit impala to single GPU training

* refine comment of scheduler

* refine comment

* fix minor probmels in the docs

* typo

* remove pip source

* fix monitor

* add performance of A2C

* Update README.md

* modify logger for GPU detection

* add learning curve for ES

* add learning curve for ES

* support new APIs of the cluster

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* rename learner.py

* Update README.md

* Update README.md

* Update README.cn.md

* Update README.md

* Update README.cn.md

* Update README.md

* ES example

* refine settings

* fix yapf

* refine documentation; remove csv logger

* fix bug

* merge learner.py and train.py; add version requirements of gym and atari_py

* unify actor num

* add new_alg.rst

* rename LiftSim_demo as LiftSim_baseline

* Update new_alg.rst

* Update new_alg.rst

* add liftsim baseline

* yapf

* yapf...

* modify acc. comments

* yapf

* yapf..........

* yapf!

why is yapf on paddle different from that on my mac!!!!!

* first pr

* start a worker when the master is started.

* First PR & Fix logger bugs.

* update docs for a2c, impala and ga3c

* update doc

* yapf modification

* update logger

* yapf correct

* yapf

* setup.py

* old setup.py

* worker 86

* new feature: save params

* add unittest for save()/retore()

* add an example demonstrating the usage

* rename the variable

* yapf

* fix comment

* fix some problems of tensorboard

* yapf

* fix the compatibility issue

* fix the comment issue

* support paddle 1.5.1 and replace PE with compiler

* yapf&copyright

* yapf

* fix the teamcity problem

* fix the teamcity problem

* fix comment

* only support paddle 1.5.1

* Cmake

* fix comment

* fix the compatibility issue

* fix the comment issue

* remove previous algorithms; use the forward function in parl.Model

* remove abundant lines

* yapf

* intra-version: move parl.framework into parl.core.fluid

* add folder: parl.core

* remove former test folders

* yapf

* yapf0.24

* make the quickstart more compact

* remove args in the main function

* yapf

* add gif

* remove render

* Update README.md

* Update README.md

* Update README.md

* Update README.cn.md

* Update README.md

* Update README.md

* Update README.cn.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* refine A2C example

* fix unittest in python2; fix codestyle

* fix codestyle

* refine comment

* add A2C benchmark; add more information in PyPI homepage

* filter picture in PyPI homepage

* fix typo

* Update README.md

* Update README.md

* Update README.md

* soft depend on fluid; add module to monitor client status

* improve performance of IMPALA example

* fix bug of some client cannot exit normally

* refine comment

* .

* add IMPALA algorithm and some common utils

* update README.md

* refactor files structure of impala algorithm; seperate numpy utils from utils

* add hyper parameter scheduler module; add entropy and lr scheduler in impala

* clip reward in atari wrapper instead of learner side; fix codestyle

* add benchmark result of impala; refine code of impala example; add obs_format in atari_wrappers

* Update README.md

* add a3c algorithm, A2C example and rl_utils

* require training in single gpu/cpu

* only check cpu/gpu num in learner

* refine Readme

* update impala benchmark picture; update Readme

* add benchmark result of A2C

* move get_params/set_params in agent_base

* add GA3C example

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* refine Readme

* add benchmark

* add default safe eps in numpy logp calculation

* refine document; make unittest stable

* add IMPALA algorithm and some common utils

* update README.md

* refactor files structure of impala algorithm; seperate numpy utils from utils

* add hyper parameter scheduler module; add entropy and lr scheduler in impala

* clip reward in atari wrapper instead of learner side; fix codestyle

* add benchmark result of impala; refine code of impala example; add obs_format in atari_wrappers

* Update README.md

* add a3c algorithm, A2C example and rl_utils

* require training in single gpu/cpu

* only check cpu/gpu num in learner

* refine Readme

* update impala benchmark picture; update Readme

* add benchmark result of A2C

* move get_params/set_params in agent_base

* fix shell script cannot run in ubuntu

* refine comment and document

* Update README.md

* Update README.md

* add IMPALA algorithm and some common utils

* update README.md

* refactor files structure of impala algorithm; seperate numpy utils from utils

* add hyper parameter scheduler module; add entropy and lr scheduler in impala

* clip reward in atari wrapper instead of learner side; fix codestyle

* add benchmark result of impala; refine code of impala example; add obs_format in atari_wrappers

* Update README.md

* add api set_params/get_params in Model; add Interface of Network and LayerFunc to solve circular imports; refine parameter_names api of Model

* remove licence in third party code; remove interface of Network and LayerFunc; move get_parameter_pairs and get_parameter_names api to Network

* refine comment

* refine commment

* Update README.md

* Update train.py

* Update README.md

* Update agent_base.py

* Update train.py

* Update train.py

* Update train.py

* Update model_base.py

* Update README.md

* Update README.md

* fix PPO bug; add more benchmark result

* refine code

* update benchmark of PPO, after fix bug

* refine code

* add more dqn benchmark result; unify train scripts

* resize benchmark picture

* resize benchmark picture, refine comments of args

* change dependence, mujoco only support python3 now