* Support Heter Parameter Server

* add doc; notest

* fix doc; notest

* update doc; notest

* refine optimizer && adam

* refine optimizer; notest

* add adam

* fix doc

* fix doc && add adamw; notest

* add error message

* bug fix

* refine rmsprop && adamax

* fix ci

* buf fix

* update comment

* unify arguments place; notest

* fix ut, test=develop

* bug fix

* fix conflicts, test=develop

* add examples code

* bug fix

* fix comments

* fix sample code

* add sample code for Optimizer

* add adamax ut, test=develop

* fix rmsprop ut, test=develop

* add ut for optimizer.py and adamw.py

* first commit of adadelta optimizer

* fix learning rate

* fix adadelta doc and add sgd momentum

* remove unused fluid

* fix codestyle

* Update test_adam_op.py

* Update test_adam_op.py

* fix SGD in 2 unittests

* fix SGD in 2 unittests

* fix ci

* fix ut
Co-authored-by: NMRXLT <xlt2024@gmail.com>
Co-authored-by: Nmapingshuo <mps2012@yeah.net>

* fix bug

* add version check

* fix docs, test=document_fix

* fix formula, test=document_fix

* refine paddle inference api
Co-authored-by: Nnhzlx <nhzlx.dragon@gmail.com>

* decouple weights and bias from fc primitive in cache

* removed reduntant update of pointers

Use `Tensor` instead of `Variable`  in the doc of paddle.grad.

* refine manual seed

* fix ci problem

* fix unittests

* fix unittest

* set is_init_py=false in manual_seed

* fix unittest

* fix bernoulli_op

* fix(unittest): change random_seed to manual_seed

* 🐞fix(unittest): fix manual_seed

* trigger ci

* fix test_sentiment

* fix test_imperative_save_load

* fix test_uniform_random_op

* fix test_uniform_random_op

* fix test_jit_save_load

* merge develop

* fix manual_seed

* fix manual_seed

* use global engine

* use shared_ptr

* fix double free

* fix bug

* fix bug

* fix bug

* fix test bug

* fix test bug

* fix test bug

* fix ci

* fix remainder, floordiv

* Fix cpu quantize placement pass

* Include string lib

* Add mkldnn bfloat16 option to C-API

* Add test for bfloat16 gpu

* Change coverage test

register cumsum Op version for compatible Op upgrades

add broadcast feature for elementwise logical op

* Fix clip input check

* Fix default min/max value

* Allow both max and min to be None

* Register op change

* Revert OP signature change

* update the demo code and the doc of varbase.backward.

* update the doc of the fake interface `paddle.fluid.Variable`.

* remove BackwardStrategy.

fix bug that can't print int8_t 

As the title

* Add the option to execute unit tests only at night

* set ut nightly label for 3 cases.

* add intepolte_v2

* fix linear interp

* polish unittest, test=develop

* update code samples to 2.0 API, test=develop

* remove warning, test_develop

* add name in attrs, test=develop

* polish code, test=develop

* change Align to align, test=develop

* fix unittest in py3,test=develop

* fix coverage, test=develop

* fix coverage, test=develop

* fix for windows ci, test=develop

* fix coverage, test=develop

* Update Paddle-Lite commit hash.

* Add BF16 data type to VarTyp protobuf message.

* update doc, test=document_fix

* add attr(dtype)

* refine code

add collective op for cpu using gloo and paddle.distributed.* apis

add the top v2 for the paddlepaddle api 2.0

test=develop

* add cudnn related strategies to DistributedStrategy